Tracking the lag between promise and payoff.

One of the reasons non-scientists see science as at all valuable is that scientific research may result in useful medical treatments. And one of the aspects of science that seems elusive to non-scientists is just how long it can take scientific research to bring those useful medical treatments about.

In the 5 September 2008 issue of Science, Despina G. Contopoulos-Ioannidis, George A. Alexiou, Theodore C. Gouvias, and John P. A. Ioannidis [1] present research that examines just how long it has taken to get from initial discoveries to medical interventions.

Contopoulos-Ioannidis et al. identified articles that reported research showing a medical intervention to be effective in clinical trials and that were highly cited in the scientific literature (receiving more than 1000 citations each in 1990-2004). These articles represent an endpoint of a sort, the culmination of many preliminary research studies in clinical trials. However, of the highly-cited studies reporting positive outcomes in clinical trials, not all were replicated or even unchallenged by the end of 2006. Some were contradicted by subsequent studies. Others were

found to have had initially stronger effects ... when larger or better controlled subsequent studies were performed (1298)

The researchers then combed the literature to trace backward from these articles to their antecedents. In particular, for each of the highly cited articles in the study, they determined

the year when the earliest journal publication on preparation, isolation, or synthesis appeared or the earliest patent was awarded (whichever occurred first). (1298)

The time interval between the first report on preparation, isolation, or synthesis (or the earliest patent) and the highly cited articles reporting successful clinical interventions -- between the report of findings with clinical potential and the determination via clinical trials that that promise is realized in a treatment -- is the "translational lag". (There is, of course, another lag that's harder to quantify this way -- that between the initial findings in the research lab and the publication of those findings.)

Contopoulos-Ioannidis et al. found that the median translational lag for the highly cited articled in their study was 24 years. That's a long time. Moreover, they noted that the translational lag

was longer on average (median 44 versus 17 years) for those interventions that were fully or partially "refuted" (contradicted or having initially stronger effects) than for nonrefuted ones (replicated or remaining unchallenged) (1298)

This is an interesting difference. My first thoughts were that perhaps the longer translational lag represented scientific tenacity in the face of a potentially useful intervention. Alternatively, maybe scientists got to the findings that stood up faster owing to a better sense of what was likely to work; perhaps this sense prompted them to pursue their research with more vigor, getting it translated to clinical studies more efficiently.

But I suspect I shouldn't be trying to interpret the results in quite this way.

I was imagining two promising preliminary findings (call them X and Y) reported in the same year, then trying to imagine why X, translated to clinical trials more quickly, would also be more likely to stand the test of time than Y, which is translated to clinical trials more slowly. But this research works backwards from the highly cited successful clinical trials to the initially promising preliminary results. In other words, the clinical trial that is the descendant of Y (which ends up weakened or partially refuted) is working from a promising preliminary finding that has been kicking along for much, much longer. Why Y takes so long to be pursued may well be connected to the amount of promise researchers judge it to have.

As the authors put it:

claims for large benefits from old interventions require extra caution as they are likely to be exaggerated. (1298)

In light of this quantification of the lag in translating early research findings to medical interventions, what lessons should scientists draw? First, they ought to recognize what kind of time is required to realize the potential of promising findings. Contopoulos-Ioannidis et al. write:

As scientists, we should convey to our funders and the public the immense difficulty of the scientific discovery process. Successful translation is demanding and takes a lot of effort and time even under the best circumstances; making unrealistic promises for quick discoveries and cures may damage the credibility of science in the eyes of the public. (1299)

Of course, looking at research strategies and practices that can make translation more efficient would also help.

The authors also recommend appropriate skepticism about promising interventions "hiding in plain sight":

Translational efforts for common diseases should focus more on novel agents and new cutting-edge technologies; for these ailments, it is unlikely that genuine major benefits from interventions already known for a long time have gone unnoticed. (1299)

This is not to say that old interventions couldn't work, simply that they generally haven't resulted in clinically proven interventions described in highly cited scientific papers.

It seems like this last issue might be relevant to the discussion of autism and attempts to treat it going on in the ScienceBlogs Book Club right now.

_______
[1] Despina G. Contopoulos-Ioannidis, George A. Alexiou, Theodore C. Gouvias, and John P. A. Ioannidis, "Life Cycle of Translational Research for Medical Interventions," Science (5 September 2008) Vol. 321, no. 5894, 1298 - 1299.