Are we playing it too safe in cancer research?

A couple of weeks ago, NEWSWEEK science columnist Sharon Begley wrote an article entitled From Bench To Bedside: Academia slows the search for cures. It was a rather poorly argued bit of polemic, backed up only with anecdotes that came across as sour grapes by scientists whose grant proposals the NIH had decided not to fund, and based on many misconceptions she had regarding basic science versus translational research, journal impact factors, and how journals actually determine what they will publish. Not suprisingly, Begley's article caught flak from others, including Mike the Mad Biologist, Tim Kreider Steve Novella, and, of course, yours truly, who laid down some serious Respectful Insolence upon the entire article--and deservedly so, I might add.

To follow up on the same theme, an article appeared on the front page of the New York Times yesterday proclaiming that the NIH is "playing it too safe" when it comes to funding cancer research. Basically, it is a variant of the same complaints I've heard time and time again. Now, don't get me wrong. By no means am I saying that the current system that the NIH uses to determine which scientists get funded is without problems or a well-oiled machine. Those who complain that the system is often too conservative have a point. The problem, all too often, however, is that the proposals for how to fix the problem are usually either never spelled out or rest on dubious assumptions about the nature of cancer research--indeed, of research itself. Indeed, the NYT article strikes me as simply "Why are we losing the war on cancer?" 2009 edition. In fact, they look like more than that. As Mike the Mad Biologist speculates, one has to wonder if there is some sort of coordinated effort to pressure the government to change the policies at the NCI to be more nurturing of "risky" projects.

Basically, the lament goes something like this: Since the War on Cancer began 38 years ago, we've spent approximately $100 billion on cancer research, but we don't have teh curez!!!!! Of course, I like to point out that we've spend over $100 billion on another effort since 1983 (OK, 12 years less than the War on Cancer). It's also a problem that, although incredibly difficult, is far more straightforward than curing the multifarious spectrum of diseases known as cancer. I"m referring, of course, to the missile defense program started under the Reagain administration and sometimes disparagingly referred to as Star Wars. After all that money, what do whe have? A system that has never been tested under real world conditions and, most worrisome, struggles mightily to hit a test missile whose flight path and arrival time are planned and known. True, that's still a pretty impressive feat, which has been likened to hitting a bullet with another bullet, but it's nowhere good enough for me to have a lot of faith that the system can be counted on to work in a real attack. I could also point out that the entire $100 billion or so spent on the War on Cancer since 1971 is less than 1/5 the current yearly military budget and that in 2008 the U.S. spent nearly twice on missile defense what it spent to fund the NCI.

The NYT article, written by Gina Kolata, comes at the question of how the NIH funds research and what research is done in our academic medical centers from a different viewpoint than Begley. Kolata argues that the system is broken but sees the answer not in the sort of clinical research and comparative effectiveness research beloved by Begley, which in her ideal world would rapidly translate basic science findings into treatments and new tests. Instead, Kolata repeats a different mantra (and, truth be told, more common mantra) that seems to be going around, namely "high impact" research, which, we're told piously and solemnly, we are not doing nearly enough of because the NIH doesn't encourage it or fund it. Indeed, whenever I see an article entitled something like Grant System Leads Cancer Researchers to Play It Safe, I know exactly how it's going to start, and this article follows the playbook to the letter:

Among the recent research grants awarded by the National Cancer Institute is one for a study asking whether people who are especially responsive to good-tasting food have the most difficulty staying on a diet. Another study will assess a Web-based program that encourages families to choose more healthful foods.

Many other grants involve biological research unlikely to break new ground. For example, one project asks whether a laboratory discovery involving colon cancer also applies to breast cancer. But even if it does apply, there is no treatment yet that exploits it.

The cancer institute has spent $105 billion since President Richard M. Nixon declared war on the disease in 1971. The American Cancer Society, the largest private financer of cancer research, has spent about $3.4 billion on research grants since 1946.

Yet the fight against cancer is going slower than most had hoped, with only small changes in the death rate in the almost 40 years since it began.

You know, I've seen these articles often enough that I could grind one out myself without breaking a sweat. Again, I'm not saying that it's ridiculous to question why all that money and all these breakthroughs have not made a greater impact on cancer mortality than they have. On the other hand, there is a false premise in the whole question. Specifically, there is an unspoken assumption that "riskier" research is inherently more likely to result in "breakthroughs" than the more incremental model of building on previous results. The expectation is that all of that money should have produced "blockbusters" that make huge dents in cancer mortality. Unfortunately, science and biology are hard, and cancer is a complex and relentless foe. The intricacies of cancer biology conspire to frustrate even the most ambitious wishes for cures, whether they come from scientists, politicians, or a journalist like Kolata reworking a script that dates back to maybe a decade after the "war on cancer" began. (In fact, I'm dreading 2011, the 40th anniversary of President Nixon's declaration; look for pretty much every major news outlet to be publishing yet more articles of this type around then.)

It also disturbs me to see the clearly derogatory tone directed at the study of diet and health, which, quite frankly, seems at odds with the argument that the NIH doesn't spend enough on research into diet, exercise, and prevention, which by its very nature tends to consist of studies of this sort which are highly unlikely to produce anything other than incremental results leading to incremental improvements to our knowledge of how to use diet as a tool to prevent disease. Quite frankly, research of this nature isn't viewed as being as "sexy" as the sort of research Kolata thinks we should be doing more of. After all, it doesn't involve new genes, new proteins, or novel science, but rather deals with the application of what we have known for decades.

Let's get to the crux of the article:

"These grants are not silly, but they are only likely to produce incremental progress," said Dr. Robert C. Young, chancellor at Fox Chase Cancer Center in Philadelphia and chairman of the Board of Scientific Advisors, an independent group that makes recommendations to the cancer institute.

Again, note the contempt for such projects. "These grants are not silly"? Talk about damning with faint praise! But I digress. Let's get back on track:

The institute's reviewers choose such projects because, with too little money to finance most proposals, they are timid about taking chances on ones that might not succeed. The problem, Dr. Young and others say, is that projects that could make a major difference in cancer prevention and treatment are all too often crowded out because they are too uncertain. In fact, it has become lore among cancer researchers that some game-changing discoveries involved projects deemed too unlikely to succeed and were therefore denied federal grants, forcing researchers to struggle mightily to continue.

Here we go again. Everything old is new again. This is the very same complaint that pops up periodically. Again, I'm not saying that it doesn't have merit, only that it tends to be made in the absence of any hard evidence that (1) innovative ideas don't eventually get funded and (2) that funding "riskier" ideas will inevitably lead to more home runs (more on this later). Also, very conveniently, this sort of complaint always seems to pop up the most in lean times. Indeed, I remember back when I was a graduate student in the early 1990s (when Tim was in grade school). That is the last time the funding situation got as bad as it has been at the NIH for the last few years, and I remember reading articles very similar to this one. Inevitably, tight fiscal times appear to lead to a sort of funding conservatism for exactly the reason above: The NIH doesn't want to risk precious grant funds on projects that are too risky.

As evidence of this problem, does Kolata dig into the grants system and try to put together data showing that conservative funding policies are leading to the shutting out of game-changing research? What do you think? Of course not! Instead she relies, just as Begley did, on anecdotes from scientists who produced great work but were not funded initially. First up in the anecdote parade is Dennis Slamon. Personally, I admire Dr. Slamon's work greatly. Through science and determination, he truly did change breast cancer therapy, and for the better. Specifically, he is the person who developed trastuzumab (trade name: Herceptin); i.e., a humanized monoclonal antibody against the HER-2/neu oncoprotein, which was discovered by Robert Weinberg's group in 1984. HER-2 in general portends poorer prognosis and is generally a marker for more aggressive cancers with lower survival rates with conventional therapy. The reason is that HER-2 encodes a cell surface receptor that is a member of the epidermal growth factor receptor family. When it is amplified, as it is in approximately 22% of human breast cancers, its activity can result in increased cell proliferation, cell motility, tumor invasiveness, a higher likelihood of regional and distant metastases, accelerated angiogenesis, and reduced apoptosis. All of these are bad things. Consistent with this biological effect, HER-2 amplification, the presence of HER-2 amplification correlates with nastier appearing tumors on histology, decreased disease-free survival, increased metastasis, and decreased overall survival. Indeed, for overall survival and disease-free survival, the relative risk of death for women with HER-2-positive cancer versus HER-2-negative cancer is in the 1.8 to 2.7 range. In other words, women with HER-2-positive breast cancer are more than two times more likely to recur and die of their disaease.

It is also true that the addition of Herceptin, which blocks the HER-2 receptor, to standard chemotherapy of HER-2-containing cancers improves the prognosis in women with such cancers, although Kolata overstates how much it does so. For all its usefulness, Herceptin is not a magic bullet; it is not a cure. Indeed, the original phase III trial reported in the New England Journal of Medicine in 2001 found that in patients with HER-2-postive metastatic breast cancer, the addition of trastuzumab to conventional chemotherapy resulted in a longer time to disease progression (median, 7.4 versus 4.6 months; p

In Kolata's article, Slamon complains that he had difficulty getting the NIH to fund his studies and that it took a grant from Revlon for him to continue his research. His story is a bit too convenient, as a perusal of the Internet using my mad Google skillz found that the story is a bit more complex than what Kolata tells. Indeed, a made-for-TV movie about Slamon and Herceptin aired on Lifetime in 2008 called Living Proof, which was based on a book by Robert Bazell called HER-2: The Making of Herceptin, A Revolutionary Treatment for Breast Cancer, tells a more complex story than just the NIH's not funding his research. For example, this passage from a review of Bazell's book is illuminating:

Then, another stroke of dumb luck occurred in 1986. Ullrich accidentally met Dennis Slamon in a Denver airport. Slamon, a practicing oncologist at UCLA's Jonsson Cancer Center, is a dogged and devoted cancer researcher. Throughout the next 2 years, Slamon and Ullrich pursued the hypothesis that HER-2 neu played a role in the growth of breast and ovarian cancers. By 1987, they published their results, which suggested that cancers overexpressing HER-2 are more likely to recur and spread more quickly.

But could their work be reproduced and their conclusion be confirmed? More delays ensued as their colleagues failed to reproduce their experiment. These delays were dumb bad luck. Two long years later, Slamon and Ullrich proved that the failure to reproduce their work was due to the "use of contaminated chemicals, faulty techniques, and idiotic mistakes by the laboratories conducting the experiments."

By 1988, Slamon and Ullrich looked to Genentech for support to take their promising experiment to the next level of development. Support was not forthcoming. Genentech was no longer focused on cancer drug development. Their oncology staff had been disbanded following the unsuccessful interferon-alfa trials.

Herceptin's development languished until more dumb luck occurred. In late 1989, the mother of a senior Genentech vice president was diagnosed with breast cancer. "Just like that, one man flipped the switch on HER-2" and convinced his colleagues that HER-2 was worth Genentech's investment. Again, and not for the last time, an unexpected player would rescue Herceptin.

It's just sloppy reporting not mention this background and simply buy into the image of the "brave maverick doctor" whose work wasn't appreciated by the NIH. Why did the NIH decide not to fund Slamon's work? Without seeing the pink sheets (the summary statement that all applicants for a grant receive containing reviewers' evaluations of the grant appplication), it's impossible to know for sure. In the absence of having read Bazell's book, however, I can still speculate a bit, at the risk of looking foolish. If, for example, Slamon and Ullrich (the latter of which was a Genentech scientist) had submitted their HER-2 grants around the time other researchers were having difficulty duplicating their results, the reviewers at the NIH study section responsible for reviewing Slamon's grants would almost certainly have known about it. Moreover, if a pharmaceutical company had been developing Herceptin, it is possible that reviewers would have been less enthusiastic about it, wondering quite reasonably why the NIH should fund the development of this drug if Genentech would no longer do so. As much as I love a story of a scientific maverick overcoming the hidebound system (why else would Judah Folkman be one of my scientific heroes?), I'm not sure that Slamon's case represents as pure an example of such a scientist as this article would have you believe. (Maybe I should get a copy of Bazell's book and read it.)

It's also rather odd that Kolata would focus on Slamon first. It is true that Herceptin represents the first successful example of a molecularly targeted therapy for breast cancer. (Well, not quite. One could equally well argue that the anti-estrogen drug Tamoxifen, which blocks the proliferative effect of estrogen due to its binding with its receptor, were the first.) However, only 20% of women with breast cancer can be expected to benefit from it, and it is an incredibly expensive drug, and it is not without its toxicities, the worst of which is toxicity to the heart, which precludes its use in many women with heart disease. In any event, the high cost of Herceptin (several thousand dollars a month, with the usual duration of treatment for breast cancer being one year). Indeed, when research suggested a benefit for extending the use of Herceptin for adjuvant therapy against early stage tumors, in contrast to its initial uses in more advanced tumors, some European governments balked at the cost.

Two more anecdotes follow, which are more interesting but less convincing, namely because we don't have the benefit of history to tell us that the investigator was definitely on to something, as we do with Dennis Slamon. Because one of the researchers whose anecdote is told didn't even bother to try to apply to the NIH for funding because he figured it wouldn't be fundable. Whether he's right or wrong, who can tell? Instead, I'll concentrate on the other anecdote, that of Eileen K. Jaffe of Fox Chase Cancer Center:

When Dr. Jaffe stumbled upon results that went against textbook explanations, suggesting that it might be possible to find an entirely new class of drugs that could disable proteins that fuel cancer cells. Now she wants to find chemicals that might be developed into such drugs.

But her grant proposal was rejected out of hand by the institutes of health, not even discussed by a review panel. She had no preliminary data showing that the idea was likely to work, something reviewers always want to see, and the idea was just too unprecedented.

And this is not the least bit unreasonable, if indeed her idea was that novel. Indeed, even Dr. Jaffe acknowledges this:

Dr. Jaffe is just conceiving her project; it is much to soon to know whether it will result in a revolutionary drug. And even if she does find potential new drugs, it is not clear that they will be effective. Most new ideas are difficult to prove, and most potential new drugs fail.

So Dr. Jaffe was not entirely surprised when her grant application to look for such cancer drugs was summarily rejected.

"They said I don't have preliminary results," she said. "Of course I don't. I need the grant money to get them."

Actually, that's not quite true. It is, for example, close to true for the gold standard of NIH grants, the R01. These grants generally provide around $150,000 to $250,000 a year for three to five years to fund a project, and it is usually about these grants that most of the complaints are made. That's because reviewers do tend to be pretty conservative about such grants. The reason is that these grants provide a lot of money for a lot of years and can be renewed at the end of each grant through a process known as competitive renewal, in which the investigator reports on the progress made in the previous grant period and proposes where he wants to go over the next five years. In other words, an R01 is a huge commitment, and, again not unreasonably, reviewers want to see a lot of preliminary data to suggest that the project is feasible and likely to produce results that improve our understanding of a disease and lead to strategies for therapy. It's exactly the sort of grant mechanism designed to look at a question like this one described in the article:

In the study asking whether a molecular pathway that spurs the growth of colon cancer cells also encourages the growth of breast cancer cells, the principal investigator ultimately wants to find a safe drug to prevent breast cancer. She received a typical-size grant of a little more than $1 million for the five-year study.

The plan, said the investigator, Louise R. Howe, an associate research professor at Weill Cornell Medical College, is first to confirm her hypothesis about the pathway in breast cancer cells. But even if it is correct, the much harder research would lie ahead because no drugs exist to block the pathway, and even if they did, there are no assurances that they would be safe.

I actually agree to some extent with Kolata's thesis, namely that much of the grant funding at the NIH is too conservative and that the NIH should find a way to fund more innovative and "risky" grants. Indeed, I tend to approve of various initiatives to fund innovative research, such as "pioneer awards," which are designed to fund research examining "ideas that have the potential for high impact but may be too novel, span too diverse a range of disciplines or be at a stage too early to fare well in the traditional peer review process," and the so-called "transformative R01 grants" for "proposing exceptionally innovative, high risk, original and/or unconventional research with the potential to create or overturn fundamental paradigms."

One difference between me and those calling for reform of the NIH grant process is that I openly admit that I don't have any data to support my bias that the current system is too conservative and that a way needs to be found to fund more innovative (or at least different) research. What bothers me is that neither Begley, Kolata, nor any of the scientists interviewed by both of them seem able to present any hard data or science to support their bias either. They believe that funding more risky projects will result in better payoffs than sticking with the slow march of incremental science. They have anecdotes of scientists whose ideas were later found to be validated and potentially game-changing who couldn't get NIH funding, but how often does this really happen? The vast majority of "wild" ideas are considered "wild" precisely because they are new and there is no good support for them. Once evidence accumulates for them, they are no longer considered quite so "wild." More importantly, we are looking through what we doctors like to call the "retrospectoscope," which, as we say, always provides 20-20 vision. We know today that the scientists whose anecdotes of woe describing the depradations of the NIH were indeed onto something. How many more proposed ideas that seemed innovative at the time but ultimately went nowhere?

We don't know.

We also don't know exactly how the NIH would choose between so many "risky" projects. Sanjay Srivastava, writing about the NYT article, asks an excellent question:

The practical problem is that we would have to find some way to choose among high-risk studies. The problem everybody is pointing to is that in the current system, scientists have to present preliminary studies, stick to incremental variations on well-established paradigms, reassure grant panels that their proposal is going to pay off, etc. Suppose we move away from that... how would you choose amongst all the riskier proposals?

People like to point to historical breakthroughs that never would have been funded by a play-it-safe NCI. But it may be a mistake to believe those studies would have been funded by a take-a-risk NCI, because we have the benefit of hindsight and a great deal of forgetting. Before the research was carried out -- i.e., at the time it would have been a grant proposal -- every one of those would-be-breakthrough proposals would have looked just as promising as a dozen of their contemporaries that turned out to be dead-ends and are now lost to history. So it's not at all clear that all of those breakthroughs would have been funded within a system that took bigger risks, because they would have been competing against an even larger pool of equally (un)promising high-risk ideas.

In other words, a lot of the current crop of criticisms of the way the NIH selects grants of the sort put forth by the NYT rest on a large measure of selective memory and confirmation bias. Science that is successful is remembered; proposals that go nowhere are lost to the mists of time. Scientists whose work was later validated after the NIH didn't fund it are remembered and make good press copy. The many more scientists whose work wasn't funded and went nowhere aren't.

Moreover, it's easy to make grand claims that all that nasty preliminary data isn't necessary, that we should fund "plausible" studies that sound promising. However, it is the preliminary data supporting them that turn studies from speculative to plausible. Without that data, the possibilities are virtually endless, with little to distinguish truly plausible proposals from interesting but implausible ideas.

Of course, given that the conservatism of the NIH grant process always tends to be more of a complaint when funding is tight, I could envision a way of testing the hypothesis that funding more "risky" research results in more breakthroughs. It would be imperfect, but it might provide enough evidence to justify further exploration of this idea. Specifically, I'm referring to the period from fiscal year 1998 to 2003, during which the NIH budget, thanks to bipartisan support, doubled. The NIH could look at funding data from that period and ask some questions:

  1. Were more "risky" grants funded?
  2. What was the result of those grants?
  3. Were "riskier" grants more or less likely to result in new treatments that impacted the survival of cancer patients?

I realize that it would be difficult to come up with measurements to answer such questions that would be truly objective, but even imperfect data on this score would be better than what we have now, which is in essence no data. Or, at least if there is such data, I couldn't find it. Perhaps that's because the assumption that more "innovative" research must be needed because we haven't impacted cancer survival rates as much as we think we should have during the last 38 years, which means we need more innovative research. Circular reasoning at its finest, at least without some hard data. Given that what happened in response to the doubling of the NIH budget was in essence a spending spree that attracted more applications than ever to the NIH and spurred more building of research facilities by universities, I rather suspect that such a study would not show what proponents of altering how the NIH decides what research to fund would want it to show.

Blogger Mike the Mad Biologist has asked on two occasions, first in response to the NEWSWEEK article and then in response to this NYT article: Are these critics mistaking the symptom for the disease? As Mike points out, the problem goes beyond funding levels. Rather, it is incentives. It is where the money goes and what is funded. On that score, the NIH is profoundly schizophrenic these days. The sorts of promising high risk proposals do not in general come from large, multi-institutional, collaborative groups. There are too many interests, and such groups tend to have too much at stake to take many risks. The very sort of researcher who will propose the risky projects that all these reformers is the small, independent researcher funded by an R01. Yet, what has the NIH been shifting its funds to lately??

Big science. Large projects. Indeed, in 2004, articles like this were a regular staple in the press, pontificating:

The NCI can begin this transformation right away by drastically changing the way it funds research. It can undo the culture created by the RO1s (the grants that launched a million me-too mouse experiments) by shifting the balance of financing to favor cooperative projects focused on the big picture. The cancer agency already has such funding in place, for endeavors called SPOREs (short for specialized programs of research excellence). These bring together researchers from different disciplines to solve aspects of the cancer puzzle. Even so, funding for individual study awards accounts for a full quarter of the agency's budget and is more than 12 times the money spent on SPORE grants. The agency needs to stop being an automatic teller machine for basic science and instead use the taxpayers' money to marshall a broad assault on this elusive killer--from figuring out how to stop metastasis in its tracks to coming up with testing models that better mimic human response.

If there's a paradigm more designed to encourage, rather than discourge, groupthink, it's this. There's a place for large, collaborative studies, but such studies are rarely that innovative. It takes so many resources to put them together that there's too much to lose, which breeds a conservatism of a kind without the NIH needing to enforce it through overly cautious grant reviewing. Walter Boron put it well in a commentary in Physiology three years ago:

But there is still one more piece to the pay-line puzzle: the allocation of NIH dollars, sometimes mandated by Congress, and often following the advice of committees of independent investigators. The fraction of the NIH budget devoted to research by independent investigators (Table 1) steadily fell from 1998 to 2003. Conversely, spending for other programs including "big science"--the sequencing of genomes, clinical trials, and other costly and lengthy projects--steadily rose. Where does one draw the line between shifting funds to big science and yet maintaining a healthy portfolio of independent-investigator research? When the NIH is afloat in money (e.g., pay lines 25th percentile for research by independent investigators), such a shift may make sense. An example is the sequencing of the human genome, which has been invaluable. But what about sequencing the squid genome, which I personally would love to see? Before addressing this question, let us examine the value of investigator-initiated research and the dangers of interrupting it for even a couple of years.

The scientific engine that drives translational research--and that drives big science as well--is the independent investigator. It is the independent investigator who trains the next generation of researchers. Moreover, discoveries almost always come about when bright independent investigators stumble over unexpected findings and then sort them out. Such stumbling is unpredictable. The bigger the discovery, the more unpredictable. As unnerving as it may seem, the best way to invest in discovery is to fund the best independent investigators and turn them loose to stumble.

Of course, Dr. Boron doesn't really have any data to support his thesis, either, but he does illustrate the conflict between "big science" and the innovation that NIH reformers are pushing for. Also, if anecdotal evidence counts, then let me place my own personal anecdote on the table. One of the two projects I'm working on now came about from a completely serendipitous discovery on the part of my collaborator, who pursued a completely unexpected observation. Based on that discovery, the NIH has funded two R01s, and the ASCO Foundation has funded me. Based on my anecdote, the NIH must fund risky and innovative research.

Yes, two can play at the anecdote game.

All snarkiness aside, though. One problem is that the public has a rather distorted view of how science works. Science generally does result in incremental progress. Sometimes, there are even periods of stagnation, during which, or so it seems, very little is discovered and few advances made. Breakthroughs, such as the discovery of HER-2 are much less common than the gradual accumulation of knowledge and understanding that builds on what has been done before. Indeed, even a breakthrough like HER-2 was built on what came before, as Dennis Slamon's work could not have occurred were it not for Robert Weinberg, who discovered HER-2 in the first place.

I think that the issue is better put by Wafik el-Deiry, the physician-scientist who discovered p21WAF1/CIP1 a very important cell cycle regulator whose expression p53 activates. What I consider to be cool about Dr. el-Deiry is that he's on Twitter, where he Tweeted in response to the NYT article:

The major advances in cancer research have come from basic research without expectations of immediate impact on patients' lives

And followed up with this Tweet:

Yes, I worry about change that may set us back by putting less value on basic science & more on hi risk pie in the sky

I agree and share Dr. el-Deiry's concern. Oncogenes, tumor suppressor genes, HER-2, intracellular signaling molecules, all of these and more were discovered by basic scientists working because they simply wanted to know how cells work and what goes wrong to turn them cancerous. What I worry about is that, in the rush to fund more "innovative" and "translational" research, basic science will be left out. Why this worries me is that, without basic science, there can be no translational science. Translational research depends upon a constant flow of new observations and new discoveries in basic science.

More importantly, it can't be predicted where those new discoveries will come from. Sometimes they come right out of left field, like the aforementioned project I'm working on now, which resulted from a serendipitous discovery by my collaborator and has the potential to result in a great new treatment for not just breast cancer but melanoma as well. It was not the sort of discovery that could have been foretold, and it may never have been noticed if it hadn't been for a basic scientist following curiosity where it led. In any case, those who advocate for funding more "risky" research must have a lot of faith in current scientists who serve on NIH study sections to identify what proposals are truly innovative. I'm not sure I share that faith. In fact, I'm sure that I don't. At the risk of belaboring the point, I will repeat that many breakthroughs could not have been identified in a research proposal beforehand. Moreover, it can't be emphasized enough that translational research depends upon a steady stream of interesting science from the laboratories of basic scientists. Dry up that stream, and translational research will slow to a crawl.

I also have to wonder whether, if critics got their way and the NIH were to put a lot more money into funding "high risk studies," there would be so many failures that the press would suddenly start saying, as Sanjay put it, "Grant System Funds Too Many Dead Ends."

Categories

More like this

I'm not a biological investigator -- I'm a social scientist -- so I can't comment specifically on the field of biomedical research into cancer. However, I do think the general view that there is an inherent conservatism built into science funding and publishing is entirely legitimate. Really innovative or radical ideas have a difficult time getting attention and developing into programs of research. That's not necessarily wrong or bad, at least at the funding end -- resources are limited and proposal reviewers have to put their bets on stuff that's likely to work. It's awfully hard to evaluate a proposal with a potentially huge payoff but an imponderable likelihood of panning out.

On the publication end, however, the problem goes beyond "extraordinary claims require extraordinary proof." Findings don't have to be upsetting to the CW or radical to be hard to publish in a reasonably visible journal; they just have to not fit very well into theoretical frameworks and research programs that are currently in vogue. The conventions for writing scientific articles require us to put our study into the context of other, ongoing work, and the deeper it sinks into that context and the more precisely it answers some narrow, outstanding question within the vogue research program the more likely it is to be published. Terra nova remains uncharted in the literature.

Again, I'm not absolutely declaring this to be wrong or dense, and it's certainly not conspiratorial. Just an observation, make of it what you will. For people who are desperate for breakthroughs, however, it is naturally discouraging. And yes, we are likely missing something, indeed quite a few somethings. We'll stumble across theme eventually.

The romantic view of a heroic scientist fighting the establishment is bogus, as real scientists know. But non-scientists love that story, and it's what is behind the push for more "risk-taking." The criticism is based on ignorance, just like McCain's critique of beaver-control research or Jindal's jests aimed at volcano monitoring.

But real science is hard work, with gains wrought incrementally over time. My main concern with the NIH funding mechanism is that it doesn't give long-term security to scientists. There's tremendous attrition in the younger ranks. Who wants a career that pays less than private enterprise and has a high risk of job loss?

Well, here's a question - how would you figure out whether NIH was being too conservative? How would you quantify that? I figure this is a question John Ioannidis would have fun examining...

By Trine Tsouderos (not verified) on 29 Jun 2009 #permalink

I'm all for the NIH funding "risky" projects, particularly since I'm preparing a grant for an idea that I'm worried will make the reviewers wonder what I've been smoking*, but I don't see by what definition we can be said to be LOSING the "war on cancer." We aren't "winning" in a "nuke the last of them into submission" sort of way, but we aren't losing either.

Cancer, as orac and others have pointed out multiple times, is not a single disease. It won't have a single cure. And cancer cells are extremely similar to normal cells, making treatment a challange. This "war" is not a quickie invasion of some little Central American country, it's the 100 years war. And we ARE winning it. Many cancers have a better prognosis today than they did 20 years ago, most have a better prognosis than they did 50 years ago. Some are curable even when discovered in stage IV (after spreading to other organs.) This isn't a TV show where all the problems are solved before the commercial break. We're not going to have a cure for "cancer" (that is, all cancers) no matter how much the NIH gambles on risky projects.

With a reasonable amount of luck and if there are no catastrophic changes in society, we'll have improving treatments for many cancers as time goes on. Some, like testicular cancer or HL, will become increasingly curable over time. Some, like cervical cancer, increasingly preventable. Others will continue to be a problem probably into the next century. That may not be "total victory", but I don't think it can be called defeat either.

*But I do have preliminary results, if I can get anyone to read that far...

Large software systems happen to be my main area of expertise, and I have to say you're seriously understating the difficulty the Star Wars program faces. It may actually prove to be a more difficult problem to solve than the cure for cancer.

The issue isn't the hardware. That's a solvable problem, albeit one that has proved much more difficult than originally anticipated.

The real problem is the software. This would be one of the largest, if not the largest, software systems ever constructed. I've seen line count estimates ranging from 15 million lines of code to over 40 million lines. And it has to work essentially perfectly the first time on a time scale meaured in a small quantity of minutes, and there's no way to really test it.

Existing software design methodologies are not capable of handling this. They aren't even close, and what's more nobody has even an inkling of how to improve them so they would be adequate. If it's even possible it's going to require a radically different approach (sound familiar?) that goes way beyond how we build large scale software now.

If you want to read more about this, I suggest starting with the comments of David Parnas, the developer of the concept of modular programming, which is what has made really large software systems (though not as large as Star Wars) possible.

Modular design works by breaking up a big programming problem into lots of smaller size problems that are easier to solve, and more importantly, test. But even if all the modules are perfectly constructed (which they won't be), additional issues will show up in the interconnects between them. Successful large scale software systems depend on the interconnectedness of the various modules being limited, but Star Wars presents a highly interconnected problem. That means there are going to be all sorts of problems that only show up at full, operational scale and I really don't think we want to start a nuclear war in order to shake those bugs out.

Large software systems happen to be my main area of expertise, and I have to say you're seriously understating the difficulty the Star Wars program faces. It may actually prove to be a more difficult problem to solve than the cure for cancer.

Quite frankly, I highly doubt that. The inner workings of a single cell are arguably more complex than the most complicated software ever designed. Only now are we starting to understand even the most basic rudiments of the network design of a single cell.

I agree with Orac, the complexity of a system like starwars is trivial compared to the complexity of living cells. Nothing about physiology was designed in a modular way. It is all non-linear and coupled with the coupling happening over multiple time and distance scales spanning multiple orders of magnitude.

Sure, cells are stunningly complex, and we're only beginning to understand them.

But two additional points, if I may.

First, it is unlikely that a total understanding of cellular chemistry will be needed to "cure" cancer, to some reasonable first approximation of "cure".

Second and more important, it's a fundamentally different sort of problem. The thing is, cells and cancer both exist. We can study them, and the tools for studying them are powerful and constantly improving. Sooner or later we'll nail down the necessary details, although of course using our knowledge to construct that "cure" may be a very difficult problem in and of itself.

Software engineering is different. The number of possible approaches is astronomical and there are no comparable systems to study.

To put it in biological terms, it's like trying to design, oh, say, a very simple organelle whose function follows a very strict set of specifications, but without having *anything* to model it on.

Journalists always get into a lather about speculative research that promises the moon - so if anyone seriously wanted to investigate where such ideas go, a good place to start would be to look up such reporting in journals, magazines, and newspapers from a few decades back and track where those ideas actually went.

Let me state very briefly what the problem with biomedical research and modern science in general (entirely my humble opinion). It's that scientists don't know how to reason with partial information. The problem with reasoning with partial information, that is, when you don't know the solution to your problem, or the problem itself is ill-defined is that it's really hard to get feedback on whether or not, or to what degree your problem solving approach(es) are effective.

The problem is that the modern university places too many demands on a prospective and current scientist. They have to manage large staffs and budgets, apply for grants, teach and hold office hours, etc., in addition to conducting research. Now I agree that specialization is very much necessary in the modern world, but it's exactly the wrong kind of specialization that is being regarded as so. We emphasize specialization by discipline and subdivision of knowledge, yet we require scientists to be generalists in that they have so many commitments that are ancillary to the main core of the scientific enterprise. If generalists in terms of knowledge and interdisciplinary research was the typical culture and scientists were given freedom to focus exclusively on their ideas, my bet is that there would be an explosion in scientific output and progress. But this is an argument for another day...

Again, a huge problem in science today is that the organization isn't evolving as fast as it probably could. And the reason why this isn't happening is because there are very few selective signals that allow the feedback from a particular way of doing things to then improve the approaches, methods, variations, and combinations that increase the probability of future success; this is because, as I've mentioned, it's hard to gauge the success of a method when the time frame for success or failure in reaching ones goals as an organization are ill-defined. But this problem isn't insurmountable. It just takes an honest look at the situation; the patience to observe an institution's ways and means of achieving its goals, and subjecting the organization itself to the scientific method - of incremental improvement.

Once the organization itself is subjected to the scientific method, one will be able to make the true significance of the word team materialize. A team that is a whole greater than the sum of its parts, not merely a collection of individuals. Presently, the level of functioning of a team or organization is not really measurable, especially in the context of scientific research, where the goal is to discover something new. It might be actually that the present level of functioning really is the best that we can do, or close to it. Then, all that we can do is keep chugging along. But if we could improve the organizational aspect of science by an order of magnitude - then our present ways of doing science could very well be the "limiting reagent" in terms of what could be generated as "products" (progress in science) given some amount of "reactants" (scientists, funding, equipment, etc.) We won't really know until we look at organizations themselves in a thoroughly scientific manner; particularly organizations that have complex goals that are difficult to measure.