The Culture of Caution at NIH and Academia's Role

Yesterday's article by Gina Kolata about cancer research mistakes a symptom--caution due to a perceived lack of funding--for the disease, which is the symbiosis between academia and the NIH. Don't get me wrong, a lot of research should involve academics. But the priorities of NIH have become overwhelmed by the priorities of academia. I think Kolata's dead on target with this part of her article (italics mine):

One major impediment, scientists agree, is the grant system itself. It has become a sort of jobs program, a way to keep research laboratories going year after year with the understanding that the focus will be on small projects unlikely to take significant steps toward curing cancer.

"These grants are not silly, but they are only likely to produce incremental progress," said Dr. Robert C. Young, chancellor at Fox Chase Cancer Center in Philadelphia and chairman of the Board of Scientific Advisors, an independent group that makes recommendations to the cancer institute.

The institute's reviewers choose such projects because, with too little money to finance most proposals, they are timid about taking chances on ones that might not succeed. The problem, Dr. Young and others say, is that projects that could make a major difference in cancer prevention and treatment are all too often crowded out because they are too uncertain.

I'm not going to comment further on the specifics of cancer research--it's not what I do. But I've observed the phenomenon elsewhere in NIH--which is why I think it's a far more general phenomenon. Instead, I will focus on genomics and antibiotic resistance, which are two areas in which I am an expert of sorts.

What I'll propose (and I'm putting this up front because it's a long proposal) is that this dysfunctional system is derived from the emphasis NIH and other funding agencies place on R01 grants (or R01-'like' grants if NIH isn't the funding agency). By R01 grants, I mean grants that are budgeted at less than $500,000 per year--and usually less--and awarded to one main PI, although there may be a few other Co-PIs. To fix this, NIH (and other funding agencies) should be funding larger collaborative projects--the dreaded 'Big Science'--and downplaying R01s. (Note: I will refer to NIH and R01s as shorthand, even though the same types of grants are available at other funding agencies--I'm trying to avoid lots of alphabet soup). The R01 grants should function more as demonstration or proof-of-principle grants.

So onto an antibiotic resistance example. A while ago, I wrote about some excellent work by ScienceBlogling Tara Smith, and that was written up in The NY Times by Nicholas Kristof. Basically, Tara established that a clone of MRSA, ST398, which spread throughout Europe's farming system, and is now entering Dutch hospitals, and has established itself as a commensal in the human population, is found at really high levels in two Iowa farm systems (covering ~20 facilities). Now, under our current system, the next step would be that Tara applies for a grant--and Tara's very smart so she would probably get it--to look at more farming systems over a longer period of time (An aside: Having been a weird hybrid of federally funded researcher/policy wonk, I have some experience in what typically gets funded in this field). This is about what an R01-like grant with several years of funding could accomplish. And if Tara does go down this route, I wish her best of luck--she certainly has earned it.

But that's not ideally how the question should be addressed (and I've actually testified in front of federal committees about this--I'm not just ranting). To address the question of the introgression of resistance genes and resistant organisms from the farm to the clinic, you would need to do work on a scale that simply can not be funded by the R01 mechanism. You would need to collect tens of thousands (if not more) of isolates from farms, abattoirs, retail meat, fecal lagoons, farm workers, human commensal isolates (both in farming communities and distant from pig farms), pigs, and clinical isolates. These isolates would need to be systematically stored, screened for resistance, and genotyped (including sequencing, plasmid profiling, virulence factor typing (PCR and hybridization). There should also be a genomic component for a subset of the strains, both gene array and genomic sequencing. Then, of course, you need the informatics to store all of these data. And this is a multi-year project, and probably should be expanded to a couple of other critical organisms.

This is expensive. Really expensive. That means if this fails, it's very risky.

The culture at NIH, both in terms of outside reviewers (most of whom are drawn from academia) and program officials, has been really risk averse**. That's only partly due to a lack of funding (after all, unless a huge percentage of proposals were to be funded, someone's going to get shafted). From a program officer's perspective, spending lots of money in on a single program is very risky--I've seen large projects fail and it's not pretty. At the same time, that officer or institute will come under fire for starving other lines of research. As long as there's no pressure from those outside the research community, why rock the boat? Simply turn the difficult policy decisions over to the scientific reviewers and keep funding small grants.

From the perspective of academia, more resources devoted to larger projects means fewer smaller grants--and grants are how untenured professors get tenure, and tenured professors become full professors. Sure, if academics are involved with large project grants, they can swing some funding (sometimes a lot of funding), but it typically won't be 'their' funding, and that also matters for tenure decisions. Publications will have many, many authors, and, again, they can't be claimed as 'theirs.' Now, in defense of this system, it is very good for training students. It is very good at coming up with novel research ideas. It's a good system for academia, but, if you want to turn basic research into translational research (never mind interventions), it's not designed to do this. At some point, some of these ideas have to be 'embiggened': we need to give them the resources needed.

An example of this is a project some colleagues are doing that attempts to find novel molecules ('natural products') that bacteria produce, and turn them into medically useful compounds (e.g., antibiotics, tumor fighting drugs, etc.). The project starts with sequencing a bunch of bacteria, and, even though genome prices are dropping, it's still expensive--much larger than an R01 grant. Then the genes of interest have to be identified and cloned into the appropriate bacteria (you don't want to use E. coli for this). After this, the compounds have to be isolated (no small feat), slightly tweaked in thousands small ways, and screened in a high-throughput fashion against a variety of targets (e.g., bacteria, tumors, and so on).

As is the case in the theoretical example above, this is expensive. It also will take some time. But most importantly, it requires institutional commitment to see all of the steps through. You can't simply publish a paper identifying potentially interesting genes and leave it at that--you then have to do the next steps. And, even after all of this, it could still be a phenomenal bust. But if anyone has a better idea for discovering new classes of bioactive compounds, I'm all ears. It's worth the risk. And if you take enough risks, there will be some spectacular successes, along with some catastrophic failures. But, right now, looking at where we are in the area of antibiotic resistance anyway, the bacteria will evolve faster than our incrementalism--bacteria don't care about the professional concerns of NIH officials or academics.

So I'm staking my marker--here's what I think needs to happen:

  1. Funding needs to be shifted from R01s to larger project grants that provide the resources to do the science the 'right' way. Even if some large projects are basic research, we need to switch away from producing lots of little pilot studies that only hint at answers.
  2. R01s need to be viewed as pilot studies--if, after five years the results are promising, then provide adequate--that is, lotsa--funding. Yes, this will be harder on the career prospects for some faculty--that competitive renewal just got harder since competitive renewals will be larger and therefore fewer (not to mention, you'll probably have to round up a bunch of other collaborators). But the primary goal should be moving the science forward, not providing tenure milestones for universities.
  3. Program officers need to move policy directions back towards the NIH and away from advisory committees. Obviously, they shouldn't be micromanaging, but, too often, the direction that programs take is determined almost exclusively by grant review boards.

Anyway, this won't be the last I'll discuss this, so I'll stop here, even though there are a lot of other ideas bouncing around in my head.

Added after writing: David Gorski has a different take:

Blogger Mike the Mad Biologist has asked on two occasions, first in response to the NEWSWEEK article and then in response to this NYT article: Are these critics mistaking the symptom for the disease? As Mike points out, the problem goes beyond funding levels. Rather, it is incentives. It is where the money goes and what is funded. On that score, the NIH is profoundly schizophrenic these days. The sorts of promising high risk proposals do not in general come from large, multi-institutional, collaborative groups. There are too many interests, and such groups tend to have too much at stake to take many risks. The very sort of researcher who will propose the risky projects that all these reformers [want] is the small, independent researcher funded by an R01.

Needless to say, I think that 'big science' can be very transformative, although I agree that, right now, the debate is focused on anecdotes, not a systematic review of data. Might have to write a post about this....

*Two additional points. First, I'm focusing on the genetic epidemiology and population genetics--adding clinical/veterinary components would increase the cost further. Second, even if you disagree with my particular sampling scheme (this is a blog post, not a white paper), the point is that the project would to be big to draw any general conclusions.

**That's partially why all of the grief Francis Collins received when he was proposed as NIH chief was such bullshit: Collins knows how to get risk averse program officers to take risks.

Categories

More like this

DrugMonkey asks "Is the NIH trying to get rid of smaller laboratories?" Before I get to my take on the answer, what I'll be talking about is based on private conversations, so I won't be attributing some things to specific people (sorry Ed!). Take this for what it's worth... I don't think NIH is…
ScienceBlogling Tara Smith has a great summary of the recent paper reporting high frequencies of multi-drug resistant Staphylococcus aureus in uncooked meat products (S. aureus is the "SA" in MRSA--methicillin resistant S. aureus, so I won't go through the whole paper here (and Maryn McKenna, as…
A couple of weeks ago, NEWSWEEK science columnist Sharon Begley wrote an article entitled From Bench To Bedside: Academia slows the search for cures. It was a rather poorly argued bit of polemic, backed up only with anecdotes that came across as sour grapes by scientists whose grant proposals the…
In response to my post about the scientist glut, ScienceBlogling Razib writes: But that aside, what's the point of funneling more math and physics graduates into math and physics instead of finance if they can't put bread on the table? Or is the issue narrower, specifically the difficulty of…

Agreed in general Mike, though I'm not personally sure how to parse the etiology of the symptom/disease known as broken-grant-itis. There is definitely symbiosis re academia and NIH, but is this because our whole way of measuring and judging scientific output for grants is based on publication number and prospect, which is the coin of the academic realm?

We need a better way to measure capability (knowledge, resources) and measure success rather than just publication number and authorship - as you allude in your blog.

Ludwig Institute (flexible, long-term funding - see NYT article in question) has been trying to come up with a way of making sure our researchers don't miss out because they are working in multi-disciplinary teams (lots of authors), because they are working on speculative, long-term projects (slow publication rate), and because many are working on early-phase clinical trials (which are rarely written up). Any ideas?

By Sarah White (not verified) on 29 Jun 2009 #permalink

I disagree strongly with your notion that NIH focuses too much on R01s, but I'll have to think about a constructive response.

In the meantime, I am curious about two parts of your post:
1) The model you describe for antibiotic discovery by genomics - are there any successful examples?
2) Your claim about Collins... I'm not disputing it; I just don't know what you're referring to. Links? Are you talking about the genome project?

I disagree with you that NIH *in general* should shift funds away from R01s to big collaborative projects. What kind of science leads to the greatest progress is *highly* field specific. While the large-scale shit you are talking about certainly makes sense for the type of scientific question you use as your example, it makes absolutely no sense whatsoever in other areas.

This is particularly the case in physiology, where advances are made by independent investigators in a room with an animal and an apparatus to measure physiological parameters. There just ain't no way to scale that shit up in the way you describe.

I also disagree that bigger grants and fewer R01s are the way to go. In fact, in my area it is the "big science" grants that are less innovative, more cautious, and are essentially jobs programs for the big labs. Science is something that is better done by "craftsmen" rather than in "factories." There is much less room for creative thought in the "factories."

@Jim Hu,

(Just prefacing this by saying that I'm not advocating either position with respect to RO1s.)

The genome sequence wasn't done with this purpose in mind, but in the organism I study we noted many secondary metabolite clusters in the genome sequence. Without the sequence we wouldn't have noticed any of these. One in particular was interesting because of its association with a quorum sensing system. Mutational analysis demonstrated that it is functional against gram positive bacteria (Quorum-sensing control of antibiotic synthesis in Burkholderia thailandensis. 2009).

A coworker also developed a program to look for secondary metabolite clusters in fungal genomes (http://www.jcvi.org/smurf/index.php). I've been begging for a prokaryotic version, but I think you can get it to work with a prokaryotic genome now if you format it properly.

For whatever its worth, you can sequence "a lot" of genomes in a single SOLiD sequencing run. The cost the run isn't cheap, but on a per genome basis it is surprisingly low. At least the pricing options we've been given.

I misread Mike's post. I know about the kind of metagenomic work you're talking about - that's been going on for a while. For some reason I thought he was talking about targets, not drugs.

Illumina sequencing is also getting better and cheaper very fast. They gave a presentation here and they're now doing paired ends and the reads should be quite a bit longer soon. Annotation is going to become the slow step, if it isn't already.