Mike the Mad Biologist

More specifically, I think more of the NIH budget needs to be much more focused and targeted, and less researcher driven. In a post about NIH proposal revisions (i.e., resubmissions after a proposal has been rejected and critiqued), ScienceBlogling DrugMonkey writes (italics original):

The reason is that this policy does nothing about the tendency of reviewers to focus on grantsmanship issues as an easy triage mechanism, instead of taking the “fish or cut bait” hard look at the genuinely new application the first time. The primary stage of review is the main driver here. The ameliorative measures should have accounted for the source of the problem and tried to address it more directly. The single amendment limit doesn’t do this.

One of the ways the original goal ["to ensure earlier funding of high-quality applications and improve efficiencies in the peer review system"] could be accomplished would be through reviewer education and instruction. Put the data figures in front of all reviewers and say “Bad dog! Stop deifying revision status and grantsmanship. Focus on the underlying science. What will really be accomplished through the review process- changes in the proposal only? or actual changes in the resulting science?“

The problem, I think, is more fundamental than this. It’s analogous to college admissions at highly selective institutions (e.g., 5-15% acceptance rates). Most of these schools would be able to accept an entire class, get rid of it, take the next ‘class’ down, get rid that, and take the third cohort…and not miss a beat. And most will admit that if they went an additional class down, the drop off would be very little*. Ultimately, admissions officers have to come up with reasons to disqualify qualified students.

A similar phenomenon applies to NIH proposals. With the current acceptance rate of around twelve percent, roughly two thirds of perfectly fine proposals wind up being rejected. By “perfectly fine”, I mean proposals that have a reasonable chance of working out, and, if they did, would be interesting to other scientists in that area (i.e., if you heard a seminar on the work, you would conclude it’s solid work). I think about another twenty percent of proposals are basically one step away from moving into this category.

So there are too many proposals chasing too little money? Tell me something I don’t know, Mad Biologist. Much of the funding that NIH provides is ultimately investigator driven. That means a researcher (or small group of researchers) develops a proposal and submits it. While NIH and its component institutes do have different sections, they are so broad as to be essentially meaningless (antibiotic resistance, inflamatory bowel disease). I once participated in a roundtable about antibiotic resistance, and one of the attendees was the NIAID program officer who oversees all of the antibiotic resistance research (around $800 million). Many of the participants were frustrated that NIAID wasn’t funding certain areas adequately (e.g., pharmacokinetics). I’ll never forget what the program officer said: “I can only fund what is sent to me.”

The point is that, as DrugMonkey noted, it’s too difficult for reviewers or program officers to reject proposals based on their unsuitability for the goals of NIH, since these goals, even within certain areas, are too broadly defined. My experience has been that with very targeted calls for proposals, there are far fewer proposals submitted, and it’s much easier to flat out reject them because many proposals are not germane to the funding objectives. This means that NIH program officers have to be far more active in defining specific research objectives than they have been–to a considerable extent, NIH is placing this responsibility on reviewers who often lack knowledge of the larger institutional objectives. That needs to be changed.

I’m not saying that there shouldn’t be any ‘open’ funding, but the ‘open’ panels should be viewed more as demonstration projects. If after five years of funding, a case can’t be made that funding that project and other related projects is good idea (i.e., target additional funding to that very specific area), then the competitive renewal should be turned down.

This wouldn’t be a perfect system–at some point, funding would have to be opened up because novel ideas will be cut off. But right now, because funding is a free-for-all, the system is rewarding grantsmanship at the expense of novelty anyway, with the added bonus of overworking the reviewers.

Discuss.

*How one defines what a ‘good’ applicant is, of course, highly subjective.

Comments

#1 Dan
October 21, 2008

I think more of the NIH budget needs to be much more focused and targeted, and less researcher driven.

Isn’t that just inviting *more* political interference in science???
#2 Jim
October 22, 2008

I too worry that NIH senior staff and invited Council members, as talented as they are, but who already make decisions about what areas of research are “interesting” or “valuable”, will have a bit too much control over the direction science moves if they define the projects so narrowly as to exclude young scientists and mavericks (sorry about that John and Sarah) with original ideas that may eventually drive future research directives.
#3 Peter Uetz
October 22, 2008

Europe tried restricting science to particular goals with their Framework Programs (FPx) and what happened? Once the “goals” were defined for the next 5 years, someone discovered RNAi, which was not on the agenda, and then there was no funding for it for the next 5 years!

No, it’s not a good idea to have a few people decide what should be funded. A certain fraction is o.k. (25% for RFAs, contracts, earmarks ?). But there must be enough room for scientists to come up with their own ideas.
#4 Warren
October 22, 2008

The problem with this idea is that you get top-down direction of the research and lose one of the main benefits of the current structure, the input of ideas from all the applicants.

If this idea was in place in the forties and fifties, there would have been a call for development of really light, portable iron lungs and lighter leg braces as the solution to the polio epidemic.

Certainly the phenmenon of bacterial restriction would have been consdered esoteric and unfundbale, since it had no obvious value to health care – whoops, there goes recombinant DNA technology.

So, you would solve the problem of too many applications, by restricting the topics that would be allowed, but you would destroy the essential strength of the system, which is to provide support for a wide variety of of research, which is the source of the innovation that is the true goal of the NIH funding program.
#5 Rich
October 22, 2008

I fear that this attitude reflects the mistaken belief that we can easily predict where the next great breakthroughs are coming from, or that we can predict the implications of one line of basic research on another. Trying to force investigators to do specific things for specific goals may be useful in more mature fields like engineering, but will eliminate a lot of the most exciting basic work because that work is inherently less predictable.

I agree that NIH funding is too low and that historically 30-50% of grants were deemed worth of funding and now ~70% of those “fundable” grants have to be eliminated. But I don’t agree that narrowing the focus of NIH is the way to do it. Increasing the funding to NIH is the way to relieve this pressure.
#6 Mike the Mad Biologist
October 22, 2008

I seemed to have touched a nerve; I’m not arguing that all investigator driven funding should be eliminated, but that it needs to be cut back, relative to more targeted areas. Saying we need to increase NIH funding isn’t politically viable (and unless funding increases at the same rate of PI creation-PI retirement, you still have the same problem).

I’m not trying to predict the next breakthroughs or telling people how to write proposals, but one of the reasons NIH hasn’t been as successful as it should be, in my opinion, is that certain areas are not able to build critical mass (and other areas suck away the potential for critical mass). There are too many ‘pilot’ projects that go nowhere (“This proposal will serve as a model for….”). At some point, there needs to be a review of some of these ‘model’ grants and then some funding needs to be allocated there.

There’s a long history of coming up with exciting ideas and findings that then aren’t powered through with funding. That’s what I want to fix.
#7 Comrade PhysioProf
October 22, 2008

More specifically, I think more of the NIH budget needs to be much more focused and targeted, and less researcher driven.

Dude, you’re fucking nuts! First, who the fuck is going to decide where it should be focused? A bunch of motherfucking governmental bureaucratic functionaries!? Second, it is from investigator-driven science that the big breakthroughs come. If anything, too much NIH money is targeted to “goals”.
#8 Steven
October 23, 2008

While I wouldn’t use the harsh language of Comrade PhysioProf, I have to same I’m in virtually complete agreement with him. I could easily list multiple top-down NIH programs under way right now – spending hundreds of millions of NIH money – that are mostly or completely wasteful. For example, NCCAM (the “alternative medicine” center at NIH) spends over $200 million per year, almost all of it on bad science and pseudoscience. NCCAM was created at the insistence of Congress – that’s the kind of research we’ll get if we encourage more “leadership” from NIH.

There are other examples – I know of some big NIAID contracts that are a complete waste of funds, for example – but NCCAM is so egregious that all else pales in comparison.
#9 vpillsws
October 4, 2010

that’s interesting. hiçbişey anlamadım.