More specifically, I think more of the NIH budget needs to be much more focused and targeted, and less researcher driven. In a post about NIH proposal revisions (i.e., resubmissions after a proposal has been rejected and critiqued), ScienceBlogling DrugMonkey writes (italics original):
The reason is that this policy does nothing about the tendency of reviewers to focus on grantsmanship issues as an easy triage mechanism, instead of taking the “fish or cut bait” hard look at the genuinely new application the first time. The primary stage of review is the main driver here. The ameliorative measures should have accounted for the source of the problem and tried to address it more directly. The single amendment limit doesn’t do this.
One of the ways the original goal ["to ensure earlier funding of high-quality applications and improve efficiencies in the peer review system"] could be accomplished would be through reviewer education and instruction. Put the data figures in front of all reviewers and say “Bad dog! Stop deifying revision status and grantsmanship. Focus on the underlying science. What will really be accomplished through the review process- changes in the proposal only? or actual changes in the resulting science?“
The problem, I think, is more fundamental than this. It’s analogous to college admissions at highly selective institutions (e.g., 5-15% acceptance rates). Most of these schools would be able to accept an entire class, get rid of it, take the next ‘class’ down, get rid that, and take the third cohort…and not miss a beat. And most will admit that if they went an additional class down, the drop off would be very little*. Ultimately, admissions officers have to come up with reasons to disqualify qualified students.
A similar phenomenon applies to NIH proposals. With the current acceptance rate of around twelve percent, roughly two thirds of perfectly fine proposals wind up being rejected. By “perfectly fine”, I mean proposals that have a reasonable chance of working out, and, if they did, would be interesting to other scientists in that area (i.e., if you heard a seminar on the work, you would conclude it’s solid work). I think about another twenty percent of proposals are basically one step away from moving into this category.
So there are too many proposals chasing too little money? Tell me something I don’t know, Mad Biologist. Much of the funding that NIH provides is ultimately investigator driven. That means a researcher (or small group of researchers) develops a proposal and submits it. While NIH and its component institutes do have different sections, they are so broad as to be essentially meaningless (antibiotic resistance, inflamatory bowel disease). I once participated in a roundtable about antibiotic resistance, and one of the attendees was the NIAID program officer who oversees all of the antibiotic resistance research (around $800 million). Many of the participants were frustrated that NIAID wasn’t funding certain areas adequately (e.g., pharmacokinetics). I’ll never forget what the program officer said: “I can only fund what is sent to me.”
The point is that, as DrugMonkey noted, it’s too difficult for reviewers or program officers to reject proposals based on their unsuitability for the goals of NIH, since these goals, even within certain areas, are too broadly defined. My experience has been that with very targeted calls for proposals, there are far fewer proposals submitted, and it’s much easier to flat out reject them because many proposals are not germane to the funding objectives. This means that NIH program officers have to be far more active in defining specific research objectives than they have been–to a considerable extent, NIH is placing this responsibility on reviewers who often lack knowledge of the larger institutional objectives. That needs to be changed.
I’m not saying that there shouldn’t be any ‘open’ funding, but the ‘open’ panels should be viewed more as demonstration projects. If after five years of funding, a case can’t be made that funding that project and other related projects is good idea (i.e., target additional funding to that very specific area), then the competitive renewal should be turned down.
This wouldn’t be a perfect system–at some point, funding would have to be opened up because novel ideas will be cut off. But right now, because funding is a free-for-all, the system is rewarding grantsmanship at the expense of novelty anyway, with the added bonus of overworking the reviewers.
*How one defines what a ‘good’ applicant is, of course, highly subjective.