A Blog Around The Clock

This is very interesting, referring to Canadian system:

Cost of the NSERC Science Grant Peer Review System Exceeds the Cost of Giving Every Qualified Researcher a Baseline Grant:

Using Natural Science and Engineering Research Council Canada (NSERC) statistics, we show that the $40,000 (Canadian) cost of preparation for a grant application and rejection by peer review in 2007 exceeded that of giving every qualified investigator a direct baseline discovery grant of $30,000 (average grant). This means the Canadian Federal Government could institute direct grants for 100% of qualified applicants for the same money. We anticipate that the net result would be more and better research since more research would be conducted at the critical idea or discovery stage. Control of quality is assured through university hiring, promotion and tenure proceedings, journal reviews of submitted work, and the patent process, whose collective scrutiny far exceeds that of grant peer review. The greater efficiency in use of grant funds and increased innovation with baseline funding would provide a means of achieving the goals of the recent Canadian Value for Money and Accountability Review. We suggest that developing countries could leapfrog ahead by adopting from the start science grant systems that encourage innovation.

I don’t know how that would work in the USA, and certainly would not work for big $million grants, but this is quite an interesting idea – skipping the peer-review of small grants entirely and just giving all the applicants a Baseline Grant. If they use the money well and are lucky with the result, they will have publications and data they can then use to apply for bigger grants. Some really cool and unusual, non-band-waggony stuff would get done that way. What do you think?

Comments

#1 Robert Grumbine
April 7, 2009

I think it would be an excellent idea.

I’ve suggested it to my colleagues, however, and the reception has been less than enthusiastic — among those who are receiving large grants. It gets a favorable response among those from small schools, folks who apply for small grants, early career scientists, and the like.

I don’t envision a permanent gift, so to speak. If an investigator signs up for the grants, then they should also, in reasonable time frame, show that they’ve done something constructive with the previous funding. Not as a matter that takes major peer-review, but more on the level of did they publish in the professional literature.

And like the Canadian suggestion, I envision it as a small grant — sufficient to cover travel and publication costs but not putting up buildings on campus, etc.. As such, it won’t suffice in my field for, e.g., tenure at a research institution (average annual grants for that group being $500,000/year as of a few years ago).

I participated in the NSF panel reviews a couple of times. The costs of the review process in the US could be markedly higher than suggested in Canada. In addition to the time spent by the scientist preparing the grant proposal, there were the (average) 3 lengthy reviews per proposal, and then the 5 of us on the panel doing our own reviews of all (call it 50) proposals, then NSF flying everybody to the panel meeting and the solid day we spent in discussion, plus the NSF program manager and staff handling of the proposals. And this for proposals where the largest single annual amount was no more than $500k, and possibly only $300k. And, as about 75% of proposals were not funded, most of those expenses were … expensive.

That was more than a decade ago, and already it was clear that there was a move to ‘safe’, or as you suggest, band-waggony, proposals. That trend can’t have stopped.
#2 Bob O'H
April 7, 2009

You can see what will happen if that’s implemented, though.
#3 Coturnix
April 7, 2009

I am glad to see that Larry thinks this is a good idea.
#4 Jonathan Eisen
April 8, 2009

I have been talking to a variety of funders about this recently. A few definitely think that if a granting program is costing more to run than it would cost to give every applicant funds, then something is wrong. (I discussed this a bit in this post ). However the issue will be if this were announced then more people would apply for such a program and you might not have enough money anymore. So it depends on the program how well this could work. The other issue (which was raised by Steve Salzberg in response to my post I ref. above) is that peer review keeps people in line in their projects and thus funding everybody to do whatever they want may not be a great idea.

I think we could streamline many review panels and save money for more science while at the same time having some peer review.
#5 Coturnix
April 8, 2009

I am with Jonathan on this. If it is given with no review at all, people will turn in all sorts of crappu proposals (or why even write proposals, just stick your hand out for a handout). But the review could be streamlined greatly and save a lot of money. How about public, digg-like, review on a website? Where the author can defend the proposal in public, take in criticisms, improve the proposal in real time, etc. It will (like digg) have problems, but having something out in the open may be a thought to ponder.
#6 Rhea Miller
April 8, 2009

agreed. I would be afraid of funding crapola. Tho it is something to think about…..
#7 Dick Gordon
April 9, 2009

I’d invite each of you to request a copy of:

Gordon, R. & B.J. Poulin (2009). Cost of the NSERC science grant peer review system exceeds the cost of giving every qualified researcher a baseline grant. Accountability in Research: Policies and Quality Assurance 16(1), 1-28.

and then continue the discussion.
#8 Dick Gordon
April 10, 2009

For a reprint contact me at: gordonr@cc.umanitoba.ca
#9 Marko
April 10, 2009

I started the whole thing by sending the article to Bora (I got it from a historian friend of mine from U of Alberta). I would urge everyone to actually read the article. There are many other very good arguments against peer review specifically at the level of smallish baseline discovery grants besides the cost. It is a very nuanced argument and the proposals are nuanced too (not all or nothing).

I want to address something I have seen in the comments here (as well as at Lawrence Moran’s Sandwalk blog) that I think is highly pernicious. I find it really sad that scientists themselves would interiorize the highly insulting view that were we not forced to write grants we would just slip into crappy, useless research. This is precisely the attitude implied by the granting agencies: if we don’t slap ridiculous amount of self-justification and scrutiny onto you, you will take the money and run to the Bahamas. I think it is highly damaging to science to consider its practitioners as extrinsically motivated (some sort of Skinner’s pigeons who perform only when given reinforcement).

I would say that science types are by and large a self-selected bunch who obviously are not motivated by mere monetary rewards (if they were they would have turned their supposedly powerful brains to more lucrative endeavors such as business, law, etc.). Second, they pass through a long and arduous apprenticeship that culminates in a PhD. By then they should have soaked up the scientific ethos thoroughly. In addition, throughout our career trajectories we are scrutinized most carefully — when we get the job, when we are promoted, when we publish, etc. The article points out, and I think quite rightly, that scientists are judged more rigorously at all these points and that the one area where they should be trusted is precisely when they are embarking on new research.

Somewhere along the line you have to allow for what Michael Polanyi called the unaccountable element in science. A system that doesn’t allow for that element is stiffling science as a “spontaneous coordination of Independent Initiatives” (see this). There will be plenty of “quality control” later on in the process (publishing, etc.). We as scholars/researchers/scientists are intrinsically motivated to search for truth, or the best approximation to it, with all we have and in the best way we could conceive of. We are the types who would do it even if we had just a pen and a piece of paper. We are like Picasso who supposedly said that if he were put in a solitary cell with nothing to draw with he would paint by spitting on the walls.
Now this is what Rosie Redfield said on Sandwalk:

I think that’s a terrible idea. Everyone gripes about preparing grant proposals, but that’s the one time we’re forced to think rigorously about our research BEFORE we do it. If I didn’t have to justify what I want to do in order to get the funding, I’d waste a lot of the taxpayers’ money doing what I would only later realize to be useless experiments.

Oh our masters (you who hold the purse-strings) force us to think rigorously because if not FORCED we won’t do it. This is very PERNICIOUS for science. Instead of sending this message, which is just parroting what politicians are telling us Joe sixpack is thinking about science (and academics) anyway, we should be sending the message that we are to be TRUSTED to determine the direction of our own research entirely on our own. We have to defend our judgement and our intrinsic motivation. We are devotees of truth and need no FORCE (or behaviorist manipulation) to be doing our very best to seek for it.
#10 Coturnix
April 10, 2009

See also this post and this.
#11 Eric Michael Johnson
April 10, 2009

I think this is a great idea, though it’s doubtful the economic logic would also apply for the United States (but it would be interesting to see a similar financial breakdown). I completely disagree that the lack of peer review for small grants would result in a dilution of good science. Those studies that didn’t produce results would simply stop after this initial funding and, in Canada at least, there would be no financial loss to the federal government. A lot of grants today are denied, not because they are shoddy proposals (though that is sometimes the case too) but because they are asking questions that may not have a lot of prior research to back up what they want to do. Through this kind of approach, preliminary research that seeks to push the boundaries could get a foothold and the next really innovative studies might get their start in this way.
#12 Zen Faulkes
April 10, 2009

A comment I saw elsewhere was, “But what can you do with $30K?” (average NSERC grant, according to the Gordon and Poulin article). I think that the grants are small compared to what many American researchers aspire to is important.

A letter in Science last year pointed out that the U.S. funding system actively selects for expensive projects, not productive projects, largely due to institutions getting overhead. (Unless things have changed at NSERC, every penny goes to the researchers, making the total grant costs lower.)

I’ve argued for a streamlined system for very small grants. Micro-loans (cf. Grameen Bank) have been very successful at developing economic wealth; micro-grants could be very successful at developing scientific wealth.

For many, many kinds of science, it doesn’t take much money to grease a lot of wheels. While not reviewing such grants at all makes me uncomfortable, there could be much to be gained from small grants with high probability of funding that go through an extremely streamlined review process. “Have you published X papers in the last X years? Did the money from your previous micro-grant get spent on bona fide research supplies? Here’s a small pot of money, go do science.”
#13 harold lee
April 13, 2009

As I remember, there were something call ” Small Grant ” administered by either NSF-USA or NIH-USA long time ago. The review procedures were simple. And there were small grants from home-universities to encourage new faculty to do research. In my days, i.e. 40 years ago, 30,000 was big big. Nowadays 30,000 is not much but good enough to start. A good researcher can be productive with that amount.
#14 Coturnix
April 17, 2009

This discussion continues here