I didn’t pay that much attention to the mini-controversy over the NSF’s proposed revision of its grant evaluation criteria when they were first released, because I was working on the book. I was asked to say something about it yesterday, though, and having gone to the trouble, I might as well say something on the blog, too.
The main source of complaint is the “Broader Impacts” section of the grant, a category that has always been sort of nebulous, but which the new standards attempt to clarify:
Collectively, NSF projects should help to advance a broad set of important national goals, including:
- Increased economic competitiveness of the United States.
- Development of a globally competitive STEM workforce.
- Increased participation of women, persons with disabilities, and underrepresented minorities in STEM.
- Increased partnerships between academia and industry.
- Improved pre-K-12 STEM education and teacher development.
- Improved undergraduate STEM education.
- Increased public scientific literacy and public engagement with science and technology.
- Increased national security.
- Enhanced infrastructure for research and education, including facilities, instrumentation, networks and partnerships.
This list has generated a lot of complaints, including a letter to Science and an editorial in Nature that are reasonably summarized in non-paywalled form by this Physics Today story. The biggest issues are, quoting from the letter in Science:
First, the list focuses on economics and national security, but excludes protecting the environment and addressing other social problems. Aside from the consequences of neglecting these areas, this new focus may undermine the attractiveness of STEM disciplines to more idealistic students who are interested in meeting human needs rather than fostering economic competitiveness. Second, under the proposed new criteria, applicants and reviewers are restricted to the provided list of national needs, which will complicate efforts to respond to new challenges as they develop.
(the Nature op-ed has a similar set of concerns, but isn’t quoted as conveniently in a place I can get it for free).
My reaction to the whole thing is pretty much: “Meh.”
First of all, the quoted list of categories is a proposed change that was posted for comments. It might very well be revised before it gets to the final rule stage (the comment period is now closed, but was open when those pieces ran).
Even if the new rules went into effect unchanged, though, I don’t think there’s really anything to worry about, because, well, look at that list. It’s hard to imagine a worthy proposal that doesn’t meet at least one of these. If funding your grant wouldn’t result in “Enhanced infrastructure for research and education, including facilities, instrumentation, networks and partnerships,” you’re not asking for the right stuff. And if you’re working on energy or environmental issues and can’t make a solid argument that your work will increase national security or economic competitiveness, then you clearly have no idea what you’re doing. There’s even the “including” loophole, allowing you to make a case for the importance of something else that isn’t on the list.
Most importantly, I suspect that the practical impact of this is basically nil. I haven’t been on the final review panels, but every year I get sent 4-5 NSF grants for preliminary review, and based on the quality of the “Broader Impact” statements I keep seeing, I can’t imagine that this category is actually deciding who gets funded and who doesn’t. I could be wrong– I haven’t looked to see if any of the proposals with remarkably half-assed “broader impact” sections got denied funding, but if you were going to cut people off for doing a lousy job on this criterion, NSF would have a hard time finding people to fund.
And none of those proposals would have any trouble checking off boxes on the proposed list of “important national goals.” The only thing this will change is the formatting of the proposals– instead of wittering on vaguely about how training grad students supports science education, they’ll directly quote one of the relevant items from the list, and get it over with faster. It’s not going to “produce more hype and hypocrisy,” as Daniel Sarewitz claims in Nature, it’s just going to slightly reconfigure the hype and hypocrisy we already have.
The actual question I was asked about this wasn’t about the impact of the new rules, though, but was more about the “broader impacts” criterion itself. And regarding that, my opinion is basically that it probably doesn’t help all that much, but can hardly hurt.
The point of the criterion, as far as I can tell, is to force people seeking research funding to devote a minimal amount of thought and effort to public dissemination of their results. Which produces a lot of empty blather, and a large number of very traditional outreach efforts, but may also provide some people with the justification they need to do some good and innovative outreach programs.
It would undoubtedly be more effective to find those people and directly give them money to do innovative outreach programs, but academia isn’t set up to do that right now. There aren’t that many channels to get funding directly for outreach activities, and even if it brought in money, there’s a sort of stigma attached to it that would deter a lot of people anyway.
The NSF rule is about as much as you can do in a “top-down” sort of way– you can require that people who are getting research funding also do a small amount of public outreach, which at least gets you something. It’s not going to be optimal, but it’s almost certain to be better than nothing (it’s possible, to put on a public science program that actually makes matters worse, but it’s not easy), and if you didn’t have the requirement at all, “nothing” is what you would get.
What we need is a shift in the attitudes of scientists in general, to recognize public outreach and communication of science as a valuable thing in and of itself. At which point you could probably get adequate resources to those people who have the talent and inclination to do a really good job of bringing science to the general public, and improve the standing of science in general. That can be done– we are science, after all– but it involves changing a lot of minds, and a good deal of work.