Excerpts from the individual and panel reviews on my latest proposal:
The proposed work is very well motivated, well organized, with clear plans and goals.
There is also:
Guaranteed access to the CTIO 1.5 m, and an established track record with WIYN 3.5 m observations, add to the strength of the proposal. Preliminary work from both telescopes are presented, indicating that the proposed aims for the project are achievable.
And, indeed, in another one:
This is a well written proposal (though I found unnecessary details in places such as section 4).
I should note that the stuff I put in section 4 was in response to comments of a previous TAC about a certain element of the feasibility.... What one panel sees as missing, the next sees as too much information when its added.
Despite all that, in another review I have:
In general the proposal is not well written and important information about the expected results and feasibility are not given.
and, in a panel summary, contradicting something above:
The resources available to the project do not appear to
be sufficient for the proposed research.
It's really not clear to me what to do with all of this. It's all over the map. Was it well written? Or not? Are there clear goals? Or not? Are the resources I have a strength of the proposal? Or are they insufficient? No way to know. I mean, I do know, but there's no way to know how I can convey that information to a putative future panel.
There were some criticisms offered that I would agree, yes, there should be more information about that in there— although these were not things mentioned in previous years. It's such a moving target that it's frustrating. But when you get such mixed messages in one year— what is one supposed to think? Clearly it wasn't good enough, but these kinds of messages make me think that it's not possible to make this project into something good enough. The only thing that would be good enough would be if I were to happen to, effectively, win the lottery— figure out what is going to be the most exciting science du jour, and write it up well and thoughtfully.
It's too frustrating for words.
Addendum: It's also worth mentioning that the public-outreach portion of the proposal was unanimously viewed as a strength this year and two years ago. Last year, however, the reviews objected that it wasn't related enough to the science in the proposal. Again, what's a strength one year may be insufficient the next....
- Log in to post comments
I realize that you know this, but its really a crap shoot when applying to the NSF. My money is on you having a more that good enough proposal, and still not getting anything. Sucks beyond words man.
Rob,
I know it's hard. I've had some proposals funded and other proposals rejected.
When I've had proposals rejected, I try to remember three things:
1. My proposal is in a batch with many others. The other proposals, in that year, might be better. Hard to believe, but it happens :-)
2. One thing that's both good and bad about NSF grant review panels, is that the people who are on the panel change every year. So, you're absolutely right, one year you may have someone likes and understands what you're doing, another year you may have someone who hates it. Many of the people that I know, who've been successful, wait it out. They submit modified versions of the same proposal multiple years in a row. This is tough when you're in a situation where your salary and your job depend on funding.
3. The last thing thing to consider is that you may get very good reviews, but there simply might not be enough money to go around.
Here's my unsolicited advice for things you can do to improve your odds:
1. Apply to the NSF to be a reviewer. You will learn an incredible amount. It will be very worthwhile and you will be working towards ensuring that good quality science is the science that gets funded.
2. If you're looking for a job, consider being an NSF program officer. The NSF hires many program officers on short term (1-2 yr) contracts.
It's too late for all of that kind of stuff -- I don't have enough go-arounds yet for being a reviewer to have time to pay off. I have one chance left, *IF* I decide I have the boneheadedness to beat myself up with one more submission, before the tenure review comes up.
Just too few people are getting funded. It really is futile.
Re: being a program officer, I fear that's really the sort of job that not only would I not like at all, but I wouldn't be good at. (See my post "Astronomer for Hire.") Plus, I have to admit, it would eat me up presiding over a part of the agency that has no choice but to doom various peoples' careers. I just don't want to have to be in that position.
-Rob
I think I've mentioned this before, but until review committees start looking at the standard deviation of reviews and learn to reject outliers, this kind of crap will continue to occurr.
Lab Lemming, that has absolutely nothing to do with it. As Rob is saying, it is futile, the funding percentages are so low, that even applying 5-6 years in a row, there is a good chance your good/very good proposal will not be funded. That is simply the situation now. It has nothing to do with great proposals not getting through because we are not treating the statistics of reviewer biases correctly.
Your science has to be judged as outstanding in the year in which you apply. It is a moving target. You cannot change topics to match what is currently in fashion. Therefore, some people will get lucky, getting a proposal funded at just the right time, and some will be unlucky, not getting their proposals funded. Some topics are seen as important for many years in a row, that's where I would steer prospective grad students. After trying to get them to go into i-banking or engineering.
Also, Lab Lemming, it doesn't make sense. In order to have meaningful statistics, you'd need to have at least 15 or 20 people read the proposals. Panels aren't that large, and can't be that large unless we want all of astronomy to do nothing ever again but read each other's proposals.
Also, the proposals that are getting funded are good, no question about that. Probably 1/3 to 1/2 of them are the ones that every astronomer, were he or she to read them, would agree were in the "definitely fund" category; some of the rest are in the grey area. There's no worry about the NSF doing good science. The worry is for those of us who don't want to be pushed out of the field getting pushed out of the field because there isn't enough money to go around.
The huge frustration for me is that there's absolutely no way I can see to make the system work for me.
-Rob
You cannot change topics to match what is currently in fashion.
...although I would note that that is sort of what the NSF program officer was recommending to me when I talked to him about this year's turn-down.
-Rob
Yes, this proposal business is highly frustrating. The panel reviews can be an all too random part of the process, and the success of your proposal can depend all too strongly on whether the primary reviewer likes your proposal. I've been on a NASA panel (but not NSF, so far), so I've seen that process from both sides, and while the panel was on was not nearly as bad as some of the horror stories I've heard, I can see how that happens. In your case, my guess would be that the author of this comment:
was on the panel, and the more favorable reviews were mail-ins.
Have you looked at the NASA ROSES offerings? Money is tight at NASA, too, but if you can sell your science programmatically it can't hurt. (Due to a brain-dead implementation of their on-the-fly PDF generation the NSPIRES site is Mac-hostile, so try to borrow a Linux or Windows machine if you have to.)
Why is the NSF the only big money? Can NASA ROSES compete? A monopoly (NSF?) makes decisions, but doesn't have to compete to make the best decisions. Why are we using up teaching and research gifts and talents on grant-writing? My tax dollars could be doing better work! (Disclaimer - I got out of the academic research track so I wouldn't have to play this difficult game. I am also Rob's mother.)
Dear Nancy "Rob's mother" Knop,
I know something about this, and it's even worse than Rob says.
My wife (a Physics professor) and I (a former Astronomy and Math professor) have won research grants at one time or another from NASA, Department of Defense, Department of Energy, and several other government agencies.
Big grants, little grants, one after another with fairly good odds.
I'm sufficiently an expert in grant writing that I was interviewed as a finalist for the position of Supervisor of the grant writing department at Rockwell International (Space Shuttles R Us), and as a "management gold team" editor of grant proposals.
It's gotten very bad, gradually, since the end of the Cold War, almost across the board, from the US Government.
One apparent exception is Homeland Security, which wastes $10 Biliion a year of R&D with our tax dollars. After asking about 100 experts in the sort of thing they SHOULD be funding, I've only found ONE professor who either has or knows someone who has a Homeland Security R&D grant. Everyone else is apparently a Friend of Halliburton.
Lip service to the contrary, the Federal government is less and less interested in any scientist actually doing research, or teaching.
Sorry.
A friend of mine at NASA/JPL, who has done first-rate rsearch at FermiLab, UCLA, and JPL, has virtually given up writing grant proposals. It's such hard work, the odds get worse each year, the grants go to those best at grantsmanship, rather than at meaningful research.
I'm about to stop seeking a tenure-track professorship myself, after 2 years of unsuccessful searching, and have intereviewed to be a high school math teacher.
It's bad out there. Really bad. Be happy that your sone has a paycheck. It's not him at fault, really. It's a collapsing American system. Now, in China and India, on the other hand...
Funny that you should mention that since, at the same time your proposal was being rejected on the grounds of "insufficient resources", my girlfriend's was being rejected on the grounds of "insufficient public-outreach".
I honestly think they're just throwing darts at a board.
Outreach is a whole 'nuther rant I have about the NSF.
I think that doing the kind of public outreach that the NSF is *trying* to do with the "broader impacts" part of their proposal is crucially important. However, I think the way that the NSF does it is boneheaded and short-sighted.... I'll save that rant for another day.
-Rob
Certainly. Science isn't particularly useful without dissemination, but I think their criteria for judging outreach is wonky.