What is a fair non-science criterion for changing proposal funding priorities?
Below I ranted on possible political reprioritization of NASA funding
but I decided I wasn't clear in what I was worried about.
Consider a hypothetical (NASA) proposal I send in.
It is of course a very good proposal, ranked somewhere in the top 10%.
Here are some possible outcomes.
- a) Top ranked proposal, it is funded.
- b) Not funded: because funding was shifted within NASA to Exploration to expedite completion of Aries launcher
- c) Not funded: slightly lower ranked proposal by underrepresented proposer in geographically disadvantaged area got funding per Congress mandate on boosting science in diverse areas
- d) Not funded: lower ranked proposals in different fields funded because NASA admin decided this field of science is not a priority any more
- e) Not funded: lower ranked proposal funded because I already had funded proposals and other competing proposal was from a junior PI with no federal funding, program officer decided sustaining junior faculty in future fields was a priority
- f) Funded: but a higher ranked proposal in same field from a liberal blue state, like Massachusetts or California, was not funded because Pennsylvania is a battleground state and we'll get warm fuzzied and vote for incumbents if they throw some funding our way
Outcomes b)-e) are annoying at some level (I don't get funded), but also at some level policy decisions that are within the right of the people in charge.
For what it is worth, I know I have run into outcome b), d) and e) (to the extent that you ever know for sure what is going on). I am reasonably sure there have been outcomes c), though possibly not for any of my proposals.
But, even if I benefit from outcome f) (and of course most of the time "I" would not), I still consider it an illegitimate tainting of the process, as opposed to just an annoying additional process I may disagree with.
It is also stupid and ineffective.
PS: I should add that I am contemplating post hoc funding rationales - where either priorities change after the Request for Proposals goes out, so there are additional selection criteria not evident at the time of the submission; or, there are "implicit" criteria, which are contained within some broader guidance given to the agency but are not explicit at the level of an individual solicitation. Sometimes these are actually in the fine print, and sometimes you just have to learn the connection through osmosis or telepathy or some such.
- Log in to post comments
One thought I got when reading this (and I don't know if it's a good one), is that precise ranking is very, well, imprecise. Between two "good enough" proposals, differing by only a little, I very much doubt that you can find a significant difference in the quality or amount of results.
To some degree I suspect it would make better sense to have just a cutoff point - are you good enough to get funded or not (at a level that will leave just enough proposals for many, but not all, to be funded). Then, to decide among those that make the cut you use policy-level criteria as well as considerations on covering the various subfields, look at how well the project is already funded from other sources and other secondary criteria.
Proposal ranking is surprisingly universal: there is a very small group, < 5% which are consenus "must do". Then there are maybe 10-15% "can't be done"; the rest are broadly good enough and are usually ranked, with a "very good - should fund" and than a borderline area - the policy levels - balance across sub-fields and other criteria are usually done in the area straddling the "fund/border" area.
So that is how it is actually done.
The worry is if political, not policy, concerns are now reaching in - bumping down proposals and raising others for an ill reason, and an illegal one, if that is in fact happening.
For what it is worth, I doubt it is happening at the individual PI level, too fine grained. But I suspect it may be happening at the mission level, it would explain some strange mission cancellations and reshufflings in the last few years.
Ah, so you are talking about the mission level. Of course it's political: the more money involved, the more political it gets.
On the individual PI level, outcomes (d) and (e) fall under the category "programmatic considerations". If your proposal gets an Excellent/Very Good rating, but many other proposals for doing similar science have Excellent ratings, you may well lose out to a Very Good in another subfield where the Very Good really was the best proposal.
At the mission level, things get a whole lot stranger. Try to keep GSFC out of the loop if you have the option: there have been some nasty side effects to Full Cost Accounting, one of which is that they will try to bleed any cash-rich program dry (this is now happening to MMS). Also, don't be surprised if the TMC review just happens to favor the better-connected group (which happened in 2 out of 2 cases that I am aware of). And did I mention that you should watch out for greedy GSFC management?
glad to see the physical sciences have the same problems we do on the biomedical side of things...