JAM redux: What do you write in your NSF annual reports?

At JAM last week, a really useful session was conducted by Nakeina Douglas, an assistant professor in the L Douglass Wilder School of Government and Public Affairs at Virginia Commonwealth. She also is involved in the Grace E Harris Leadership Institute, and teaches public policy and research methods.

"What was so useful about her talk?" I hear you ask. Well. Let me tell you. She talked to us about how to write those darn annual reports for NSF. Let me share...

Shockingly, the two main objectives of reporting for NSF are accountability and decision usefulness. I know; who would have thought? Although Douglas pointed out that generally annual reports aren't all that useful to the program officers (also, late), who need them to argue that NSF is meeting its goals and objectives. So they would really like you to show that your research is value-added, that it was a good decision they made to invest money in your project.

While annual reports depict your general performance on your project and provide "indicators of quality," final reports connect outcomes to goals, describe your achievements and how you overcame challenges, talk about institutionalization, and "contribute to a culture of evidence that supports the NSF mission."

In Fastlane, there are a series of textboxes you are expected to fill in, whether or not you include supplementary pdf material. Let me go through Douglas's suggestions for each in the context of HRD projects:

1. Participants, Organization Partners, Collaborations: NSF would like you to describe your participants (including their demographics) as well as different organizational partnerships your project has struck up. How were they useful? What financial relationships were there? What were the results? Consider things like how long they participated, what they contributed, what other orgs they were affiliated with. Tables might be good here. On organizations -- what other departments, units, universities, local/state/federal agencies did you work with? Or industry?
2. Activities and Findings: provide a summary of your goal-related activities, and your significant accomplishments by project members. Are there activities that participants are organizing on their own, as a result of their participation in your program? What about the quality of your research - is it replicable? On the findings' side -- tell the reader about your implementation, research design, methods (relevant to different kinds of data) and what your data outcomes were. This is also the place to talk about "training and development," such as workshops, seminars, student research projects and so on can go. Include a discussion of "outreach activities" too, in terms of both planned and completed opportunities.
3. Publications and products: this seems pretty straightforward, but remember to include web resources you generate, and instructional or research tools that you could share.
4. Contributions: Douglas said this is the section projects most often leave blank, but should be the place you link your project to other projects and people. She broke this category into multiple subcategories:
   1. Within STEM disciplines compared to other disciplines -- have you increased accessibility as a result of your project? Had an impact in STEM policy development? Developed resources that were innovative in some way? Share them here.
   2. Human Resource Development -- it's the name of the division, right, so it must be important. So don't ever leave it blank. Talk about the numbers of students who you've helped graduate, or how people have reached career goals because of engaging with your project. Talk about faculty who you have helped support, or your other impact on "pipelines" or critical junctures.
   3. Resources for research & education: I'm still not sure what goes here that is different from earlier sections - I think this is where you show how you have transfered your research into educational changes. Kind of a "best practices" showcase, I think.
   4. Beyond science and engineering -- what kind of impact did you have on your institution's culture and climate? How have you met the needs of particular target populations?

Her overall recommendations:

• Compare your accomplishments with your goals for the reporting period
• Describe your accomplishments in both qualitative and quantitative terms (although anecdotes are not good enough -- the difference between ancedotes and qualitative research is systematicness)
• organize your data by the specific goals of your project
• if your project goals haven't been met,
  explain why, and what steps were taken/will be taken to get you back on track
• Get an external evaluator to help you evaluate your project!
• Stay in constant contact with your program officer for guidance.

She didn't have any recommendations for the "highlights" of the Fastlane section, but encouraged us to talk with our program officers.

One final note... make sure the URLs you embed in your document actually work.

So. That's the bar we're trying to meet with our annual report. Kind of. Any advice out there from the interwebs on how to write these reports? Dos and don'ts? Good ideas and pitfalls to avoid? Share your annual report stories with us in the comments...