Grading Federal Agencies on Scientific Openness

On Friday, the Union of Concerned Scientists released a "report card" on media practices and scientific speech in federal agencies. According to UCS,

Both good policy and good practice in the communication of scientific results to the media are achievable goals for federal agencies. Yet there is no consistency among agency policies, and the ability of government scientists to speak freely about their research depends on the agency that employs them.

Here are the "grades" UCS gave each agency:

But what does this actually represent?

Honestly, what bothers me about this "report card" is that it appears at first glance (as in this WaPo article) to represent documented infringements of scientists' speech, or inaccuracies UCS sussed out in how the agencies are representing their science.

Instead, the methodology for this report involved (1) reviewing the media policies of each agency, and (2) sending an email questionnaire to agency scientists that included questions about their ability to speak to the press about their science, the policies that govern that interaction, etc. (UCS got an overall response rate of about 10%). In their report card, UCS dinged agencies for not having explicit policy protection for scientists to speak freely, whether or not scientists said they had actually experienced pressure to toe the line.

Clearly scientific outcomes ought to be reflected accurately and honestly in all situations, and scientists should never feel pressure to misrepresent their work or keep silent. Hopefully we can all agree on that. And this report could be useful for the incoming administration to consider in revising agency policies, which could state these goals more emphatically.

However, I don't think that the particular inputs this report is measuring correlate well with actual misrepresentation of science. . . nor that good media policies always ensure that, in practice, science is represented accurately.

Check it out for yourself and see what you think:

UCS
The full report (pdf)
The methodology (pdf)