In a post last week, I was trying to work out whether science journalism can do something more for us than just delivering press releases from the scientists. Specifically, I suggested that journalists with a reasonable understanding of scientific methodology could do some work to assess the credibility of the research described in the press releases, as well as the credibility of the scientists issuing those press releases.
Although the post was concerned with the general question of whether science journalism can do this bit of evaluative work for a lay audience that, by and large, is both rusty on the basics of scientific methodology and at least a little scared of thinking hard about technical issues, it was prompted by the particular strategy Chris Mooney set forth for his reporting in The Republican War on Science, and by Steve Fuller's critique of that in his essay for the Crooked Timber seminar on Chris's book. And, in a comment on my post, Steve Fuller brings us back to the question of the particular strategy Chris was using, and of the sorts of standards to which journalistic work guided by this strategy ought to be held. I think this is a fair question, but it brings us into the turbulent waters at the confluence of science and politics, where journalism may go beyond presenting an objective picture of the terrain and may exhort us to choose a side.
So you might want to grab a life-jacket before we begin.
From Steve Fuller's comment:
First, I read The Republican War on Science as just as much an attempt to make as to report news - very much like George W. Bush's declaration of a 'War on Terror'. In both cases, one can see a wide range of evidence for the conflict indicated in the corresponding phrase but now collected together and focused for ideological effect. There also a self-fulfilling air to both phrases: Both Mooney and Bush want people to realize there is a war - and to takes sides. As a journalistic invention, it is potentially very illuminating because it seems to offer a distinct perspective from which lots of things at once in the public eye start to make sense. But given that promise, it becomes imperative that the journalist has staked out an independent point-of-view. 'Independent' here simply means using standards of judgement that are his/her own, and not entirely parasitic on that his/her informants.
'Trust' is a really bad word to use in this context because it suggests an either-or choice the journalist needs to make with regard to (in this case) the political and scientific informants: A journalist who aspires to the panoramic sweep suggested by The Republican War on Science should be engaged in more of a wheat-and-chaff exercise with regard to informants. Maybe Chris did do this, but it's not evident from the book. He sometimes - but rarely - faults the guys on his side. Indeed, Chris pretty much did a George Bush and painted the situation as the Forces of Light (Democrats and Real Scientists) against the Forces of Darkness (Republicans and Pseudo-Scientists), which ends up only preaching to the choir - which, admittedly, nowadays is pretty large. ...
(By the way, it's possible to adopt an 'independent' viewpoint in my sense without being a 'centrist', just like a judge can weigh the merits of a case and still come down on one side.)
One thing that troubles me about the discussion so far is that you guys seem to be portraying the Chris' journalistic task as simply a matter of evaluating the merits of rival scientific sources, and then looking to the philosophy of science to solve your problem. Chris claims to be doing something more creative here - and so deserves to be held to a different standard, one that takes science journalism out of the realm of 'Methodology for the Masses'. In the end, the independence of Chris' standpoint should be judged by how he disentangles and analyses the relationship between scientific claims and political motives. In that spirit, I suggested in my review that he should have asked both sides of this 'war' to troubleshoot each other's claims and motives, and then formally evaluate them, as a judge might do in a case.
Fuller is right that my post was concerned with the general problem of whether a non-scientist can evaluate the credibility of scientific sources (and the comments that followed it) -- I was responding to a comment that seemed to raise the general question. He may also be right that Chris is aiming to do more than to use his "judgment to evaluate the credibility of different sides and to discern where scientists think the weight of evidence lies", although I take it that Chris would judge this evaluation of credibility as a crucial first step to the larger task. (Indeed, even if Chris were engaged in the more modest project of discerning where the community of science thinks the weight of the evidence lies, it might be informative for a science journalist to see how scientists on different sides of a scientific controversy respond to each other's evidence and claims.)
So how does the journalistic task change if we're not just interested in what the scientists know and how they know it? If we're evaluating scientific knowledge in a political environment, how does this shift the standard to which we should hold the journalist making the evaluation?
Fuller suggests, for one thing, that the journalist needs to develop standards of judgment that are not entirely parasitic on those of his or her informants. Does this mean that the science journalist who is not a scientist ought to establish his or her own standards for evaluating sound science -- whether or not working scientists would agree with these evaluations? I take it this is a move Chris explicitly rejects. Perhaps, though, the independent standards Fuller has in mind pertain to judging how the scientist's political interests might be entangled in his or her scientific work or scientific judgments -- in the questions asked, the ways data are interpreted, the course of action the results suggest, etc.
The task for the journalist, then, would be to sniff out whether political factors might be acting to bias scientists' views of what counts as good science. If scientists with more Democratic leanings see the science produced by scientists with more Republican leanings as wanting, is this due to actual methodological problems, or due instead to a political disagreement that manifests itself in the kinds of scientific knowledge produced by the two camps? Is it possible to take a piece of scientific knowledge and discern from it the political biases of the scientists who produced it? Is there any independent point of view a journalist could stake out from which he or she could come to the conclusion that one political side tends to produce better science than the other, or does a Forces of Light/Forces of Darkness conclusion indicate that the journalist's own biases are driving the story?
Doing objective science is hard. There are lots of ways political leanings could affect the way a scientist approaches a problem -- perhaps in the sorts of background assumptions one holds, how one frames the scientific question, or what kinds of outcome one is expecting to observe. I suspect something like politics may play a role in what scientists see as the "moving parts" of the systems they study (e.g., genes or environment in studies of diseases or of intelligence), and that this may play a significant role not only in what is manipulated in the research, but also in the kind of accounts scientists give of the potential for practical applications arising from their work. To the extent that politics might bias individual scientists in their scientific research, however, I take it that these scientists rely on the community of science as a whole to weed out these biases as much as possible. The hope is that the objective facts of the system under study are those that a diverse community of scientists can agree upon.
Put another way, scientists aspire to build knowledge that is so well supported by the facts that even scientists with wildly different background assumptions and biases will accept it as knowledge. Convincing those who are already inclined to see the world as you see it is not enough.
So, my inclination is to say that good science strives to be free from political biases -- at least insofar as "good science" is a matter of the methodology a scientific community uses to produce scientific knowledge. How that scientific knowledge ought to be used once it has been produced is a separate matter. Ought we to use our knowledge of nuclear reactions to build power plants or bombs (or both, or neither)? Ought we to use our knowledge about how intelligence develops to change the distribution of societal goods, or to offer gene-level interventions? Answering questions like these relies on other commitments scientists have -- on values that, strictly speaking, need not play a role in their labors producing scientific knowledge. Even if the scientists themselves sometimes forget that questions of how to use science are separable from what science lets us know, a careful science journalist ought to be able to draw that distinction for us.
But I take it the disagreement Chris looks at in his book is not a simple quarrel about how to use what, thanks to science, we know. Rather, it seems to be a battle about what even counts as good science, a battle that is driven by political forces from outside of science. In other words, it's not a situation where Democratic scientists come to conclusion X and Republican scientists come to conclusion Y (because of how their political commitments shape the knowledge they build). Instead, Chris seems to be describing a situation where, within the community of scientists actively engaged in research, the majority agree that the evidence points to conclusion X, but forces from without (driven by political aims that are potentially threatened by conclusion X) actively boost the dissenting voices that argue for conclusion Y -- indeed, boosting these dissenting voices in ways that misrepresent the soundness of conclusion Y as judged by the methodological commitments shared by the community of scientists.
What makes the Forces of Darkness evil, in this scenario, is not that they would prefer it if the scientific consensus were behind conclusion Y, nor even that they would prefer to ignore the scientific consensus behind conclusion X (because conclusion X is inconvenient for the policy objectives the Forces of Darkness would like to pursue). Rather, the evil lies in trying to distort the public's picture of what the scientists think they know -- to make conclusion X look less well supported than it actually is, or to make conclusion Y look better supported. The evil lies, as well, in misrepresenting to the public the process by which scientists try to achieve consensus. (It is striking how many minority-view advocates in the debate have claimed the mantle of Galileo rather than working to find the evidence or the arguments that might convince their scientific peers.) And, there is a perverse kind of evil in at once claiming scientific support for one's political and policy aims (by acting as a booster for minority scientific views that seem harmonious with those aims) and simultaneously ignoring (or, in the case of scientists in one's employ, silencing) the voices of those in the community of science whose conclusions are not to your liking, despite the fact that the scientists themselves judge the methodology used in reaching those conclusions to be sound.
While I don't think doing good, objective science (or working from the knowledge produced by that science) forces a particular political view upon anyone, it does seem that there are particular political strategies that may be incompatible with one's commitment to the methodology of science. A political strategy that required annointing as "knowledge" whatever view best advanced the policy agenda would be at odds with scientific commitments about how best to build knowledge of the world. If one side in the political discourse really did make a decision that inconvenient scientific facts were getting in the way of winning the hearts and minds of voters, and if that side decided that it would be political expedient therefore to cast the scientific facts themselves into confusion and to make it difficult for members of the public to get an accurate view of what the scientists thought, I don't think it would be unfair to cast that side as against science. Indeed, a political faction could be anti-science from the point of view of casting into confusion certain bits of scientific knowledge while still being quite enthusiastic about potential applications from other branches of science.
If this were the scenario playing out, what kind of standards of independence ought we to demand of the journalist trying to report on it? Must he or she maintain agnosticism about whether suppressing or misrepresenting scientific knowledge for political ends (whatever they may be) is a good thing? That might be an awful lot to ask (and I think it's more than Fuller is asking). Must he or she locate political operatives who are willing to state openly that this is the strategy they are pursuing? If they're good operatives, and the strategy relies on a certain kind of dishonesty, this just might not be possible. Must he or she find evidence that scientists who agree with the (inconvenient) scientific consensus are being silenced? That scientist who hold the (more harmonious) alternative view nonethless have qualms about how their scientific dissent is "spun" for the public, because it subverts proper scientific channels for adjudicating the dispute? It may be possible to find some such voices, but there is also the possibility of finding political operatives in scientists' clothing -- at which point, an assessment of their methodology and degree of engagement in the debate within the scientific community might be useful in distinguishing serious scientists who disagree with the majority position from hacks.
For the record, I'd be perfectly happy to live in a world where being in favor of good science is compatible with any political position, where we can start with reasonable agreement about what the facts are and work from their on our competing programs as to how to act on those facts. But when claims of how the world ought to be depend on certain claims about how the world is, you run the risk that science will undermine your premises. Finding different ways to support your oughts from the facts is one way to respond. Arguing that scientific knowledge is not the only kind of knowledge that matters to support your oughts is another. But acting to distort the picture of the scientific facts in order to claim scientific support for your oughts smacks of an intellectual dishonesty that's hard to reconcile with the reality-based epistemological project of science. Those entitled to "trust" are not the actors with whose politics you most agree, but those unwilling to sacrifice honesty to support even their dearest political goal.
What perfect timing! We just had our annual advisory board meeting of a med/sci journalism program at the local U. The program director is initiating a new class on ethics in science journalism and asked for (unpaid) lecturers. I'd like to talk about how sci/med journalists evaluate the expertise/authority of their sources.
I'm a half-decent cancer researcher and can speak authoritatively about my little area. But I feel that the print and broadcast media latched onto me a little too easily and uncritically when I became a source for herbal med/dietary supplements. This experience made me think that reporters weren't as critical about checking creds of their sources - they were just happy to find a univ-associated prof who could speak intelligently and non-judgementally about dietary supplements. I'd love to do a tag-team lecture with you out here, so perhaps we can have a discussion off-blog.
"Fuller is right that my post was concerned with the general problem of whether a non-scientist can evaluate the credibility of scientific sources"
If science can only be understood by experts, then it will ultimately fail in the political and social arena. If science can only be understood by experts, then the experts are failures because they can't communicate to normal people.
writerdd, you'll notice from the post I linked that I come down on the position that non-experts can plausibly evaluate the credibility of scientific sources. (It helps a lot, of course, if those non-experts have learned something about scientific methodology and valid patterns of inference, but I talk about all that in the other post!)
Does this mean that the science journalist who is not a scientist ought to establish his or her own standards for evaluating sound science -- whether or not working scientists would agree with these evaluations?
The lesson I took from the previous post was that there are "extra-scientific" methodologies that are useful and important in determining what is sound science -- e.g. finding out where someone's funding comes from. We can't expect everyone to be experts on the experiments involved in every question: we might ask that of journalists, perhaps, but then we'd have about one journalist per issue, and I'd argue that we also want Joe and Jane Citizen to do their own thinking. So we are left with:
1. consensus within the specific scientific community (e.g. among climatologists)
2. expert opinions from scientists and others outside the narrow community in question but with relevant expertise
3. media reporting; for the layperson, this is a crucial source of "follow the money"-type information
4. critical thinking: broad familiarity with scientific methodologies and logic (valid inferences, control strategies, and so on) will, as per the post that sparked this one, allow non-experts to make decently rigorous appraisals of strategy if not of data. This is intimately linked with #2 and #3, since outside experts and reporters can do lay audiences a lot of good by helping with this sort of strategy appraisal -- consider Tim Lambert's indispensable coverage of the Lancet Iraq study, much of which consists in knocking down methodological strawman critiques that might otherwise have confused a math-impaired reader like me.
scientists with more Republican leanings
I note in passing that there are not many of these. Something about a habit of responding to the weight of evidence perhaps...?
Thanks again for doing such great writing on this topic. You seem already to understand where I'm coming from. All I would add is the following. Steve Fuller has been a good critic to tangle with, but it's not clear to me that he credits the possibility that maybe, just maybe, I looked at all the evidence from an independent standpoint and then decided that, lo and behold, there was something extremely troubling going on with respect to attacks on science. Does this make me an apologist for the scientific community? As one who has criticized that community's political ineffectuality, I don't think so:
I hate to start on a 'meta' note but Chris is presuming there is such a thing as 'the scientific community' that upholds the same values, regardless of political context. The very phrase 'scientific community' only comes into vogue in the 1950s, as US scientists start to worry that their agendas might be influenced by state-dictated Cold War concerns, as was suspected to be the case in the USSR. (Lots of people have written on this, not just me, most notably David Hollinger at Berkeley.) However, this has always been a case of "my enemy's enemy is my friend" thinking. Scientists have been historically very difficult to organize for any political purpose, including the insurance of their political neutrality. The most fruitful historical option has been for academic scientists to defend the institution of tenure as a guarantor of independent judgement. But academic scientists are by no means the majority of scientists, and tenure is under constant assault.
Barring what I believe to be the rare cases of outright misrepresentation of facts, most of the so-called "Republican War on Science" is about perfectly competent scientists who simply draw different policy inferences from much the same facts as their opponents - or, more interestingly, include a somewhat different range of facts as relevant to their policy inferences. For example, libertarians are much more risk-tolerant (a.k.a. reckless) than welfarists when looking at the same data. There may even be differences of judgement about the likelihood that the basic data will change in the future, not only as new data come in but also as measuring techniques change. Much of the scepticism about global warming seems to be founded on the latter - no doubt reinforced by paymasters, etc.
It's important to note these ties, but they're not necessarily a bad thing. Scientists may simply need to be more transparent about their values and projected futures, so that people can understand better what is at stake. Admittedly, this means that scientists should better communicate their politics just as much as their science - and that does run the risk (such as it is) that scientists come to be seen as just another political interest group. But I believe that's ultimately that's what democracy demands, beyond whatever attempts are made to make the science clearer.
A friend of mine who's been following my various wranglings about ID over the past few months has suggested that this latest outbreak of 'science wars' is simply a 'leftist revenge fantasy' by those unable to unseat George W. Bush after two tries. The best way to overcome that objection is for people of the left to show how the science they support would imply a better future for all, which implies admitting that the 'war' is indeed political through-and-through. What I find a bit disturbing about the Seed article that Chris links is that he seems to be recommending that leftist scientists conceal their motives - e.g. by hiding behind religious evolutionsts like Ken Miller.
Ummmm.....there is no better future for mankind, or better world if you believe that the Rapture is imminent. Think about that. When we have Ann Coulter saying it is our RESPONSIBILITY to rape and pillage the earth (parapharased) then that needs to be addressed. That is religious and political with social, economic and scientific implications.
There is no reasoning room here. Lefties are not going to go centrist with the RIghties. And if they did we would end up with nothing anyway. It would be watered down poo poo science.
What needs to be done is SCIENCE: not looking for glimpses of God's hand in everything. If there is an agenda, which there always is, explain and move on. I believe that Darwin was against vivisection in his early days, but as he got older he became a supporter as it became clear how amazing the physiological implications for people were. Time and money and patience and more stick -to- it- iveness just need to happen. Eventually moe good science will build upon more good science.
Every age has its bugaboos. There is always resistance from fundies about God's will. Not so long ago there was a majority of religious people who beleived that it was immoral to use pain killers for childbirth. Yeah- how many religious people do you know who have turned down painkillers recently?? Other than tc and Katie.... There are more C Sections and spinal thingies done today than I would have imagined the last time I gave birth twelve years ago(without painkillers- but it was my third).
We seem to repeat the one lesson that we see happen over and over in history- technology always outpaces morality, legality and theology. I am sure with the advent of the wheel there were misgivings.
Patience and calmness to all-------- unless we are on a mission to blow up the world for real. Then scream yell and holler!!!!!!!!!!!!!!!!!!
The relationship between scientists and the media is certainly interesting. I am a science doctoral student working concurrently on a MA in science/health communication -- caught between both worlds. One of the main issues is the fact that scientists are not generally rewarded for interacting with the press. A study from the mid-80's showed that the first priority for a scientist is to share information with other scientists, not the public. While this practice makes sense to a scientist, it is difficult for a reporter to grasp because the general public is still under the impression that people in science have dedicated their lives to curing diseases. We have a semi-joking sentiment in our lab -- if we ever cure a disease we will all be without jobs.
Both scientists and journalists should be working a little harder to meet in the middle. Scientists need to learn how to communicate with the media and journalists who report on science and health issues should have the skills to understand how to interpret the information being reported. Most of the science/medical news in the press comes from the newswire and trickles down the press hierarchy. Many people in the media defend this by complaining about how difficult it is to get scientist sources and, consequently, the errors in the news cause scientists to continue complaining about science news. It is a difficult task to take a scientist's discovery and turn it into an interesting story for nonscientists. This is certainly the case with stem cell research where rather than reporting on the actual science, reporters focused on subjects with which all audiences could relate -- morality and politics. Journalists write stories because audiences like narratives and drama... a scientist discovering some epigenetic mechanism for regulating tumor suppressor genes just isn't going to cut it -- the writer has to add some universal element like a child undergoing cancer therapy, a famous celebrity undergoing some medical treatment, etc. Can you blame them? I don't - I will be the first to admit that pure science makes my brain numb.