"Timothy Caulfield has spent years listening to scientists complain that the media does a poor job of explaining science. As research director of the Health Law Institute at the University of Alberta, he has heard this so often, he says, that he started to believe it too. Finally, he decided to find out for himself.
Caulfield pored over the print media's coverage of genetic discoveries from around the English-speaking world and compiled a list of 627 newspaper articles reporting on 111 different scientific journal articles. Together with a team of coders, all of whom had scientific backgrounds, he compared the newspaper articles with the original journal studies for signs of technical errors or exaggerated claims of the research findings.
Contrary to perceived opinions, he found that only 11% of the media stories could be categorized as inaccurate or exaggerated ( Can Med Assoc J, 170:1399-407, 2004). "I was genuinely surprised that the media does a fairly good job of reporting genetic discoveries," says Caulfield. His results not only astonished him, they contradicted him: Years earlier, he had published an article in a law journal about how the inaccurate reporting of genetic research, a phenomenon he calls "genohype," was hurting the public's understanding of science. "You can tell I'm a law professor and not a scientist," he says, "because I wrote a long essay about genohype and only later went to do the study."
Read the rest. What do you think? Has anyone read the actual paper and the nitty-gritty details of methodology and results? As this is certainly counter-intuitive.
Perhaps the media reports are basically correct about any single new paper they cover, but, where they miss the boat, is in placing the new papers in a broader context, in history, perhaps in a way that reinforces some old incorrect dogmas?
- Log in to post comments
Well, we curmudgeons can still console ourselves with the study which found medical journalism to be rotten.
I am suspicious. You have to note that he's selecting from a highly unrepresentative subset - specifically, genetic research articles based on published journal articles.
In contrast, complaints about media misreporting is usually focused on reporting (and perhaps even more often, non-news opinion pieces/special features) on unpublished research, pseudoscience, off hand statements by notable 'experts', issues in the public eye and so on. Also, I get the impression that reporting varies across fields - things like theoretical physics, and also medicine/food scares are far more commonly misreported.
"he found that only 11% of the media stories could be categorized as inaccurate or exaggerated"
Skimming the article... they actually found that 11% had "moderate to highly" exaggerated claims and another 26% had "slightly" exaggerated claims - for a total of 37% having some sort of exaggerated claims.
They also found that 18% had "significant scientific or technical errors"
I'm not sure how widely the interpretation of this study can be applied, since it looked at only one subfield and only at newspaper articles.
FhnuZoag:
Yes, with something like the EmDrive incident, where New Scientist's conduct was downright reprehensible, there wasn't a peer-reviewed paper to begin with.
As someone with a foot in both camps (I am a scientist and I freelance), I have some sympathy with scientists and journos. One problem that journalists face is that they are expected to write about things that they really have no educational background for, so they trust the scientist and the PR surrounding the research to clue them up. I have seen some very poorly written PR generated by universities and research organisations, so it is no surprise that reporters get it wrong.
I think the big problem is not inaccuracies, but rather the lack of follow through. Basically today their might be an interesting story on the development of techniques in optical quantum computing (say really stable interferometers), tomorrow their might be a break through in something else related to quantum computing (say storing qubits in a piece of diamond), but there is no attempt to place the two in relation to each other. The fault here lies with both reporters, who don't refer back to their own stories on similar topics, and also scientists who don't explain how their work builds on previous work and relates to current work by others. (the funny thing is that original research paper always does a much better job of putting the research in perspective than does the PR).
The result is that the public is left with the feeling that science is a set of independent breakthroughs with no logical or causal connections. And who the hell can make sense of that?
Well, I don't really pay attention to media coverage of genetics, but I know that in terms of paleo, they don't know the difference between a bird and a pterosaur, which is just as basic a deliniation (mispelled) as the difference between a turtle and an alligator.