I’m almost done grading a massive mound of papers by my freshman. There’s the usual assortment of dismal writing, hilarious colloquialisms, and insight. It’s been an exhausting task (minnow’s teething hasn’t helped), but also a useful one, because the papers have exposed the continuing misconceptions my students have about -ology.
The assignment was to select a recent news article that was relevant to -ology, summarize it, and analyze it. Specifically, I wanted them to put the article in context, offer an opinion on the topic, and assess the credibility of the piece. Most students had no problem offering opinions and almost everyone could place the article in some sort of context (global, personal, or within the class). But very few even attempted to discuss credibility, and of those who did, many simply said things like “[Magazine] is a credible source.”
Maybe that’s my fault. Maybe it’s too much to ask of freshman. Or maybe I wasn’t clear enough about what I expected. Maybe the students didn’t know what credibility means. So the rest of this post is intended to be a guide for my current and future students. I would greatly appreciate any feedback my readers have for me.
What does “credibility” even mean? For the record, Merriam-Webster defines credibility as “the quality or power of inspiring belief.” As far as science reporting, I think that credibile means that the science was reported accurately without distortion for political or social reasons.
Here are some questions to keep in mind when you read a science news article.
Where is the article published? If an article is published in the “mainstream” media, it intrinsically has more credibility than something self-published by an individual or organization. That’s because we understand that media organizations have internal standards to ensure at least minimal standards of accuracy and ethics are met. Newspapers and magazines hire professional journalists that have had formal writing training and they may have fact-checkers to help them out. This is not to say that individuals or advocacy groups can’t publish factual articles, just that the underpinnings of bias- and error- prevention aren’t always known to be there. Thus, a science article published in the New Yorker has higher credibility than an article published by Greenpeace.
Who wrote the article? If the article is written by someone with a degree related to the general topic of research, they probably understood the science well enough to correctly report it. Most science reporters do not have science degrees, and some do excellent jobs, but without strong science backgrounds they may be reliant on press releases or fail to notice inaccuracies. There are also people and organizations that view the news as a tool to advance their political or social agenda, and they will selectively report or distort the science so that it supports their cause. Sometimes the people and organizations will be open about their agenda, but other times you may have to do a little digging to discover where the money is coming from. For example, the Greening Earth Society was created by the coal industry to cast doubt on the scientific consensus about global warming.
Who funded the research? Again, this comes back to whether the research and the resulting news story might be biased because the organization funding the research was pushing a particular agenda. Not all research funded by industries or non-governmental organizations is untrustworthy, but you need to evaluate the conclusions. For example, if research funded by the Fisheries Industry concludes that pregnant and breast-feeding women should eat more fish, despite mercury effects, I’d think twice before loading up on tuna. Unfortunately, in the general media, the funders name is often unreported, but sometimes with a little internet sleuthing you can find out.
Was the research published in the scientific literature? This is a really important thing to look for in the news report. Articles should have a sentence that says something like “The paper will be published in next week’s issue of the Journal of -ology.” If the research is published in a scientific journal, it means that three other scientists with the same specialty thought that the research was sound enough to be published and accepted as part of the body of scientific knowledge.
Were there corroborative sources in the article? Has more than one person seen the elusive ivory-billed woodpecker? Did the reporter get comments from researchers at other universities or government agencies? When other scientists can verify the results, or at least comment on their importance, that’s an indication that the news is credible.
Where was the research done and who did it? Was the research done at a well-known university, medical institution, or government lab? If so, there were procedures in place to ensure the well-being of the research subjects. Also, the researchers had to be competent enough to be hired at the institution in the first place.
How does the reported science fit in with what you know about how the world works? While some scientific discoveries come out of the blue and revolutionize the way we think about the world, most science is incremental and builds on what was already known. Based on what you’ve learned in your classes, what you’ve read in other newspaper and magazine articles, and what you have observed by looking at the world around you, you can start to evaluate whether the reported science makes sense.