In today’s Chronicle of Higher Education there’s an article about the methods journal publishers are deploying to detect doctored images in scientific manuscripts. From the article:
As computer programs make images easier than ever to manipulate, editors at a growing number of scientific publications are turning into image detectives, examining figures to test their authenticity.
And the level of tampering they find is alarming. “The magnitude of the fraud is phenomenal,” says Hany Farid, a computer-science professor at Dartmouth College who has been working with journal editors to help them detect image manipulation. Doctored images are troubling because they can mislead scientists and even derail a search for the causes and cures of disease.
Ten to 20 of the articles accepted by The Journal of Clinical Investigation each year show some evidence of tampering, and about five to 10 of those papers warrant a thorough investigation, says [executive editor] Ms. [Ushma S] Neill. (The journal publishes about 300 to 350 articles per year.)
Maybe that frequency isn’t alarming — at worst, this is less than 7% of the articles they publish in a year where the images have been doctored. However, if one of those doctored images is in a paper upon whose accuracy you are relying (say, as a staring point for your own promising line of research), it will likely seem like too high an incidence. The images, after all, are supposed to be conveying something useful about the results actually obtained — not about the results the researchers were expecting to get, or wished that they had gotten.
Back to the article:
Experts say that many young researchers may not even realize that tampering with their images is inappropriate. After all, people now commonly alter digital snapshots to take red out of eyes, so why not clean up a protein image in Photoshop to make it clearer?
“This is one of the dirty little secrets–that everybody massages the data like this,” says Mr. Farid. Yet changing some pixels for the sake of “clarity” can actually change an image’s scientific meaning.
Why does this sound so much like, “Everyone fibs on their taxes” or “Everyone lies about sex” to me? I’ll grant that there may be an awful lot of people doing it, but it’s surely not everyone. Moreover, to the extent that researchers are not advertising that they have massaged their data or doctored their images, it seems to be an indication that they at least suspect that they ought not to be doing it. Otherwise, why hide it?
And really, in a discourse where you are supposed to make your case based on the data you actually collected, doctoring an image to “improve” the case you can make only works by misleading the scientists you’re trying to convince. If your data actually supports your claims, you can present that very data to make your case.
To the article again:
The Office of Research Integrity says that 44 percent of its cases in 2005-6 involved accusations of image fraud, compared with about 6 percent a decade earlier.
New tools, such as software developed by Mr. Farid, are helping journal editors detect manipulated images. But some researchers are concerned about this level of scrutiny, arguing that it could lead to false accusations and unnecessarily delay research.
You know what else can unnecessarily delay research? Relying on the veracity on an article in the peer reviewed literature whose authors have tampered with a crucial visual representation on their results.
“Only a few journals are doing full image screening,” says Mike Rossner, executive director of Rockefeller University Press. Mr. Rossner became a leading crusader for such checks after he accidentally stumbled upon manipulated images in an article submitted to The Journal of Cell Biology six years ago, when he was the publication’s managing editor.
He worked with researchers to develop guidelines for the journal outlining proper treatment of images, and several other journals have since adopted them. Some enhancements are actually allowed–such as adjusting the contrast of an entire figure to make it clearer. But adjusting one part of an image is not permitted, because that changes the meaning of the data.
He says all papers accepted by The Journal of Cell Biology now go through an image check by production editors that adds about 30 minutes to the process. If anything seems amiss, the authors are asked to send an original copy of the data–without any enhancements.
So far the journal’s editors have identified 250 papers with questionable figures. Out of those, 25 were rejected because the editors determined the alterations affected the data’s interpretation.
Having your original data available is always a good idea, and it seems like the best defense against a “false positive”. I don’t doubt that it might be inconvenient or upsetting for an honest scientist to be asked by a journal to prove that her images are not doctored — that it could feel like a journal editor is doubting her integrity. However, journal editors have an interest in protecting the quality of the scientific record (at least the piece of it published in their journal), and the whole scientific community depends on the goodness of that shared body of knowledge.
The people to get mad at are not the journal editors who are scrutinizing the manuscripts, but the cheaters who have made such scrutiny necessary.
At Nature Publishing Group, which produces some of the world’s leading science journals, image guidelines were developed in 2006, and last year the company’s research journals began checking two randomly selected papers in each issue for image tampering, says Linda J. Miller, U.S. executive editor of Nature and the Nature Publishing Group’s research journals.
So far no article has been rejected as a result of the checking, she says.
Ms. Miller and other editors say that in most cases of image tampering, scientists intend to beautify their figures rather than lie about their findings. In one case, an author notified the journal that a scientist working in his lab had gone too far in trying to make figures look clean. The journal determined that the conclusions were sound, but “they wound up having to print a huge correction, and this was quite embarrassing for the authors,” she says.
Ms. Miller wrote an editorial for Nature stressing that scientists should present their images without alterations, rather than thinking polished images will help them get published. Many images are of gels, which are ways to detect proteins or other molecules in a sample, and often they are blurry.
No matter, says Ms. Miller. “We like dirt–not all gels run perfectly,” she says. “Beautification is not necessary. If your data is solid, it shines through.”
In the real world, data are seldom perfect. And, to support your scientific claim, they don’t need to be perfect — just good enough to meet the burden of proof. If the data you’ve collected aren’t clean enough to persuade other scientists in your field of the conclusion to which they lead you, this usually means you need to be a little more skeptical yourself, at least until you can find better evidence to support the conclusion that appears so attractive.
Cadmus Professional Communications, which provides publishing services to several scientific journals, has also developed software to automatically check the integrity of scientific images.
The Journal of Biological Chemistry, which uses Cadmus for its printing, sends 20 to 30 papers a year through this system, at a charge of $30 per paper, says Nancy J. Rodnan, director of publications at the American Society for Biochemistry and Molecular Biology, which publishes the journal. She says the journal cannot afford to send every paper through (without passing the cost on to authors), so its editors send only those that they suspect, usually because some figures look like they have gel patterns that have been reused. Last year about six of the checked papers led to more serious investigations, and a couple of those were eventually found to have been altered inappropriately.
This almost makes me wonder whether scientists would be inclined to exert more peer pressure on each other not to cheat if the journals decided to screen every manuscript and to pass that cost on to every author.
In any case, the Chronicle article notes that not every journal has undertaken such screening measures — and, that there have been manuscripts rejected on account of suspect images by journals that do screen that were then submitted to (and published by) journals that do not do the screening. Given the lack of uniformity in the scrutiny applied by different journals, the misleading data can find its way into print. Moreover, not all cheaters are so risk averse that they won’t take a chance on slipping through the spot checks.
This is not a new problem, nor are journals’ efforts to deal with it new. (I discussed it back in January of 2006.) But clearly, it’s not a problem that has gone away in response to journal editors indicating that doctoring your images amounts to lying about your results.
I commend the journal editors who have stepped up to try to combat this problem. I assume that PIs are also making a point of communicating to their trainees that lying with a pretty image is not OK in scientific discourse.