As I've noted previously, there have been attempts to question the scientific peer review process following the Hwang Woo Suk scandal. But a Rick Weiss article in the Washington Post over the weekend helpfully explains why it's naive to think that peer reviewers can catch this kind of chicanery:
Despite all the recent hand-wringing, there may be precious few new lessons to be learned from the Korean debacle, several experts said. Even the journal editors who promised to beef up their screening of submitted manuscripts say privately they doubt there is a practical way to intercept the small proportion of scientists determined to cheat.In the end, several noted, most research misconduct that comes to light, including Hwang's, does so for the most old-fashioned of reasons: Colleagues or former co-workers turn in the cheaters.
This should have been obvious--but unfortunately, too many conservatives have seized on the Hwang scandal and tried to use it as a club against the scientific community (which they have their problems with for other reasons). So it's important to set the record straight. Frauds will always exist, but that doesn't make anyone else responsible for their transgressions--unless they are willing accomplices. Scientific journal editors certainly aren't that.
- Log in to post comments
The problem, of course, is that many of the people who may be able to turn in a cheater are lower in the hierarchy and thus are taking a honking big risk to reveal the cheating. Often, given the reputation of the very successful scientist they're working under (Hwang was practically a national hero), they fear -- quite rightly -- that they won't be believed, that their complaint will be put down to a personal grudge or emotional disturbance or just not being an experienced enough scientist to understand how things are properly done.
So maybe the journal editors are off the hook for this one (although being really clear in what kind of contribution is required to have your name listed as an author on a paper, and what kind of responsibility coauthors have to ensure their collaborators' contributions are legitimate, might be a good thing). But it seems to me the community of science needs to take a look at its hierarchies and work out how to break them down at least a little. The ideal, after all, is a scientific community where junior colleagues can safely call shenanigans on fabricated or falsified research before it's left the lab and made a splash in the journals.
What is interesting about the critical reflection on peer-review, is that from my observation, most of it has come from scientists in their remarks to journalists, in op-eds, in interviews on NPR, or in news analysis articles by leading science writers such as Wade or Kolata.
In contrast, most of the conservative attacks have focused specifically on stem cell research, and have not moved to more institution-wide criticism.
Scientists may be getting themselves into the same trouble that journalists have. Conservatives attacked journalists for being liberal, and then journalists, especially after major focusing events, have devoted a lot of attention to meta-coverage i.e. are we really liberally biased, if so, what should be done about it? In the process, journalists only served to amplify the claims of conservatives. Scientists may now be doing the same.
Over at www.framing-science.blogspot.com, in the next couple days, I'm going to post a longer round-up evaluating this form of meta-coverage of the peer review process. Stay tuned.
I think I'm in agreement with Matt - that most of the questions about peer review that I've seen have been coming from scientists themselves. See Donald Kennedy's comments at the end of this article, for example. The only conservative I've seen you comment about was Smith and as I pointed out he wasn't talking about the scientific community in general, but peer review specifically.
A point I've personally read elsewhere, but probably bears repeating is that even with all the negative attention that this debacle/fiasco has received, it does prove the point after all that the system DOES work. Perhaps Science and Nature's review standards may not be as rigorous as the could be, but there is also a point at which you can be as careful as you want, but can't possibly catch everything. Having subordinates or collaborators or competitors blowing the whistle is part of the more system-wide effort of "peer review" and the how scientific discovery is accepted or refuted. One should never assume that everything in print is gospel, so to speak.
On Dr. Freeride's point above, I think (and this is purely speculation) that the woman who emigrated to Dr. Schatten's lab from Hwang Woo Suk's may have been the whistle blower in this case. Perhaps she felt it ethically imperative to leave, and this is how Dr. Schatten's hackles went up in the first place (I'm sure the patent issue didn't help), recieving inside info of some transgressions. Granted that leaving your job (and country) is not a very easy and comfortable way to deal with wrongdoing, but at least there is (potentially) a way for a subordinate to blow the whistle without fear of retribution.
Thanks everyone. I agree this shows the broader system of science does work. I was probably a bit overzealous in saying that the peer review criticism was just coming from conservatives. Thanks for setting me straight on that.
I agree that little can be done on the part of editors or peer reviewers to catch determined cheaters. The peer review system is simply not designed to detect fraud. When a journal receives a paper to evaluate, they have been handed a finished product. Since it would be enormously time-consuming and costly to request and evaluate the many intermediates which preceded this product, reviewers must trust that the data are presented honestly and accurately.
That being said, I do not feel we should throw our arms in the air and say, "Frauds will always exist, but that doesn't make anyone else responsible for their transgressions--unless they are willing accomplices." The majority of scientific misconduct is performed by graduate students and postdoctoral researchers without any knowledge of the lab head. Aside from an embarassing retraction, the boss often suffers no long term professional damage. If he had stepped off the lecture circuit for a few moments and asked a few questions before rubber stamping that manuscript, there is a high probability the fraud would have been detected. In my opinion, a great deal of fraud can be prevented at the institutional level, before the paper ever lands on an editor's desk. Research institutions need to make it clear that even if the lab head is not a "willing accomplice" in the fraud, he can and will be held responsible for research misconduct in his lab.
Check out a recent egregious example:
http://www.nature.com/nature/journal/v437/n7061/full/nature04181.html
I think the reason the criticism has focused on stem cell research in particular rather than on the larger system of peer review is that there has been so much other deception in the stem cell field--the California initiative, we now know, was sold on the back of some flat-out lies about how funding and royalties would work, and of course the promise of the research more generally has been astonishingly and shamefully oversold, leading many politicians and others who want to support the science to parrot things that are not true and make total fools of themselves, thereby setting the field itself back (for instance, what my governor Tom Vilsack said in his State of the State speech last week).
We should also recognize that new anti-fraud measures built into the peer review process will have costs--in terms of journal resources, time required to get papers into print, etc. A decent argument could be made that we are currently not in such a bad place in the cost/benefit equation--major frauds seem to be relatively rare and, at least in this case, caught quickly and decisively.