Science After Sunclipse

But even academics are influenced by media coverage: Phillips et al. showed, in a seminal paper from the New England Journal of Medicine in 1991, that if a study was covered by The New York Times, it was significantly more likely to be cited by other academic papers. Was coverage in the NYT just a surrogate marker for the importance of the research? History provided the researchers with a control: for 3 months, large parts of the NYT went on strike, and while the journalists did produce an "edition of record", this was never published. They wrote stories about academic research, using the same criteria of importance as ever: the research they wrote about, in articles which never saw the light of day, saw no increase in citations.

Of course, we can think up plenty of caveats: some sciences might show a stronger effect than others — physics might be less skewed than medicine, say — and so forth. Still, it's a point worth considering (and gathering additional data upon), particularly since media coverage draws heavily on press releases, which are known to go wrong in amazing ways. Nowadays, though, we can get summaries and paraphrases of current research prepared by scientists for scientists. As more of these nefarious ivory-tower elitists join the game and initiatives like ResearchBlogging find better ways to "let the cream rise to the top," will science blogs take over this role, thereby eliminating deleterious steps in the chain?

THE WARDA/HAN AFFAIR

As January gave way to February, several bloggers called attention to a puzzling review article in the journal Proteomics, available online and slated for publication in the paper version. Mohamad Warda and Jin Han's paper was entitled, "Mitochondria, the missing link between body and soul: Proteomic prospective evidence." As PubMed and Proteomics now note, that paper has been retracted, but not, surprisingly, because it offered no actual evidence for its stated claim — that some grandfalutin' higher power had been at work inside mitochondria, designing the ways their proteins worked together. Instead, the paper was retracted due to "substantial overlap of the content of this article with previously published articles in other journals" — in plain language, plagiarism.

What concerned the scientist-bloggers most was not so much the transparently flawed allegations of Warda and Han themselves, but the sloppy practice of the journal Proteomics in letting those claims get through peer review into publication. Now, nobody expects peer review to be perfect — like any human institution, it's not going to be — it's just a procedure for telling, as Cosma Shalizi says, that "a paper is not obviously wrong, not obviously redundant and not obviously boring." Still, this incident was rather beyond the pale.

While the Warda and Han paper was itself obviously wrong, the developments from it have been far from boring — and not entirely in a good way. The Korean newspaper The Hankyoreh picked up the story, and in consequence machine translation gave us the delightful phrase, "OK, the power of science blog!" Shortly thereafter, Fabienne Gallaire wrote it up in the French publication Rue89. Gallaire's piece describes how these shenanigans have played out, from the beginning until now. Of particular interest is its accurate description of how the plagiarism was first discovered:

Les internautes, au contraire, ont la langue bien pendue: deux heures après la mise en ligne de cette critique, l'un d'eux remarque que la différence de style entre les parties techniques et celles plus théologiques suggère un plagiat. Encore deux heures, et une première occurrence de plagiat est identifiée: un plein paragraphe, copié mot à mot.

Moins de 24 heures après la publication de l'analyse du professeur Myers, au terme d'une curée d'une redoutable exhaustivité, les commentateurs auront prouvé qu'au moins 20% de cet article de quinze pages est un collage de passages, tirés d'une demi-douzaine d'articles non crédités dans la liste de références.

Des courriers indignés sont envoyés aux auteurs de l'article et aux responsables de la revue, ainsi qu'aux victimes pillées par ces "copier-coller" intempestifs. Le lendemain, Proteomics annonce le retrait de l'article incriminé pour cause de "redondance substantielle de son contenu avec d'autres articles". Il n'apparaîtra donc pas dans la version papier.

A year in France did my French less good than one might imagine: thanks to American scientific hegemony, several of my classes were in English, and my fellow physics students preferred to practice their Americanese on me rather than being testbeds for my French. Much of the experience I did get came from ordering dinner at the local kebab joints, but for what it's worth, my translation goes like this:

On the other hand, surfers of the Internet speak easily. Two hours after this critique had been put online, one of them remarked that the difference in style between the technical portions and the more theological portions suggested a plagiarism. Another two hours, and the first instance of plagiarism had been identified: a whole paragraph, copied word-for-word.

Less than 24 hours after the publication of Prof. Myers's analysis, after a trial of formidable exhaustiveness, commenters showed that at least 20% of the fifteen-page article is a collection of passages pulled from a half-dozen articles not credited in the list of references.

Indignant letters were sent to the authors of the article and the managers of the journal, as well as the victims looted by these inappropriate "copy-and-paste" actions. The next day, Proteomics announced the withdrawal of the incriminated article due to "substantial overlap of the content of this article with previously published articles". Thus, it will not appear in the paper version.

(Warda and Han wrote that a "mighty creator" tinkered with mitochondria; Rue89 translates this phrase as "puissant Créateur." It is my solemn duty to report that at least two people read this as "pissant Creator." I, of course, am above such behavior. . . . "Avoir la langue bien pendue" is apparently an idiom meaning to speak with ease or facility; literally, it means to have a well-hung tongue.)

After the initial heady rush of bloggers and commenters howling, "We Can Fact-Check Yo' Ass!", things kind of quieted down. As Mike O'Risal found, no bad paper is so obviously, glaringly wrong that a creationist won't seize on it for support, but we haven't seen much gabbing in the usual cdesign proponentsist circles about it. More importantly, Proteomics retracted the paper but to date — well over half a year later — has issued only a vacuous non-explanation for how it survived peer review in the first place. PZ Myers wrote, a month after the affair broke loose,

We want to know how this paper slipped through the cracks, because we want to know how large the cracks in the peer review process at Proteomics are. It's a journal with a good reputation, and we are not presuming that there was any wrong-doing or systematic failure of peer review there, but we do think that a lack of transparency is of concern: there is no assumption of a crime, but the ongoing cover-up is grounds for suspicion. Let's see some self-criticism from the journal editor, and an open discussion of steps being taken to prevent such errors from occurring again.

Alternatively, if the journal wants to outsource its quality control to a mob of bloggers, that works, too ... but we tend to be less formal and much more brutally and publicly critical than an in-house process might be, and we're also going to be less well-informed than the actual principals in the review process. Better explanations are in order. Let's see representatives of the journal provide them.

That was back in mid-March, and to my knowledge, nothing has come public since.

So: blogs meet academia, a minor goal is achieved, and then matters float without resolution on into the indefinite future. Maybe if more blogs carried the weight of institutional names, we'd see more progress in such affairs?

The epigraph to this post comes from a similar incident two years ago, where the science blogs also caught a miscreant — only that time, it wasn't a journal which published nonsense, but the magazine New Scientist. Just as with the Warda/Han Affair, the nonsense was remarkably blatant, and, just as with the later fracas, the concession in the face of criticism was, in a word, minimal. If you can't stab businesses in the pocketbook, I guess they're just not going to change their habits. And if we can't change the behaviour of the journals which academics use or the magazines on which the interested public relies, where are we going?

Some material for this post was reconstituted from the dehydrated remains of this one and also this other one.