Belated retraction of Seralini's bad anti-GMO paper

Last year, the Elsevier journal Food and Chemical Toxicology by Gilles Seralini and others that purported to show that rats fed genetically modified corn were more prone to get cancer. The cranks loved it; Mike Adams thought it was great, it was touted on the Dr Oz show (I don't know why they were concerned; these are the people who think cancer can be cured with herbs, urine, and drinking hydrogen peroxide).

But right from the beginning, scientists were appalled — not by the conclusion, but by the incredibly shoddy protocol used by the researchers. Biofortified went through the paper, step by step; would you believe that in a study with a control group and multiple experimental groups fed on GMO corn, the authors did not use any statistical tests to tell if there was a significant difference between any of the groups?

Let that sink in.

Here's the first figure from the paper, and Ashley Ng's breakdown of the data.

In the study, Figure 1 shows Kaplan Meier plots the number of rat deaths by “control group” and other “treatment groups”.

What do these mean? Well, not much because the authors failed to use a statistical test to tell if there was a difference between the control groups and treatment groups.

This is important, as all their claims relate to the incidence of cancers (and other “diseases”) in the “treatment group” compared to the “control group”. These comparisons can only be made if a statistical test shows that what you observe is not happening by chance.

Still on Figure 1, we see that several “treatment groups” of male rats receiving GM NK603 corn (the 22% group and 33% group) actually had fewer cancers than the male control group.

Similarly, a treatment group of male rats receiving 33% GM corn and Roundup had no difference to the control group, and two treatment groups receiving Roundup (A and C) had the same or less incidence of cancer compared with the control group.

I just eyeball the data, and what I see is typical noisy cancer mortality data (these are rats with a genetic predisposition to get cancer: 70% of males and 87% of females get it.) The one thing that would have looked significant to me is the higher likelihood of females coming down with cancer…but that's a predilection already built into the strain. The problem is compounded by very small ns — there were only 20 rats in each group. I wouldn't be surprised if the researchers had done some statistical analysis, but didn't report it because the paltry statistical power of their study meant nothing was significant.

At the time the paper came out, Carl Zimmer also raised holy hell because it was another case of science by press conference. There were all kinds of complaints by scientists about the study, but journalists who got the paper in advance had to sign confidentiality agreements that prohibited them from consulting with experts — they were expected to flounder about in the dark and simply accept what they were told.

Here's a little good news, though: the paper is being retracted. The editor-in-chief of the journal has made a rather weasely statement denying any wrong-doing by the authors, but that the paper is being retracted solely because of the ambiguity of the results.

Unequivocally, the Editor-in-Chief found no evidence of fraud or intentional misrepresentation of the data. However, there is legitimate cause for concern regarding both the number of animals in each study group and the particular strain selected. The low number of animals had been identified as a cause for concern during the initial review process, but the peer-review decision ultimately weighed that the work still had merit despite this limitation. A more in-depth look at the raw data revealed that no definitive conclusions can be reached with this small sample size regarding the role of either NK603 or glyphosate in regards to overall mortality or tumor incidence. Given the known high incidence of tumors in the Sprague-Dawley rat, normal variability cannot be excluded as the cause of the higher mortality and incidence observed in the treated groups.

He then goes on to praise the peer-review system, which is weird, because here's a paper with huge obvious holes that fell right through the system. And furthermore, it's a paper with gigantic political implications — right now, it is the linchpin of anti-GMO movements around the world — and should have gotten extra-careful scrutiny.

Jon Entine at Forbes has an especially thorough dissection of the implications of the paper. It reveals that other problems have emerged.

“The study appeared to sweep aside all known benchmarks of scientific good practice and, more importantly, to ignore the minimal standards of scientific and ethical conduct in particular concerning the humane treatment of experimental animals,” concluded a prominent group of scientists in Transgenic Review. They noted the rats in the study were exposed to extreme and unnecessary cruelty. None of the results depended on the size of their tumors or how long they lived after the tumor appeared. This unethical treatment of animals was a direct violation of accepted research protocol and was by itself grounds for the article being rejected initially or withdrawn.

It was rather peculiar that the paper reported only on mortality. They were studying the appearance of cancer, so a more relevant and direct measure would have been to assess by the appearance of tumors of a particular size, and then to humanely euthanize severely affected animals. This study had them languish in a cage until they died and could be scored. There was no description of the cancers in the control group! They did seem to have a number of rats with huge, grossly disfiguring tumors that were handy for photo ops, though.

So it was a terrible, sloppy paper with gaping deficiencies that somehow slipped past peer review but made scientists gape in surprise when they finally saw it published, and it's finally being retracted. But too late: anti-GMO propagandists are now seeing the retraction as a sign that there is a conspiracy to Hide the Truth™, and are using the efforts to apply standards of evidence to the work as proof that Big Science is out to give everyone cancer.