There has been a major dust-up in the climate denialist world. A study published in late July made false claims and was methodologically flawed, but still managed to get published in a peer reviewed journal. The Editor-in-Chief of that journal has resigned to symbolically take responsibility for the journal’s egregious error of publishing what is essentially a fake scientific paper, and to “protest against how the authors [and others] have much exaggerated the paper’s conclusions” taking to task the University of Alabama’s press office, Forbes, Fox News and others.
Let me break it down for you
The paper, by Spencer and Braswell, was called “On the Misdiagnosis of Surface Temperature Feedbacks from Variations in Earth’s Radiant Energy Balance” and it made the claim that the Earth’s atmosphere releases more heat into space than climate scientists had estimated, thus removing concern about the warming effects of fossil CO2 being released into the atmosphere. The following things were also true:
1) The paper was published in a journal, Remote Sensing, that normally does not address climate science, although there were some atmospheric scientists on the editorial board.
2) The authors, in particular Spencer, had a reputation for being “climate change denialists” which is not a kind of scientist, but rather, a politically motivated contrarian pretending to be a scientist, in this case with some scientific credentials.
3) Author Spencer was known to have made major mistakes in his research in the past.
4) The research in the paper had glaring errors, discussed in more detail below.
At the time, I wrote in a Research Blogging review of the paper:
“On the Misdiagnosis of Surface Temperature Feedbacks from Variations in Earth’s Radiant Energy Balance” is a big ol’ bunch of hooey. I eagerly await an explanation from the journal’s editors, Dr. Wolfgang Wagner and Mr. Elvis Wang and the editorial board as to what they are up to with this paper.
Dr. Wagner’s resignation as Editor-in-Chief, which is available in print here (pdf), is a rather startling and definitive explanation! In short, the paper should never have been published.
What was wrong with the paper?
There were two major things wrong with the paper. First, the conclusion that the Earth’s atmosphere could not heat up with extra CO2 contradicted the very important facts that the Earth’s atmosphere has heated up and this heating up correlates to increases in atmospheric CO2 very much in the manner expected if the “greenhouse model” was correct. In addition, the basic idea of a greenhouse effect is pretty simple, solid, and well understood science. If something other than the greenhouse effect was happening, that would be major news.
But that sort of “flaw” — a claim that contradicts what we are very certain of — could be a virtue. A paper contradicting what everyone knows to be true would be brilliant, an amazing discovery, the stuff of awards and accolades. But, unfortunately for the paper’s hapless authors, there were other things wrong with it as well.
The numerical results presented in the paper lack statistical significance, but this is hard to detect because error bars or estimates of statistical uncertainty are presented poorly or left out. The methods used in the paper are not described well enough to verify that they could work.
When these results were examined more closely they were found to be not replicable.
The statistical strangeness of the results are explained in part by looking at the scale at which the work is being done. Standard climate models look at climate variables over various time scales from less than a decade to centuries of time. The Spencer and Braswell research inappropriately mixed time scales in a way that seems to have given them results they were looking for rather than a valid finding.
What they did, essentially, was watch a car veering towards the curb because it was trying to avoid hitting a cat, extrapolating the direction that car was moving at that moment to predict a long term pattern (which would put the car in a neighbor’s back yard rather than grocery store, where it was actually going).
In this case, Spenser and Braswell used observational data from a short time period (veering around the cat) in a model involving long term variation (the whole drive to the grocery store averaging out all the little backs and forths one effects while driving anywhere).
Clearly the research was flawed. Likely, it was intentionally flawed to support an unscientific politically motivated denialist view. This would make the paper a scientific fraud. As Editor-in-Chief of the journal, Dr. Wolfgang Wagner might have seen himself a little like a bank manager who accidentally left the vault open so crooks could steal the gold, thus his resignation. But how much of a failure of the peer review process was this?
There is another element at work here, I think, that should be considered. Remote Sensing is one of a new breed of journal, called “Open Access” which has a very different model for how journals should work. It is, frankly, the much preferred model over the traditional way things are published, but the fight between “closed” and “open” styles of publishing has been rather vitriolic. Indeed, the term “Open Access” uttered in some academic settings will produce sneers and disgusted looks among those who don’t understand what it is or how it works. And, to make things worse, it may even be the case that there are some commercial Open Access journals that are over-commercialized (though I have no credible evidence of this at hand … it is just something that “people say” as far as I know).
Had a major well established traditional “Closed Access” journal published this paper, it is possible that the Editor-in-Chief of that journal would not have resigned just because of a major dust-up over one paper. However, in this case, it may have been necessary because of the somewhat tenuous nature of this sort of publishing venture. Dr. Wagner does not explicitly state this in his resignation but he does make direct reference to the challenges of earning a good reputation in the scientific publishing field and the qualities of the two and a half year old journal.
In the end, the peer review processed worked because a paper clearly recognized as something that should not have been published has been rather spectacularly identified as such with this resignation, which is published in the very same journal in the form of an editorial. It could make sense to also withdraw the paper but it may be the case that there is no mechanism for this. And, this is the scientific literature after all. The paper is a testament to the efforts, worthy or not, of its authors. It should stay there amid the literature surrounding it, for posterity.
There is an explanation for why this paper was published that applies generally to all bad papers as well as to good papers. The peer review process is designed to meet several different objectives. Relevant to the present case are two of them: 1) Filtering out true drek — A zoology journal would not even consider the latest summary of bigfoot sightings from the north woods, and a medical journal would not even consider a study comparing different ways to make healing solutions from homeopathic crystals; and 2) Ensuring the quality of the research itself, methodologically, logically, substantively, and so on with carefully done and thoughtfully managed peer critique.
The first objective is sometimes summarily met by editors who simply do not consider manuscripts that are inappropriate, or by reviewers to whom the manuscripts are sent. When one receives a manuscript there is the option to return it unreviewed or with a note that it is out of range for the publication being considered. (This step is often avoided by sending potential reviewers an abstract, asking if they would be able and available to conduct a review.) The second objective is met by having appropriate reviewers … people who know the relevant specialty and literature very well … carefully go over the paper and critique it, and along with the detailed critique, provide a recommendation about publication.
In this case, according to Dr. Wagner’s resignation letter, three reviewers looked at the paper and had only minor criticisms. Given that this paper is deeply flawed, this means that either the reviewers did not really look closely at the paper (meaning, frankly, that they did not do their jobs) or they are also climate denialists and this was all some sort of conspiracy. I can think of no other alternatives to explain this pattern.
How likely is it that a given reviewer would simply glance at a paper, pretend to have read it carefully, and send back a poorly done review having ignored the details? This is not likely but I would guess that it does happen. What are the chances that three reviewers would do the same thing, by random chance? Very very unlikely, but it is also possible that all the oxygen molecules in a room could randomly migrate to one corner, suffocating everyone present. Well, OK, the latter is significantly more unlikely, but the chance of three poorly done reviews happening at once for a paper is not large.
If there happen to be three bogus reviews from slacker reviewers, one would expect the editor managing the paper to notice this. There would be signs. The editors read the papers and must have some idea of the quality of the reviewers’ critiques when they come back.
However, there is another way that this could happen, if an editor is not really on top of the game, a way that reviewers (or some subset of them) end up providing an inappropriately positive ranking for a paper on purpose. The authors could have submitted the paper to an inappropriate journal but made a reasonable argument that the journal should publish it anyway. That leaves open the possibility of the authors writing their own ticket for passage through the peer review process.
Many journals allow, or even encourage, authors to submit names of potential reviewers. For that matter, authors can submit names of people who either should not review a paper, or if they do, should be watched closely by the editors because of potential bias against the authors. This is a reasonable and even necessary part of the peer review process because there are factions and there is infighting in science, and there are historical quibbles or institutional rivalries or other similar cultural phenomena that should not stand in the way of science, and need to be worked around by sensitive and thoughtful editors.
It is quite possible that this paper was submitted to a journal that wouldn’t quite know how to handle it, along with “helpful” information of the kind that in other cases might have been, well, helpful, but in this case served to derail the normally earnest and honest process of peer review. That something like this happened was certainly on my mind when I first saw this paper in this journal. Since certain parts of the process of review are kept confidential (for good reasons) we may never know this. Ultimately, though, Dr. Wagner may have felt that the gate-keeping (in a good way) function of the editorial staff was inadequate, and thus his very powerfully symbolic resignation.
It is possible, I suppose, that the research in Spencer and Braswell’s “On the Misdiagnosis of Surface Temperature Feedbacks from Variations in Earth’s Radiant Energy Balance“, way down deep beneath the trickery, the bad methodology, and the scandalous politically motivated lack of scientific rigor has in iota of scientific merit. If so, this paper is on the table and available for examination, and the hypotheses embodied there could be further considered by climate scientists.
As you know, I’ve just started blogging at a second venue, called “The X Blog.” I’m going to use this opportunity to put a list of links related to Wagner’s resignation and the demise of Spencer and Braswell’s credibility over there.
Spencer, R., & Braswell, W. (2011). On the Misdiagnosis of Surface Temperature Feedbacks from Variations in Earth’s Radiant Energy Balance Remote Sensing, 3 (8), 1603-1613 DOI: 10.3390/rs3081603
Wagner, Wolfgang. (2011). Taking Responsibility on Publishing the Controversial Paper “On the Misdiagnosis of Surface Temperature Feedbacks from Variations in Earth’s Radiant Energy Balance” by Spencer and Braswell. Remote Sensing 2011, 3, 2002-2004; doi:10.3390/rs3092002