Realclimate has a good explanation of the latest battle in the hockey stick wars. It looks to me like McIntyre & McKitrick’s claim (that the hockey stick is the product of an erroneous calculation) is not correct. That doesn’t mean that the graph is correct of course, since the proxies the graph is built on may not measure temperatures very well.

In an editorial, the Wall Street Journal systematically misrepresents the whole affair:

In 1998, Willie Soon and Sallie Baliunas of the Harvard-Smithsonian Center for Astrophysics published a paper in the journal Climate Research, arguing that there really had been a Medieval warm period. The result: Messrs. Soon and Baliunas were treated as heretics and six editors at Climate Research were made to resign.

In fact, the editors resigned because they felt that Soon and Balanius’s paper was so badly flawed that it should not have been published.

In 2003, Stephen McIntyre, a Toronto minerals consultant and amateur mathematician, and Ross McKitrick, an economist at Canada’s University of Guelph, jointly published a critique of the hockey stick analysis. Their conclusion: Mr. Mann’s work was riddled with “collation errors, unjustifiable truncations of extrapolation of source data, obsolete data, geographical location errors, incorrect calculations of principal components, and other quality control defects.” Once these were corrected, the Medieval warm period showed up again in the data.

This should have produced a healthy scientific debate. Instead, as the Journal’s Antonio Regalado reported Monday, Mr. Mann tried to shut down debate by refusing to disclose the mathematical algorithm by which he arrived at his conclusions. All the same, Mr. Mann was forced to publish a retraction of some of his initial data,

If you actually, I don’t know, **read** Mann’s correction you’ll find that he didn’t retract his initial data, but corrected the description of it. And that mathematical algorithm that the WSJ alleges that Mann refused to disclose? It’s right here.

Statistician Francis Zwiers of Environment Canada (a government agency) notes that Mr. Mann’s method “preferentially produces hockey sticks when there are none in the data.”

This strikes me as a big red herring. If you do a linear regression on random data, you’ll produce a straight line. Does that mean that linear regression is invalid because it preferentially produces straight lines when there are none in the data? Of course not. What is important is whether the result of the regression is statistically significant—for random data it won’t be. William Connolley did some experiments and reports:

What that appears to demonstrate is that M&M are right about one thing: it often does lead to a “hockey stick” shape in random data. But the problem is that the variance-explained of the PC1 done this way is tiny: the first eigenvalue is about 0.03. Whereas when you run it on real data the first eigenvalue is about 0.55 (back to 1000) or 0.38 (back to 1400). Which means the two problems are very different.