The Island of Doubt

The Good Old Hockey (stick) Game

Posted by James Hrynyshyn on July 17, 2006
(11)
More »

Chris Mooney is sick of the stick. The hockey stick, that is. I don’t blame him. How often should we have to revisit the tired argument over whether today’s climate is warmer than any time in the last 400 years or 1000? But here we are again, thanks to Joe Barton’s House Committee on Energy and Commerce, which commissioned three statisticians with no expertise in climatology to give Mann’s graph yet another once-over.

In the absence of anything else of substance to chew on this morning, I bothered to look at what Barton’s three wise men came up with. Same old, same old, it would appear. But wait. Not so fast. Turns out there is way more at stake. From the Barton report:

In this case we judge that there was too much reliance on peer review, which was not necessarily independent. Moreover, the work has been sufficiently politicized that this community can hardly reassess their public positions without losing credibility.

Read that again.Three allegedly distinguished academics — Edward J. Wegman of George Mason University, David W. Scott of Rice University and Yasmin H. Said of Johns Hopkins University — have taken it upon themselves to to attack not just paleoclimatologist Michael Mann, whose work was largely vindicated by the National Academies of Science just the other week, but the tendency of climate scientists to rely on peer review.

Their argument, at its core, is that climatologists are lousy at statistics and so should be working more closely with us, the real experts:

As statisticians, we were struck by the isolation of communities such as the paleoclimate community that rely heavily on statistical methods, yet do not seem to be interacting with the mainstream statistical community. The public policy implications of this debate are financially staggering and yet apparently no independent statistical expertise was sought or used.

Dr. Mann isn’t taking that lying down. Nor should he. He writes:

The panel makes the odd claim that there is “too much reliance on peer-review” which goes against every principle of current scientific practice. Barton in his ‘factsheet’ goes further and suggests that the anonymous peer reviewers themselves are in some way biased, a claim that he cannot possibly support since peer reviewers are in fact anonymous and this was not studied in the report

Indeed. While it is almost certainly true that any specific field of science is characterized by a small circle of researchers who spent a lot of time interacting with each other, that doesn’t mean they can’t do their jobs as scientists.

And yet that’s exactly what we get in the (non-peer-reviewed) Barton report:

In our further exploration of the social network of authorships in temperature reconstruction, we found that at least 43 authors have direct ties to Dr. Mann by virtue of coauthored papers with him. Our findings from this analysis suggest that authors in the area of paleoclimate studies are closely connected and thus ‘independent studies’ may not be as independent as they might appear on the surface.

To my mind, this is about much more than the validity of one graph. It’s about the very nature of scientific research in the 21st century. Much has been written of late about the flaws in the peer review process. But as others have noted, it’s like what Churchill said about democracy — the worst form of government except for all the others. Are the authors of Barton’s latest assault on the climate change consensus really suggesting we get rid of peer review?

Probably not. All we really get from them is “you should have talked with us first.” But this sort of disdain for the scientific community is entirely consistent with the lack of respect for science that typifies the Republican and Bush approaches to public policy. I suggest we should be very concerned when any elected official with any significant degree of power supports the not so subtle message that scientists by their very nature, can’t be trusted.

We have to nip this one in the bud, folks.

(11)
More »

Comments

  1. #1 Ben M
    July 17, 2006

    As statisticians, we were struck by the isolation of communities such as the paleoclimate community that rely heavily on statistical methods, yet do not seem to be interacting with the mainstream statistical community

    I’d hate to see what these guys think of, say, nuclear and high-energy physics. We rely extremely heavily on statistical methods, often highly elaborate ones; we not only have very little contact with modern statistics research, but most of us have never taken a statistics course. Stats is something you learn as you go along.

    These guys’ objection seems to be the standard response to a scientist reading papers outside of his or her field. I know that, when I read (e.g.) a biology paper, I have no way of evaluating the biological reasoning, the molecular probes, etc. But—and this is a very reliable response on my part—as soon as they make mention using a radionuclide and a thermoluminescent plate, I’ll pick it out as The One Thing that needs to be looked into, because they didn’t do it the way I would, and I know more about radioactivity than the authors do. It’s simply a matter of deferring on matters unknown, feeling that you’ve deferred too much, and looking for something on which you feel qualified to comment. I bet it happens at every grad student journal club in the country.

  2. #2 Barry
    July 17, 2006

    And that’s assuming that Barton picked these three statisticians at random. Which I really, really doubt.

  3. #3 Koray
    July 17, 2006

    I don’t get it. Have they used statistical methods wrongly? Do Barton’s three wise men point out the errors explicitly, or is their complaint limited to not being invited to play?

  4. #4 James Hrynyshyn
    July 17, 2006

    Koray: No, their argument does cover what they consider to be errors of statistics. However, much of what they identify as problematic was previously reviewed by the NAS panel, and found more or less solid. That’s the nature of statistics — there is a fair bit of room for debate over the wisdom of applying one tool over another. I’m not arguing they’re wrong, only that by including their criticism of within-the-climatology-community peer review, they reveal a curious bias.

  5. #5 Jasmine
    July 17, 2006

    I too originally stumbled across this report after trying to see what the Energy cmt had scheduled (we are at war after all…) and was shocked to see this old, tired bit of business on the agenda of this committee.

    After reading the “report,” I was even more amazed at its unreferenced and unsubstantiated attack on the peer review process. It’s ludicrous to draw conclusions on the identity of anonymous reviewers based on a network of individuals who have published on “temperature reconstruction.”

  6. #6 Dano
    July 17, 2006

    t’s about the very nature of scientific research in the 21st century.

    Yes.

    They are trying to marginalize science that doesn’t track with their worldview.

    Just another noodle flung at the wall to see what sticks.

    Best,

    D

  7. #7 CT
    July 17, 2006

    Let me not put words into mouths here but it seems to me that this assesment from these 3 statiticians has clumsily expressed only what has become fashionable amongst global warming deniers.

    The argument du jour has been that this issue has been politicized and usurped by a small group of researchers. That in turn has skewed not only the debate on it (basically silencing it) but the actual facts and figures presented.

    I don’t think this attack on Mann’s figures is a broader attack on the whole of the peer review process. We need to take their sentence in full, as they went on to explain that the employed peer review process was overused because it “was not necessarily independent.”

    This is a very specific attack, as has become the norm assault from those who booed ‘An Inconvient Truth,’ that the inbred, non inclusive nature of the community of researchers working on global warming makes objective peer review exceedingly difficult. What with professors like Bill Gray and Richard Lindzen more and more in the public eye it is becoming more difficult to deny them and many other anti-alarmists a more prominent position at the table.

    Finally, there’s no doubt that climate change is one of the most politicized scientific issues of all time. And it may need to be because of the nature of the problem. Even so, politics is NEVER a good thing for actual science. At the least, there’s something to take from the report for the House Committee, that the public nature of the claims and the politics at stake should make all slightly more wary (that’s all, there’s no denial here) of the science (either in support of or challenging the theory of global warming) for the difficulty the the nature of the lens focused on climate change has probably made in maintaining subconcious objectivity.

    p.s.
    I like run on sentences and poor punctuation…

  8. #8 dogscratcher
    July 18, 2006

    “Are the authors of Barton’s latest assault on the climate change consensus really suggesting we get rid of peer review?”

    Yes they are. Its the “Readers Digest” method of science. Amusing anecdote anyone?

  9. #9 Hank Roberts
    July 19, 2006

    Abbreviated from what I posted at Stoat:

    Wegman, as quoted by Barton, on what statisticans do:
    http://energycommerce.house.gov/108/News/07182006_1995.htm

    “With clinical trials for drugs and devices to be approved for human use by the FDA, review and consultation with statisticians is expected. Indeed, it is standard practice to include statisticians in the application-for-approval process. …”

    New England Journal of Medicine:
    http://content.nejm.org/cgi/content/full/353/10/969
    FDA Standards — Good Enough for Government Work?
    Jerry Avorn, M.D.

    “…there is one area of biomedicine in which the government allows — even defends — a minimal standard that would be unacceptable anywhere else in research. It is the set of evidentiary requirements maintained by the Food and Drug Administration (FDA) for the approval of new drugs.”

    This is why they want to control science — to sell shit.

  10. #10 per
    July 27, 2006

    there appear to be a couple of minor errors in this piece.

    1) the wegman report was peer-reviewed. moreover, I believe that there is an open offer that anyone on the committee can arrange their own, independent peer-review (i.e. both democrat and republicans)

    2) professor north, the chair of the NAS/NRC panel, gave testimony immediately after wegman. I recall that he gave testimony that the nas panel agreed with the statistical criticisms that wegman raised.

    to be explicit, the nas panel also found that decentring the PCA affects the reconstruction and mines for hockey-sticks. The nas panel has reservations on the linearity of tree response to temperature, and recommends that bristlecone pines should not be used for reconstructions. The NAS panel pointed out that the whole of mbh is dependent on the effect of the bristlecone pines around the california basin.

    oh and specifically; the nas panel agreed that we know temperatures back to 400 years ago; but they were very clear that you cannot reliably evaluate the accuracy of studies that go back before then. So if you think that the temperatures of 1000 years ago are merely a tired old argument, you are disagreeing with the NAS panel.

    yours
    per

  11. #11 TTT
    July 27, 2006

    Liars, damned liars, and statisticians.

    Bjorn Lomborg–anti-science author of “The Skeptical Environmentalist,” which claimed among other things that since we are catching more fish in the ocean, fisheries productivity must have increased–is a statistician.

    So too is Steven Milloy of junkscience.com, who claimed that the World Trade Center fell down because environmentalists took the asbestos out of it.

    They’re like lawyers but with math instead of words. They can argue for any position, no matter how factually bankrupt it is.