The Frontal Cortex

Science and Climategate

Jon Stewart on the stolen Climategate emails:

I have two responses to the release of these admittedly unflattering emails. Firstly, they shed virtually no light on the actual climate science. Tyler Cowen says it best:

I see science, including climate science, as very much a decentralized process, based on the collective efforts of thousands of researchers. The evidence for our current understanding of climate change also comes from a wide variety of disciplines, including chemistry, meteorology, oceanography, geography, tree ring studies, ice sheet studies, and a good body of theory, which has held up well. These results all point in broadly similar directions. Call me naive but, with apologies to Robert Sugden, I don’t think many scientific results depend on what comes out of East Anglia, even if you include its emailing affiliates from Penn State and the like. Even very, very simple climate models generate many of the basic results.

I’d add peer-review to the list of decentralized institutions that help ensure the steady progress of scientific knowledge. Scientists complain endlessly about the peer-review process (snarky anonymous reviewers, conservative journals, etc.) but such a system ensures that every paper gets screened by researchers who have no vested interested in the thesis. They want to find its mistakes; those who falsify get the glory.

My second response is that I’m surprised that people seem so surprised that scientists are human beings, stuffed full of all the usual flaws and biases. Here’s a sneak preview of my next Wired article, which examines these cognitive blunders in detail:

Kevin Dunbar is a scientist who studies how scientists study things­–how they fail and succeed. In the early 1990s, he began an unprecedented research project: observing four biology labs at Stanford University. Philosophers have long theorized about how science happens, but Dunbar wanted to get beyond theory. He wasn’t satisfied with abstract models of the scientific method–that seven-step process we teach school kids before the science fair–or the dogmatic faith scientists place in logic and objectivity. A scientist himself, Dunbar knew that scientists often don’t think the way the textbooks say they are supposed to think. He suspected that all those philosophers of science–from Aristotle to Karl Popper–had missed something important about what goes on in the lab. (As Richard Feynman famously quipped, “Philosophy of science is about as useful to scientists as ornithology is to birds.”) And so Dunbar decided to launch an “in vivo” investigation, attempting to learn from the messiness of real experiments.

He ended up spending the next four years staring at post-docs and test tubes: The researchers were his flock, and he was the ornithologist. Dunbar set up tape-recorders in the coffee room and loitered in the hallway; he read grant proposals and the rough drafts of papers; he peeked at notebooks, attended lab meetings, and conducted countless interviews. “I’m not sure I appreciated what I was getting myself into,” Dunbar says. “I asked for complete access and I got it. But there was just so much to keep track of.”

Dunbar came away from his in vivo studies with an unsettling insight: Science is a deeply frustrating pursuit. Although the researchers were all using established techniques, more than 50 percent of their data was unexpected. (In some labs, the figure exceeded 75 percent.) “The scientists had these elaborate theories about what was supposed to happen,” Dunbar says. “But the results kept contradicting their theories. It wasn’t uncommon for someone to spend a month on a project and then just discard all their data, because the data didn’t make sense.” Perhaps they hoped to see a specific protein but it wasn’t there; or maybe their DNA sample showed the presence of an aberrant gene. The details always changed, but the story remained the same: The scientists were looking for X, but they found Y.

Dunbar was a little disturbed by these statistics. The scientific process, after all, is supposed to be an orderly pursuit of the truth, full of elegant hypotheses and control variables. (Twentieth-century science philosopher Thomas Kuhn, for instance, defined “normal science” as the kind of research in which “everything but the most esoteric detail of the result is known in advance.”) However, when experiments were looked at up close–and Dunbar interviewed the scientists about even the most trifling details–this idealized version of the lab fell apart, replaced by an endless supply of disappointing surprises. There were models that didn’t work and data that couldn’t be replicated and simple studies pockmarked with mistakes. “These weren’t sloppy people,” Dunbar says. “They were working in some of the finest labs in the world. But experiments rarely tell us what we think they’re going to tell us.”

How did the researchers cope with all this unexpected data? How did they deal with so much failure? Dunbar realized that the majority of people in the lab followed the same basic strategy. First, they would blame the method. The surprising finding was classified as a mere mistake; perhaps a machine malfunctioned or an enzyme had gone stale. “The scientists were trying to explain away what they didn’t understand,” Dunbar says. “It’s as if they didn’t want to believe it.”

The experiment would then be carefully repeated. Sometimes, the weird blip would disappear, in which case the problem was solved. But the weirdness often remained, an anomaly that just wouldn’t go away.

This is when things start to get interesting.

You’ll have to buy the magazine, or at least check the Wired website in a few weeks, to find out what happens next. (And which brain mechanisms are responsible for our irrational attachment to erroneous theories.) The larger point, though, is that the effectiveness of science has never depended on the inhuman objectivity of scientists. Instead, science works – and it really does work – because of the institutions that help correct for our innate imperfections. Scientists don’t have to be rational, because science is. Here’s Richard Rorty:

There is no reason to praise scientists for being more ‘objective’ or ‘logical’ or ‘methodical’ or ‘devoted to truth’ than other people. But there is plenty of reason to praise the institutions that they have developed and within which they work, and to use these as models for the rest of culture. For these institutions give concreteness and detail to the idea of unforced agreement.

Comments

  1. #1 Vida
    December 3, 2009

    I think the reason people are surprised scientists are human beings is because they often don’t present themselves that way. It’s not often that you’ll hear members of the scientific community admit, “Sure, we manipulate data and aren’t bastions of absolute objectivity. We’re just human after all.” In fact, a lot of scientists go out of their way to criticize others who don’t hold a dogmatic view on objectivity and logic. In this criticism, they emphasize the myriad ways that science improves life and dictates truth. In this way, they seem to differentiate themselves from the lay community, emphasizing their superior rationality and objectivity. Why would it surprise you that most people believe them?

  2. #2 OftenWrongTed
    December 3, 2009

    Piltdown Man and the hoax that went on for years is another betrayal of the public trust by the scientific community.

  3. #3 Will
    December 3, 2009

    This is typical. I think people do understand they are human. In fact, that humans design methods (some of which are only later overthrown, like tree rings) and interpreting results (objective but subjective process) makes the climate change arguments more suspect. The skeptics, sadly enough, are often true believers. The results they are skeptical of are results interpreted by other true believers. The only rational voices in this debate will be those who speak of it historically. There are too many interests involved. Money, politics and reputations are an awful mix. For now, we’re all pissing in the wind with a fundamentalist fervor. I’m happy to have ridden the fence on this issue.

  4. #4 Noah David Simon
    December 3, 2009

    are you talking about science or a priesthood? the objective data should of been shared and not hoarded from the centralized network. what you are describing is worse then fudging the data.

  5. #5 Noah David Simon
    December 3, 2009

    it’s science, not ARPA. Decentralized systems like the internet are designed for information defense and not asking the citizens for their tax dollars.

  6. #6 Lyle
    December 3, 2009

    To see details about how the scientific clan works, see Structure of Scientific Revolutions by Kuhn. What we have in the climate change community is a strong paradigm of what is happening, and that paradigm causes the community to have a strong immune response to criticism. The process is the model is built and applied to issues, and it works until enough issues/problems arise that the community decides that the consensus is wrong. See the Earth Sciences and plate tectonics in the 1960s, or the story of the Spokane Wa floods by Bretz running from the 1920s to the 1950s.
    Of course even a lot scientists are not aware of how scientific revolutions happen, let alone students who are taught a nice cut dried and picked science.

  7. #7 pete
    December 4, 2009

    Any time someone questions the so called science of global warming they get shouted down. It is Science! It is proven!

    If you hold the science of a matter out as unquestionable then you shouldn’t be surprised when people are upset over the shenanigans that go on.

    Funny, the alternative theories to explain the warming, good scientific theories from scientists at Universities, those theories are dismissed as lies propagated by ‘deniers’. It must only be science when it supports your own opinion I guess.

    No the global warming faithful have propogated a religion. And just like when preachers get caught in scandal people are jumping all over your hypocrisy.

  8. #8 Lyle
    December 5, 2009

    Re #7 you just stated the case for science in the domination of a Paradigm. Alternative theories are rejected because there is a paradigm, this is the way science really works.
    To take one example from the 20th century. J Harlan Bretz looked at eastern Washington states landscape in the 1910-1925 period and saw evidence of super floods. He presented the theory to the geological society of Washington dc, and was politely laughed out of the room. After a long period and more people looking at the landscape by the 1950s the theory was accepted. The source of the floods was the catastrophic emptying of Glacial Lake Missoula when its ice dam broke. Bretz lived long enough to get praise for his ideas, but for a while he was a laughing stock.
    As noted this is the way science works, but not the school version. The post makes the point well scientists are humans not emotionless calculating machines, and have human foibles.

  9. #9 Richard
    December 5, 2009

    Not your finest hour Jonah – what some of the emails clearly show is the manipulation of data to support the paradigms of an often fanatical and intolerant political climatology. This may be what some scientists do, but some lawyers conspire to steal by skilfully breaking the law..

  10. #10 Dacks
    December 5, 2009

    #8: Your point is? I don’t see the analogy. What is the alternative theory?

    So far, there is a strong paradigm (as you note) which is still being shaped by the push and pull of new data. Climate warming has been corroborated ad nauseum by thousands of studies. Some of these studies show surprising results – an increase in ice where a decrease was expected, for instance. These results do not invalidate the paradigm, as the scientists would be first to acknowledge.

    What the global warming skeptics have not done is produce a viable counter theory that explains the extant data. Until that happens, drawing attention to real or imagined misbehavior (I don’t know which category the current kerfuffle falls into) has no impact on the validity of the AGW thesis.

  11. #11 Steve Silberman
    December 5, 2009

    Thanks for posting this, Jonah. Imagine if R.J. Reynolds and company had been able to hack into the email of cancer researchers in the ’50s and extract a few phrases to be presented out of context, allegedly proving that “fanatical and intolerant” doctors were trying to create a socialist nanny state where a man couldn’t simply enjoy his after-dinner smokes.

    What’s really chilling is how the Murdoch/GOP noise machine has jumped on this to discredit science as a whole (WSJ example: http://bit.ly/4E23A5) and paint scientists as just another partisan special-interest group motivated by greed; as if creating thousands of US-based green jobs was somehow insidious and unpatriotic.

  12. #12 Milt Lee
    December 6, 2009

    I find the issue of human behavior and science to be a bit nuts. Of course some scientists are nuts – as we all are. But to leap on this and then say that this proves the climate isn’t changing – is even more nutty.

  13. #13 Steve Silberman
    December 6, 2009

    There’s nothing in those emails that suggests those scientists are “nuts.” Under tremendous pressure, yes, which escalated to Grand Guignol heights this week.

    Meanwhile, the NYT’s Andrew Revkin just published a useful update on the state of the science: http://bit.ly/51svol

  14. #14 davidbmc
    December 8, 2009

    What is the optimum temperature of the earth? Or of each hemisphere? Or of each continent? What is our target temperature here and who says so and how do we know?

  15. #15 Sans Talbot
    December 16, 2009

    I’m not a Republican, and in general, I’m in favor of reducing pollution, so a world terrified of global warming doesn’t bother me.

    But for the love of all that’s holy, can anybody explain to me how climatology is in any way a science?

    In science, we test our hypotheses. How on Earth (if you’ll please forgive the pun) do you test climate models?

    There’s only one Earth, after all, which discounts even the possibility of a control group. Plus, if you had two earths, you’d need the ability to make changes in one of them, of large enough scale to have an effect. Then you’d have to wait the decades or centuries it would take to get results.

    Climatologists ‘test’ their theories with computer programs which are, after all, nothing more than an embodiment of their theories.

    The sciences climatology draws from may be empirical, but climatology itself has zero inherent empiricism. There is literally no way to test a climate model.

    When that happens in a ‘science,’ it should be no surprise if you get thousands of researchers all confirming each other’s opinions.

    How many paid climatologists, after all, disagree with the main ideology?

    Answer – only the five or six paid by the Republicans.

    Everybody, on both sides, is effectively paid to support the side they support, and there can never be any empirical data proving anybody right or wrong.

    I don’ know what that is, but it’s NOT a science.

  16. #16 John
    December 23, 2009

    Your ornithologist friend is far from the first to observe scientists in action (not that you claimed he was)–you might read the work of anthropologists of science, who tell some fascinating stories about real lab work as well–check out the work of Bruno Latour, Paul Rabinow (author of _Making PCR_ and _French DNA_, and Margaret Lock (on organ transplants), just to name a few.

  17. #17 Andrew Berman
    January 1, 2010

    John,
    The peer review process was clearly being manipulated– journals were being put under pressure to not approve certain articles. And while we’re at it, peer review is not some abstract mathematical machine that guarantees correctness in an article– it’s designed to catch obvious flaws. Incorrect data passes through peer review quite easily. Experiments are not re-done– nobody who approved any of the suspect articles went and looked at the computer programs that massaged the data, for example.

    I challenge the notion of independence of data sets. What does your neuroscience background tell you about what happens to other scientists when you have a powerful cluster of scientists who have a certain view of the world?

    Finally, here’s a proposition: Man-Made global warming theories are incorrect and the scientists are working with bad data and are too ensconced in their positions to admit to it. Whatever the odds were that that proposition is correct, Climate gate has raised those odds.

The site is currently under maintenance and will be back shortly. New comments have been disabled during this time, please check back soon.