The value of polling scientists

Posted by William M. Connolley on August 29, 2007
(4)
More »

There have been various attempts to survey scientists opinions about climate change. Wiki has an article on this: Scientific opinion on climate change. Check the recent history for another attempt… :-)

All of these attempts have various methodological problems which I’m sure you can think of for yourselves; but they also suffer from a structural problem which is what I’ll discuss here: which is: does it matter?


Of course in terms of public relations it does matter. People like to quote these things one way or the other, and learned societies put out statements either because they feel they ought to or in the spirit of public outreach or for some other reason. If public opinion went with the learned societies then there would be no doubt at all over GW (see the wiki article). This rather suggests that either the PR doesn’t work (or that it would be even worse without them?).

But is this the correct way to assess scientific opinion? I would say mostly no: what counts for science is what is published in the literature (and, I suppose, spoken at conferences etc; but any of that of any value get published eventually (unless you’re JA arguing against the freqentists, of course)). Therefore the correct way to survey “opinion” is not to ask people what they think but to read their papers. Oreskes attempted this with abstracts, and anyone wanting to do more of this should do the same, but more thoroughly.

There are some problems with this argument, of course.

One is that of timeliness: if things are fast-changing, the literature won’t reflect that. Arguably the possibilities of large sea level rise from polar ice sheets comes in this category. But if you are trying to assess agreement or otherwise with the IPCC, which is/should be largely based on pre-2006 literature, thats not so much of a problem.

Another is that you have to assume that the literature is “fair”. Some would say that it unfairly rejects “sceptic” material; others can just as plausibly argue that it unfairly priviledges speculative papers that disagree with the consensus.

But probably the main problem is that if the question you want to ask is The scientific basis for human impacts on climate is well represented by the IPCC WG1 report. The lead scientists know what they are doing. We are warming the planet, with CO2 as the main culprit. At least some of the forecast consequences of this change are based on robust evidence. then you can’t do it easily that way, since people don’t write papers that say this. Instead I suppose you have to pluck out various statements from the SPM and see if you can find people to disagree with them. For example:

The understanding of anthropogenic warming and cooling influences on climate has improved since the TAR, leading to very high confidence7 that the global average net effect of human activities since 1750 has been one of warming, with a radiative forcing of +1.6 [+0.6 to +2.4] W m-2

or

Warming of the climate system is unequivocal, as is now evident from observations of increases in global average air and ocean temperatures, widespread melting of snow and ice, and rising global average sea level

or (more controversially :-)

Palaeoclimatic information supports the interpretation that the warmth of the last half century is unusual in at least the previous 1,300 years.

All of those could be testing by looking for papers that disagree.

The other problem is the good old fashioned “science isn’t done by consensus” arguement. Which is true enough, but not very illuminating. You can only go with what we have. In an effort to think about that: Suppose, just at the start of the twentieth century you had attempted to decide scientific opinion on “is the universe essentially Newtonian”? what would you have found? Not being a historian of science, I’m not sure. Had you been a person of the time, likely you would have interpreted things one way and come to the conclusion that there were a few quibbles but most people did feel that way. Looking back now, you might well find all the papers that were beginning to find problems with the conventional classical interpretations. I’m not sure that helps one way or the other.

[Update: of course, you could try asking reality instead -W]

(4)
More »

Comments

  1. #1 mugwump
    2007/08/29

    By Gavin Schmidt’s own admission, climatology is above reproducibility. Hence it should now be accorded the same respect as other fields that consider themselves above reproducibility – eg paranormal studies.

  2. #2 fergus
    2007/08/29

    Does scientific opinion on climate change matter, beyond the PR value? Of course it does. Not so much within science itself, perhaps, though there are those who would argue this too, as it might help liberate the ‘voices’ of individuals who could be intimidated by the prospect of opprobrium and thus prevented from voicing their true opinion.

    Where the value in understanding scientific opinion lies mostly is in the critically important interface between scientific results and public attitudes. If the survey of public responses to CC cited today on the radio is correct, 90% of the public are aware that they should be doing something about climate change, but only 10% actually do it. At least one plausible explanation for why people aren’t changing their habits (and this applies to governments, too) is that there is no clearly established and trusted ‘authority’ to act as a guide. It is a given that a rational person would want reason to be the guide, and in CC, this means climate scientists. Whether the expectation of a definitive, clear message is misplaced or not doesn’t bear: the public perception is that the science is still ‘uncertain’ (a loaded, multi-meaning term).

    If the public or policy makers are in a position to see that climate scientists can speak with ‘one voice’, even on a relatively straightforward matter, this offers a reassurance and a motive for ending apathy. Perhaps this shouldn’t be how things work, but it is (at least in part), how it seems to pan out in the real world. In brief: we want to be told, without prevarication, ‘the truth’.

    As to how you establish what that opinion is: there is a lot of value in reviewing the findings of a collection of papers (and the embedded research) on a regular basis, and keeping the public up-to-date with the ever-progressing understanding of what is going on and how the body of evidence is developing. OTOH, if you want to know what someone thinks, surely the easiest way is to ask them.

    Polls and reviews alike need careful construction and considerable thought for any meaningful and valid conclusions to come from them. They also need to be robust to criticism and open to discussion and revision. This may be a difficult task, but it is not beyond the competence of a dedicated team of researchers to achieve.

    For the benefit of scientists and the public alike, I suggest then that having a clear notion of what scientists think is one of a number of desirable, if not necessary ways of getting the message across in a way which will improve the prospects of CC action being timely and potent.

    Regards,

  3. #3 jules
    2007/08/29

    You suggest that we should read all the papers to work out what scientists believe, but scientists don’t usually write what they believe in their papers. They write what is publishable. Often when you talk to them in person their actual beliefs appear far stronger (or even weaker – a year or so later :-) ) that you would have been led to believe by the host of maybes ifs and buts that they correctly include in their papers.

    [Thats not really what I'm arguing. I'm arguing that what they believe is less important than what they publish -W]

    Also it’s the IPCC’s job to report on all the papers. Obviously that’s a huge task which can’t reasonably be undertaken by one person, and can it even be undertaken by the IPCC without bias? So asking us all to get a feel for what scientists think by reading the papers seems rather odd. I think you are taking this rather too seriously, and that seeing what how scientists respond to a poll is quite an entertaining experiment.

    [I'm not asking *everyone* to do this; I'm asking one or a few people to do it and report the results. Or at least I'm saying it would be better than polling, entertaining as that might be -W]

    I find it interesting what climate-scientists think mostly because climate science is so political that it is barely a science. Non-scientist people ask me all the time about global warming – and they jolly well expect me to have an opinion! I’m looking forward to when all this future-climate-change stuff blows over and we – or those that are left after all the politically motivated funding disappears – can get back to doing science again.

    [Ah well, I can sympathise with that -W]

    Obviously the poll in question is fundamentally flawed because I didn’t get asked. Or maybe I did, and it ended in my spam box, or maybe I delete all emails that start with a very large first paragraph that goes something along the lines of, “you don’t know me but I am a really nice guy and I need you help….” :-) Definitely a bias against people with well educated spam filters…

  4. #4 fergus
    2007/08/30

    Someone has drawn my attention to this: http://www.dailytech.com/Survey+Less+Than+Half+of+all+Published+Scientists+Endorse+Global+Warming+Theory/article8641.htm

    Which I’ll be looking for shortly. Slightly peculiar that it should have been produced by an endocrinologist, but can’t really comment without access to the paper.

    Regards,