Useless on-line survey of climate scientists

In a report on a climate change seminar, Bernd Ströher and Benny Peiser write:

Particularly revealing were the almost sensational results of a survey conducted by Prof. Bray among some 500 German and European climate researchers. The results show impressively that the much-repeated claim of a "scientific consensus" on anthropogenic global warming is a carefully constructed piece of fiction: According to the survey results, some 25% of European climate researchers who took part in the survey still doubt whether most of the moderate warming during the last 150 years can be attributed to human activities and CO2 emissions.

Robert Matthews writing in the Telegraph claims the scientists were even more skeptical: (my emphasis)

Prof Dennis Bray, of the GKSS National Research Centre in Geesthacht, Germany, submitted results from an international study showing that fewer than one in 10 climate scientists believed that climate change is principally caused by human activity.

As with Dr Peiser's study, Science refused to publish his rebuttal. Prof Bray told The Telegraph: "They said it didn't fit with what they were intending to publish."

Both stories are referring to the same survey. Clearly they can't both be right. The relevent question and result of the survey:

40. Climate change is mostly the result of anthropogenic causes.
strongly agree 1 2 3 4 5 6 7 strongly disagree
9% 25% 21% 14% 8% 11% 10%

The Telegraph is only counting the 9% who chose 1 on the seven-point scale as agreeing with the statement. This is not correct. Clearly, responses 1,2 and 3 all indicate agreement with the statement. Still, a significant fraction disagree with the statement.

Is global warming skepticism amongst climate scientists as widespread as this survey indicates? To answer this we need to look at how the sampling was conducted:

The 2003 survey was conducted as an on-line survey. The existence of the survey was posted in the Bulletin of the American Meteorological Society, the Climlist server, and was sent to institutional lists in Germany Denmark and the U.K. As an effort to prevent general access to the survey, the survey was password protected. The password was contained in the informative message distributed according to the above.

However, the information about the survey was reposted (mail list membership required to read link) to the climatesceptics mail list by Timo Hämeranta on Sep 20 2003: (my emphasis)

the survey (below) is directed to those involved in the natural sciences related to climate change and not, for example, those involved in policy analysis or economic issues.

I suggest that you participate by completing the questionnaire (instructions to participate below).

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Lähettäjä: CLIMLIST Climatology Distribution List
[mailto:CLIMLIST@...] Puolesta CLIMLIST
Lähetetty: 19. syyskuuta 2003 18:12
Vastaanottaja: CLIMLIST@...
Aihe: Survey of Climate Scientists
CLIMLIST Mailing Number 03-09-24
Origin: "Dr. Dennis Bray"

Due to the nature of the distribution of electronic surveys some recipients of this message might have received the same previously. If that is the case, my apologies. Please do not submit the survey twice, although, once would be much appreciated. The survey is directed to those involved in the natural sciences related to climate change and not, for example, those involved in policy analysis or economic issues. Your discretion in choosing to participate in the survey on this basis would be greatly appreciated, as would your cooperation in making the survey a success. If you know of colleagues not contained on climlist but involved in the climate sciences, it would be appreciated if you could bring this survey to their attention with the suggestion that they too might like to participate by completing the questionnaire. Simply forwarding this message is likely the most convenient method.

... If you do choose to participate, the survey can be reached by opening your web browser and going to the following link:

http://w3g.gkss.de/G/Mitarbeiter/bray.html/

When the page opens click the link to "survey of climate scientists" Here you will be asked for a username and password.

For username enter "respondent" (without quotation marks)

For password enter "ccsurvey" (again without the quotation marks).

The survey is password protected as an effort to limit the respondents to those involved in the climate sciences. There is also the option to print the survey from a PDF file and submit though regular postal services. Electronic submissions do not transmit your email address and consequently anonymity is ensured. Thank you in advance for your cooperation.

Since the survey was anonymous, there is no way to ensure that only climate scientists participated and no way to prevent people from submitting the survey multiple times. Furthermore, the survey was distributed on the climatesceptics list which has over 200 members, almost all of them strongly skeptical about global warming. Since the total number of participants was just 557, this could serious skew the results. I don't believe that the results of this survey are representative of the views of climate scientists.

Update: Bray has some on the methodology here. The only thing Bray did to check for multiple responses by the same person was to see if there were any responses that were exact duplicates.

Update 2: Bray replies.

Categories

More like this

Remember Dennis Bray's useless survey of climate scientists? The URL and password were posted to the climatesceptics mail list, so the results were biased and included responses from people who were not climate scientists. Bray refused to concede that this meant that the survey was hopelessly…
At the grandly named Adam Smith Institute blog, Alister McFarquhar (an economist who was one of the sixty scientists denying that climate change was real) asserted: Surveys show two-thirds of scientists either don't know or don't believe man can influence climate Jim from Our Word is our Weapon…
Remember Dennis Bray's useless survey of climate scientists? The URL and password were posted to the climatesceptics mail list, so the results were biased and included responses from people who were not climate scientists. Bray refused to concede that this meant that the survey was hopelessly…
The Australian government's conclusion that the climate change debate is over has prompted a column from Andrew Bolt, who insists that there is to a big debate still going on. Bolt writes: Just look at the big Greenhouse 2005 conference [environment minister Ian Campbell] department is sponsoring…

Furthermore, this survey methdology would certainly not be up to the standards for publication in Science. Also, I'd be very suprised if a rejection letter from Science said anything remotely like "it didn't fit with what they were intending to publish."

Yet another example of how scientists can be remarkably stupid when it comes to dealing with people. They are too used to studying nature that doesn't try to cheat.

Note that Hans von Storch is also behind this survey. His name does pop up with great regularity among material used by contrarians. I wonder to what extent he likes that?

By Thomas Palm (not verified) on 01 May 2005 #permalink

I think this mostly demonstrates there is a lot of uncertainty regarding GW. If you asked them about another only indirectly testable phenomenon, for instance whether evolution is a correct theory, I'm guessing you would find 90% in the strongly agree category.

From what I've seen, there is also growing evidence for solar factors in GW. I think while there is strong evidence the Earth has warmed recently, it's too early to say for sure what amount of that warming is actually anthropogenic. I think we'll have a much clearer picture in 20 years or so.

I also like to note that while the effects of GW are still being debated, we know what the effect of global cooling would be. Could human civilization survive an Ice Age? Some models suggest only anthropogenic factors are preventing one now (although again there is a lot of conflicting data). It might actually be more irresponsible to curb emissions.

I think we should all agree more study needs to be done, and should be funded.

I'm afraid the survey doesn't tell us anything useful. Yes, scientists aren't as certain about AGW as evolution, but they are really certain about evolution.

If recent warming isn't mostly anthropogenic, there must be two unknown factors operating. One unknown factor that stops greenhouse gasses from having effect, and another one that is actually causing the warming.

There is already enough CO2 in the atmosphere to prevent any Ice Age for thousands of years.

Tim,

I do agree the methodology appears suspect, but keep in mind there are some people out there advocating GW policies as though GW was as well-proven as evolution.

The CO2 levels probably are high enough to stave off an Ice Age for a very long time, but I don't think we can say that definitively, at least not with our current understanding of GW, and obviously there are dangers in making that assumption. What would be the effect if we experienced an extended temperature reduction like the
Little Ice Age of a few hundred years ago?


Western Europe experienced a general cooling of the climate between the years 1150 and 1460 and a very cold climate between 1560 and 1850 that brought dire consequences to its peoples. The colder weather impacted agriculture, health, economics, social strife, emigration, and even art and literature. Increased glaciation and storms also had a devastating affect on those that lived near glaciers and the sea.

...mostly due to unfavorable climates with the most notable peak in the year 1816 - "the year without a summer." One of the worst famines in the seventeenth century occurred in France due to the failed harvest of 1693. Millions of people in France and surrounding countries were killed.

The effect of the LIA on Swiss farms was also severe. Due to the cooler climate, snow covered the ground deep into spring. A parasite, known as Fusarium nivale, which thrives under snow cover, devastated crops. Additionally, due to the increased number of days of snow cover, the stocks of hay for the animals ran out so livestock were fed on straw and pine branches. Many cows had to be slaughtered.

In Norway, many farms located at higher latitudes were abandoned for better land in the valleys. By 1387, production and tax yields were between 12 percent and 70 percent of what they had been around 1300. In the 1460's it was being recognized that this change was permanent. As late as the year 1665, the total Norwegian grain harvest is reported to have been only 67 - 70 percent of what it had been about the year 1300 (Lamb, 1995.)

Even sub-Ice Age cooling could be devastating.

TallDave, A fair poll would give well over 90% favoring evolution, but try circulating it on any of the creationist networks, just as this was circulate among GW contrarians, and the result would be very different. You can get any result from a poll if the sample is sufficiently biased.

Obviously we will have a clearer picture of what is going on with climate in 20 years, and we will know even more in 60, 80 100 years. So what? The question is what we know today and how we will act based on the information we have now. As we learn more we may have to adjust our plans, but that is no reason not to do anything today.

The idea that we happened to start emitting large amounts of CO2 *just* when a new ice age was about to start is sheer fantasy. Ice ages tend to start slowly and one isn't expected for some tens of thousands of years anyway.

By Thomas Palm (not verified) on 01 May 2005 #permalink

You say, "Since the survey was anonymous, there is no way to ensure that only climate scientists participated and no way to prevent people from submitting the survey multiple times." Actually, there's a number of voting schemes that provide for anonymous, one-time only voting restricted to a particular userbase... for example, look at Chaum's mix-networks.

They're more work to set up , and people unfamiliar with cryptography may not be comfortable, but they've got some nice provable security properties.

By Brian Palmer (not verified) on 01 May 2005 #permalink

Thomas,

Well, I think we can agree creationists by definition are not scientists. They are theologians. GW skeptics are not claiming divine intervention is responsible for GW.

TallDave,
Lots of creationists and "intelligent design" people would disagree with you. They do everything they can to at least appear to be scientists. Anyway it doesn't matter, this isn't a question of whether people are scientists or not but whether they may have answered that survey.

By Thomas Palm (not verified) on 02 May 2005 #permalink

It is worth noting as well that the survey question presented above is not framed so as to be explicitly refering to climate change today, rather it is very generally applicable to climate change at any time.

I also note as evidence of this problematic ambiguity the fact that some questions use the phrase "global warming" which more strongly suggests current climate change whereas this question says "Climate change is mostly the result of anthropogenic causes"

By Coby Beck (not verified) on 02 May 2005 #permalink

Thomas Palm,

Well, I'd say any "science" that relies on divine intervention is better labeled theology (at least, outside of Charles Stross stories). But I can see your point.

OK, how about another only indirectly testable phenomenon, say the Big Bang/expanding universe cosmology model? This is something relatively recently proven, and again I think you would be hard-pressed to find a survey sample of scientists where you would get only 9% strongly agreeing it is basically correct.

The question is what we know today and how we will act based on the information we have now.

I would argue the question is how reasonable is it to act on a theory that is not that well understood or accepted, especially when the prescribed actions have significant negative economic consequences and could even turn out to be counterproductive.

This is all of a piece, isn't it? Anyone who would think that this survey, given the history described here, is of any value at all, would demonstrate an off-kilterness that makes it no surprise that they would also feel that anthropogenic global warming is an unsupported theory, only believed by a vocal minority, etc.

By gerald zuckier (not verified) on 02 May 2005 #permalink

gerald,

Do you think any poll of scientists would find support for anthropogenic GW as universal and strong as for evolution or the expanding universe? There is still a lot of uncertainty in the AGW argument. I wouldn't say it's totally unsupported, just that it's not totally supported, and that its predictive value is low (at least for now).

But again, I think more funding and more research is called for.

I participated in this survey. I also discussed the methodology with Bray, specifically the problem that there appeared to be no real control for multiple responses and no way to check for ballot-stuffing by the septics (and then I didn't even know about the message on the septics mailing list; that makes it far worse).

Note, BTW, that this survey is a repeat/follow-up to a 1996 survey.

Well, I think we can agree creationists by definition are not scientists

No. Whose definition are you using to say that TallDave? If you're going for Karl Popper's falsificationism, you'd be wrong (but then again, loch ness monster believers, yeti sighters and flat earthers are all scientists according to his criteria).

Creationists do not apply scientific criteria. They reject many physical laws.

T. Palm, you are right.. this is not the method for a reliable survey. " Scientists.. They are too used to studying nature that doesn't try to cheat".. absolutely.

BTW, Eli Rabbet's laconic post is excellent.

As far as I can tell, Peiser changed the categories that Oreskes used. She said that there were no papers that explicitly disagreed with the consensus view. Peiser responds with a claim that "there were 34 abstracts reject or question the view that human activities are the main driving force of 'the observed warming over the last 50 years'".
It's perfectly possible for both statements to be true, with none of the 34 rejecting the consensus but just raising questions.

I'll see if I can get the list of 34 papers from Peiser so that everyone can judge for themselves.

The percentage of papers that Benny found still does not overturn the consensus, nor does it completely falsify Oreskes' work. Oreskes not stating a null hypothesis makes my last phrase somewhat problematic, however.D

Peiser's letter (including the editors' letters) is here:

http://www.staff.livjm.ac.uk/spsbpeis/Scienceletter.htm

I checked the abstracts of the two papers mentioned in Ref. 5- the C. M. Ammann paper (that's the same Caspar Ammann that is a contributor on Realclimate BTW) just says that there is a relationship between temperature and solar output over the last four centuries- which is hardly news and is consistent with the consensus. The Reid paper says that "solar forcing and anthropogenic greenhouse-gas forcing made roughly equal contributions to the rise in global temperature that took place between 1900 and 1955. The importance of solar variability as a factor in climate change over the last few decades may have been underestimated in recent studies." So nothing there either that obviously disagrees with the consensus that most of the warming over the past 50 years is due to human activities.

Peiser's piece is another example of contrarian cherry picking. Take for example his claim that, "Quite a number of papers emphasise that natural factors play a major if not the key role in recent climate change (4)." The footnote that supposedly supports this claim is a clear misrepresentation. Peiser claims, "C. M. Ammann et al., for instance, claim to have detected evidence for "close ties between solar variations and surface climate", Journal of Atmospheric and Solar-Terrestrial Physics 65:2 (2003): 191-201." First, the citation is incorrect. The article is properly attributed to Hee-Seok Oh, Caspar M. Ammann, Philippe Naveau, Doug Nychka and Bette L. Otto-Bliesner (Oh et. al.), not Ammann et. al. Second, the sentence from the abstract from which Peiser extracts the phrase, "close ties between solar variations and surface climate," mentions NOTHING about "recent climate change" or any other period for that matter. The full sentence reads, "While the low-frequency component shows close ties between solar variations and surface climate, a relationship between the 11-year sunspot cycle and temperature reconstructions is more difficult to attribute." Looking into the article proper, it is quickly discovered that the "low frequency component" refers to the 85 year "Gleissberg" cycle and the significant correlation occurred during the time period 1610-1960 - a period clearly not relevant to "recent climate change."

Interesting, Brian.Your link is different than the one I read that went to Science - surely he wouldn't use that inflammatory rhetoric in a journal submittal.inspired by Brian's example, and quickly checking Benny's work, surely coincidentally Benny's footnote 5 by Linden is not an empirical work, and indeed states:

Today, proponents of catastrophic anthropogenic climate change, again claiming scientific consensus, threaten to create even greater energy market distortions at large social and economic costs. The author traces his conversion to energy contrarian to the general failure of consensus and to his own misjudgments in these critical policy areas.

so it's problematic, as he is an engineer and from what you can see of his CV, doesn't do empirical work on the environment (where does this slippery slope begin).Benny's footnote 6 ref is the same as 5. That surely falsifies Benny's work. Oh, wait: let's not perpetuate the rhetoric. The only paper by those 2 authors in 1996 has an abstract that states:

An overview of the ozone issues is given including the following aspects: 1. The impact of tropospheric ozone on climate as a greenhouse gas (GHG), 2. Solar activity effects on TO and ozone concentration vertical profiles in both the troposphere and stratosphere (in cases of solar radiation absorption by the stratosphere, an unexpected problem arises via a coupling between processes of increased absorption due to ''bursts'' of solar activity and an enhanced destruction of ozone molecules due to the same increase resulting in weakening UV radiation absorption) and 3. Surface ozone concentration variations under conditions of polluted urban atmospheres which lead to episodes of photochemical smog formation (dangerous for human health).

which sounds nothing like lil' Benny's characterization of it. Footnote 7 is actually as Benny says and reflects the knowledge of the time (1993):

This article examines the status of the scientific uncertainties in predicting and verifying global climate change that hinder aggressive policy making. More and better measurements and statistical techniques are needed to detect and confirm the existence of greenhouse-gas-induced climate change, which currently cannot be distinguished from natural climate variability in the historical record. Uncertainties about the amount and rate of change of greenhouse gas emissions also make prediction of the magnitude and timing of climate change difficult. Because of inadequacies in the knowledge and depiction of physical processes and limited computer technology, predictions from existing computer models vary widely, particularly on a regional basis, and are not accurate enough yet for use in policy decisions. The extent of all these uncertainties is such that moving beyond no-regrets measures such as conservation will take political courage and may be delayed until scientific uncertainties are reduced.

I hope Benny doesn't mean to imply that footnote 8 appears in ISI - it does not. It is an opinion piece.So Benny's work is sloppy, thus rejectable. You can say there's discrimiation here, but only because the work is subpar. Those biased journals - never allowing a dissenting view. Tsk, tsk. Best,D

Even if the survey had an effective barrier to multiple responses (which it apparently didn't) and even if it could be ensured that only actual climate scientists responsed (which also appears not to have been the case), the survey would still have the problem of being a self-selected sample, which would make the results useless for drawing any conclusions about the existence or non-existence of a scientific consensus on AGW.

I've emailed Peiser, asking for the list of 34 abstracts that "reject or doubt the view that human activities are the main drivers of the 'the observed warming over the last 50 years'."

A commenter over at this http://chriscmooney.com/blog.asp took a close look at one of Peiser's references. I'm reproducing the relevant parts below:


Peiser claims, "C. M. Ammann et al., for instance, claim to have detected evidence for "close ties between solar variations and surface climate", Journal of Atmospheric and Solar-Terrestrial Physics 65:2 (2003): 191-201."

.............

Second, the sentence from the abstract from which Peiser extracts the phrase, "close ties between solar variations and surface climate," mentions NOTHING about "recent climate change" or any other period for that matter. The full sentence reads, "While the low-frequency component shows close ties between solar variations and surface climate, a relationship between the 11-year sunspot cycle and temperature reconstructions is more difficult to attribute." Looking into the article proper, it is quickly discovered that the "low frequency component" refers to the 85 year "Gleissberg" cycle and the significant correlation occurred during the time period 1610-1960 - a period clearly not relevant to "recent climate change."

By caerbannog (not verified) on 03 May 2005 #permalink

I trust you'll share, Tim, and I'll happily look in ISI if Benny's letter linked above is any indication of the quality of his work...D

So, after spending weeks throwing mud at the statistical methodology of the Lancet article, the most TallDave can muster in critique of this online survey is, "the methodology appears suspect" (emphasis mine) while then going on to support the conclusions of the survey anyway.

I can't remember... does this make TallDave a crank or a hack?

Hi Disputo,

You have set up a false dichotomy here about TallDave. Why can't he be both a crank and a hack at the same time? He seems to aspire to both at once.

Disputo, it's a fair point, but does not justify calling him a crank or a hack.

Everyone, no personal attacks on other commenters please. They do not help the discussion.

Tim,

Benny Peiser's paper rejected by Science contradicts your allegation here.

Oh, thanks for the Gong - I must ask Ray Evans to arrange another meeting of the Lavoisier Group in the nearest phone box for a meeting to discuss where we should display all our Lambert Gongs.

By Louis Hissink (not verified) on 05 May 2005 #permalink

Benny's paper appears to be unable to stand scrutiny, as I show above Louis. If you mean to say Tim's allegation is that Benny was rejected due to bias, you are correct in your 'contradiction' statment. If Tim's allegation is that Benny's paper was rejected because it was weak, your statement is incorrect. At any rate, Tim is doing an audit on Benny's work and that should clear things up. Or start a flame war on Usenet. Or both.Best,D

just a couple of additional points. I agree that this is, at best, a rather irresponsibly administered piece of social science. I'd expect better of Nico Stehr & co. He should stick to hifalutin sociologising.
1. For an international study, to assume equal distance between the response variables is not necessarily a valid assumption
2. Question 40 is hopelessly ambiguous, and doesn't really tell us anything about the scientists' real opinions - you can not agree and still think that there are anthropogenic factors, or even that anthropogenic factors count overwhelmingly for changes.
3. I fear the damage has already been done with the exploitation of these results in the popular press in Germany (and by Tony Blair) muddying the waters.