Deltoid

Doyou want totry to writethat again without the oddpauses and fracturedenglish, Tim?

Posted by: Wow | August 29, 2010 3:07 PM

The survey title just says 'science'!

Regulated capitalism is OK. And free market socialism.

The point being that any existing political ideology isn't going to survive any future climate change scenario, whether we mitigate or not.

Posted by: Paul UK | August 29, 2010 5:21 PM

I needed a "don't know" choice rather than being forced to say probably one way or the other who was involved in assassinating who.

Kenendy's head as pushed the wrong way for the Book Deposit.

Posted by: jakerman | August 29, 2010 7:19 PM

I would have thought that inviting Deltoid readers to participate in a questionnaire of this sort is likely to produce statistically skewed results since it likely to encounter a preponderance of respondents who are "pro-science" rather than "skeptical of science".

Posted by: Anonymous | August 29, 2010 7:59 PM

Not very well-written. What on earth is that question about Coca-Cola all about? I don't even care enough to Google it.

As for "Does burning fossil fuels warm the atmosphere measurably", the answer is surely "no": burning fossil fuels releases CO2, which is responsible for causing a measurable rise in atmospheric temperature. The heat from the burning of fossil fuels itself is not particularly consequential.

Posted by: Vince Whirlwind | August 29, 2010 8:30 PM

I didn't like the lack of a don't know option, either. I knew what the Coke question was about, but Timothy McVeigh? Remembering that he was involved with various right wing crazies doesn't help me with remembering whether or not they were involved in the bombing itself.

I s'pose the questions were "designed" to check on what people "believe" regardless of what they know, but some of it made me uncomfortable.

Posted by: adelady | August 29, 2010 9:26 PM

I didn't mind the economic standing questions. As a pair they work quite well. Many people have very little understanding of just how poor poor people are and how astoundingly rich others are.

Posted by: adelady | August 29, 2010 10:27 PM

This seemed to be a pretty ordinary survey. Difficult to criticise the infamous Oregon petition without putting this survey into the same bucket.

Posted by: Adam | August 30, 2010 4:13 AM

It is a seriously weird questionnaire. I stopped doing it when I realized that the UWA logo directed one to the kwiksurveys site rather than UWA. Is it some sort of attempt to spish? Quite apart from the inept conspiracy questions that don’t allow “no idea” as an answer, and the two-part questions where you might have different response to the two parts. BTW - they say the experimenter is Charles Hanich, not Stephan Lewandowsky as you put Tim....

Posted by: Ian | August 30, 2010 5:17 AM

the questions are fine.

lack of a neutral reply is a good idea. people tend to choose such an option too often, to avoid making decisions.

a "don t know" button might have been a good idea, but isn t really necessary.

the good thing about conspiracy theories is, that you will recognise it, even without ever heard about it!

i am looking forward to the results of this one. but i fear very few denialists will take it.

Posted by: sod | August 30, 2010 7:07 AM

I find it amusing that I think it is more likely that there are aliens in US military bases, and that the CIA conspired to kill Kennedy, than climate science being wrong.

Posted by: Mattb | August 30, 2010 10:39 AM

Mike, although I didn't go into details, it would be correct to assume that as a highly experienced programmer I'm aware of how online surveys protect against multiple submissions, and my very simple script would circumvent them. It would be illegal and unethical to use such a script and I'd never do so, but the skills required are not difficult to acquire.

Posted by: Tony Sidaway | August 30, 2010 11:02 AM

"spish" was the noise people made on tasting the New Coke. Probably this whole thing is a cola marketing survey. I dunno. I got through it, but it hung when I tried sending it, so likely didn't get counted.

Posted by: Hank Roberts | August 30, 2010 11:28 AM

Tony @ 25. The hole is getting deeper. Best advice is stop digging.

Posted by: Mike | August 30, 2010 7:09 PM

Anonymous:

it likely to encounter a preponderance of respondents who are "pro-science" rather than "skeptical of science".

What sense does "skeptical of science" make, considering that science itself is skeptical. The same as saying skeptical of skepticism.

Posted by: Chris O'Neill | August 30, 2010 10:53 PM

Hank Roberts, with his RAS Syndrome whatever that is, is probably one of those people from a TLA acronym agency who use jargon to convince people that things like the HAARP Project aren't endangering us all.

Posted by: Marion Delgado | August 31, 2010 7:08 AM

That's supposed to be solved by asking LOTS of questions. That artifact will exist in all questions and the answer to one question that is shaded off-colour will shade in an earlier question with what you REALLY mean.

It's why a good survey is DAMN hard and really long.

Most surveys are not worth shift except on simple questions like "Is Barak Obama a Nazi Communist Muslim Extremist? Yes or Hell yes!"

:-)

Posted by: Wow | August 31, 2010 3:09 PM

I have to admit that I've seen worse surveys, though I don't think it's the best example I've seen. The question of whether or not to include a neutral response option is a bit of a vexed issue, and isn't as simple as it seems. Like sod says, when you include neutral or don't know options, people are disproportionately more likely to use them, especially if there's some social stigma associated with the answer that they would have given had the neutral option been absent. So there's a sense in which including neutral options acts as an excuse for people not to think too hard about the question. Worse, it can bias people against giving certain types of answers. On the other hand, there's an equally compelling argument that it really is the case that people just don't have any opinions, knowledge or beliefs relevant to some questions, and no matter how hard they think about those questions, they can't give a sensible answer. When that happens, people get annoyed, which is never desirable; and from a methodological standpoint, it over-inflates your estimates of the number of people who hold weak beliefs. Given that there are real arguments each way, there's some disagreement on this topic: my personal view is that if you are going to remove the neutral response options, you should use more than 4 points in the response scale: if you use 6 or 8 options, it makes it easier to use the shape of the response distribution as a (weak) cue that a particular question actually had a lot of people making "near-neutral" responses because they don't have any real knowledge.

As for the question choice, I thought the "burning fossil fuels causes climate change" phrasing was probably the right choice - it's scientifically clunky, since the mechanisms involved are much more complex (as everyone on this site knows better than I do), but if the survey is targeted at a general audience it's very rarely a good idea to be technical. You'll just confuse and annoy people rather than elicit actual opinions. And if you do know the technically correct version, then most (but not all) respondents will endorse the clunky version, though often with a certain amount of irritation. So the trade-off becomes "irritate 1% of the sample because you're scientifically imprecise" versus "irritate 40% of the sample because you're being too sciency and talking over them". If your goal is to elicit sensible answers from as many people as possible, the answer is pretty obvious - irritate the small proportion of people who happen to have technical knowledge. Oddly, technical precision is actually methodologically irresponsible in this context.

On the other hand, I think the "conspiracy theory" section is too heavy-handed to be useful. There's no chance that people won't figure out what the survey is looking for here, and everyone knows that "conspiracy theory!!" is pejorative. So I suspect that a lot of people who actually do think that climate scientists are rigging the data will hide their beliefs, even in an anonymous survey, because they're worried that they're going to be painted as conspiracy theorists. Maybe with good reason, maybe not. Curiously, there's actually ways of getting around this issue: the classic solution is to ask people to privately flip a coin, and then endorse the conspiracy theories if it's heads, answer honestly if it's tails; that way the researcher can't know which specific people actually believe the conspiracy theories, and the respondent knows this. So those people who actually do believe them feel safer (quite correctly) and are more likely to respond honestly. It means that you need a bigger sample size for your survey, since it adds a lot of random noise to your data, but it's often a good trade if it reduces the rate of people hiding socially-undesirable beliefs. On the other hand, there are issues with this method too. Probably it's the sort of thing that they'd have looked at while piloting the study. Without knowing all the details about what other risks they were worried about when designing it, it's hard to have strong opinions.

Anyway, I really must stop procrastinating. I'm running late for work.

Posted by: Dan | August 31, 2010 6:52 PM

What attitude could there be except its all just a money making scam. Have a look at how the UNIPCC fabricated the data and is now saying it is not a science organisation ROFL

Posted by: twawki | September 4, 2010 10:50 AM