1. tfk
  2. The survey is over

The survey is over

By tfk on July 6, 2011.

Thanks to everyone for their help with the Miss USA survey. There were 713 responses, which should give me what I need. I'll post the results shortly. I closed the survey and have started crunching numbers.

There was a question raised about the question wording, so I changed how the questions were introduced about halfway through. It was a modest change in the wording, clarifying that I wanted respondents to assess the contestants' answers in terms of their objective merits, not just whether they answered the question asked. I didn't think it was a big impact, but I was curious whether that caused a significant change in how people rated the Miss USA responses. To check, I graphed the average scores given for each contestant after the change against the scores given before the change. As you can see, the impact is small, and well within the error bars, which represent the median difference between ratings among all 713 answers (there were 368 responses before the wording change, and 345 after).
i-b9a36625c4d8944d65fbd6d351d9a59f-201107061901.png
The faint diagonal line is the line with slope of 1, which is where the data should cluster if the wording change had no effect. As you can see, the data do cluster there, and do so fairly consistently and within the error bounds. It looks a little bit like the answers after the change are higher for higher values and lower for low values, but the slope is not statistically different from 1, the correlation between the results before and after is 0.99, and I'm confident that it's safe to pool all 713 results.

Thank you all for your help.

Tags
creationism
Culture Wars
Policy and Politics

More like this

How Well Do Miss USA Contestants Represent Their States?
Remember when I invited readers to take a survey on the Miss USA evolution answers? And I was kinda vague about why I was doing it? At last it can be told, I was working on a guest blog post at Scientific American. You should read the whole thing, but here's the bit about how I used the survey…
Casual Fridays: Why those annoying menus are here to stay
Last week's Casual Fridays study was inspired by my annoyance at a website form which required me to constantly switch between typing in information and selecting it from a menu. I wondered if there was really any significant benefit to requiring the use of menus, when typing (for me, anyways)…
Casual Fridays: You think THAT is romantic?
Last week's Casual Fridays study started out innocently enough: A contact on Twitter asked if most women would prefer fresh fruit or chocolates as a romantic gift. That's a fascinating question, I said -- maybe even interesting enough to merit study on Casual Friday. With Valentine's Day coming up…
Kids are influenced by their peers in eyewitness testimony
In every courtroom drama, the most dramatic scene is always when the star witness points her finger at the villain and proclaims that "he did it!" The confidence with which an eyewitness describes the perpetrator of a crime is often the most convincing evidence in a court battle. But how accurate…