Last night, several folks on twitter expressed similar concerns about the wording of the survey I created and blogged about a couple days ago. Some folks taking the survey were confused about whether I wanted respondents to rate the answers by whether or not they answered the question, or by how well the ideas expressed by the contestants matched the raters’ own beliefs (even if they didn’t quite answer the question).
I wanted the latter, and have updated the instructions to clarify. If you took the survey before, and if this clarification changes how you would have answered, please consider taking the survey again.
As I said before, this survey will help me come up with a less subjective measure of the quality of the different replies, a measure that will be useful for a project I’m working on and hope to reveal as soon as next week.