Last week we asked our readers to predict the result of the election. How did they do?
Out of the 474 people who guessed the results of this year’s presidential election, only six got the electoral vote right – 365 votes for Obama (assuming Missouri goes for McCain and Omaha goes for Obama). None of these respondents was accurate on the popular vote, but one anonymous respondent got close, guessing that McCain would get 47 percent (the actual figure was 46.3 percent). Only one person who guessed 365 left his name, so let’s give Wayland credit as the unofficial “winner” of our prediction contest.
What we were really interested in is how information sources relate to predictions. Do people rely on these sources, or do they just give it a guess? Several websites attempted to analyze poll results this election season, and some of them became extremely popular.
First off, let’s consider how accurate the polling sites turned out to be. The site that came closest to predicting the electoral vote was electoral-vote.com, which predicted 365.5 votes for Obama — just 1/2 vote off the actual total. The worst of the major sites was the New York Times, which predicted 330.5 votes – 34.5 votes off the actual total. FiveThirtyEight.com, favored by more readers than any other site, did the worst of all the dedicated poll aggregation sites, predicting 346.5 votes for Obama. But all the sites did better than our readers, whose average prediction was 318.3 electoral votes.
But did readers’ preferred polling site have an impact on their predictions? This graph shows the results:
Readers of FiveThirtyEight.com did significantly better than readers of other sites. They also made guesses closer to what their preferred website predicted. But nearly all readers would have been better-served by guessing closer to the polling sites’ predictions. Another way of looking at the results is to consider the average absolute error made by readers and their favorite websites:
Even though FiveThirtyEight.com had the worst prediction of the dedicated polling sites, its readers were significantly more accurate than readers of any other site.
Does reading more polling sites help you predict the results? Take a look at this graph:
Reading more different aggregators was associated with better prediction accuracy. But even reading more than four different poll aggregators was no better than just reading FiveThirtyEight.com.
Did political partisanship affect prediction accuracy? Check this out:
McCain supporters were significantly less accurate than readers who supported other candidates. This is probably to be expected — people are more likely to predict their own candidate will win. Yet Obama supporters were still significantly less accurate than readers of FiveThirtyEight.com.
Why were FiveThirtyEight.com readers so much more accurate than everyone else despite the fact that FiveThirtyEight was the worst of the poll aggregation sites in terms of predicting the electoral vote? I suspect the reason is that FiveThirtyEight is the only aggregator that offers analysis of the polling on the front page of its site. The other aggregators only provide links to other sites or web pages where analysis is provided. So readers of FiveThirtyEight might have been more interested in the analysis that leads to predicting the vote.
Some other interesting tidbits from the data:
Whether you donated, volunteered, or commented on political blogs or forums didn’t have a significant correlation to predication accuracy. However, there was a small significant positive correlation between watching the Daily Show and accuracy. Watching the Colbert Report was not associated with accurate predictions. But watching both programs was the an even better predictor of accurate predictions.