If you’re one of the probably four people who haven’t heard about Nate Silver, you’ve missed out. He’s a statistics guy who runs the always interesting 538 Blog at the New York Times. He made his name in baseball statistics, Moneyball style, and moved into politics where he made state-by-state statistical forecasts of the last couple national elections. His track record is pretty good. The final prediction for this election was an Obama win with 90.9% probability, and it duly came to pass. On a state-by-state basis, he seems to have been correct in every case. (Much of this was blind luck, as he’d be the first to tell you. His model yielded coin flips in a few cases like Florida, and the fact that his 50.3% chance there happened to come out right was a matter of good fortune.)
So I like the guy a lot, and his Monte Carlo methods are near and dear to me and pretty much any physicist who’s ever done a computationally-intensive calculation. But I do want to be the guy who sounds the sad trombone and pours just a little cold water on his well-deserved celebration, for two reasons.
1. It’s not science.
Well, it might be. We can’t tell. His model is closed. He hits the go button on his computer, it spits out numbers according to some algorithm that isn’t public, and they tend to be right most of the time. That suggests that he’s doing something right. But science has to be testable, and while and accurate prediction is a test, it doesn’t advance our understanding of the world unless we know just what it is we’re testing.
Which isn’t to say he’s under any obligation whatsoever to make his algorithm public. If I ever discover an algorithm that can predict the stock market with uncanny accuracy, I’m certainly not going to tell any of you how it works. It won’t be science, and because I love science I’ll just have to console myself with a fruity drink beside my gigantic pool. While I doubt a presidential prediction model is so lucrative, he’s perfectly justified in keeping it to himself if he wants.
2. It’s not necessarily an improvement on simpler methods.
If you had just made a prediction based on the average of each swing state poll over the last week or so, you’d have probably made the same call in every case except Florida. Are further refinements to that simple “take the polls at face value” model actually improvements, or just epicycles? Beats me, but the answer is not obvious.
These two objections are pretty minor in the grand scheme of things, and I reiterate that I’m a big fan of statistical methods in politics, and with those caveats also a fan of Nate’s work with them. It’s nice to see computational statistics get its day in the sun, even if just once in four years.