Political Experts

Since the 2008 election appears to be in full swing, and the political prognosticators have started peddling their predictions, I thought it was worthwhile to remind everybody that political experts are not to be trusted. The psychologist Philip Tetlock has spent decades following the predictions of these so-called "experts," and seeing if their predictions are prophetic. The results are pretty dismal:

People who make prediction their business -- people who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtables -- are no better than the rest of us....Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an expert's predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote.

Of course, not all political experts are this bad. Some actually perform (slightly) better than random chance:

Low scorers look like hedgehogs: thinkers who "know one big thing," aggressively extend the explanatory reach of that one big thing into new domains, display bristly impatience with those who "do not get it," and express considerable confidence that they are already pretty proficient forecasters, at least in the long term. High scorers look like foxes: thinkers who know many small things (tricks of their trade), are skeptical of grand schemes, see explanation and prediction not as deductive exercises but rather as exercises in flexible "ad hocery" that require stitching together diverse sources of information, and are rather diffident about their own forecasting prowess.

For those interested in more reasons why they should not listen to anybody who offers political predictions (especially if they seem particularly confident), I highly recommend Tetlock's recent book.