In recent days, there has been a lot of discussion about Sarah Palin's lack of experience in foreign policy. These criticisms all depend on the same assumption: that knowing more about foreign policy is always better. (Experience is typically used as a stand-in for knowledge, so when people say that you're inexperienced what they're really saying is you're ignorant.) But is that true? What is the payoff of expertise when it comes to political judgment?
Philip Tetlock has conducted the gold-standard study of political expertise. In the early 1980's, he picked two hundred and eighty-four people who made their living "commenting or offering advice on political and economic trends" and began asking them to make predictions about future events. He had a long list of pertinent questions. Would George Bush be re-elected? Would there be a peaceful end to apartheid in South Africa? Would Quebec secede from Canada? Would the dot-com bubble burst? In each case, the experts were asked to rate the probability of several possible outcomes. Tetlock then interrogated the pundits about their thought process, so that he could better understand how they made up their minds. By the end of the study, Tetlock had quantified 82,361 different predictions.
After Tetlock tallied up the data, the predictive failures of most experts became obvious. Although they were paid for their keen insights into world affairs, they tended to perform worse than random chance. Most of Tetlock's questions had three possible answers; the pundits, on average, selected the right answer less than 33 percent of the time. In other words, a dart-throwing chimp would have beaten the vast majority of professionals. Tetlock also found that the most famous pundits tended to be the least accurate, consistently churning out overblown and overconfident forecasts. Eminence was a handicap.
Furthermore, Tetlock found that knowing a lot about a place - being, say, an expert on Iraq - can actually interfere with one's judgment on questions involving Iraq. "We reach the point of diminishing marginal predictive returns for knowledge disconcertingly quickly," Tetlock writes. "In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals--distinguished political scientists, area study specialists, economists, and so on--are any better than journalists or attentive readers of the New York Times in 'reading' emerging situations."
To summarize: I'm not sure there's good evidence that Sarah Palin's lack of experience or knowledge when it comes to foreign affairs will actually be a handicap. (Many experts, after all, seriously misjudged the cost and duration of the Iraq war.) Let's just hope she attentively reads the New York Times.
That said, Tetlock's study should make us worry about some aspects of Palin's decision-making process, at least as revealed in her recent ABC News interview. Although his study is best known for its demonstration of expert failure, Tetlock also found that a few experts performed above average. Tetlock explained the difference between successful and unsuccessful pundits with an allusion to an ancient metaphor, made famous by Isaiah Berlin, who distinguished between two types of thinkers: hedgehogs and foxes. (Tetlock did not find any significant correlation between political ideology and thinking style.) A hedgehog is a small mammal covered with spines that, when attacked, rolls itself into a ball, so that its spines point outwards. This is the hedgehog's only defense. A fox, on the other hand, doesn't rely on a single strategy when threatened. Instead, it adjusts its strategy to fit the particulars of the situation. Foxes are also cunning hunters. In fact, they are one of the hedgehog's few predators. According to Tetlock, the problem with people who think like hedgehogs is that they are prone to bouts of certainty - their big idea is irrefutable - and this certainty causes them to misinterpret the evidence.
So when Palin gave this answer in response to a question:
...you have to be wired in a way of being so committed to the mission, the mission that we're on, reform of this country and victory in the war, you can't blink.
Is it bad to blink? When did second-guessing yourself become a sin? As Tetlock writes, "The dominant danger [for experts] remains hubris, the vice of closed-mindedness, of dismissing dissonant possibilities too quickly." Palin may not be a foreign-policy expert, and that may not be a problem, but if the last eight years have taught us anything it's that the world is a little too complicated for a hedgehog in the White House.
Brain & Behavior
Comments (19)
Why doesn't somebody just ask the big "STEM CELL" question, so they can inadvertently in their comments prove the necessity of separating political control over science on this issue. Are you aware of the possibilities of the latest research in Down Syndrome? As a mother, were I in such a situation, I would move the earth to uncover any scientific advances for my child. I wouldn't find a serenity prayer enough comfort in any situation where I might need to become involved and make changes for the future of many physically and mentally ill people.
Posted by: Lee Pirozzi | September 12, 2008 9:51 AM