One of the lingering questions in decision science is the extent to which game theory – an abstract theory about how people can maximize their outcomes in simple interactions – is actually valid. It’s a lovely idea, but does it actually describe human nature?
As usual, the answer depends. With few exceptions, lab tests of game theory find that real human beings sharply deviate from the predictions of game theory. We don’t maximize gain, often because of misperceptions about risk. But people have actually performed better (i.e., we act like Von Neumann tells us to act) in the field. For instance, Mark Walker and John Wooders analyzed several thousand serves at Wimbledon:
We have constructed a data set that contains detailed information about every point played in ten professional tennis matches. Each match provides us with four 2 2 “point games” with which to test the minimax hypoth- esis, giving us a total of 40 point games. In each of the 40 point games we use the server’s “win rates”–the observed relative frequencies with which he won points when serving to the receiver’s left or to his right–to test whether his winning probabilities are indeed the same for both serving directions, as the theory says they should be. In only one of the 40 point games is minimax play rejected at the 5-percent level. This rejection rate is actually slightly below the rate predicted by the random character of equilibrium play.
In other words, tennis players (like soccer goalies) are rational agents, at least when choosing where to serve. (The big exception is that even the best tennis players tend to switch the location of their serves a little too often.) This has led some to argue that the irrationality observed in the lab is a side-effect of lab artifice. Perhaps the incentives are too insignificant? Perhaps the subjects don’t understand the rules of the game? Or maybe they’re intimidated by the presence of the scientist, who is measuring their behavior?
The latest attempt to rectify this schism between lab data and real life observations comes from Kenneth Kovash and Steven Levitt. They began by seeking out vast data sets, since one of the central problems with the studies of soccer players and tennis stars was the smallness of their n. (For instance, the game theory goalie study looked at fewer than 500 penalty kicks.) And so Kovash and Levitt moved on to other sports:
In the case of baseball, we observe every pitch thrown in the major leagues over the period 2002-2006 – a total of more than 3 million pitches. For football, we observe every play in the National Football League for the years 2001-2005 – over 125,000 plays. In both settings, the choices being made have very high stakes associated with them.
The results obtained from analyzing the football and baseball data are quite similar. In both cases, we find clear deviations from minimax play, as evidenced by a failure to equalize expected payoffs across different actions played as part of mixed strategies, and with respect to negative serial correlation in actions. In the NFL, we find that offenses on average do systematically better by passing the ball rather than running. In baseball, pitchers appear to throw too many fastballs, i.e., batters systematically have better outcomes when thrown fastballs versus any other type of pitch.
In football, teams are more likely to run if the previous play was a pass, and vice versa. This pattern is especially pronounced when the previous play was unsuccessful. Negative serial correlation in actions is consistent with a large body of prior laboratory evidence. Pitchers also exhibit some negative serial correlation, particularly with fastballs, i.e., they are more likely to throw a non-fastball if the previous pitch was a fastball, and vice versa.
The magnitude of these deviations is not trivial. Back-of-the-envelope calculations suggest that the average NFL team sacrifices one point a game on offense (4.5 percent of current scoring) as a consequence of these mistakes. In baseball, we estimate that the average team gives up an extra 20 runs a season (about a 1.3 percent increase). If these estimates are correct, then the value to improving these decisions is on the order of $4 million a year for the typical baseball team and $5 million a year for an NFL franchise.
This suggests that the lab data is actually correct, and that people (or at least MLB pitchers and NFL coaches) aren’t particularly good at obeying game theory. It’s a rational model we don’t naturally emulate. But why not? Wouldn’t natural selection want to endow us with a mind capable of maximizing competitive outcomes?
One possibility is that failing at game theory is the price we pay for having emotions. Just look at NFL coaches, who according to Kovash and Levitt don’t call pass plays often enough. Since 1960, quarterbacks have managed to increase their average gain per pass attempt by nearly 30 percent, from 4.6 yards to 6.5 yards. (Running backs only get about 4 yards per attempt, a number that hasn’t changed in thirty years.) Furthermore, even as quarterbacks have gotten more yards per pass, they have managed to throw fewer interceptions. In 1980, passes were picked off more than 6 percent of the time. By 1995, the rate of interceptions had been halved, which meant that passing the ball wasn’t any statistically riskier than rushing.
Given these statistics, NFL teams should pass the ball the vast majority of the time. That, at least, is the conclusion of Ben Alamar, a professor of sports management at Menlo College. He argues that the NFL exhibits a “Passing Premium Puzzle”. (This is the sports version of the Equity Premium Puzzle, which is the mystery of why investors hold so many low yield bonds when stocks have performed so much better over the long-term.) Alamar notes that, despite the significant increase in the expected utility of the passing game, coaches still run the ball about 46 percent of the time. While this represents a decrease from the 1960′s – the average NFL team use to run the ball more than 58 percent of the time – a perfectly rational coach would pass even more frequently, since passing represents a higher rate of return. So why do coaches still run the ball? Alamar admits that a successful running game contributes to the success of the passing game, since you want the defensive backs to have to worry about the possibility of a run. But he isn’t convinced that coaches need to run the ball quite so often. “For all of their planning and late nights,” Alamar writes, “NFL coaches do not act in a fully rational manner.” Just like investors choosing bonds over stocks – stocks have a much higher rate of return over the long run – coaches are swayed by the illusory perception of risk, or what’s often known as risk aversion. Although passing the ball isn’t statistically riskier than running the ball, it feels riskier, as the ball is lofted into the air and is up for grabs. (I’d argue that a similar calculation applies to breaking balls, which might also feel riskier since we have less control over where the ball ends up.) The end result is that teams gain fewer yards than they might otherwise.
The point is that an inexplicable feeling – a judgment of risk we can’t justify – interferes with the rational calculation. The play might not be optimal, but it feels less likely to trigger a worst case scenario (interception, home run, etc.), and those scenarios carry a disproportionate weight. While our emotions might lead us to do many silly and stupid things – not obeying John Nash is the least of these quirks – they’re also amazingly effective tools when it comes to quickly evaluating the world, especially when we’ve got experience in that domain. And so we call a run play and opt for the fastball. We’re not rational, but we’re rational enough.
Thanks to Eric Barker for the tip.