The Frontal Cortex

Too Many Fastballs

Posted by Jonah Lehrer on February 11, 2010
(30)
More »

One of the lingering questions in decision science is the extent to which game theory – an abstract theory about how people can maximize their outcomes in simple interactions – is actually valid. It’s a lovely idea, but does it actually describe human nature?

As usual, the answer depends. With few exceptions, lab tests of game theory find that real human beings sharply deviate from the predictions of game theory. We don’t maximize gain, often because of misperceptions about risk. But people have actually performed better (i.e., we act like Von Neumann tells us to act) in the field. For instance, Mark Walker and John Wooders analyzed several thousand serves at Wimbledon:

We have constructed a data set that contains detailed information about every point played in ten professional tennis matches. Each match provides us with four 2 􏰄 2 “point games” with which to test the minimax hypoth- esis, giving us a total of 40 point games. In each of the 40 point games we use the server’s “win rates”–the observed relative frequencies with which he won points when serving to the receiver’s left or to his right–to test whether his winning probabilities are indeed the same for both serving directions, as the theory says they should be. In only one of the 40 point games is minimax play rejected at the 5-percent level. This rejection rate is actually slightly below the rate predicted by the random character of equilibrium play.

In other words, tennis players (like soccer goalies) are rational agents, at least when choosing where to serve. (The big exception is that even the best tennis players tend to switch the location of their serves a little too often.) This has led some to argue that the irrationality observed in the lab is a side-effect of lab artifice. Perhaps the incentives are too insignificant? Perhaps the subjects don’t understand the rules of the game? Or maybe they’re intimidated by the presence of the scientist, who is measuring their behavior?

The latest attempt to rectify this schism between lab data and real life observations comes from Kenneth Kovash and Steven Levitt. They began by seeking out vast data sets, since one of the central problems with the studies of soccer players and tennis stars was the smallness of their n. (For instance, the game theory goalie study looked at fewer than 500 penalty kicks.) And so Kovash and Levitt moved on to other sports:

In the case of baseball, we observe every pitch thrown in the major leagues over the period 2002-2006 – a total of more than 3 million pitches. For football, we observe every play in the National Football League for the years 2001-2005 – over 125,000 plays. In both settings, the choices being made have very high stakes associated with them.

The results obtained from analyzing the football and baseball data are quite similar. In both cases, we find clear deviations from minimax play, as evidenced by a failure to equalize expected payoffs across different actions played as part of mixed strategies, and with respect to negative serial correlation in actions. In the NFL, we find that offenses on average do systematically better by passing the ball rather than running. In baseball, pitchers appear to throw too many fastballs, i.e., batters systematically have better outcomes when thrown fastballs versus any other type of pitch.

In football, teams are more likely to run if the previous play was a pass, and vice versa. This pattern is especially pronounced when the previous play was unsuccessful. Negative serial correlation in actions is consistent with a large body of prior laboratory evidence. Pitchers also exhibit some negative serial correlation, particularly with fastballs, i.e., they are more likely to throw a non-fastball if the previous pitch was a fastball, and vice versa.

The magnitude of these deviations is not trivial. Back-of-the-envelope calculations suggest that the average NFL team sacrifices one point a game on offense (4.5 percent of current scoring) as a consequence of these mistakes. In baseball, we estimate that the average team gives up an extra 20 runs a season (about a 1.3 percent increase). If these estimates are correct, then the value to improving these decisions is on the order of $4 million a year for the typical baseball team and $5 million a year for an NFL franchise.

This suggests that the lab data is actually correct, and that people (or at least MLB pitchers and NFL coaches) aren’t particularly good at obeying game theory. It’s a rational model we don’t naturally emulate. But why not? Wouldn’t natural selection want to endow us with a mind capable of maximizing competitive outcomes?

One possibility is that failing at game theory is the price we pay for having emotions. Just look at NFL coaches, who according to Kovash and Levitt don’t call pass plays often enough. Since 1960, quarterbacks have managed to increase their average gain per pass attempt by nearly 30 percent, from 4.6 yards to 6.5 yards. (Running backs only get about 4 yards per attempt, a number that hasn’t changed in thirty years.) Furthermore, even as quarterbacks have gotten more yards per pass, they have managed to throw fewer interceptions. In 1980, passes were picked off more than 6 percent of the time. By 1995, the rate of interceptions had been halved, which meant that passing the ball wasn’t any statistically riskier than rushing.

Given these statistics, NFL teams should pass the ball the vast majority of the time. That, at least, is the conclusion of Ben Alamar, a professor of sports management at Menlo College. He argues that the NFL exhibits a “Passing Premium Puzzle”. (This is the sports version of the Equity Premium Puzzle, which is the mystery of why investors hold so many low yield bonds when stocks have performed so much better over the long-term.) Alamar notes that, despite the significant increase in the expected utility of the passing game, coaches still run the ball about 46 percent of the time. While this represents a decrease from the 1960′s – the average NFL team use to run the ball more than 58 percent of the time – a perfectly rational coach would pass even more frequently, since passing represents a higher rate of return. So why do coaches still run the ball? Alamar admits that a successful running game contributes to the success of the passing game, since you want the defensive backs to have to worry about the possibility of a run. But he isn’t convinced that coaches need to run the ball quite so often. “For all of their planning and late nights,” Alamar writes, “NFL coaches do not act in a fully rational manner.” Just like investors choosing bonds over stocks – stocks have a much higher rate of return over the long run – coaches are swayed by the illusory perception of risk, or what’s often known as risk aversion. Although passing the ball isn’t statistically riskier than running the ball, it feels riskier, as the ball is lofted into the air and is up for grabs. (I’d argue that a similar calculation applies to breaking balls, which might also feel riskier since we have less control over where the ball ends up.) The end result is that teams gain fewer yards than they might otherwise.

The point is that an inexplicable feeling – a judgment of risk we can’t justify – interferes with the rational calculation. The play might not be optimal, but it feels less likely to trigger a worst case scenario (interception, home run, etc.), and those scenarios carry a disproportionate weight. While our emotions might lead us to do many silly and stupid things – not obeying John Nash is the least of these quirks – they’re also amazingly effective tools when it comes to quickly evaluating the world, especially when we’ve got experience in that domain. And so we call a run play and opt for the fastball. We’re not rational, but we’re rational enough.

Thanks to Eric Barker for the tip.

(30)
More »

Comments

  1. #1 david
    February 11, 2010

    Interesting.

    So one of the most rational sports is pro basketball, at times, because when the game is within 2 or 3 points and the clock is running out, they will try and usually will get the ball to the strongest player and he will shoot his highest percentage shot. For a person who knows who and what for a team they can predict this every time. That person is usually not a sportscaster, who is idiotic, but a player such as Bill Russell, used to be.

    So when a person is under stress do they move to their strength, like the basket players? Only as they perceive it in their circumstance, not as it is necessarily. So they may move to prayer for example, or visit a relative’s tomb. Emotion plays a role–you said that— what’s happened recently. And so does physiology. A person’s heart function may vary for example and they get “cold feet.”

    And I have heard that chimpanzees when being captured and cornered will pick up a few sticks as if building a nest, apparently finding comfort. Makes perfect sense to me, very human. Humans under stress will often go home and straighten up the house. They are getting ready for a later game.

    So, so many additional factors are removed from game theory that it becomes just as helpful to go back to bird augurs for the battle of Troy as to follow game theory.

    For example, you know that the Rand corporation applied game theory to the Vietnam war, which was a disaster. They put out a book on it. Then, you could find soldiers who were pumping gas for hire a few weeks before, saying, “If we can just get the ratio to ten to one…” Rand may have been right in a stupid sort of way, but they did not have all the factors obviously. What’s more, do we know why they did not? Arrogance, which can help you, but can also make you blind, and game theory becomes worthless.

  2. #2 Conny
    February 11, 2010

    Well, aren’t our emotions shaped by our genes to obey the laws of the game theory of life (think iterated prisoners dilemma of cooperation, hawk-game of agression and so on)? In other words: humans do not necessarily optimize but evolution does.

  3. #3 gwern
    February 11, 2010

    I wonder how much of the fastball and passing deviations can be explained by agents’ self-interest: players do not have the good of the team solely at heart. Running & fastballs would seem to be much flashy and self-promoting than dull passes or normal pitching.

  4. #4 inboulder
    February 11, 2010

    You’re comparing random schmoes in a lab vs PROFESSIONAL tennis players? I’m sorry but doesn’t it seem blindingly obvious that tennis pros would be more ‘rational’ with regards to GAME theory as they’ve self selected for WINNING the game of tennis. Good grief. Take 1000 lab schmoes, whittle them down to the top 10 at the lab game and guess what you’ll find.

    Further, this analysis of other sports seems far too simplistic to even determine what the mimimax model should be (although I do not have their study details). In football it is thought ‘the run sets up the pass’, in baseball a pitcher can place a fastball with more accuracy than other pitches, did they take into account the need for a strike vs walks etc?

  5. #5 royniles
    February 11, 2010

    The irony here is that game theory, which treats life’s strategies as gaming within a set of theoretically presumptive rules, isn’t even predictably accurate when compared with our most entertaining varieties of actual games where rules are made to fit. And yet here you seem to be arguing that if strategies were changed, the teams that change them would win more often. Ridiculous. These theories are wrong precisely because they don’t account for the innumerable counter strategies that each team can “choose” from within the rules of the particular game.
    Then apply this to the game of life where we don’t yet know some of the most basic rules, and thus some of the most basic strategies and counter strategies so far devised by biological forms. Game theory which presumes to be operational within those rules can never be more predictively accurate than the accuracy of those presumptions.
    (Which I might add, don’t even presume there’s a significant switch at some point between our short term and long term predictive apparatus.)

  6. #6 david
    February 11, 2010

    you know that on the dropdown menu this blog is alphabetized under “The”?

  7. #7 david
    February 11, 2010

    in the listing The Frontal Cortex is in order of Fr but to punch it up requires either reading the whole list or punching T.

  8. #8 Ray in Seattle
    February 11, 2010

    Just as in “How We Decide” you establish that we use intuition to guide our behavior, esp. when there high rewards or punishments at stake – like Tom Brady’s passes.

    And then you go and suggest that maybe the reason we make less than optimal decisions in some cases is because our emotions interfere with our rationality. You suggest, “One possibility is that failing at game theory is the price we pay for having emotions.”

    I would ask if a major league pitcher really selects his throws differently from how a subject in the Iowa Gambling Test selects the deck – and I’d suggest he does not. The apparent difference you notice I believe is an error in assumptions.

    You assume that pitchers brains are only seeking to win the game. I’m no baseball expert but it seems to me they are also seeking other, possibly more important things, such as a secure high-paying contract when the current one expires. That means that the pitching philosophy held by the people who hire and pay pitchers places limitations on how the pitcher can win the game and still succeed professionally.

    They not only have to earn the out they must consistently do it in a way that their owners and fans see as smart – or their career lifetime earnings can suffer. I suspect their intuitive pitch choices probably reflect that.

  9. #9 Mike
    February 11, 2010

    Just a little FYI – The vast majority of plays in football are called by the offensive coordinator or the head coach. Occasionally, a QB will audible but the other players (WRs, RBs, etc) NEVER have immediate say in what play gets called. The same can be said for baseball where the manager signals the pitches to the catcher, who then relays them to the pitcher. The pitcher can shake them off if he choses but it’s still only a small % of the pitches. So an analysis of the players’ motives in these situations is not applicable. In basketball, on the other hand, players have far greater control over the flow of the game and therefore their own stats.

    Rather than focusing on why we’re not operating at 100% success rate, it amazes me that the performance of human beings is only improved by between 1 and 5 % by a THEORY in these cases. Seems pretty darn good really.

  10. #10 DR
    February 11, 2010

    The datasets are impressive, but I wonder if they really should be expected to reflect game theory. Take pitching (in the Cubs’ case, “take their pitching … please.”). From time to time, a pitcher may not have “good stuff,” meaning that his pitches don’t move optimally, causing him to throw more fastballs on that day, and over the course of time, biasing pitch-segmentation in the dataset. This isn’t a matter of optimizing game theory, but of physical conditions.

    The counter-bias (a pitcher having a subpar day with his fastball and therefore relying on breaking pitches) is the dream of major league hitters and doesn’t seem to happen very much — at least not among pitchers who remain employed. Why? Curveballs work not because they curve, but because they’re not fastballs, or stated in baseball game theory: when you have no fastball, you have no curve. Yes, there have been successful pitchers with poor fastballs, but they are exceptional by definition.

    A final point, just for fun. Pitching strategy is not separate from baseball strategy, and relief pitching is of paramount importance to a winning team. As abhorent as walks may be in the early part of the game, they are catastrophic for a relief pitcher. In the immortal words of Tommy Lasorda to Joaquin Andujar in a very brief World Series mound trip: “Throw f’cking strikes!”

    The prime directive of relief pitchers could not be stated more eloquently. A skillful relief pitcher can certainly throw curveballs, and is well-advised to do so now and then, but never at the expense of walking a batter. His job, if he can keep it, is to reduce the number of outs remaining to the opposition. The only practical way to do that is to throw strikes, which is easier to do with a fastball, which therefore predominates among relief pitchers (who may also tend to be so pumped up with adrenalin when they enter the game that they dare not attempt a curveball among their first pitches).

    So here’s a question. Does the dataset break down to pitches thrown in innings 1-6, and those thrown in 7-9? In other words, is a modern baseball contest comprised of two games, each of which may very well comport separately with the expectations of game theory? (cf. david’s comment about basketball end-game).

  11. #11 Ray in Seattle
    February 11, 2010

    Thanks Mike and DR for teaching me something about baseball – but aren’t you guys pretty much saying the opposite?

  12. #12 Paul Mineiro
    February 11, 2010

    What about observation bias?

    How do we know that the situations in which the alternate strategy could be employed (e.g., more passing, less fastballs) are statistically identical to ones where the historical data suggested suboptimal strategizing?

    My bias on these matters is that the coaches know something the scientists do not, and not the converse :)

  13. #13 DR
    February 11, 2010

    Ray, I think I’m agreeing (or certainly not disagreeing) with Mike in that I attribute pitch selection not only to statistically-based decisions (hugely emphasized in major league scouting, which is probably why we see 95% adherence to game theory), but also to various physical conditions and supervening strategies at large (e.g., what’s the best way to finish this game, who is likely to make the best reliever).

  14. #14 david
    February 11, 2010

    @ Ray in Seattle

    How we decide what? And is deciding a series different from making a single decision?

    I think I know about types of decisions. My first hint came from Henry James, whom I disliked heartily so caught in many inane statements. He had said we decide what we want to do and then we look for reasons for it. I watched to see about that for many years.

    As you see, Henry did not mention “emotion” but did mention “want” which could also perhaps be “desire.” Desire would be a closer word than emotion, but desires involve emotions, not so? So the scalpel cutting vocabulary is not available for what we mean, in some languages anyway.

    Applied to juries Henry is correct, they want to decide a certain way, then they decide. Note that it is a group dynamic. Getting the jury through that process favorable to one side and not the other is sort of like a three-ocean war. That process is complicated, takes experience, and ethos is important. Some lawyers never understand it. If a person wanted to watch that battle by reading he or she could watch the lawyers in the Lizzie Borden trial. However, the Borden case can become addictive. There are clubs, etc.

    Henry James was right once, though I still heartily dislike him, and you may be right as well (probably would like you), and the words available are so clumsy that the possibility exists that both you and the blogger are right and some distinctions are just a matter of words.

    But in general, not in every case, I think you will see persons want a certain decision and then look for reasons to support it. We may need to know them well to detect this, such as a spouse.

    That’s my opinion anyway and I have not read “How We Decide” but perhaps should.

    And then there’s the matter of how we make voting decisions. We know that not everyone decides by the same process. We may be dumbfounded to meet people on election day who have not made up their minds. I’m sure I do not know how they decide.

  15. #15 Donald Dade
    February 11, 2010

    I’ve done a lot of reading on the quantitative side, e.g. proving the Min-Max theorem, the qualitative side, e.g. Schelling’s “Arms and Influence”, and the applied side, e.g. Ed Waltz’s “Counterdeception Principles and Applications for National Security”. From all of these I’ve gleaned that Game Theory in it’s purest form is not particularly helpful because of the intractable task of quantifying all the cause-and-effects into a game matrix, and mapping from payoffs to utility. If one had the cognitive ability to do that, then one probably wouldn’t need the tool that is Game Theory, either…

    But when I read the U.S. Army’s FM 3-14.3 on Military Deception, I realized that much of the art of crafting a deception plan (which is an application of Game Theory) is encouraging your adversary to think in a way that makes him susceptible. For example, if I have a deception goal which gives me an advantage, I create a deception story which hopefully builds on the preconceptions he already has.

    If he doesn’t have deception channels through which he perceives me, I need to create some for him and encourage him to listen at them. More importantly, if he is not predisposed towards strategic thought, I must encourage him to be so inclined. In foreign relations, this is called “diplomacy”, and that’s one reason why diplomats go to dinners and such.

    I suppose my point is that FM 3-14.3 shows that even if Game Theory appears to not be immediately useful, one needn’t give up; the context and mode of interaction with one’s adversary is plastic. If one is creative enough, one can find ways to make it more applicable. But I didn’t find that particular insight in any book on Game Theory.

    That was an interesting question; thanks for it.

  16. #16 royniles
    February 11, 2010

    Counter strategy is counter deceptive. As far as I can tell or find, Game Theory barely touches that aspect of gaming or of behaviors in general.

  17. #17 royniles
    February 11, 2010

    Here’s a good paper on deceptive strategy:
    http://all.net/journal/deception/Framework/Framework.html

    How would I know? I worked as a professional investigator in the deception business for 40 some years.

  18. #18 David W. Fenton
    February 11, 2010

    1. I think it’s interesting that “game theory” doesn’t even work for actual GAMES, let alone for the areas it is applied to (stock market, war, etc.).

    2. The analysis as related in the blog post doesn’t really say whether maximizing points earned would have won more games. It’s likely that when a team is already winning, they will RATIONALLY play conservatively to protect their win, sacrificing possible additional (unnecessary) points in order to protect a winning margin.

    Would that extra 1 point per game for football and the 20 runs per season have actually improved the record of the teams in question?

  19. #19 Dan Peterson
    February 12, 2010

    In support of the emotional over rational theory; Sean Payton’s onside kick call to start the second half of the Super Bowl. I don’t know the actual stats, but is seems to not be a high percentage, maximum benefit play. Then again, he slept with the Lombardi Trophy on Sunday night.

  20. #20 Lenny C
    February 12, 2010

    It seems like a lot of people commenting here know more about the brain than they do about sports (which is fine, because this is not a sports blog, but still, I thought I would set some records straight):

    Whitey Herzog coached Joaquin Andujar in the World Series, not Tommy Lasorda. Managers don’t call every pitch and relay it through the catcher to the pitcher. That happens only maybe 20% of the time, if that. Pitchers and catchers have more control pitch to pitch. A comparison was made saying basketball players have more control of the game and their stats — but don’t basketball coaches call plays occasionally, too? I’d say they do at least as often as baseball managers. Onside kicks are not very successful percentage-wise, when they are done in the last minutes of the game when the receiving team EXPECTS one. However, the timing of the superbowl kickoff made it so unexpected, it had a much higher probability of working out for the Saints (as the result showed). I did concur with one sports-related truth, on these comments, though — and that’s the quality of the Cubs pitching!

    Good work, Jonah, I loved the book and the blog.

  21. #21 Brian
    February 12, 2010

    i see a big assumption in this article that rational maximization of results would lead to increased fitness. i do not see this as necessarily the case. as seen in basic questions of altruism and fitness we often see that behavior and it’s outcomes has more dimensions than its direct consequences. it could be the case that games theory has insights into fitness but it may not be a direct correlation.

    i also see a false dichotomy between “emotion” and “rationalization”. i understand that from a neurological point-of-view these are distinct responses in the brain but from a sociocultural perspective this is not the case. actors do try to maximize fitness, or in this case agency, but they do so by reading the social topography. it’s the people who ignore or are unable to read the topography and are able to see the rational and act within their own system or understanding of it that are the innovators, but they often pay an exceptional inter-personal price ie are often social outcasts. most of us must function socially and to do so is that actual maximization of fitness. we play by rules even if the rules are “flawed” from games theory perspective.

  22. #22 Mike M
    February 12, 2010

    It looks like there’s something missing here in the analysis. That is, that all yards on a football field are not equal, and all pitches in a count are not equal.

    Very simply, a fastball is used when there are already 3 balls and the pitcher is in a tough spot, so naturally fastballs will be correlated with improved success by batters. By that I mean: fastballs are used more frequently when a pitcher is already losing to the batter!

    Similarly, in football, the last yard to go for a touchdown, or even for a first down, is much more important than the yard beyond the line, or the first yard in a 3rd-and-10 situation (where yards 1-7 are almost meaningless). In addition, I’d imagine that the standard deviation of yards gained is much greater for passes, and there’s a higher chances of gaining 0 yards on a pass than a run. Thus, passes are employed in situations where yards are cheap, and runs are utilized when yards are hard to get. A team may try four runs from the 1-yard-line, be successful on the last, have an average gain of only .25 yards, but have a touchdown to show for it. That situation can’t be statistically compared to one 60-yard pass play simply on yards gained.

  23. #23 Lenny C
    February 12, 2010

    Excellent points, Mike M. There’s too much depth to the strategy of both of these sports to reduce it to a simple generalization that says that fastballs and running plays are statistically less successful than the alternatives.

  24. #24 Matthew C.
    February 12, 2010

    BTW, the equity risk premium doesn’t exist anymore after the horrible returns for stocks (and excellent returns for bonds) from 2000 – 2010.

  25. #25 inboulder
    February 12, 2010

    I just came back to look at the comments, it seems we’ve pretty much torn any points Jonah Lehrer had to shreds. Attention gets him pages view, but perhaps not all publicity is good. I’d be interested to hear him respond to some of the criticisms of his reasoning in this article.

  26. #26 David
    February 14, 2010

    Interesting that the main metaphor of “gaming theory” as written, mostly relates to sports analogies.

    In business school I studied it as a potential sequence of events taken, and various responses from outside sources–competitors–based on trying to predict risk/reward scenarios for purposes of financial gain.

    What’s not apparent, though, is no matter how faulty gaming theory may be in the real world–when it comes to nuclear weapons and the use of gaming theory, that’s what we’ve been focusing on since the inception of the bomb.

    Surely this angle should be discussed and analyzed. My 2 cents as poster and post #15 misses the point:

    A nuclear war has no emotion and/or diplomatic variables, and that’s why pure gaming theory is vital in and of itself as a value proposition.

    And to counter an expected response that nobody wins in a nuclear war is well, not yet known.

    More need then, for gaming theory studies that are truly applicable for the real world, and not ridiculous assertions as applied to sports or everyday human endeavors, where the variables are much less predicable because of human emotion.

  27. #27 DR
    February 14, 2010

    No emotion in nuclear strategy!? Yikes! Don’t people decide these things, David? And not just randomly selected people, but people with a drive to obtain power. For only the most recent commentator on MAD, see Steve Coll in this week’s New York Review of Books (Feb 25, 2010).

  28. #28 David
    February 18, 2010

    Of course, DR. I meant once the missiles start flying. That’s when the human emotion leaves the game theory theories, and start to play a role in what happens afterward.

  29. #29 Helen
    February 19, 2010

    Yeah. I think I need to filter out some “friends of friends” from my facebook. I’m getting overwhelmed will Farmville updates.

  30. #30 Chris
    February 20, 2010

    Very interesting stuff. Regarding football, there are emotional benefits of running the ball. It punishes the defense and can boost the confidence of the offense. This is pretty intangible, and thus immeasurable, but it’s real. The offense line especially likes to run the ball, to “jam it down the throat” of the defense. You can’t do that by passing. Also, scoring as quickly as possible (by passing and gaining more yards per play) is not the optimal strategy in football, as it won’t allow your own defense time to rest between your opponent’s possessions. Anybody who knows anything about football knows that establishing a running game and preventing your opponent from doing the same is crucial to winning the game, which is ultimately what teams and coaches are judged upon (not yards per play, or even total yards).