In honor of the start of football season, I thought I'd blog about one of my favorite economics papers. It's by David Romer of UC Berkeley, and it should be mandatory reading for every sleep-deprived NFL coach out there. The question Romer was trying to answer is familiar to every NFL fan: what to do on 4th down? Is it better to bring on the kicking team or to go for it? Under what conditions should coaches risk going for it?
To answer this profound question, Romer analyzed every fourth down during the first quarter in every NFL game between 1998 and 2000. (He had help from a computer program.) The first thing Romer did was figure out the fluctuating value of a first down at each point on the football field. After all, a first down was more valuable for a team if it occurred on an opponents two yard line than on their own twenty yard line.
Then Romer calculated the statistical likelihood of going for it on fourth down under various circumstances and actually getting a first down. He also calculated the probability of kicking a successful field goal from various spots on the field. So let's say you are NFL coach, and you have a fourth and three on your opponent's 30 yard line. Romer could tell you that 1) you have a 60 percent chance of getting a first down, and that teams with 1st downs inside the thirty yard line score a touchdown 40 percent of the time, for an expected point value of 1.7 and 2) that field goal attempts from the 32 yard line failed almost 65 percent of the time, which meant that going for a field goal only had an expected point value of 1.05. In other words, it's almost twice as effective to go for it than to attempt a field goal.
So what do most coaches do? Well, NFL coaches consistently make the wrong decision. According to Romer's analysis, teams would have been better off going for it on fourth down during the 1st quarter on 1100 different drives. Instead, coaches decided to kick the ball 992 times. This meant that NFL coaches made the wrong decision over 90 percent of the time. Romer summarized his counterintuitive results: "This analysis implies that teams should be quite aggressive. A team facing fourth and goal is better off on average trying for a touchdown as long as it is within 5 yards of the endzone. At midfield, being within 5 yards of a first down makes going for it on average desirable. Even on its own 10 yard line - 90 yards from a score - a team within three yards of a first down is better off on average going for it." Romer conservatively estimates that a more aggressive approach on fourth downs would make a team 5 percent more likely to win the game. This is a significant advantage: a coach willing to endure the risks would win one more game in three seasons out of every four.
But if kicking a field goal or punting on fourth down is such a bad idea, then why do coaches always do it? To explain the consistently bad decisions of NFL coaches, Romer offered two different answers. The first is risk aversion. If coaches followed Romer's strategy, they would fail about half the time they were within ten yards of the endzone. This means that instead of kicking an easy field goal and settling for three points, they would come away empty handed. Although that's a winning strategy in the long-run, it's awfully hard to stomach. After a long drive down the field, fans expect some points. A coach that routinely disappointed the crowd would quickly get fired. Nothing upsets us more than losing a risky gamble, and going for it on fourth down is perceived as a risky decision. As Daniel Kahneman notes, "Worst case scenarios overwhelm our probabilistic assessment, as the mere prospect of the worst case has so much more emotional oomph behind it."
The second reason coaches stink at making decisions on fourth down is that they stink at statistics. As Romer politely writes, "Many skills are more important to running a successful football team than a command of mathematical and statistical tools...It may be that individuals involved want to make the decisions to maximize their teams' chance of winning, but that they rely on experience and intuition rather than formal analysis." Of course, this problem isn't limited to professional sports. All of us rely on experience and intuition instead of equations and carefully analyzed probabilities. As the surfeit of field goal attempts demonstrates, the human brain is a terrible calculator.
PS. NFL coaches remain skeptical of Romer's findings.
- Log in to post comments
I think this points out the problems of saying what the right decision or wrong decision is from a single paper.
Anyone who understands statistics can say that yes, an expected value of 1.70 is better in the long run than an expected value of 1.05, but how accurate are those numbers? It's not my memory that field goals from the 32 yard line fail almost 65% of the time. Nor is success on fourth down an unvarying probability like radioactive decay. Teams vary. Situations vary. There are more variables than are listed in play-by-play accounts. What would happen to Romer's numbers if trying on fourth down increased? Was that an unusual time period being studied? Is there some other quirk in those numbers? With any study it's easy to decide that the researcher is competent and therefore take some bottom line number as absolute, when it isn't.
At least one can be objective about picking apart data. When it comes to subjective analysis, it can get so much worse. Who says a coach would get fired for disappointing fans or that fans would be disappointed by going for it and failing? Fans cheer for that decision. They may be fickle, but not instantaneously. As another East Bay denizen once said, "Just win, baby." That might mean getting a more sure 3 points on this one try rather than trying for the 7 that requires multiple tries to pay off. If you have a lead, you want to extend it, by 3 or by 7 isn't necessarily as important. San Diego's tactics last night would have been much different if they had needed 50 points to win.
Cliches are often helpful. "You don't want to come away with no points in that situation," has some truth to it. 32 head coaches can test that truth vs. Romer's hypothesis and see how well either one works. If they don't want to, there are more explanations than saying they're afraid or ignorant. OK, so Romer has numbers that suggest that it pays off in points to go for it on fourth down more. That's not hard to understand. I'm sure it's already encouraged some coach somewhere to go for it sometime. People do try things.
But is it so hard to understand that Romer might not be right?
David brings up such excellent points. Clearly, being a football coach isn't just a matter of fiddling with the probabilities. (Although I don't think Romer's data is flawed, or that 1998-2000 were unusual years.) That said, even Romer admits that there are way too many intangibles to count. I think the biggest factor Romer fails to consider is momentum. Going for it and failing inevitably shifts the momentum, and not in a good way. But I still think this study should at least be considered by NFL coaches. (Some, like Bill Belichick, already take Romer's analysis into account.) However, I'm not optimistic that's going to happen anytime soon. Bill Cowher explains why: "It's easy to sit there and apply a formula, but it's not always the easiest thing to do on a Sunday. There's so much more involved with the game than just sitting there, looking at the numbers and saying, 'OK, these are my percentages, then I'm going to do it this way,' because that one time it doesn't work could cost your team a football game, and that's the thing a head coach has to live with, not the professor. If we all listened to the professor, we may be all looking for professor jobs."
As David said, changing the strategies could change the probabilities. The given probabilities are for a fairly conservative NFL.
Also, there is some aspect of situational strategy. For example, if you don't get many chances to put points on the board in a low scoring game, then kicking field goals or playing for field position may be in your best interest. In higher scoring games, a field goal may not be worth much.