Bill Belichick has never been the most popular coach in the NFL, but his Sunday night decision to go for it on 4th and 2 on his own 28 with two minutes remaining in the fourth quarter has even his fans crying foul. I bring up this football decision not because I'm interested in a debate - as a Pats fan, the last five minutes of that game were excruciating - but because I think it illustrates the difficulty of making rational decisions, even when the evidence supports the call.
I've blogged about the research of UC Berkeley economist David Romer before, but his basic thesis, based on an exhaustive statistical analysis of 4th down scenarios, is that NFL coaches are irrationally risk-averse. They punt the ball way too frequently and kick far too many field goals.
Belichick was an econ major, and has expressed a familiarity with Romer's research. Nicholas Beaudrot has persuasively shown how, from this econometric perspective, Belichick's bizarre decision actually makes perfect sense:
On 4th down, with 2 yards or fewer to go, New England has gained a first down on approximately 66% of its attempts with Tom Brady as quarterback. The Colts had one timeout. If the Patriots gain a first down, the game ends; they can slowly walk to burn a few seconds, then take a knee on each down to end the game. If they don't gain a first down, the Colts would still need to score a touchdown to win the game. Let's give the Colts a probability P of getting the six if the ball starts at the 28 yard line. So if the Patriots try for the first, their chance of losing is
(Probability of 4th down failure) x
(Probability of Colts scoring a TD from the 28 Yard line) = 0.33PThe average New England punt nets about 40 yards. Let's give the Colts a probability Q of scoring a TD on a driving starting at the Indianapolis 32. Then, the chance of the Patriots losing is simply Q. For Belichick's decision to make sense, we just have to believe that he gave his team a lower chance of losing. In math terms, that would mean 0.33P < Q. Doing some algebra leaves you with P < 3Q. In other words, for the Patriots to have made the right decision, we only have to believe the Colts odds of scoring a TD on a drive starting 28 yards from the end zone are less than three times the odds of the same outcome starting from 68 yards out. The win probability graph for the game suggests that, given 1st-and-10 from New England's 29, the Colts had roughly a 51% chance of winning in the actual situation. We have to believe that their chances under the punt scenario were above 17% for Belichick to have made a bad good decision. Considering the Colts' have scored touchdowns on 30% of their offensive possessions, my guess is that this was a good one.
The reason I bring up this analysis is to demonstrate that even defensible decisions can have wrenching emotional consequences. Belichick's call might have been statistically correct, but it felt horribly wrong.
And this kind of contradiction isn't just relevant for football coaches. Just consider health care: the only way we're ever going to reduce medical costs is to restrict procedures that haven't passed evidence-based efficacy tests. Maybe that means 40 year old women don't get mammograms, or that we treat prostrate cancer less aggressively, or that we stop performing spinal fusion surgeries. Although there's solid evidence to question all of these medical options, such changes provoke intense debate. Why? Because our emotions don't understand statistics. Because when we have back pain we want an MRI. Because when it's our father with prostate cancer we want the most aggressive possible treatments. And so on.
The point is that there's often an indefatigable gap between the rigors of cost-benefit analyses and the emotional hunches that drive our decisions. We say we want to follow the evidence, but then the evidence rubs against a bias like loss aversion, and so we make an exception. We'll follow the evidence next time.
So here's my cheeky proposal for lowering the cost of health care: Put Belichick in charge of Medicare. Nobody likes him anyways, and he's clearly able to follow the math even when it feels like a mistake.
PS. Razib addresses a similar issue from a slightly different angle.
Update: Here's more evidence that Belichick's decision was eminently rational, and made them 9 percent more likely to win than punting the ball.
- Log in to post comments
I've never seen a single football play get so much coverage from public intellectuals before!
Steven Levitt covered the topic a couple of years ago here:
http://www.u.arizona.edu/~dreiley/papers/ProfessionalsMinimax.pdf
As I've said elsewhere, I'm a longtime Pats fan & I agree with his decision.
It was the outcome that was bad, not the decision.
What happens to P if 11 Pats walk back onto the field thinking that their coach willfully set them a too burdensome task? thinking that he has put his ego above their extreme exertions of the previous hour, thus forfeiting, if even for one critical moment, his claim upon leadership?
What happens to Q if 11 Colts think, rightly or wrongly, that the opposing coach has insulted them in front of 60,000 home fans or that the goal line has moved, magically, as if in response to their inner wishes, 60% closer than it ought to have been?
Maybe coaches punt "too much" on fourth down because statisticians know too little about motivating football players.
"Success has a thousand fathers; failure is an orphan."
- John F. Kennedy
I agree with DR. The calculations in the linked NYT article assume the Colts' ensuing offensive drive and Pats defensive efforts are independent from the failure of the 4th and 2 try. So the fact that it FEELS so wrong means that the psychology is different (i.e. are the defensive players thinking Beilichick doesn't trust them? Are they less motivated because they think the blame will go to the coach instead of them?) than the typical situations the stats were calculated from and maybe don't generalize.
Honestly it was probably still the right decision (maybe the psychology HELPED the defense) but the point is that people's perceptions of 'irrationality' also affects the outcome.
I bet you if Grady Little used that same statistical analysis he would have taken Pedro out!
Belichick is an arrogant jerk but besides that he was completely inconsistent (not wrong).
I didnât think going for it was necessarily wrong â I wouldnât have done it, but it's not a bad call. I understand that he is either
1. afraid of Manning
2, has total confidence in his passing offense
3. both
(I think the answer is 3 â with 2 being greater than 1)
Before analyzing it, let me say regardless of that call he got away with coaching malfeasance that people haven't talked about because of the call. What he did with his time outs was terrible. No excuse for that, he needs those two timeouts he blew on nothing, especially if he is afraid of Manning. Bespeaks arrogance.
As for the decision to go for it - if he makes the first down, game over. Takes it out of chance. Okay
The counterargument not to go is just as good, his move works better against bad teams that can't stop him or would have trouble from the 30. Wonât argue going for it â unknowable answer.
But- here are two questions I have no one brought up because they are hung up on the call itself. Buy the premise that itâs not a bad move.
1. Have you no confidence in your run game? If you, and no one else, know you will go for it on fourth down it means you have two downs to run for two yards. They canât do that? This now tells everyone the Patriots are not a running team. Valuable info.
2. More importantly â once the move failed, and the Colts went down to the one with 45 seconds to go, it makes no sense not to let them score. If you remember they put up a stand and stopped them on the first run from the one. Think about this. The Colts have three more plays and a timeout. Youâve just essentially said I donât trust my defense to stop them from going 70 yards, you mean you think you are going to stop them three times for one yard? Come on. IF he lets them score with 45 seconds on Addaiâs first run, at least he gives Brady 45 seconds to move for a field goal, not out of the realm of possibility - five or six plays before a kick. Otherwise the Colts will do exactly what they did, run the clock down so there is only time for three plays. I canât see any rationale that is consistent there. Happened in the Jets game â and the running back messed them up by taking a knee at the one to run the clock to the field goal . But in this situation the Colts had to score a touchdown immediately -couldnât wait to kick a field goal. Pats could have had the ball with time to move it.
What it says I think is that Belichick has unlimited confidence in his passing game. He can pass any time on anyone. Problem is he canât do that against a really good team, who will put pressure on Brady. Thatâs how he lost against the Giants in the Super Bowl -unless his offensive line can contain the rush, then the fact that Brady is the best (and I think you saw that last night) and Moss and Welker canât be covered is a moot point. The Colts just pressured Brady so that he couldnât automatically complete the passes.
Belichick figures that passing attack can win any game regardless of his running game or defense- it will beat bad teams but they canât win the Super Bowl like that because good teams can stop them enough times to outscore them. The call proved he needs either a defense or a running game to complement his passing attack to beat a really good team.
DR and Tiffany are tapping into something I thought about following Belichick's decision: how this plays out in terms of game theory. The decision to go for it on 4th and 2 impacts other variables, including, for example, these two:
1) Attitude of New England's Defense - many former players weighed in on this topic this week, and most of the one's I heard indicated this decision implied a lack of confidence by Belichick in his defense. While impossible to measure, one wonders what the mindset was of the defensive players when 1) the coach just made a truly unorthodox call that 2) implied a lack of confidence in them, and 3) left them with a really short field to defend.
2) Sense of momentum - the possibility of an increase in the Colt's confidence/energy resulting from stopping the Pats on 4th and 2, and having a relatively short field in front of them, versus having to drive most of the field if the Pats had punted. The Colts already had some momentum; did the defense being put into a position to make that 4th-and-2 stop, and succeeding, further increase that sense of momentum?
I think some of this has to do with going from the general to the specific: the statistics cited (i.e. 51% chance of scoring following a punt) are taken from a broad analysis, and can't take into consideration the specific factors of this one decision at that one particular point in time, particularly in terms of less measurable variables. I suppose that is what a coach is for: to develop a feel for assessing the variables that can't simply be calculated via stats (i.e. momentum, player fatigue, etc.).
None of this means it was the wrong call; I just think that the statistics cited above don't account for the whole story, because the statistics can't account for significant portions of the situation.
Jonah writes: the only way we're ever going to reduce medical costs is to restrict procedures that haven't passed evidence-based efficacy tests
This statement makes no sense. If the cost of procedures come down, then the overall costs will be less. In Japan, and MRI of head costs $105. In the US it's between $1000 and $1400. The Japanese visit doctors and get MRIs at three times the rate Americans do, and yet they pay less because the *price* of medical goods and services is much less there than in America.
I recently got a colonoscopy. It took about 20 minutes. At the clinic they do one after another of these all day long, five days a week. The insurance company paid $2200 for it.
My primary care doctor gets $55 from the same insurance company for a 5-10 minute consult The colonoscopy takes maybe 3-4 times longer, but it costs 40 times more. That is the price per minute of the colonoscopy is more than 10 times more. Why cannot the clinic be as efficient as the primary care doc?
The answer is they can, but why should they of the insurance company is perfectly willing to give them loads of money.
In Japan, the ministry of health bargains with the Japanese medical society, the hospital association and the major drug companies over what the price for their products and services will be in Japan. The price charged in Japan much lower that those charged in the US.
I work for Pfizer. We sell lots of Lipitor in Japan, but our revenue is pretty small because the price we charge there is much less than the price we charge in the US.
If we want to sell at all in Japan, we have to sell at their price, it's kind of like Walmart, you have to meet their price or they won't do business with you.
We still make a profit on sales in Japan, we could sell at a tenth the US price and still make some profit. But we like our US business much better because Americans are willing to pay through the nose for their medical products and services.
And *that* is the main reason why health care is so expensive in the US.
I don't know about this; The Japanese are some of the most aggressive users of preventative care and exploratory, but non-invasive, procedures like mammograms and MRIs in the world, and they manage to keep their costs per person below ours. I think the real question is this; are we willing to make doctors strictly middle class citizens and cut significantly into medical manufacturer profits by 1) putting caps on medical salaries 2) aggressively negotiating down procedure & device costs, and 3) subsidizing the wide-spread existence of community clinics for basic care as the Japanese have done?
There are, of course, other ways and other models open, but even under the more market oriented German and Swiss models, doctor salary and procedure cost are just as negotiated as procedure access. That the debate in this country has immediately jumped from whether to have a public plan to the question of what procedures we will restrict to limit costs shows, yet again, just how powerfully the monied, and not the public, interests are driving this debate.
Another point to make regarding medical care and cost in this country, particularly salient to #3 in my above comment, is the extent to which unregulated industry consolidation has driven costs in the U.S. Practically every hospital these days has to be some gigantic research and care campus, and the reasons for this have little to do with the provision of care and everything to do with market competition and profit margin. Is it any wonder that vast swathes of the mid-West, South East, Deep South, and Appalachia are filled with towns bereft of doctors? With the sort of personnel concentration and focus on specialization which drives medical-care-as-industry, and thus, medicine-as-education, these days, is it any wonder that vast inequities have come to exist between the health of our poor, who cannot afford to drive 2 hours to Des Moines to see a dentist whenever they get a tooth ache, and our wealthy? Is it any wonder that, when one has no general practice doctor to have access to, one would develop crippling maladies? Any serious reform of our health care system would at least make an attempt to fight the regional and procedural monopolies which have come to typify our medical industry, and as part of that, try in some way to get doctors out of the ER and into general practice. The transport savings alone would save the majority of U.S. citizens hundreds of dollars yearly.
I think this post makes a really good point. Coming to the conclusion that current practices do not agree with the available evidence base can be extremely difficult for so many reasons. And even if we can scientifically come to that conclusion, what amount of time and effort will it take to change popular perception and then practice of a treatment? It reminds me how 'human' the field is...and I think often medicine and science is held to an unrealistic standard.
Great post. "Doing the right thing feels bad."
IMHO if investors had more emotional intelligence, as well as a basic understanding of Mathematical Expectation, they'd be better armed to sell/sell short into the hysteria and buy/cover shorts in the panic.