Someone should really tell the NCAA tournament television commentators that "the hot hand" doesn't exist. I've gotten pretty tired of hearing these tired cliches about Texas going cold, or Stephen Curry catching fire yet again. Never has a cognitive illusion gotten so much play.
The illusory nature of basketball shooting streaks was first demonstrated by Amos Tversky (of kahnemanandtversky fame) and Thomas Gilovich, a psychologist at Cornell. They began the investigation by sifting through years of Philadelphia 76er statistics. They looked at every single shot taken by ever single player, and recorded whether or not that shot had been preceded by a string of hits or misses. If "the hot hand" was a real phenomenon, then players should have a higher field goal percentage after making several previous shots. The streak should elevate their game.
So what did the scientists find? There was absolutely no evidence of "the hot hand". A player's chance of making a shot was not affected by whether or not their previous shots had gone in. Each field goal attempt was its own independent event. The short runs experienced by the 76ers were no different than the short runs that naturally emerge from any random process. Taking a jumper was like flipping a coin. The streaks were a figment of our imagination.
The 76ers were shocked by the evidence. Andrew Toney, the shooting guard, was particularly hard to convince: he was sure that he was a streaky shooter, and went through distinct "hot" and "cold" periods. (Toney is still regarded as a great clutch player. Charles Barkley has called him "one of the best kept secrets in the history of the NBA.") But the statistics told a different story. During the regular season, Tooney made 46 percent of all of his shots. After hitting three shots in a row--a sure sign that he was now "in the zone"--Tooney's field goal percentage dropped to 34 percent. When Tooney thought he was "hot," he was actually freezing cold. And when he thought he was cold, he was just getting warmed up: after missing three shots in a row, Tooney made 52 percent of his shots, which was significantly higher than his normal average.
But maybe the 76ers were a statistical outlier. After all, according to a survey conducted by the scientists, 91 percent of serious NBA fans believed in "the hot hand". They just knew that players were streaky. So Tversky and Gilovich decided to analyze another basketball team: the Boston Celtics. This time, they looked at free throw attempts, and not just field goals. Once again, they found absolutely no evidence of hot hands. Larry Bird was just like Andrew Tooney: after making several free throws in a row, his free throw percentage actually declined. Bird got complacent, and started missing shots he should have made.
Why do we believe in streaky shooters? The danger of random processesâ¯things like slot machines, stock markets and basketball shotsâ¯is that they are full of intermittent rewards that feel really good. The problem arises when we try to fit these stochastic events into a neat pattern, when we try to explain why the stock market went up 50 points, or why Stephen Curry just made three shots in a row. In both cases, the answer is the same: random chance.
I always assumed that there was one major exception to this psychology of streaks: Joe Dimaggio's 56 game hitting streak in 1941. Here is the late, great Steven Jay Gould, summarizing the evidence in 1988:
There is one major exception [to the hot hand rule], and absolutely only one--one sequence so many standard deviations above the expected distribution that it should not have occurred at all. Joe DiMaggio's fifty-six-game hitting streak in 1941. The intuition of baseball aficionados has been vindicated. Purcell calculated that to make it likely (probability greater than 50 percent) that a run of even fifty games will occur once in the history of baseball up to now (and fifty-six is a lot more than fifty in this kind of league), baseball's rosters would have to include either four lifetime .400 batters or fifty-two lifetime .350 batters over careers of one thousand games. In actuality, only three men have lifetime batting averages in excess of .350, and no one is anywhere near .400 (Ty Cobb at .367, Rogers Hornsby at .358, and Shoeless Joe Jackson at .356). DiMaggio's streak is the most extraordinary thing that ever happened in American sports. He sits on the shoulders of two bearers--mythology and science. For Joe DiMaggio accomplished what no other ballplayer has done. He beat the hardest taskmaster of all, a woman who makes Nolan Ryan's fastball look like a cantaloupe in slow motion--Lady Luck.
Sorry, DiMaggio. Looks like your streak was also a by-product of random chance. A new, more comprehensive statistical simulation has demonstrated that even a 56 game hitting streak will naturally emerge from the history of baseball:
Using a comprehensive collection of baseball statistics from 1871 to 2005, we simulated the entire history of baseball 10,000 times in a computer. In essence, we programmed the computer to construct an enormous set of parallel baseball universes, all with the same players but subject to the vagaries of chance in each one.
To tease out the meaningful lessons from random effects (fluky streaks that happen by luck), we redid the whole thing 10,000 times. In each of these simulated histories, somebody holds the record for the longest hitting streak. We tabulated who that player was, when he did it, and how long his streak was.
And suddenly the unlikely becomes likely: we get a very long streak each time we run baseball history. These results are shown in Figure 1. The streaks ranged from 39 games at the shortest, to a freakish baseball universe where the record was a remarkable (and remarkably rare) 109 games.
More than half the time, or in 5,295 baseball universes, the record for the longest hitting streak exceeded 53 games. Two-thirds of the time, the best streak was between 50 and 64 games.
In other words, streaks of 56 games or longer are not at all an unusual occurrence. Forty-two percent of the simulated baseball histories have a streak of DiMaggio's length or longer. You shouldn't be too surprised that someone, at some time in the history of the game, accomplished what DiMaggio did.
- Log in to post comments
"Tooney made 46 percent of all of his shots. After hitting three shots in a row--a sure sign that he was now "in the zone"--Tooney's field goal percentage dropped to 34 percent."
Um, doesn't that mean the shot following the misses is affected by the results of the previous shots? This is clearly in opposition to the thesis (at least for this player).
You write that the same thing drives the stock market and shooting streaks: random chance. But while streaks might be random, overall shooting percentage isn't - with learning, practice, and talent, some individuals have higher shooting percentages than others. That doesn't explain streaks - but it does suggest that Toney's going "cold" (for instance) is a reversion not to the mean, but to his overall shooting percentage, which is higher than would be suggested by random chance.
I'm really enjoying your blog - thoughtful and thought-provoking. Thanks!
Interesting, but *something* bothers me about the arguments. So far all I get is that 'intuitive' feeling that usually means that my (subconscious?) math ability has spotted something, now I need to figure out what it is. In this case I think it is going to be some unstated assumption they are making in the studies.
The 'hot hand' or 'streak' is also seen in poker, where it is a 'rush' or being 'hit with the deck' when someone is winning a lot (especially when the wins are due to catching low-probability cards or combinations of cards to do it) or getting 'cold decked' when the opposite happens. But good poker players tend to be ultimate pragmatists, playing the odds when they are in their favor because they know that, in the long run, they will make money doing so.
lol!~
a perfect example of why science should stay out of religion (and religion out of science)! ;)
science vs. religion, logos vs. mythos, cold hard numbers vs. a beloved slugger's art.
being human is messy. data on human beings is far from complete.
i think it's great fun to run numbers and pull stats, but sometimes a game is just a game, with its own rituals, language, mystery, and art.
paraphrasing william james, the "hot hand" is as real as one needs it to be.
cheers!~
You might want to check into some of the new research on this topic. The hot hand phenomena might actually exist.
it still seems to me that when my son makes a basket, it increases his chances of making another, because his confidence is lifted.
I assume that the Cubs losing streak is also a common occurrence in those 10,000 baseball history simulations.
It doesn't matter if the statistics are real or not. When you're on the court, the "hot hand" is really about confidence. Even if the numbers aren't real, the feeling of being on a role is very real, and confidence helps you win basketball games. So if you're watching a game, and a player hits two or three in a row and starts to look like he's feelin' good about himself (or herself), please don't yell down, "Hey, statistical studies have shown that the odds of you missing a shot are going to increase." They'll throw you out for bringing math where it isn't welcome. And you'll be booed.
I've never understood this argument against the "hot hand." Let me try an analogy: Suppose a scientist says, "We used to think that height was affected by genetics, but statistical analysis of the population has shown that the number of 7-foot-tall men is no more than would be expected from pure chance. Thus, height is just a matter of pure chance."
Valid argument? Yet that is the equivalent of what I read in Tversky/Gilovich, whose point was that "streaks" happen no more often than predicted by chance. To which the answer is, so what? Lots of things can be described by a typical bell curve, but that doesn't mean that genetics, basketball ability, etc., doesn't exist.
For a given shooter, isn't an expect probability (shooting percentage) required in order to calculate if clusters / streaks / hot-hands exist, right? And using that player's avg. shoot percentage over the year isn't the proper thing to do, because what if that average actually undergoes real hot / cold fluctuations over the year (or even game).
We know that tossing 6 tails in a row w/a coin is due to pure chance b/c we can reference that streak / cluster of tails against the probability of throwing head or tail on each toss (.5).