Implicit Racial Bias in NBA Referees

Even though study after study has shown that implicit race bias is pretty much ubiquitous in American society, I'm still occasionally surprised when a study comes out demonstrating it in an area I hadn't previously thought about. That was the case when a friend emailed me a New York Times article on a study of racial bias in NBA referees (the working paper on the study is here. Why did the study authors, Joseph Price and Justin Wolfers, choose NBA referees? Here's their explanation, from the paper:

Our setting provides intriguing insights into own-race bias; relative to social, judicial, or labor market settings, the evaluators in our sample (NBA referees) are a particularly expert group, with substantial experience, continual feedback, and they face robust career incentives to beaccurate. Indeed, NBA Commissioner Stern has claimed that these referees "are the most ranked, rated, reviewed, statistically analyzed and mentored group of employees of any company in any place in the world."

In other words, all the factors that you think might control for racial bias are in place. Furthermore, NBA referees provide a particularly good study group because their assignment to games is essentially random.

What Price and Wolfers did is pretty simple. They first collected three types of data: player and game statistics using box scores from every regular season NBA game played from the 1991-1992 season to the 2003-2004 season. During that period, there were 29 NBA teams (there are 30 now, but the Charlotte Bobcats were added in 2004), two of which started playing in the 1995-1996 season (the Toronto Raptors and Memphis Grizzlies), with each team playing 82 regular season games. According to Price and Wolfers, this yielded more than 250,000 "player-game observations." Next, they classified the race of NBA players using several sources, and NBA referees by using press photographs. As an initial analysis, they then looked at the number of fouls white and black players get as a function of whether the majority of officials (there are 3 in every regular season game) were white and black. They found that black players were called for about the same number of fouls per 48 minutes (the length of an NBA game) regardless of the race of the majority of the officials, while white players were called for significantly fewer fouls when the majority of the officials were white than when the majority was black.

Since they'd found a difference in this initial analysis, they decided to do a more complex analysis using regression. What regression analysis does, in essence, is measure the relationship between several variables and one the measure you're interested in (in this case, the number of fouls called on players). This allows you to "control" for the influence of some variables so that you can focus a few target variables (in this case, the race of players and officials) on that measure. For their analysis, Price and Wolfers included control variables like the position of the players (guard or forward), the players' height and weight, whether the player was an all star, whether the player was a starter, whether the team was in contention to make the playoffs, along with the stadium where the game was played (including a variable for whether a player's team was the home or away team). This analysis showed that controlling for several potentially relevant factors, the number of fouls called on black players increased between 0.12 to 0.21 (or 2.5-4.5%) when the the number of white referees increased from 0 out of 3 to 3 out of 3.

Subsequent regression analyses focusing on statistics like points scored found that as the number of "opposite race" referees increased, players' scoring, blocks, and steals per game decreased, while their number of turnovers increased. How important are these differences? Based on their player-level analyses, Price and Wolfers calculate that "a team's winning margin would rise by up to half a point if they could simply change the race of a player so that it matched that of the refereeing crew" (p. 14). Over the course of an 82 game season, that difference could mean a change in the win-loss outcome of a game or two, which, in a tight playoff race, could have a big impact on the outcome of a team's season.

Once again, then, we learn that racism is everywhere. It shouldn't be surprising, I know, but judging by the fact that this as of yet unpublished study made it into the New York Times, I'm not the only one who was surprised by it. Unfortunately, it's impossible to know how the implicit biases demonstrated by Price and Wolfers analyses (presumably referees are consciously focusing on race) correlate with referees explicit attitudes toward race, because the referees were not studied directly. However, this study does suggest that explicit biases are not necessary to produce real-world effects -- in this case, effects on players' performance and even the outcome of games and seasons. It also shows that it's very difficult, without using fairly sophisticated statistical techniques, to observe the effects of implicit bias in real world situations, because the NBA's referee evaluation procedures presumably failed to discover it.

UPDATE: Statisticians discuss the study.