Called Strikes and Race: Et tu, Baseball?

i-4a29bdfc7fac0eb655931b1abe23cba2-ph_400089.jpgA few months ago, I posted about a study showing implicit racial bias in NBA referees' calls. Now it's baseball's turn, because yesterday reports of study by Parsons et al.1 that shows analogous results for home plate umpires began popping up all over the media.

The study is pretty straightforward, though the data analysis must have taken forever. I'll let Parson's et al. tell you what they did:

There are 30 teams in Major League Baseball, with each team playing 162 games in each annual season. During a typical game each team's pitchers throw on average roughly 150 pitches, so that approximately 730,000 pitches are thrown each season. We collected pitch-by-pitch data from for every MLB game in the three years 2004-2006.2 For each pitch we identify the pitcher, pitcher's team, batter, batter's team, pitch count, score, inning, and pitch outcome. We classify each pitch into one of seven mutually exclusive categories: Called strike, called ball, swinging strike, foul, hit into play, intentional ball or hit by pitch. We supplement each pitch observation with game-level information from box scores including the stadium name, home team, away team, team standings, and the identities and positions of all four umpires. In addition, for each pitcher's appearance in each game we collect the exact number of innings pitched and the number of allowed hits, walks, strikeouts, homeruns, runs and earned runs. Finally, for each starting pitcher in each game we collect the game score, a composite index designed to summarize a starting pitcher's performance. (p. 4)

Those three seasons yielded a total of 2,120,166 pitches. I feel sorry for the poor undergrads who had to sit and watch all of those! Forty-seven percent of the pitches were excluded because the batter swang or was hit by the pitch, or the pitch was intentionally thrown for a ball, leaving 53% of 2.1 million pitches to analyze. In addition to looking at whether each pitch was a ball or a strike, they also coded the race of the home plate umpire, batter, and pitcher. Using a regression model (see the basketball post for a brief description of regression) that takes only these into account, they find that when the umpire and pitcher are of the same race, the umpire is about .34% more likely to call a strike. That's only a little more than half a pitch a game, but over all of those pitches, and when you take consider that it's only one source of potential discrimination, it's a potentially important effect.

Interestingly, the effect of same-race umpires goes away when the umpire's calls are evaluated by QuesTec's computerized system (only 35% of MLB ballparks had the system between 2004 and 2006), when crowds were larger, and when there were either two strikes, three balls, or both (meaning that the call could potentially end the bat). Parsons et al. argue that these results suggest that the cost of a decision will affect the influence of implicit racial bias. Specifically, the data implies that costlier decisions will display less bias, a finding that could have a wide range of implications in research on implicit biases.

1Parsons, C.A., Sulaeman, J., Yates, M.C., &Hamermesh, D.S. (Unpublished Manuscript). Strike three: Umpires' demand for discrimination.

More like this

Matsuzaka looked impressive in his MLB debut. He had 10 strikeouts in 7 innings and only threw 108 pitches. I'm still not convinced he's worth $103.1 million, but the weak Kansas City lineup looked pretty dazed and confused. Matsuzaka's genius, I think, is to create as much batter uncertainty as…
Ah, yes, a real game (kidding, Scrabble people). If you've watched many baseball games or baseball movies, you know that one of the things that makes for a successful hitter is the ability to predict what the next pitch will be. Is it going to be inside or outside? Will it be a fastball or a…
Found some Koufax footage. About halfway through this short clip he Ks Mantle, looking, and a bit later, in the dark footage toward the end, is a good strip of him throwing the devastating curve. Note there the emphatic downward motion of his shoulder -- which brought down his hand the faster,…
On the way to get SteelyKid from day care last night, I flipped on ESPN radio in the vain hope of getting a baseball score, but wound up listening to former Mets manager and freelance jackass Bobby Valentine talking about how difficult batting is, which included the statement: And the whole thing…

A third of one percent? Half a pitch a game? That's nothing. Are you (or the researchers) seriously suggesting that umpires are so racially biased against minority players that they ring them up to the tune of one more strike every other game? That's ridiculous. It's either the most subtle (not to mention mild) form of racism every discovered, or it's an innocent statistical anomaly. Surely it's the latter.

eh? Less than half a pitch a game? i.e. about 1 pitch every second to third game?
I'd like to see the significance tests on that. Since they ignored all the other data they collected, which could also be correlated, this is actually showing pretty unbiased umpiring really. I mean, whether the ump was insulted by the manager the last inning might have something to do with things too, and at a one pitch every 2 or 3 game level, that correlation would be very confounding, to my mind anyway. That does look like an upper limit on what would be visible bias.

A couple things: the difference is, of course significant, but with a sample size like that, any difference would be. But it's important to note that the difference is collapsed across races. It's actually higher for some.

But the most interesting finding is that the effect goes away in high pressure situations.

Chris, just a quick question - isn't the effect going away in costlier decision situations associated with a lack of power (because of, I presume, a radically lower number of observations)...?

Whatever the case, interesting study.

I don't claim to understand the statistical methods used to show this bias. I think the most interesting part of the paper is figure 4 (, which looks like it says that minority pitchers win 9% more often when there is an umpire of their ethnicity. That's really shocking.

Here's my first question for people who know more about statistical methods than I do. Is 8 minority umpires too few to compare to 85 white umpires and find significant results? They report including 5 black and 3 hispanic umpires. I know they looked at millions of pitches, but they probably looked at about 10 times more pitches called by white umpires, which seems significant, especially when we're talking about less than one-half percent.

Second question: If there was a subtle bias, why would it not show up with batters as well? Why would an umpire unconsciously hurt or help a pitcher yet not do that for a hitter? I guess that's less of a statistical question than a suggestion that I am mystified by the analysis.