Some Notes on Gender Bias in Elementary School Math

I've seen a lot of reshares of this report about the long-term effect of gender bias in elementary math, which comes from an NBER working paper about a study of Israeli schools. The usual presentation highlights one specific result, namely that on a math test graded by teachers who knew the names of the students, boys outscored girls, but a blinded test saw girls outscore boys.

This sounds pretty damning, but also kind of puzzling-- is there really that much room for partial credit in elementary school math? Looking at the actual paper (which you can get emailed to you if you have a .edu address) clears things up a bit. The relevant scores are on two very different tests: the blind-graded one was a national exam given to fifth-graders, and the non-blinded one was a mid-year exam given in sixth grade to students in a particular district (they have the same teachers for fifth and sixth grade). That's a lot less damning than the initial impression-- these are two very different tests, graded by entirely different sets of people. The paper doesn't go into any detail about the format and content of these tests, which seems like a pretty important question-- it would be interesting to see some follow-up from people in math education about how directly comparable these are.

The other important caveat to the story as frequently presented is that the gender effect varies widely. It's not that every teacher is systematically "over-assessing" boys-- in fact, they say that the math results are, on average, gender neutral. But they have "quite a large heterogeneity" among teachers-- there's a truly awful set of histograms at the very end of the paper showing the distributions (seriously, did they get one of the fifth-graders in the study to make these ugly graphs?). This is, in fact, the thing that lets them get to the meat of the paper, which is a study of the long-term effects. Because there's a wide variation among teachers they can look at the difference between fifth-graders who had a teacher whose scores show a big gender gap, and those whose teacher was more neutral.

And that genuinely is bad (and, incidentally, strengthens the case that what they're measuring with the score gaps is something real). They report a significant and long-term negative effect on girls who wound up with teachers who have larger gender gaps-- they're less likely to take more advanced math, less likely to go into STEM subjects, etc. This is an important and interesting result, but kind of gets buried under the "math teachers grade boys higher than girls" stuff.

So, what do they know about the teachers? They report that large gender gaps are more likely for older (they use a dummy regression variable for teachers over 50), un-married teachers. Which is not too terribly surprising. They also mention in passing that most of the math teachers in their sample also teach Hebrew, which might be a part of the problem, depending on how much weight those receive-- someone who really wants to be teaching Hebrew but is stuck also teaching math might well be doing a lousy job on the math part.

So, anyway, a slightly less damning indictment of elementary school teachers than it might initially appear, but also a clear demonstration of a very real problem. Biased math instruction starts early, and has long-lasting effects.

(I did a bunch of Twittering about this this morning, but that's a terrible medium, and it was very early, so I figured I should type up something longer in a less ephemeral location.)