I've seen a lot of reshares of this report about the long-term effect of gender bias in elementary math, which comes from an NBER working paper about a study of Israeli schools. The usual presentation highlights one specific result, namely that on a math test graded by teachers who knew the names of the students, boys outscored girls, but a blinded test saw girls outscore boys.

This sounds pretty damning, but also kind of puzzling-- is there really that much room for partial credit in elementary school math? Looking at the actual paper (which you can get emailed to you if you have a .edu address) clears things up a bit. The relevant scores are on two very different tests: the blind-graded one was a national exam given to fifth-graders, and the non-blinded one was a mid-year exam given in sixth grade to students in a particular district (they have the same teachers for fifth and sixth grade). That's a lot less damning than the initial impression-- these are two very different tests, graded by entirely different sets of people. The paper doesn't go into any detail about the format and content of these tests, which seems like a pretty important question-- it would be interesting to see some follow-up from people in math education about how directly comparable these are.

The other important caveat to the story as frequently presented is that the gender effect varies widely. It's not that every teacher is systematically "over-assessing" boys-- in fact, they say that the math results are, on average, gender neutral. But they have "quite a large heterogeneity" among teachers-- there's a truly awful set of histograms at the very end of the paper showing the distributions (seriously, did they get one of the fifth-graders in the study to make these ugly graphs?). This is, in fact, the thing that lets them get to the meat of the paper, which is a study of the long-term effects. Because there's a wide variation among teachers they can look at the difference between fifth-graders who had a teacher whose scores show a big gender gap, and those whose teacher was more neutral.

And *that* genuinely is bad (and, incidentally, strengthens the case that what they're measuring with the score gaps is something real). They report a significant and long-term negative effect on girls who wound up with teachers who have larger gender gaps-- they're less likely to take more advanced math, less likely to go into STEM subjects, etc. This is an important and interesting result, but kind of gets buried under the "math teachers grade boys higher than girls" stuff.

So, what do they know about the teachers? They report that large gender gaps are more likely for older (they use a dummy regression variable for teachers over 50), un-married teachers. Which is not too terribly surprising. They also mention in passing that most of the math teachers in their sample also teach Hebrew, which might be a part of the problem, depending on how much weight those receive-- someone who really wants to be teaching Hebrew but is stuck also teaching math might well be doing a lousy job on the math part.

So, anyway, a slightly less damning indictment of elementary school teachers than it might initially appear, but also a clear demonstration of a very real problem. Biased math instruction starts early, and has long-lasting effects.

(I did a bunch of Twittering about this this morning, but that's a terrible medium, and it was very early, so I figured I should type up something longer in a less ephemeral location.)

- Log in to post comments

most of the math teachers in their sample also teach HebrewWait--is this district in the US, or Israel? Having Hebrew teachers would not be unusual in Israel, but it's not something you would see many places in the US. And it suggests a confounding cultural issue. If the results hold up, it would be very bad indeed, but I'd like to see a confirmation with a more representative US teacher sample.

The schools studied are in Israel; I believe it's the Tel Aviv city school district, but don't have time to check that. They looked at national tests versus local tests in math, Hebrew, and English classes; math tests showed a gap between local and national scores in favor of boys, English a gap in favor of girls, Hebrew no gap.