I continue to struggle to avoid saying anything more about the Hugo mess, so let’s turn instead to something totally non-controversial: gender bias in academic hiring. Specifically, this new study in the Proceedings of the National Academy of Science titled “National hiring experiments reveal 2:1 faculty preference for women on STEM tenure track” with this calm, measured abstract that won’t raise any hackles at all:
National randomized experiments and validation studies were conducted on 873 tenure-track faculty (439 male, 434 female) from biology, engineering, economics, and psychology at 371 universities/colleges from 50 US states and the District of Columbia. In the main experiment, 363 faculty members evaluated narrative summaries describing hypothetical female and male applicants for tenure-track assistant professorships who shared the same lifestyle (e.g., single without children, married with children). Applicants’ profiles were systematically varied to disguise identically rated scholarship; profiles were counterbalanced by gender across faculty to enable between-faculty comparisons of hiring preferences for identically qualified women versus men. Results revealed a 2:1 preference for women by faculty of both genders across both math-intensive and non–math-intensive fields, with the single exception of male economists, who showed no gender preference. Results were replicated using weighted analyses to control for national sample characteristics. In follow-up experiments, 144 faculty evaluated competing applicants with differing lifestyles (e.g., divorced mother vs. married father), and 204 faculty compared same-gender candidates with children, but differing in whether they took 1-y-parental leaves in graduate school. Women preferred divorced mothers to married fathers; men preferred mothers who took leaves to mothers who did not. In two validation studies, 35 engineering faculty provided rankings using full curricula vitae instead of narratives, and 127 faculty rated one applicant rather than choosing from a mixed-gender group; the same preference for women was shown by faculty of both genders. These results suggest it is a propitious time for women launching careers in academic science. Messages to the contrary may discourage women from applying for STEM (science, technology, engineering, mathematics) tenure-track assistant professorships.
Especially not those last two sentences. Nope, not provocative in the least…
I actually knew this one was coming, having been alerted to it by a colleague when I last wrote about gender in academic hiring. So, what is the deal with this?
Well, as is often the case, this is a paper study– they sent a bunch of faculty in four different fields (engineering and economics, where women are badly underrepresented, and biology and psychology where they’re not) “hiring committee reports” on sets of three imaginary candidates for a faculty position and asked them to rank the three candidates. Two of these were the experimental test, the third a “foil” deliberately designed to be somewhat weaker than the other two; the “foil” was nevertheless chosen first in about 2% of the tests, which contrary to the popular saying, lets you precisely account for tastes…
The test candidates were drawn from a set of 20 packets randomizing a bunch of features, but all designed to be equally strong. The most important random feature was the pronoun used for the candidate (some got male pronouns, others female; if I read this correctly, none got names), but they also varied “lifestyle” factors: marital status, number of children, past parental leaves (these were mentioned as spontaneously offered by the candidate in the “report” from the “search committee”). One other important feature was that they varied the specific adjectives in the letters of recommendation, describing some candidates with words most commonly associated with letters for men (“analytical, ambitious, independent, stands up under pressure, powerhouse”) and others with words generally associated with letters for women (“imaginative, highly creative, likeable, kind, socially skilled”). They swapped pronouns on both sets of letters, so that some “women” got “male” adjectives, and some “men” got “female” ones.
Each test got one fictitious woman and one fictitious man from the experimental set to rate; the “foil” was always a man. Taking out the tiny fraction of oddballs who picked the foil, they found that the paper reports identified as women were preferred by a two-to-one margin, across the full range of conditions. The only sub-group of faculty who didn’t have a strong preference for the woman in the test sample were male economists, who were slightly pro-male (55%-45%) in contrast to their female colleagues (68%-32% in favor of the woman).
In follow-on experiments they try to sort out a bunch of the individual factors involved– presenting faculty with full fake CV’s rather than just “search committee reports,” varying just the family status of the candidates, and asking their test faculty to rate just a single candidate rather than doing a head-to-head comparison. There are a few small quirks– this is, after all, social science– but the general result is very consistent: when asked to evaluate this set of paper candidates, the faculty in their sample show a strong preference for hiring women.
This runs counter to a lot of conventional wisdom, and just as with the abstract, the concluding discussion features some language that borders on the combative regarding the current state of academic hiring, including the sweeping claim that:
Our experimental findings do not support omnipresent societal messages regarding the current inhospitability of the STEM professoriate for women at the point of applying for assistant professorships (4–12, 26–29). Efforts to combat formerly widespread sexism in hiring appear to have succeeded. After decades of overt and covert discrimination against women in academic hiring, our results indicate a surprisingly welcoming atmosphere today for female job candidates in STEM disciplines, by faculty of both genders, across natural and social sciences in both math-intensive and non–math-intensive fields, and across fields already well-represented by women (psychology, biology) and those still poorly represented (economics, engineering).
From a slightly more impartial perspective, though, there’s one obvious way to reconcile these results with the picture of academic hiring as highly biased, which has to do with a difference in experimental design. The much-cited big paper-resume study showing bias in favor of male students was deliberately designed to use kind of marginal candidates in their fake resumes– good-but-not-great grades, and so on. This study, on the other hand, uses fake candidates who are carefully designed to be equally strong— that is, they present the faculty in their sample with a difficult choice between two excellent candidates.
The obvious way to reconcile these is to push the gender bias back to an earlier stage in the process. That is, to say that while this study may show a strong preference for hiring women who are rated as exceptionally qualified, bias in letter-writing, interviewing, etc. mean that very few actual women are rated this highly in real hiring situations, so the direct head-to-head comparison they test here only rarely comes into play in the “real world.”
They sort of attempt to address this with the adjective thing I mentioned earlier (though I don’t see any detailed breakdown of those results in there). The probably more useful counter (or, I guess, counter-counter-argument) is in the data they give about actual hiring rates, namely that while women are less likely to apply for faculty jobs, they get hired into assistant professor positions at about the same rate that they get Ph.D.’s. That would allow you to reconcile this study’s strong preference for highly qualified women with previous studies’ tendency to under-rate women. The former basically undoes the latter, ending up with hiring that is mostly neutral in a statistical sense.
Which, I guess, is the next angle for a paper-resume study: put together packages that under-rate the imaginary female candidates, and see what happens then.
In terms of allocating grant funding, though, it might be better to put together a study of why male economics professors are such assholes relative to their colleagues in other fields…