If you find yourself in the condition of being unavoidably female, and you aren't willing to undergo a sex change operation, then your best publication strategy may be to hide the XX affiliation.
The title of a recent publication on this issue is self-explanatory: "Double-blind review favours increased representation of female authors" by Budden, Tregenza, Aarssen, Koricheva, Leimu, and Lortie. Sadly, as the authors note, double-blind review is "rarely practised". If your name screams out "woman", you may be better off with an initial.
Of course, this is nothing terribly new; just a very nice and thorough documentation of the effect in one journal, Behavioral Ecology. The authors observed a 7.9% increase in female first-authored papers after double-blind review was implemented at BE. That's an increase three times greater than the increase in female ecology graduates across the same time period of the study. No similar increase was observed in comparable journals that continued with standard review practices. The authors also note that the double-blind review process may eliminate bias against less well-established researchers, where women in the field are concentrated.
This article has been discussed elsewhere in the blogosphere, including over at Grrl's place and Nature's Peer-to-Peer blog.
Grrl comments:
Of course, this is only the tip of the iceberg, as they say. I would like to point out that this bias against women not only affects publishing scientific papers, but it also applies to the review, grading and funding of scientific grants.
Indeed, a 2005 report, Gender Differences in Major Federal External Grant Programs makes just this point. No differences were found at NSF and USDA, but NIH was another story. Even after allowing for all sorts of confounding factors - age, academic degree, institution, grant type, institute, and year - "the gender gap is still 17 percent, which means that women still receive only 83% of what men receive when it comes to grant writing."
Wenneras and Wold showed that females had to be 2.5 times more productive as male applicants to be judged equally competent in a fellowship competition. Virginia Valian extensively documented how gender influences evaluation in her book Why So Slow?
It's not like any of this is news. We just keep collecting more and more data delineating how gender influences evaluation - negatively for women, positively for men.
Peer-to-Peer acknowledges
If referees know the authors' identities, it may leave the latter vulnerable to biases about them or their previous work, their gender, their nationality or their being new to an area of research.
yet goes on to outline reasons for maintaining the status quo:
But the PRC survey supports the contention of Nature and others that identifying authors stimulates referees to ask appropriate questions (for example, differentiating between a muddy technical explanation and poor experimental technique). Knowing author identities also makes it easier to compare the new manuscript with the authors' previously published work, to ensure that a true advance is being reported. And knowing rather than guessing the identities of authors encourages reviewers to raise potential conflicts of interest to the editors.
I am not clear on the mechanism that transmutes knowing an author's name to ability to differentiate between "muddy technical explanations" and "poor experimental technique". What, exactly, about not knowing an author's name keeps you from being able to differentiate thus? Because you know your buddy Joe would never have poor experimental technique, he's just a bad writer - but Sally there, we're pretty sure she screwed up.
And sure, knowing the author's name makes it easier to compare to their previous work - this is exactly the sort of bias toward established researchers that hurts newcomers, i.e. women who tend to be less-established researchers. I think the question of conflict of interest could be adequately addressed after the review is completed.
On whole, these arguments do not impress me as reasons for avoiding double-blind review. But Peer-to-Peer's Maxine Clarke goes even farther: she suggests that double-blind review would be more readily accepted by editors only if it "generated more constructive comments in the minds of editors and authors". How about if it just performed equally well? Clarke acknowledges that this has been shown in studies of 7 medical journals. So what's keeping editors from adopting double-blind review, which we know would reduce unconscious (and perhaps conscious) bias?
Well, it's too much extra work. And reviewers can guess authors' identity in a lot of cases anyway. So why bother? Clarke concludes:
...Nature's policies over the years have generally moved towards greater transparency. Coupling that with the lack of evidence that double-anonymity is beneficial makes this journal resistant to adopting it as the default refereeing policy any time soon.
WTF? this comes shortly after Clarke says
The one bright light in favour of double-blind peer review is the measured reduction in bias against authors with female first names (shown in numerous studies)...
Well, I guess if it only benefits women it isn't "beneficial".
In light of evidence that double-blind review does NOT negatively affect the quality of review, and that it DOES eliminate gender bias, the cry of "greater transparency" sounds hollow.
Since the established powers show themselves unmoved by data detailing gender bias and how it can be avoided, I recommend that you continue your strategy of trying not to be female for the foreseeable future. Initials, not first names, if you want a shot at equitable treatment. Maybe you can put your name on the final, published version of the paper. That's what Maxine Clarke recommends you do: you bend, because the system won't.
UPDATE: I forgot to say, hat tip to reader Christina Pikas for alerting me to the Budden article.
- Log in to post comments
This was interesting to read since Jocelyn Bell Burnell (of Little Green Men and pulsar fame) was visiting our physics department today. Part of what she talked about was women in science and mentioned a study (may have been Wenneras and Wold) where people (both men and women) were more likely to hire John Smith than Jane Smith based on CVs that were exactly the same.
So not only do we need more publications, it is harder for us to get published.
Yeah, and if they do change the rules and women get more publications, they will complain that the system is biased towards women.
Like the British education system now. Today girls do much better in exams (at 16 and 18) than boys, so it must be biased towards them. Ignoring all the years that it was biased towards the boys and still is biased post 18.
One day, one day...all things will be equal. Won't they?
IMO Nature's reviewing (even more so than most journals) seems to be much more based on who you are than what you write, so it's hardly surprising that they are hostile to any blinding of the names - perhaps surprising that they will admit it openly though.
Perhaps this can be considered a faint ray of light - following the BE paper, the journal I work for is considering switching to double-blind. Unless someone at our editorial board meeting in March can come up with a compelling reason (such as Elsevier telling us that EES simply can't cope with double-blind reviewing), we're likely to switch across later this year.
Now, we're no Nature - we're a midfield journal in a comparatively small field - but the inertia on our side is pretty intense, and I'm expecting some not insignificant resistance from the editorial board along the lines of "But we've always done it this way", "But everyone else uses single-blind" and "Are you saying I'm sexist/racist/a bad reviewer?" (suggestions for countering these are happily welcomed). Switching peer review systems is also a damn big logistical challenge for us, but it's doable, and it's worthwhile.
If it works for this journal (and I can't see why it wouldn't), then I can carry that forward for the rest of my career. I can point out to other people in the industry that it's been a success for us, and ask the editorial training programs I'm connected to to promote double-blind as the more egalitarian option. It's a slow, incremental, bottom up approach, but eventually it could build into something worthwhile.
Even when a journal's policy is double-blind review, it doesn't always follow through in practice, especially when manuscripts are so easily circulated online. At least twice in recent years, journal editors have sent me a pdf of an article to review, that had the author's name clearly visible. When I asked about that, the reply was, essentially, "Oops, disregard that one, we'll send you the right version." Uh-huh. And "the jury will please disregard that dramatic confession from the defendant." Yeah, sure they will.
I was very unimpressed by that article on double-blind peer review. See comment #15 at:
http://scienceblogs.com/grrlscientist/2008/01/women_science_and_writing…
As you know many other variables are confounded by gender including seniority, faculty location, etc. They made no attempt to see whether these also explain the different. I also propose a better paper method in the linked comment.
While I recognize, gender biases exist, I'm very skeptical that gender vs. knowing who people are is the cause of a bias. As for Nature, I can't see how they could do a double-blind system even if they wanted. The editors make the first cut after only skimming the papers and only a fraction even get sent out to full review. How could you blind the editors who are supposed to select the reviewers? I guess it's possible to blind them until they decide to send out to review and then unblind them, but they still have the final say on accept or reject and they are then unblinded.
Nature definitely has serious bias issues towards republishing the same authors. This hurts women, but it isn't a gender bias.
I generally think there are enough benefits to unblinded review (yes knowing the authors' history and being able to look at past papers does put a reviewed paper in better context) and until I see clear biases caused by not blinding, I'd be hesitant to change anything.
bsci, I read your comment over there. Reply: The paper compared papers with female FIRST authors to papers with male FIRST authors. In most disciplines, first authors are either the most senior scientist (rare) or one of the junior scientists (more common). In both cases, people of similar rank are first authors in all papers. Your comment might have a point in the second case, because "senior" can mean more or less senior, but it is clearly unfounded if this field has the convention of putting the senior author last, because the first author is ALWAYS a junior person regardless of gender. Finding out which convention holds in this particular field is actually your business, because it's your argument that is based on it.
BSCI, gender bias in review/evaluation is EXTENSIVELY documented. Please take a look at Why So Slow? by Virginia Valian. It's not like the jury's still out on this one.
Schlupp, I saw several confounds in that article. Another one is that their two samples were paired with an increase in the number of women in academic science (since the double-blind sample was later). This could be better done using the list of submissions, but the question is a journal brave enough to do some internal research on this and publish it.
Zuska, As you know, the biases vary in magnitude and frequency from field to field. I know biases are better documented in some areas than others, but I've yet to see something that clearly shows gender bias in modern (last 20 years) peer review. I don't have the Valian book, but does she reference any articles on publication peer review bias that don't have the same confounds and this BES study?
bsci, your statement is simply not true: They did not only compare before and after, but also compared publication data for two journals - one with blind review and one with conventional - for the SAME time span, see their table 1. Result: female first authorship increased in the journal that changed towards with blind review and not in the other.
I agree that double-blind paper reviewing ought to be implemented at most, and especially at elite, journals. It just makes sense, although as a reviewer it's a little annoying. Arguments against it are just weak. I have submitted manuscripts with initials, only to have my full information show up on the header sheet anyway, so the journals would have to make an effort. The fact that the head editors see full author information is probably unavoidable, but minor compared to the nitty gritty review.
The report about gender differences in NIH funding does say there is a 17% difference in amount funded; however, they did not have information on amount _requested_. So it's possible women ask for less (which would be in line with lots of research suggesting women demand less; perhaps run smaller groups; etc.) There remains a difference in the rate of funding, corrected for other factors, of 28.7% for women, 31.3% for men, meaning women get funded at ~91% the rate of men. Not 100% but hey, considered everything else, not so bad.
considering the research that pretty clearly shows gender bias in ranking of fellowships, CVs, etc, suggests to me the review panels are doing a pretty good job of countering a tendency of bias in favor of men.
Schlupp,
Look at the actual data in table 1. The female:male ratio increased over time for BOTH journals.
For BE (the double-blinded journal) it increased from .38 to .58. For BES, the single blind journal, it increased from .45 to .51. There was a higher ratio of names with unknown gender in BES. The greater # of unknown names would lower the p-value for calling the change significant in the second journal.
I think this paper presents interesting observational data, but it far from convincing in its conclusions. More work can and should be done to show this better.
bsci, I did look at the data in detail: Yes, it increased for both journals. And no, statistics can never be 100% convincing. (As a Bayesian by training and inclination, I completely agree.) So no, you do not have to accept 'statistically significant' as 'they got a result'. BUT, and this is important: You cannot flatly assert that they had no results, as you did in your comment. You would have to write: "While they did have statistically significant results, these results do not satisfy my standards for significance for the following reasons...."
I also agree with you that the "unknowns" can be important: In a journal with non-blind review, women might prefer to use only their initials and thus fall into the "unknown" category. However, the unknowns increased by 48% for BE (double-blind) vs. 40% for BES (conventional), i.e., the change of the unknowns does not at all support the hypothesis I just put up.
Schlupp,
I agree with you and I don't think I ever said they had no results. In the "Living the Scientific Life" thread, I wrote, I find the referenced paper very weak" in comment 15.
In comment 27, I wrote, I posted a detailed criticism above of the paper. They present some data that might be evidence of a bias which needs to proven or contradicted by additional studies. Just because a paper isn't the final word on a topic isn't a reason to call the entire study wrong.
First of all, I do realize that there exists a real gender bias in science. It is for that very reason that I think it is necessary to be extremely careful about how we attempt to document it.
That being said, I find several problems with many of the studies cited. Actually, many of the studies themselves acknowledge their weaknesses, but this uncertainty isn't reflected by those who report the data. In other words, these studies almost certainly do reflect an actual bias, but the numbers themselves are highly misleading.
For instance, the double-blind study compared numbers before and after the implementation of a double-blind review process for a single journal. That's a sample size of 1, which is insufficient to determine the efficacy of double-blinding in the review process.
They compared the double-blinded journal with another non-blinded journal, and found that the non-blinded journal did not experience the same rise in numbers. But the non-blinded journal was not on the same topic, although they are relatively closely related.
A separate issue in the same study is the actual percentage increase in papers accepted by female first authors. Although they pointed out other confounding factors, they did not attempt to include them in the analysis. Specifically, they mentioned that 7.9% increase is 3 times the recorded increase in female graduate students. The obvious next step to take is to subtract that from the 7.9% estimate (actually, you'd have to do more complicated statistics than that to account for uncertainties in the data). The number would be somewhere around 5.4%.
In the end, although the study may have documented a real effect, it is highly problematic to draw conclusions from that study about the source of that effect.
bsci, sorry, I did not read all your comment over there. My impression was based on your sentence here "Another one is that their two samples were paired with an increase in the number of women in academic science (since the double-blind sample was later)." Which gave me the impression that you had overlooked the comparison between the journals.
Zuska,
You said:
One possible reason why knowing the identity of the author would be helpful is if the author was foreign. Having a native command of the language is huge benefit when trying to communicate complicated ideas. If the reviewer knew that the author might not be as comfortable in English, then they could take that into account when evaluating the paper.
I find it a slight bit troubling that you immediately assume that reviewers would routinely use the author's identity to actively discriminate based on gender. Unless you have some actual data, neither of us can do any more than speculate as to why knowing the identity of the author could be beneficial.
It is worth mentioning that I think a double-blinded review process sounds like a good idea. There are other ways to account for non-native papers, such as allowing the reviewer to only see that English is not the author's first language (although that perhaps opens the door for potential racial discrimination).
My opinion doesn't mean much, however, since I have not been on a review board and don't have the experience necessary to debate the matter. For that reason, I am extremely cautious in evaluating studies that support one side or the other.
There's nothing about an author's name that would tell you how well they speak English, unless you assume that ever name that's a bit odd belongs to someone who doesn't speak English well. Which is not a valid assumption.
I do not assume reviewers automatically acively discriminate on the basis of gender. What I am talking about is bias that enters in without our realizing it, unconscious bias, because of gender schemas we are all subject to, male and female alike.
http://xkcd.com/385/
What I take from this study is that it finally sheds some light on an unspoken evil that is befalling women (of course, it caught me off guard at my last publishing job.) Women DO participate in discrimination against other women. I've found this to be personally painful, as no one believed me when I came forward with situations at work where women were biased against other women.
Women are not saints; we are created in the same culture, after all, that men are raised in. So why is it so surprising that we have low self-esteem, so that we transfer that to our assessments of other women's work? Or that we mimic what men have done to get ahead?
This woman-on-woman discrimination is therefore nearly impossible to root out because women are seen as being automatically anti-discriminatory (and just by including one quota woman in one high place everyone believe things are OK). Each of us needs to work on it within ourselves.
There are reasons to not have double-blinded manuscript reviews. Knowing whether or not the data being presented is novel does require going back to the literature to see whether or not it is novel. I've sat in on a review session where the author had extensively taken from previous papers he had published and was trying to present the data as 'novel.' This is not entirely uncommon. And for a lot of fields, not knowing the author's identity would require sifting through hundreds of papers in order to determine the novelty of the work (unless you work on the same thing as the author, in which case you probably already know who s/he is).
This is not to say that the gender bias is not real or that double-blinded reviews may not be worthwhile after all, but there is a real reason why knowing the author's identity is worthwhile.
And as far as that Gender Differences in Grant Funding study goes, they did not have a lot of the data for the NIH that they did for the NSF and USDA (such as the research ranking of the institution where the PI works). Again, this is not saying that gender bias doesn't exist (clearly it does, as women were only about 25% of PIs in all the funding applications), but I wonder why there would be a 17% difference in the NIH but not the NSF/USDA? The review process is pretty similar. It may have a lot to do with the fact that the situation in biomedical research is more complicated, in terms of a wider variety of outside funding sources and the gender differences found between medical schools/universities.
Brian, the 17% difference was unexplained by other factors - it's the part that can't be accounted for by known differences, the part that is attributable to gender.
Regarding checking a paper for "novelty": that's not incompatible with double-blind review. You could have one reviewer who only checks for novelty, while the others review blinded for quality. Or, you could have reviewers do the novelty check after the quality review is done and submitted.
sister of physics brothers, good call. I have seen this around and even noticed it in myself. Internal slash-and-burn of the rainforest of prejudice is my way out, but I hear heartening tales of one referee demanding genderblinding on all the papers she reviews. Just one, in an entire university, but it is true.
After reading this, I may have to buy her a cake.
Well, it's all attributable to gender, right? (I'm not being sarcastic here). But it is odd that there was no difference in the NSF/USDA funding amounts, whereas there was with the NIH, indicating that there may be something different in biomedical research. The fact is, the NSF is pretty much the only funding mechanism for a lot of the basic sciences (I don't know so much about the USDA). In biomedical research, however, there are a *lot* of alternative funding mechanisms. Pretty much every disease has its own foundation that supplies researchers with money, and there are things like the Bill & Melinda Gates Foundation.
Women applying to the NIH are asking for significantly less money relative to men, whereas that is not so much the case with the NSF/USDA. Are women seeking out alternative funding mechanisms (the reapplication rate among women is also significantly less than men)? Or is it simply that, given the time constraints on women (family, etc.), do they have smaller labs and thus not ask for as much money?
The researchers in that study couldn't control for a number of things, because the NIH did not supply the data. For example, the research ranking of the institution, and the amount that women received relative to what they asked for. If women were indeed receiving substantially less than what they asked for, relative to men, I think that that would be an example of a bias in the applications process. However, simply receiving less than men is not an example of a bias. Given that NSF/NIH/USDA all treat the grant application process pretty similarly, and only in the NIH was a difference found in funding rate, I don't think that that necessarily speaks to a funding bias specifically in the biomedical field. This is not to make light of the problem - women are, after all, receiving substantially less than men. But I think that that is a result more of the systemic bias in the biomedical field generally, where there are a lot more 'megaprojects' and very large labs relative to those that are receiving NSF/USDA funding. And the constraints on women I think would make it difficult for them to head the same number of these 'megaprojects'/labs as men. And it may be a result of a 'good old boys' network that I think is pretty pervasive in medical schools, which may discourage women from entering medical schools, where a lot of the NIH funding goes (especially if you have an MD or an MD/PhD).
As far as the novelty side of the paper review process - for a lot of the higher-tier journals, that's the key to getting a paper published. How new and exciting the work is is the difference between a second or third (or fourth) tier journal. The technical side of things is not so important - a poorly performed study is not going to be published in second-tier journals like Blood or Journal of Experimental Medicine (most of the time), irregardless of how interesting the findings are. And the fact is, without knowing who the author is, it becomes very difficult to really check how novel the work is (not so much for the first-tier journals, as only the really obviously new stuff is published there). Is it worth the gender bias that's apparently creeping in? Probably not, but a lot of the higher tier journals are never going to adopt a double-blind review process for this reason, and pretty much this reason alone (all the talk about "conflicts of interest" notwithstanding).
I wonder if there was really such a difference between the NIH and the NSF/USDA results. My experience is that a male PI with a team of co-investigators is much less likely to be discriminated against (or at least receive patronising reviewers' reports) than a female PI with a team of co-investigators. Hence, while dividing the funding received between all the investigators on the grant (as I understand was the method applied by the study in the case of NSF/USDA grants where coinvestigator details were available) may address the question as to whether there is gender equity in terms of total funds allocated, it doesnt address what I think is the most important question: namely, is there discrimination against women in terms of getting their projects funded if they are the named PI. In other words, if for all agencies (ie NSF and USDA as well as NIH) they had considered only the PIs and not the co-investigators, would they have actually found the same consequences for women as they found for the NIH (where coinvestigator details were not available so they could only consider the PI)? Or did I miss where they reported this?
One thing that might be worth mentioning is that when you submit a paper electronically, you're usually asked to enter your first name in a compulsory text box. So it's not always possible to conceal your gender from an editor by using initials, at least without making it obvious that you're hiding.
I just had some more thinks about this and blogged about it.
The link for Miss Prism's post is here.
I have seen this around and even noticed it in myself. Internal slash-and-burn of the rainforest of prejudice is my way out, but I hear heartening tales of one referee demanding genderblinding on all the papers she reviews.