Adventures in Ethics and Science

Regular commenter Blair was kind enough to bring to my attention an article from The Globe and Mail, reporting research done at the University of British Columbia, that illustrates how what we think we know can have a real impact on what we can do:

Over three years, researchers gave 135 women tests similar to those used for graduate school entrance exams. Each woman was expected to perform a challenging math section, but not before reading an essay that dealt with gender difference in math.

Of the four essays, one argued there was no difference, one argued the difference was genetic and a third argued the difference stemmed from the way girls were taught in elementary school.

The fourth essay covered the subject of women in art; it has long been held by researchers that simply reminding a woman of her gender will negatively impact her test performance.

Any guesses as to the outcomes?

The women told prior experience determined their math ability got twice as many answers right on the exam as women told their genetics were to blame. …

The women who were simply reminded of their gender also performed worse than those told their was no difference in gender abilities in math.

Let’s repeat that: reminding the test subject that she’s a woman impairs her performance on the math section.

The article calls this phenomenon “stereotype threat”, which seems to amount to living up to social expectations of the group of which you are a member. And this, it seems to me, is one more reason to be particularly careful about research that purports to demonstrate that some group of humans really does display some behavior (less aptitude for math, lower average IQ, what have you) that pretty much fits what people generally believed about that group prior to the research. To break it down:

The expectations of the researchers may influence their experimental design and interpretation of results.
The expectations of the subjects may influence how they behave in the study — again, potentially skewing the data.
The expectations of both researchers and subjects may be influenced by societal stereotypes — maybe consciously, maybe unconsciously.
The results of “scientific studies” (which may well be biased by stereotypes) can serve to strengthen those stereotypes.

Lather, rinse, repeat.

The UBC researchers offer a possible explanation for why certain kinds of “scientific backing” for stereotypes may be especially influential on the behavior of the people top whom the stereotypes are supposed to apply. As reported in the article:

“The influence of genes on behaviour is enormously complex but unfortunately the way these messages are conveyed are in grossly simplified terms,” said Steven Heine, an associate professor of social psychology at UBC and the co-author of the study.

He cited the example of the discovery of a link between genetics and obesity.

“People seem to interpret it as meaning that if I have this gene I must become obese,” he said.

“The relations between genes and behaviour are very complex and unfortunately people do view them in more deterministic terms than they ought to.”

Yup, people seem to be genetic determinists. It’s much more complicated to take account of the fact that even when genes are part of the picture, environment matters — and that social environment is a real part of the environment that ought to be taken into account.

What to do? I don’t think the answer is to ban certain lines of research, but rather to be aware of how easy it is for these lines of research to be biased by what “everyone knows” from the stereotypes. It seems like there ought to be a pretty high burden of proof to establish that bias hasn’t crept into the results. In the meantime, I hope that women and others affected by “stereotype threat” can figure out how to be appropriately skeptical of the claims of researchers to allow them to kick some ass on their math tests.

Comments

#1 Sunny
October 21, 2006

Would this imply that me being reminded that I’m Chinese before I take the GREs would boost the score on my quant sections?
#2 Michael Anes
October 21, 2006

Sunny, the answer is absolutely yes. See Shih, Pittinsky and Nambady (1999), Psychological Science. Good reviews of stereotype threat are found in Ben-Zeev’s work, which is very well done.
#3 gwangung
October 21, 2006

Would this imply that me being reminded that I’m Chinese before I take the GREs would boost the score on my quant sections?

Actually, I think they’ve found this very effect.

Not that this helped a few of my cousins…
#4 Baratos
October 21, 2006

Would it be a good idea to include essays like that at the beginning of any important test? It would seem that reminding students their race or gender doesnt matter, would be an easy way to improve test scores.
#5 JYB
October 22, 2006

In BLINK by Malcolm Gladwell he writes about a similar study. My memory is a little hazy but I seem to recall they were black students taking a GRE or GRE type of test. I don’t think they even read an article, half were just asked what their race was before the test was taken. The group that was asked their race performed half as well as the group not asked.

It made me think about how you are often asked to bubble in your race (optionally of course) before taking many standardized tests.
#6 RyanG
October 22, 2006

Perhaps instead of the ‘Women in Art’ essay discouraging them by reminding them of their gender, the ‘Women can do just as well as men’ essay was actively encouraging them.

Did they not think to have a control group reading no essays?
#7 Lab Lemming
October 23, 2006

“Let’s repeat that: reminding the test subject that she’s a woman impairs her performance on the math section.”

Is the obvious corollary of this effect that group identification programs will have a negative effect on their member’s academic performance, and thus should be discouraged at progressive universities?

At the very least, should students be warned about the deleterious effects of self-identifying with historically underperforming groups?
#8 Richard Simons
October 23, 2006

I have seen several reports along these lines in the last few months. One was in a recent issues of ‘Science’ (I’m sorry, I can’t put my hands on it right now), reporting on a study in which a racially mixed group of students, at the beginning of a course and just before a quiz, were asked to spend 15 minutes writing down their values. As I recall, this had no effect on the performance of the White students but had a dramatic effect on the performance of the better Black students, bringing them close to par with the better White students. Not only that, but the effect lasted throughout the entire course.

People involved in education, more than anyone else, have to be aware of the impacts of these stereotypes and make an effort to negate their effects.
#9 XXlabrat
October 28, 2006

I wonder what the results would be like if men were reminded of the theory that the Y chromosome (obviously the determining factor of male-ness) is simply a broken X chromosome; therefore, they are not as complete as their female counterparts.
#10 llewelly
October 28, 2006

I wonder what the results would be like if men were reminded of the theory that the Y chromosome (obviously the determining factor of male-ness) is simply a broken X chromosome; therefore, they are not as complete as their female counterparts.

Occasional negative remarks do not have nearly the impact of repeated oppression. I suspect only men who had suffered severe ostracization would be significantly affected. The amount of ‘you’re dumb because you’re a guy’ messages men receive is much smaller than the ‘you can’t do math ’cause you’re a girl’ messages that woman receive. Furthermore, whereas messages to women that they are mathematically challenged are always negative, messages to men that they are stupid are often framed positively – among much of the population, it is even laudable for a man to be stupid. It seems to me that attempts to an insult to men that is ‘equivalent’ to a given insult to women nearly always fail to achieve equivalence, because the context is so different.
#11 greensmile
October 31, 2006

Can I take away a bit of hope from this research and your cautions about it Dr. Stemwedel? Can I hope that if indeed little in a person’s behavior is completely innate then we can look at upbringings and other influences broadly categorized as “nurture” for ways to redress unbeneficial behavior?
#12 Talia Mana
November 1, 2006

What this really shows is that self-efficacy has a significant impact on performance (ie what you believe about yourself and your abilities) irrespective of whether that belief is rooted in gender, race or simply limiting beliefs.

I liked the comparison with people thinking that genetic predisposition to obesity automatically determines that they will become obese (as if there is an all you can eat pizza gene lol)