Mixing Memory

Gross, right? Those are the same two photos (the one on the right is now on the left, but rightside up this time. The distortions to the one face, which didn't look so bad when it was upside down, now look... grotesque. What's going on here? Well, in the inverted photo on the right (on the left when the faces are rightside up), the mouth and eyes are upside down. This doesn't bother us all that much when the faces are upside down. In fact, we often don't even notice it, and the expression looks pretty close to the normal face. Only when we view the face rightside up do the upside-down facial features strike us. The illusion is often called the Thatcher illusion, because its discoverer, Peter Thompson2, used a photo of Margaret Thatcher in his original experiment. Here are his photos (from this site):

Why does this occur? Clearly, our ability to process faces is impeded by inverting them, but what exactly is impeded in the Thatcher illusion? There are three competing hypotheses3. The first, called the expression hypothesis, goes as follows4:

(a) the grotesque appearance of a Thatcherized face is due to its expression, (b) inversion impairs the encoding of expression, and, therefore, (c) inversion disrupts the perception of grotesqueness of a Thatcherized face. (p. 284)

In other words, we have a harder time encoding expressions when faces are inverted, so we don't notice the problem with the expressions until we turn the faces rightside up.

The second hypothesis has to do with reference frames. It says that when we view an object like a face, we use two reference frames, one of which is based on the object, and the other on our egocentric or contextual sense of orientation (e.g., what's up and what's down). When the faces in the Thatcher illusion stimuli (generally called "thatcherized faces") are upside down, the top of the mouth and eyes differs for the two reference frames, but when the thatcherized faces are rightside up, the two reference frames are in agreement, and we get an even uglier Margaret Thatcher.

These first two hypotheses have some empirical support, but they don't fit with all of the data, so researchers have come up with a third hypothesis based on dual process theories of facial perception, which is now well supported empirically4. It begins with the assumption that we process faces by looking at "local features" (e.g., eyes, nose, mouth) and their configuration (how they're organized relative to each other). When the faces are inverted, it is difficult to process the configural information; we just can't seem to process the relationships between the features. So we rely on the local features, which don't appear to be off in the thatcherized faces. However, when the faces are rightside up, both the configural information and the local features are screwy, causing them to look grotesque. Evidence for this hypothesis comes from experiments in which participants are easily able to perceive alterations to local features (e.g., blacking the teeth) when the faces are inverted, but, as the above photos demonstrate, are unable to perceive even large deviations from the normal configuration of those features6.

Interestingly, studies using an electroencephalogram to measure the brain's response to the different photos indicate that our brains do recognize a large difference between the thatcherized and unthatcherized faces, even when they're inverted, despite the fact that participants are rarely consciously aware of the differences7. Since this difference in event-related potentials likely occurs early in the processing of the faces, it may be that higher-order visual processing of the local features overrides the differences that the visual system initially perceives, making it difficult for us to consciously perceive those differences. However, it's not quite clear how this works, and futher neuroscientific research on this aspect of our processing of thatcherized faces may provide interesting insights into face perception.

¹Schwaninger, A., Carbon, C.C., & Leder, H. (2003). Expert face processing: Specialization and constraints. In G. Schwarzer & H. Leder (Eds.), Development of Face Processing, pp. 81-97. Göttingen: Hogrefe.
²Thompson, P. (1980). Margaret Thatcher: A new illusion. Perception, 9, 483-484.
³The competing explanations are detailed in Bartlett, J.C., & Searcy, J. (1993). Inversion and configuration of faces. Cognitive Psychology, 25, 281-316.
⁴Ibid
⁵Carbon, C.C., Schweinberger, S.R., Kaufmann, J.M>, & Leder, H. (2005). The Thatcher illusion seen by the brain: An event-related brain potentials study. Cognitive Brain Research, 24, 544-555.
⁶Searcy, J.H., & Bartlett, J.C. (1996). Inversion and processing of component and spatial-relational information of faces. Journal of Experimental Psychology: Human Perception and Performance, 22, 904-915.
⁷Carbon et al. (2005).