Detecting faces: People use some of the same strategies computers do

How does our visual system decide if something is a face? Some automated face-detecting software uses color as one cue that something is a face. For example Apple's iPhoto has no trouble determining that there are two faces in this color picture:

i-0b564a7ea8864fe3cd2b2e829757a72c-bindemann1.jpg

That's Nora in the back, and her cousin Ginger in front. In this picture, however, iPhoto can't identify a face:

i-572d15b49d92a2971ade9c707daa3ce1-bindemann2.jpg

That's a vintage black-and-white photo of Nora and Ginger's grandfather, but the computer can't find any faces in it. Do people, like computers, use color to help decide whether something they see is a face? Humans are excellent at identifying colors, and while faces can be many colors, there are also many colors that are very rarely seen in faces (e.g. blue, green, orange). Could we use skin-tones to help identify faces?

Markus Bindemann and Mike Burton created a set of images with faces placed in random locations, like this:

i-62416ada842ffb844fb0a1b962f76b6b-bindemann3.jpg

Some were full-color, some black-and-white, some had faces in black-and-white and color backgrounds, and some had black-and-white backgrounds and color faces. The faces also varied in size and position within the pictures. One-third of the photos contained no faces at all. Twenty-four volunteers watched as these images flashed in front of them, indicating as quickly as possible whether they saw a face. Did the color of the faces matter? Here are the results:

i-baac7fc454bd5e0a929a16ba6ba55235-bindemann4.jpg

Color faces were detected significantly faster than black-and-white faces. Even when the black-and-white faces were on a color background, they were still detected significantly more slowly. Similarly, more errors were made on black-and-white faces compared to color faces.

In a second experiment, Bindemann and Burton showed viewers face pictures that were half-color and half- black-and-white, in addition to the normal full-color faces. Once again, full-color faces were detected significantly faster.

The researchers say this means the visual system must be searching for skin-colored areas of a roughly elliptical or oval shape--much like computers do.

Bindemann, M., & Burton, A.M. (2009). The Role of Color in Human Face Perception Cognitive Science, 33, 1144-1156 : 10.1111/j.1551-6709.2009.01035.x

More like this

Regular readers will know that I'm not an intertubez triumphalist. But I read that the Harvard Book Store has bought itself a fancy gizmo to print any book in about four minutes: Battered booksellers, especially independent ones, have so far withstood the punishing shock-and-awe offensive of…
tags: Great tit, Parus major, birds, mystery bird, bird ID quiz [Mystery bird] Great tit, Parus major, photographed in Helsinki, Finland. [I will identify this bird for you tomorrow] Image: GrrlScientist, 24 November 2008 [larger view]. Please name at least one field mark that supports your…
All the day long whether rain or shine she's a part of the assembly line she's making history working for victory Rosie the Riveter! One of the most iconic images, spawned by this song: is that of Rosie the Riveter. For the first time in the United States, women were not only entering the…
Standup comics have long made vice president Dick Cheney the butt of their jokes, suggesting that he's never seen in public with the President because he inhabits some fortified underground bunker so as to avoid terrorists or some other unidentified threat, or that he's actually a cyborg, secretly…

iPhoto is much better at recognizing faces when two eyes are clearly delineated, so the problem with the second photo is likely that his face is turned to the side and the frame of his glasses have turned his eyes into a straight line. I've had good luck with iPhoto recognizing faces in black and white photos if the faces are clear.

Steve: Good point. Ideally to test you'd take several pictures and make BW and color versions of them, then see which faces it found. But regardless of whether iPhoto uses color, Bindemann and Burton say that many commercial face detection software packages do rely on color.

Is this really specific to faces and skin-tone? I would expect the same qualitative pattern if asked to identify trucks or soda cans regardless of their color.

By Eric Dimperio (not verified) on 19 Nov 2009 #permalink

Eric:

I didn't report on another part of the second experiment, where faces were colored in reverse (hence, unnatural) pigmentation. Viewers were slower to detect those faces. So it's not just a color / BW thing; it's specific to skin colors.

What range of color were they defining as 'skin tone'? I tend to think of skin tone as the hard-to-describe light-peachy-pinky-beige of my Caucasian skin. I, of course, fully recognize that my friends with complexions that I would describe as olive or tan or brown or dark brown do have 'skin tones', I just would tend to describe them using the available alternate words.

And I suspect this would make a difference. I have noticed that on TV or when seeing people in dim light, I do have a much more difficult time spotting the faces of people with darker skin. I suspect the relative lack of contrast between skin and lips as well as skin and hair contributes to this problem. I'm just wondering whether these computer programs have an easier time with all full color faces regardless of skin tone or whether they are better with full color Caucasian faces than those with full color darker tones or whether there is less of a differance between full color faces with dark tones or black and white of the same faces.

By DinaFelice (not verified) on 20 Nov 2009 #permalink

Do you think this could be because humans design software in the first place - if we have coded our own ability to recognise the face into computers, it's no wonder we both recognise faces the same sort of way.