What does the shadow know?

How do we tell where an object is in a three-dimensional world when our eye only gives us two dimensions worth of information? Today's reading ("Moving Cast Shadows Induce Apparent Motion in Depth" by Daniel Kersten, Pascal Mamassian, and David Knill of the University of Minnesota [Perception, 1997]) explores one aspect of that question: the role of an object's shadow.

Video game designers faced this issue in the 1980s when they began attempting to make "3-D" arcade games. One classic example was the game Zaxxon, where you flew a spaceship diagonally across the screen. The trick was, you not only had left and right controls, but also up and down. It was difficult to tell exactly how high your spaceship was, but the programmers made it easier by adding a shadow below your ship. The farther away the shadow was, the "higher" the ship was. Somehow, it still didn't help much.

Kersten and his colleagues replicated this effect in the lab by using a simple square displayed above a three-dimensional grid. Interestingly, they found that a crisp shadow didn't give nearly as much an impression of depth as a blurry one. They have generously allowed me to reproduce their demo movies here. This first one shows a crisp-shadowed square followed by a blurry-shadowed one (quicktime required).

It's pretty clear from this demo that Zaxxon would have been a much easier game if the shadow was blurrier—the sense of depth you get from the video is much more dramatic with a blurry shadow. (If you're nostalgic for the game, there's a rather crummy online version of it here. Should cure you of your nostalgia real fast.)

But to really thoroughly study this phenomenon, Kersten et al. needed a more sophisticated visual. They developed a "ball in a box" example, where a ball moves around in a virtual box on a computer screen. In the simplest example, the ball's actual path on-screen remains the same; what changes is the path of its shadow. Watch this next demo carefully:

You might need to watch it several times to convince yourself that the ball's shadow is the only thing that changes between the two displays.

Now take a look at one more demo. In this one, as in the previous example, the ball's actual path on-screen remains the same. However, the shadow follows a zig-zag route which makes it appear that the ball first goes diagonally toward the back, then moves forward and up, then down and back, then forward and back up again. In the second sequence, the shadow follows the same path, but the motion of the shadow is caused by moving the light source (watch the shading on the ball, and the lighting of the rest of the box).

Even when observers are aware that the only thing changing is the lighting, it still appears that the ball is moving forward and backward in the box. Kersten et al. tested several different examples, with an elliptical ball, and with more objects in the box, so as to make it perfectly clear that the only thing moving was the light source, and still it appeared to viewers that the ball was moving.

So our perceptual system assumes a stationary light source, even when we are conscious that the light source is moving. This may be part of the reason that movie scenes where the light source is moving around, such as the famous "flashlight" scene in Metropolis, can be so disorienting.

More like this

It be nice to know what parts of the brain are being used for these distinctions. I'm very interested in learning about what part of the brain is implicated in the shadow and ball study. What makes our brain see a difference when we consciously know that there is no difference? There is only a made up shadow that changes but the ball itself is constant.