When the visual system can't adjust for the motion of our bodies

One of the amazing things the visual system does is to compensate for the motion of our bodies. Consider, for example, the difference between the apparently smooth view of the world you get when you're talking a walk, and the shaky image you see if you record the same walk while holding a camcorder. Compensating for the motion of your body is a complex process, but it seems to occur almost automatically.

Similarly, if you quickly dart your eyes from one object to another, or even several objects in succession, the motion is seamless, while a videocamera recording the same motion would be extremely disorienting.

But clearly there are limits to the visual system's ability to compensate for motion. If you spin around in place a dozen or so times, you'll quickly find one! But where, exactly, does the limit lie? Will spinning once cause some disorientation? What about walking in a circle instead of spinning?

There's been a considerable amount of research on the subject, but the results haven't been completely clear. When viewers try to recognize changes in a group of objects, some studies have found that when the viewers rotate a small amount but the scene stays in place, they're just as fast and accurate recognizing the change as when they stay in the same place. But when the observer stays in the same place and the scene moves an equivalent distance, accuracy and reaction time worsen.

But for larger rotations (225 degrees), other research has found that people are slower and less accurate in remembering the locations of objects, whether they are rotating or the scene they've memorized is.

No single study, however, had systematically varied the amount the scene and / or the observer rotated until last year, when a team led by Michael A. Motes did just that.

Volunteers were led into a dark room with an eleven-sided box in the middle. The viewers could only see into the box through little doors on each side. Only one door was open. Inside was a table with 11 objects. The first frame of the figure below should give you some idea of the layout (though I've only included five objects):

After studying the arrangement of the objects, they went into a different room where duplicates of the objects were on a table. The observers tried to arrange the objects to match the pattern in the original room. If they failed, they were allowed back into the first room to study the objects. This continued until they had reliably matched the pattern. They went back to the first room and stood at the door they had used to memorize the position of the objects (the door was now closed). Next they walked to a specified door, which was opened, revealing the objects again, from a different perspective. Their task was to indicate, as quickly as possible, whether two of the objects had switched place, as they have in frame 2 above.

The task was repeated several times, with observers led in and out of the room as the objects were rearranged. In addition, instead of walking to a new viewing location, sometimes the table inside was rotated, with the task being the same: indicate whether a pair of objects had been switched.

Here are the results:

Whether the scene or the observer moved made no difference! There was no difference in accuracy between when observers walked around the scene, and when the scene rotated and the observers stayed in the same place. Reaction times followed a similar pattern, although for both types of rotation, responses were slower when the table was rotated farther from the original viewpoint.

But perhaps the visual system only compensates for the motion of the body for a short period of time after the scene is memorized. To see if the time delay was the problem, the researchers designed a new experiment. This time, the display was generated by a computer, and there were just five objects, so observers could memorize them after viewing for just three seconds. At this point the image disappeared and they moved to a new position; seven seconds later, the image reappeared and they were asked if one of the objects had moved, and if so, which one. Here are the results:

Unlike in the first experiment, this time accuracy declined as distance from the starting point increased. As before, reaction times also worsened as viewers walked farther.

So Motes' team concludes that compensating for motion, at least in cases such as this, is not automatic. But clearly we do automatically compensate for some motion--otherwise our view of the world while walking would be no different from a shaky video. When is compensation for motion automatic? The researchers make a compelling case that it's when the motion occurs inside the scene. In their experiments, Motes et al. studied only motion outside of the scene. But other studies have found that when we rotate inside of a room, we're accurate at remembering the locations of objects in the room, but we don't remember the locations of objects outside of the room (nearby roads, trees, and so on).

I wonder if this is partly the reason why dolly shots (where the camera moves from place to place rather than just panning) in movies are so compelling and somewhat disorienting: they represent a type of motion we can't adjust to as easily as a simple pan.

Motes, M.A., Finlay, C.A., & Kozhevnikov, M. (2006). Scene recognition following locomotion around a scene. Perception, 35, 1507-1520.