# How our vision system adapts to a crooked world

If you’re a perception teacher, a great way to show how the vision system adapts is to use prism glasses to shift a volunteer’s vision. While various types of glasses are available (the most common is designed to allow a person to read a book lying on her back), the most effective for this demo is a pair that makes the world appear shifted about ten degrees to one side—so what was directly in front of your victim now appears ten degrees to the left (or right, depending on the particular pair of glasses). The best volunteers are athletes—quarterbacks or pitchers. Suppose the starting softball pitcher is in your class. You get her to throw plush balls at the you while wearing the glasses—initially, she will miss by several feet. It doesn’t take long to adapt, however, and soon the volunteer has learned to hit you with every pitch. Then you ask her to remove the glasses and try again: for the first several tosses, the throws miss in the opposite direction (for true hilarity, get the coach to show up at this point and threaten to drop her from the team!). Fortunately, the victim soon adapts, throwing strikes again and saving her full-ride scholarship.

This demo only works if the volunteer can’t see her hand and the target at the same time. If you have a new volunteer sit at a table wearing the glasses and attempt to reach for an object a few feet away, he’ll easily reach for it, just as quickly as if his vision wasn’t distorted at all (provided his hand and the target are both visible).

Curiously, if you try to duplicate this second demo using a video monitor, the results are rather different. Consider the following setup:

The volunteer is behind a screen and can’t see his hand, only a TV monitor displaying a picture of his hand and the target, but rotated by 40 degrees. It’s a slightly different setup from the prism glasses, because the camera is showing a view from above, but the problem is essentially the same—how to adapt when the input you see is different from the real world. Yet with this setup, the participant never completely adapts—though he does get a bit better, he’s never perfect. He keeps making the same mistake, over and over again. Instead of moving in a straight line to the target, the hand starts off in the wrong direction, and the path must be constantly corrected in order to reach the target. But even after 40 trials, it doesn’t completely return to normal. When the image is rotated back to normal, the mistake is made in reverse. With prism glasses, as long as both his hand and the target are in view, these transitions are managed with ease, and he never makes a mistake.

So is viewing your movement on a video screen inherently different than using prism glasses? There are many cases when people need to use a video screen to monitor their work—robotic surgery, handling hazardous substances, remote control of probes, and so on. If motion errors in these tasks are unavoidable, it’s an important issue to study.

A group led by Isabel Pennel devised an experiment to attempt to uncover just what’s different about video monitors compared to prism glasses. They noted that most prism studies involved rotations of much less than 40 degrees—generally, they were closer to 10 degrees. So perhaps if the video image was rotated less, observers would make errors and adapt in a pattern similar to what earlier research had found for the prism glasses.

Pennel’s team tested three groups of people. The first group saw images rotated by 40 degrees; the second, 10 degrees; and the third’s images weren’t rotated at all. All groups started with a pre-test, where the camera was not rotated. In this part, everyone was able to reach directly for the target. Next, during the testing phase, the camera was rotated by 10, 40, or 0 degrees. Finally, during the post-test, the camera was again set to 0 degrees for all groups. Here’s a summary of the results:

In the testing phase, as expected, participants in the 40° condition were initially off by a lot—over 20 degrees. After 5 trials, their results stabilized, but they were still not accurate. However, participants in the 10° condition were also inaccurate at the start. Their results, too, stabilized after 5 trials, but they were as accurate as the 0° condition. So, unlike wearing prism glasses, when a video monitor is used to skew vision, an initial period of adaptation is required before people can accurately reach for a target.

In the posttest phase, those in the 40° condition again required a few trials before their responses were accurate. The 10° condition was statistically indistinguishable from the 0° condition, but even this group had some problem adapting to “normal” vision: when the room was darkened and they were asked to point straight ahead relative to their body position, responses were off by several degrees.

So something about a remote monitor is fundamentally different than using prism glasses. It can’t be that accurate reaching is impossible, because those in the 0° condition performed just fine. Pennel et al. believe that the different viewing angle on the TV monitor may be partly to blame. If a camera angle is dramatically skewed, adapting in a short period of time is nearly impossible. As technology offers more opportunities for remote controlled devices, engineers will need to carefully test them to make sure humans are capable of safely operating them. And a remote-controlled softball pitcher is probably out of the question.

Pennel, I., Coello, Y., & Orliaguet, J.P. (2003). Visuokinesthetic realignment in a video-controlled reaching task. Journal of Motor Behavior, 35(3), 274-284.

1. #1 Doug Hoffman
October 21, 2005

Fascinating, Dave, and it indeed has relevance not just to remote surgery but all endoscopic surgery. In sinus surgery, we routinely use scopes which offer views thirty degrees and seventy degrees off horizontal. They’re a bitch to use, and we all know to use them with extreme caution (in sinus surgery, going off course could result in serious complications).

I’m also aware that there’s a steep learning curve for certain endoscopic procedures, such as endoscopic cholecystectomy. You don’t want to be the general surgeon’s third or fourth (or tenth, or twelfth) patient. It’s never a bad idea to ask, “How many of these have you done?”

2. #2 Dave Munger
October 21, 2005

Interesting, Doug. Based on this research, it seems like it would be a really good idea to “warm up” using a video simulator or some other apparatus that mimicked the offsets of the scopes used in surgery, with at least 10 trials (and preferably more) before doing the real thing. Do you know if that’s the practice in the field?