If you’re like me, when you read a book, you form a picture of what’s happening in your mind. I try to imagine not only what the characters look like, but also their surroundings. Eventually, I’ve created a picture of an entire world in my head. Then, when the book gets made into a film, I’m often disappointed when things don’t look the way I imagined them. Could it be that the visual environment I’ve created in my head is just as appealing as the one created by the filmmaker?
It makes some sense: if we’ve gone to the trouble of creating a visual environment, why not use the part of our mind that’s designed to handle visual information — the visual perception system itself? Wouldn’t it be more convenient to do it that way instead of clumsily describing visual representations using words?
But how can we know whether readers really process language in this way — processing the visual images the same way we process images that we sensed directly with our eyes? One way is to see if these two types visual representations are treated in the same way. For example, when you see a real scene, you don’t keep a precise representation of it in your visual memory; you rely on the fact that the items are present in front of you. If an item disappears when you’re not looking at it, you’re not likely to remember it was there — this is change blindness. While it’s impossible to make an item “disappear” in a story, a room can go dark, a character can be blinded by bright light, or something can move in front of her so that she can’t see some or all of the objects she once saw. If we really represent stories using the visual system, it should be more difficult to recall items that the main character can’t see.
Consider this brief story:
1. Russ was in the hospital recovering from minor surgery.
2. In the bed next to him was an older man named Marty.
3. A television was attached to the ceiling between them.
4. At Marty’s side was a tall vase of flowers.
5. Russ’s friends had given him only a get well card.
6. A nurse came in and drew the curtain around Marty’s bed.
7. She wanted privacy while she did an examination.
Now, try to answer this question about the story:
Did Marty have a vase of flowers?
It’s an easy question, but is it harder to answer knowing that the vase is “concealed” from Russ by the curtain? William Horton and David Rapp presented 24 seven-sentence stories, including this one, to 32 volunteers, measuring their reaction time as they answered the questions. In every story, Sentence 4 contained the critical piece of information, and Sentence 6 described some means of visually occluding it. Each story also had an alternate version, where a different action was substituted, one which didn’t block the critical object from the protagonist’s field of vision. For example, the hospital story was modified this way:
6. A nurse came in and attached a monitor to Marty’s bed.
7. She needed to take his blood pressure.
So that the participants didn’t notice the pattern, an additional 48 filler stories were mixed in. These stories varied in length, and the critical information was presented in a variety of spots in the story. All stories were presented one sentence at a time, with a yes/no question following. Readers had 2.5 seconds to respond to the question. Did blocking the object from view affect reaction time? Here are the results:
That’s not a huge difference — after all, these are easy questions — but reaction time was significantly faster when the object in question was in view of the protagonist, suggesting that readers are indeed using visual processing to answer the question!
But it’s also possible that a change in the environment that’s large enough to occlude an object is simply more distracting than other changes, and that’s why reaction time was slower. To address this concern, Horton and Rapp developed a new experiment. They used the same stories, but sometimes they asked questions about objects that weren’t occluded. For example in the hospital story, they asked Was a television attached to the ceiling? If the curtain was more distracting than the monitor, then respondents should be slower to respond in the curtain version of the story, even though the TV wasn’t blocked from view. Here are those results:
Only when the object was actually blocked from view was the reaction time significantly slower. Even though none of the stories mention the fact that the relevant item is no longer visible, readers respond as if they no longer “see” the item in question. Readers appear to literally be taking the visual perspective of the stories’ main characters as they try to recall the details of each story.
So if it seems that you’re actually seeing a story in your head as you read, this study provides compelling evidence suggesting that’s really what you’re doing. No wonder the movie’s never as good as the book!
Horton, W. S., & Rapp, D.M. (2003). Out of sight, out of mind: Occlusion and the accessibility of information in narrative comprehension. Psychonomic Bulletin & Review, 10(1), 104-110.