Cognitive Daily

i-eca0cf2af9fc3ac4445c7dff7d8aab70-research.gifIf you’re like me, when you read a book, you form a picture of what’s happening in your mind. I try to imagine not only what the characters look like, but also their surroundings. Eventually, I’ve created a picture of an entire world in my head. Then, when the book gets made into a film, I’m often disappointed when things don’t look the way I imagined them. Could it be that the visual environment I’ve created in my head is just as appealing as the one created by the filmmaker?

It makes some sense: if we’ve gone to the trouble of creating a visual environment, why not use the part of our mind that’s designed to handle visual information — the visual perception system itself? Wouldn’t it be more convenient to do it that way instead of clumsily describing visual representations using words?

But how can we know whether readers really process language in this way — processing the visual images the same way we process images that we sensed directly with our eyes? One way is to see if these two types visual representations are treated in the same way. For example, when you see a real scene, you don’t keep a precise representation of it in your visual memory; you rely on the fact that the items are present in front of you. If an item disappears when you’re not looking at it, you’re not likely to remember it was there — this is change blindness. While it’s impossible to make an item “disappear” in a story, a room can go dark, a character can be blinded by bright light, or something can move in front of her so that she can’t see some or all of the objects she once saw. If we really represent stories using the visual system, it should be more difficult to recall items that the main character can’t see.

Consider this brief story:

1. Russ was in the hospital recovering from minor surgery.
2. In the bed next to him was an older man named Marty.
3. A television was attached to the ceiling between them.
4. At Marty’s side was a tall vase of flowers.
5. Russ’s friends had given him only a get well card.
6. A nurse came in and drew the curtain around Marty’s bed.
7. She wanted privacy while she did an examination.

Now, try to answer this question about the story:

Did Marty have a vase of flowers?

It’s an easy question, but is it harder to answer knowing that the vase is “concealed” from Russ by the curtain? William Horton and David Rapp presented 24 seven-sentence stories, including this one, to 32 volunteers, measuring their reaction time as they answered the questions. In every story, Sentence 4 contained the critical piece of information, and Sentence 6 described some means of visually occluding it. Each story also had an alternate version, where a different action was substituted, one which didn’t block the critical object from the protagonist’s field of vision. For example, the hospital story was modified this way:

6. A nurse came in and attached a monitor to Marty’s bed.
7. She needed to take his blood pressure.

So that the participants didn’t notice the pattern, an additional 48 filler stories were mixed in. These stories varied in length, and the critical information was presented in a variety of spots in the story. All stories were presented one sentence at a time, with a yes/no question following. Readers had 2.5 seconds to respond to the question. Did blocking the object from view affect reaction time? Here are the results:

i-c7be270b739e89d6de568a44c74dc245-Horton.gif

That’s not a huge difference — after all, these are easy questions — but reaction time was significantly faster when the object in question was in view of the protagonist, suggesting that readers are indeed using visual processing to answer the question!

But it’s also possible that a change in the environment that’s large enough to occlude an object is simply more distracting than other changes, and that’s why reaction time was slower. To address this concern, Horton and Rapp developed a new experiment. They used the same stories, but sometimes they asked questions about objects that weren’t occluded. For example in the hospital story, they asked Was a television attached to the ceiling? If the curtain was more distracting than the monitor, then respondents should be slower to respond in the curtain version of the story, even though the TV wasn’t blocked from view. Here are those results:

i-fc6011269039dc6fe39031790116ca13-horton2.gif

Only when the object was actually blocked from view was the reaction time significantly slower. Even though none of the stories mention the fact that the relevant item is no longer visible, readers respond as if they no longer “see” the item in question. Readers appear to literally be taking the visual perspective of the stories’ main characters as they try to recall the details of each story.

So if it seems that you’re actually seeing a story in your head as you read, this study provides compelling evidence suggesting that’s really what you’re doing. No wonder the movie’s never as good as the book!

Horton, W. S., & Rapp, D.M. (2003). Out of sight, out of mind: Occlusion and the accessibility of information in narrative comprehension. Psychonomic Bulletin & Review, 10(1), 104-110.

Comments

  1. #1 Aaron Couch
    April 3, 2007

    A very interesting find! Personally, I’ve always liked the movies better, but that’s often because I see the movies before I read the book; so then I create the images in my mind based more around what I’ve seen in the movie.
    Just to clarify: on the second figure, the “Non-blockable” object refers to the television on the ceiling and the “Blockable” refers to the vase of flowers, correct? (And the blocker in both cases is the curtain around Marty’s bed.)

  2. #2 Dave Munger
    April 3, 2007

    “Non-blockable” object refers to the television on the ceiling and the “Blockable” refers to the vase of flowers, correct? (And the blocker in both cases is the curtain around Marty’s bed.)

    Yep. You got it.

  3. #3 Dave Group
    April 3, 2007

    Random thoughts on this post:

    1. When I took the Evelyn Wood Speed-Reading course some three decades ago, part of it involved downplaying subvocalization (saying the words in your head) and emphasizing visualizing the events as they unfolded.

    2. How many mystery writers, subconsciously or otherwise, use this technique to throw readers off , creating a “surprise” conclusion to their story.

    3. I would say that, in most cases, the book is better than the movie, partly for this reason and partly for the fact that the book can go into far more detail and can more easily enter the thoughts of the characters. Sometimes, though, bad books can make great movies. Examples of cases where (in my opinion, at least) the movie was better than the book include JAWS, THE TAKING OF PELHAM 1 2 3 (original version), Hitchcock’s PSYCHO and FRENZY, and 2001: A SPACE ODYSSEY.

  4. #4 Dave Munger
    April 3, 2007

    I *really* like The Taking of Pelham 123 but haven’t seen the movie — that’s quite an endorsement!

    Interesting meme — movies that *were* better than the book.

    Unfortunately, I can’t think of any. Here’s a partial list of movies I think were huge disappointments compared to the book:

    Dune
    Bonfire of the Vanities
    The World According to Garp
    Huckleberry Finn
    Pride and Prejudice (pick your version)
    1984
    The Hunt for Red October

    I do have a couple movies I think were roughly equal to the book (mainly because I didn’t care much for the book):

    Lord of the Rings
    Harry Potter

  5. #5 Alexei Polkhanov
    April 3, 2007

    Strange, I used only phrase “Russ’s friends had given him only a get well card” to answer the question if Marty has a vase of flowers, because I thought that the fact that vase of flowers located besides Marty’s bed, does not mean it belongs to him. And I was expecting that purpose of the question itself was to trick me into believing that Marty has flowers when in fact he did not. Note that visualization part did not play any role. Would love to see rest of tests :-)

  6. #6 Richard
    April 3, 2007

    Also, check out this article about mental imagery and stories:
    http://www.psych.unito.it/csc/cogsci05/frame/poster/3/f491-johansson.pdf

  7. #7 subcorpus
    April 3, 2007

    i always like the books better …
    minus jurrasic park …

  8. #8 Ted
    April 3, 2007

    No wonder the movie’s never as good as the book!

    The answer is to watch the movie in 1080p on a plasma and forget the book. I think it’s environmentally sound and ethical. Books are so 20th century.

    There are movies that I think are better than the book; specifically political thrillers based on actual events.

    1. Seven Days in May
    2. Z
    3. The Battle of Algiers
    4. The Parallax View
    5. All the President’s Men
    6. All the King’s Men (1949)

  9. #9 Alan Kellogg
    April 4, 2007

    Yes, You Sparked Some Thinking

    We tend to ignore what we expect to see in a scene, it’s the novel that gets our attention. We may note a vase with flowers next to a hospital bed, but that information can quickly slip our mind as time goes by. Writing brings such details to the forefront, and can even make us aware of additional details we might otherwise miss.

    For instance, take these two examples. In the first you enter someoen’s hospital room. You see your friend in his hospital bed and notice a vase of flowers on a stand to the left of his head. Nothing unusual, nothing out of the ordinary.

    In the second, “Bob walked into the room. The patient lay sleeping on the bed, arms in casts and face thick with salve and ointment. On a stand next to his head was a vase. Blue with twining vines of pale green. In the vase were rhododenrons and daffodils. The patient himself was nude, his stab wounds freshly stitched, though the one in his groin still seeped blood.”

    A lot more detail, and detail you really can’t miss. Now, if you asked somebody, “What was the patient wearing?” they’re apt to tell you, “A hospital gown.” Yet if you had them read the passage again they’re apt to say, “Oh, I didn’t notice that.”

    Thing is, they did notice, all you did is remind them of what they had read. What happened is that they glossed over that part, because it’s so out of place in such a scene.

    So you have two dynamics here, our tendency to notice something because someone or something has made it impossible to ignore a detail, and our tendency to elide a detail because it doesn’t jibe with our expectations. So we watch as an expert on video demonstrates that humans can walk as the subject of the Patterson/Gimlin (1967) film does, and miss the fact that humans don’t walk that way normally.

    (That last part incorporated deliberately to see how many people respond to that and that alone. This part included to see how many people accuse this comment of being trolling.)

  10. #10 Peter Hollo
    April 4, 2007

    It’s interesting that some respondents imagined the room in such detail that they were able to tell that the vase was blocked from the protagonist’s view and identify with the protagonist to the extent of actually not “seeing” the vase.
    I honestly don’t think I’d ever do that (and I’m speaking as a constant reader of fiction). I don’t think my visual imagination is up to the task; I rarely have a clear picture of what the characters in a book look like, past a kind of blurry idea of some of their features. I can still have the reaction of “that’s really not what Character X looks like!” when I see the movie version, and perhaps even “That’s not what that room looked like in the book”, but this level of detail is, I think, quite beyond me.

    Even so, when studying Ramachandran & Pylyshyn on mental imagery and so on back at Uni, I recall being adamant that there *is* some sort of “enactment” of the seeing act going on in one’s head when imagining a scene, so it’s not that I don’t believe that we can “form a picture of what’s going on in [our] mind”.

  11. #11 Aaron Couch
    April 5, 2007

    I think I like Ted!

  12. #12 Dave Group
    April 5, 2007

    Okay, I’ll take the bait. After researching the Patterso/Gimlin film of bigfoot, one has to conclude that there is something genuine there, or that it is one of the most brilliant hoaxes ever perpetrated. Though Patterson was evidently someone who regarded the truth as a rather flexible concept, so to speak, no one has positively identified who made the suit (assuming a hoax, that is) or who wore it. The creature was definitely not of human height or proportions.

    As for the book/movie issue, I guess it boils down to who has the better imagination, the writer or the director!

  13. #13 Nate
    April 9, 2007

    Well I’m a little late to the discussion, but I think The Godfather is the classic example of a movie that transcends the book on which it is based (although the novel still wasn’t a bad read).

    I’ve also been told the movie The Princess Bride is much better than the book, but though I adore the film I haven’t gotten around yet to reading the book to judge for myself.

  14. #14 llewelly
    April 25, 2007

    Dave Group:

    … no one has positively identified who made the suit (assuming a hoax, that is) or who wore it. The creature was definitely not of human height or proportions.

    uh, wait, are you saying there saying the Patterson/Gimlin was a non-human creature in a bigfoot suit?