Cognitive Daily

When you look out the window and then look away, how do you remember what you saw? Do you hold a picture of the window in your head, frame and all? What about a photo? Do you remember the physical photo, or do you imagine the real scene it represents? If you remember the scene, and not the photo, then how do you form the boundaries of the scene? Does your memory end precisely where the photo does?

Here’s a little test to see how accurate your short-term memory of a photo is. When you play the movie below, you’ll have a second to get ready, then a photo will flash for just a half-second. It will be replaced by a random pattern for two seconds, and then the photo will reappear. It might be the same as the original, or it might have been cropped or enlarged slightly. Your job is to say if the area depicted in the second photo is the same, larger, or smaller than the original.

What do you think? Let’s collect the answers in a poll (don’t repeat the movie until you’ve responded):

Even playing the movie repeatedly it might be difficult to tell, so I’ll display both photos side-by-side at the end of the post. What we’re exploring here is a phenomenon that’s been investigated for several years by Helene Intraub and her colleagues:

It’s called boundary extension, and it has been robustly found in a variety of circumstances — even in blind and deaf people. Intraub believes the phenomenon is related to the way we construct memories of scenes. Rather than remembering scenes on a pixel-by-pixel basis, we remember just enough information to reconstruct that scene later. Since items in a picture may extend beyond its border, our memory, too, usually extends beyond the boundaries of a picture. If we see the same picture later on, we usually believe it has been cropped. Did I crop the picture in the movie I showed you? I’ll let you know at the end of the post.

Even when we see a picture for a very short period of time, and even when the picture is removed from vision for just two seconds, boundary extension is still observed. In a new experiment, Intraub’s team asked viewers to focus just on the center of the photo (and tracked their eye movements to make sure they did). Even in this case, significant boundary extension occurred — viewers believed the picture they saw was 50 percent bigger in area than it actually was. When allowed to move the borders of an image to reconstruct the view they thought they had seen, each edge was extended by around 20 percent.

But perhaps the boundary was only extended because viewers weren’t allowed to look at it. To explore this possibility, Intraub’s team designed a second experiment. 250 milliseconds after the photo was displayed, an arrow appeared, directing viewers towards an object on the left or right side of the picture. Before viewers could move their eyes all the way to the object, the photo disappeared. Again, they were allowed to reconstruct the boundaries of the picture. Here are the results:

There was significant boundary extension on three of the four borders of the picture: the top and bottom, but also the side viewers were cued to look at. The only side where no extension occurred was the side where they didn’t look!

Boundary extension occurred precisely where viewers were looking. Thus, the researchers argue, it’s not due to inadequate information about boundaries, but an active process whereby our memory actively extends beyond the boundaries of a scene. In a third experiment, viewers were sometimes cued to look one direction or the other, and sometimes cued to remain focused on the center of the photo. When focus remained on the center, there was no significant boundary extension to either side of the picture, but the top and bottom boundaries were still extended.

So boundary extension does not occur when we’re actively “not looking” in a particular direction, but it does occur when we’re looking in a particular direction. This again supports the notion that boundary extension is an active process of the mind, and that our memories are actively constructed, rather than mere passive reflections of reality. In other words, you make your own memories; they aren’t made for you.

So what about the photos I showed you in the movie above? The second photo was zoomed out 5 percent wider on each side, for a total of 21 percent greater area depicted than the first photo:

So even if you thought that the two photos were the same, you were still showing boundary extension (and, of course, if you said the second photo was cropped, you were clearly extending the boundary).

Intraub, H., Hoffman, J.E., Wetherhold, J., & Stoehs, S. (2006). More than meets the eye: The effect of planned fixations on scene representation. Perception & Psychophysics, 68(5), 759-769.

Comments

#1 Laura
February 22, 2007

I recognized that the second picture was zoomed out, but not by any variation on where I presumed the boundaries to be. In the second picture, the child’s face (the most attention-grabbing part of the photo) appears smaller, logically reflecting a zoom out.
#2 Dave Munger
February 22, 2007

Laura –

What you say is true, and yet as of now 50 percent of respondents still show boundary extension. Intraub’s experiment allowed for a more precise measure of boundary extension, but I suspect we’ll still find a significant bias towards boundary extension even with our very unscientific poll.
#3 R E Morley
February 22, 2007

Couldn’t use the test since the film would not run
#4 Dave Munger
February 22, 2007

R E –

Sorry about that. Do you have the latest version of QuickTime?
#5 Amy
February 22, 2007

I figured that it was zoomed out for the same reason–the girl seemed smaller in the second photo. It would be interesting to do the same thing with a photo that has just a landscape.
#6 Mark
February 22, 2007

what are you trying to prove about memory? “We” remember the girl, her arm, the mountain etc. The second picture looks just like the first. We don’t have a scale/ruler drawn on the image in our memory, to compare with the second , that also has no scale.

We make our own memories? Who else makes them?

Can you alter the middle of the photo 5 to 18%? no
#7 Dave Munger
February 22, 2007

Mark,

If you don’t think the research is interesting, I suppose that’s your prerogative.

I think it’s interesting that visual memory is not simply an exact representation of what we see. I think it’s even more interesting that memories differ from what is seen in predictable ways, and that there are reliable ways to influence those memories.

I would dispute your contention that the second picture “looks just like” the first. The two pictures are clearly different, and even when alerted to the precise nature of the potential difference, most people get it wrong.

What’s more, this is a smaller difference than that found in many boundary extension experiments, where more than 20 percent may be added to each boundary of an image.

Furthermore, your befuddlement at the utility of this study reflects a common misconception about psychology research — that most psychology is “common sense” that can easily be understood by introspection. Whether an experimental result matches with introspection can sometimes be relevant, but simply dismissing all research that agrees with introspection as “obvious” is quite wrongheaded. If we don’t study things scientifically, we won’t be alerted to the many instances in which our introspection is wrong. I’d submit, further, that the boundary extension effect is one that never would have been uncovered by introspection alone.

But if you’re not interested in this study, that’s fine with me. There are plenty of other things you could be doing right now.
#8 Sean Thompson
February 22, 2007

I was correct, I had looked at the girls face (which I guess would be expected when you display a image that includes a face) and because it was neatly framed by the dark rock behind her head (note a symmetrical amount of rock visible either side of her head), the loss of the symmetry stood out..
Was quite surprised to see people voting the opposite way though!
#9 austin
February 22, 2007

Hmm isnt this about preattentive processing rather than some visuospatial owrking memory stuff? I think the presences of a face confounds the theory presented
#10 Dave Munger
February 22, 2007

Austin:

It could be if this demo was the only study on boundary extension. However, lots of studies, including ones which give viewers much more time to memorize a scene, come to the same conclusion.
#11 Scott
February 22, 2007

Everyone has the right to an opinion! Images can be very useful in explaining a message by reducing “noise” and avoiding issues that are often a consequence of jargon. This wee example suggests that even pictures aren’t perfect!
#12 Amie
February 22, 2007

Oddly, it’s not the size of the girl’s face, but the upper right corner that looked off to me – I recall seeing a different arrangement of shapes in that area, and once the new picture came up, it looked like the sky was a little broader. The girl was the least informative part for me, except for how the shape of her interacted with the other parts of the composition. Of course, my husband could predict that phenomenon – when we go to the mall, he sees 10 visually outstanding people for every 1 I see.
#13 Sammy
February 22, 2007

I also realized it had changed because the girl’s face looked smaller. It’s interesting that in the tiny fraction of time the picture was visible, I immediately focused on the face, rather than the scenery behind her. It seems quite a few others did the same. I suppose being able to quickly identify a face as a face (even an animal face) was helpful for survival?
#14 Kapitano
February 25, 2007

What happens if the scene is just a bland landscape – no figure, trees, building etc? Is the boundary extension effect weakened?

My intuition is that it would be, but as you point out, intuition is pretty useless for these questions.

Also, does the delay between the two images make a difference? Will more people see the difference if the delay is 1 second?

What about those viewers who are unsure whether the second image is cropped, zoomed or the same? If the test has asked me how sure I was that the images were the same (with, say 1 for “sure they’re different” and 10 for “sure they’re the same”), I’d have chosen around 7
#15 Colin
February 28, 2007

I saw the change pretty readily. Though being graphic designer I’ve got a lot of training for this sort of thing. I immediately ignored the face and concentrated on the edges and shapes created by the negative space and the foreground background difference. Most salient was the cropping difference of the triangle created by her akimbo arms.