Neurophilosophy

Eye movements reveal unconscious memory retrieval



THIS short film clip shows two images of the same scene. Watch it carefully, and see if you can spot the subtle differences between them. As you watch, your eyes will dart back and forth across the images, so that you can perceive the most important features. And even though you might not be consciously aware of the differences, your brain will have picked up on them. This implicit form of remembering is referred to as relational memory; in this case, the brain is encoding the perceptual associations between items in the image. And recent studies have shown that relational memory retrieval is evident in eye movements – even if you did not explicitly detect the change in the second image, you will, without knowing it, have spent more time looking at the changed area than at others.

However, it was unclear whether relational memory retrieval involved the same neural substrates as retrieval of other, more familiar types of memory, such as your recollection of what you had for breakfast this morning, or what you did last weekend. This latter form of memory (known as declarative memory) is well known to involve the hippocampus, a brain structure buried deep within the brain, in the medial temporal lobe. Now researchers from the University of California, Davis, report that unconscious retrieval of relational memory also involves the hippocampus, and that activity in this part of the brain can predict the eye movements associated with expression of this type of memory.

That the hippocampus is critical for memory formation was first established by the pioneering neuropsychologist Brenda Milner in the 1950s. H. M. It is generally agreed that the hippocampus is also involved in conscious recollection, although one recent study showed that memory retrieval becomes less dependent on it, and more dependent on the frontal cortex, with time. But the role of the hippocampus in relational memory is unclear, and it has been suggested that activity in this brain region is associated only with memory recollections that enter conscious awareness.

Deborah Hannula and Charan Ranganath therefore set out to investigate whether the eye movements associated with relational memory would be correlated with hippocampal activity. Using functional magnetic resonance imaging (fMRI), they scanned the brains of 18 participants during a simple memory task. In the first block of trials, the participants were shown a series of scenic images, each of which was presented for 1 second, before being superimposed with a face. The scenes were then shown a second time, as a cue, and each was followed, after a short delay, by three of the faces, presented simultaneously. The participants were required to indicate which of the faces had been associated with each of the scenes in the previous earlier trial, by pressing a button. During the experiment, the participants’ eye movements were simultaneously tracked.

Hannula and Ranganath predicted that the scenic images in the second block of trials would elicit retrieval of relational memories, and that this would be manifested as a disproportionate viewing of the faces that were previously paired with each scene. And as expected, the participants spent more time looking at correctly identified faces than incorrectly identified ones. This was not due to explicit recollection of the face-scene pairings, because the correct faces were viewed for longer even during trials in which the wrong faces were selected, and in those in which the participants reported not knowing which face had previously been paired with the scene.

Analysis of the fMRI data confirmed this. Activity in the hippocampus and surrounding medial temporal lobe regions was found to predict disproportionate viewing of the faces matching the scenes, even in trials in which the participants could not explicitly recollect which face matched the scene being viewed at the time. The increase in hippocampal activity was greatest during trials in which the correct face was identified, but increased activity was also measured in which the wrong face was selected, and when the participants reported not knowing which was the correct face. 

It was also found that the accuracy of face selection was associated with a specific pattern of brain activity. All trials were associated with increased hippocampal activity; however, during correct, but not incorrect, trials, there was increased activity in the perirhinal cortex (which is adjacent to the hippocampus) as the scene cues were presented, as well as in the left dorsolateral and ventrolateral prefrontal cortex (dlPFC and vlPFC, respectively). Further analysis of the imaging data revealed an increase in functional connectivity between the medial temporal lobe and prefrontal areas during the simultaneous presentation of the three face displays in the second block of trials. This suggests that while the hippocampus alone may be sufficient for retrieval of implicit relational memories, this information must be communicated to the prefrontal cortex in order for conscious recollection and overt decision-making to take place.

Activity in the hippocampus is therefore closely correlated with the eye movements associated with retrieval of relational memories, even in the absence of conscious awareness. Implicit and explicit memory retrieval also appear to be correlated with distinct brain activation patterns. The authors suggest that their findings could have far-reaching implications. It is possible that eye-tracking could be used to investigate memory function in infants, or to obtain information about past events from those who might be unwilling to recall them, or are unable to. Schizophrenics, for example, often exhibit memory deficits, and Hannula has already used memory-related eye-movements to show that they have difficulty noticing small changes in a previously learned scene.

More on memory:


Hannula, D. E. & Ranganath, C. (2009). The Eyes Have It: Hippocampal Activity Predicts Expression of Memory in Eye Movements. Neuron 63: 1-8. DOI: 10.1016/j.neuron.2009.08.025.

Comments

  1. #1 Adrian Morgan
    September 14, 2009

    In the video, I spotted three changes. In the order that I noticed them: (1) Only one photograph has a pot plant in the window. (2) One photograph shows more of the chair legs which in the other are cut off by the lower edge of the photograph. (3) In the photograph without the pot plant, a lot more light shines through the window onto the table.

    I spotted (1) immediately, and (2) and (3) soon after by looking at a different area of the picture each time it changed.

  2. #2 Jordi
    September 14, 2009

    Interesting article!

    I spotted the missing plant immediately too and noticed that the camera is pointed slightly more upward when there is no plant. I spotted this around the 3rd or 4th flip. The extra light on the table seems to be due to the fact that the plant is not blocking any light when it is removed.

    Are we missing something or is the movie just not a great example of us being change blind?

  3. #3 Nigel
    September 14, 2009

    I agree. Worst change blindness demo ever! I noticed the missing plant straight away. Maybe that is because the light on the table (which I did not consciously notice, but clearly it is a direct result of the plant being moved) tips off your visual system about where to look. I wonder if this could be a confound in the experimental results. If they are changing the actual scene (rather than photoshopping the image) they may be altering the optical structure of the scene in unintended, and relevant, ways.

  4. #4 Mo
    September 14, 2009

    It’s the only clip I could find. It has nothing to do with the researchers who carried out the study I discussed in the post.

  5. #5 david
    September 14, 2009

    I hope this is relevant to what you talk about. I think I knew this phenomenon a little. I’m pretty good at unscrambling words in whatever order, even getting more than one word out of them if it exists. This is not entirely conscious. Study the word, then look up vaguely and to the right and the solution(s) almost always comes to you. I have taught others this trick and it works for them too. It is some type of pattern recognition. The same works in chess if you have played enough to have patterns to go by. The eye movement is essential to call the phenomenon into play.

  6. #6 William Lu
    September 14, 2009

    Interesting article…reminds me of Logan and Roediger’s 2008 experiment finding that strong right handers who performed an exercise moving their eyes from left to right improved their performance on retrieval of word lists.

  7. #7 David
    October 1, 2009

    Adrian, Jordi- agreed, the change in camera angle confounded the effect… also the plant shot was taken first and earlier in the day (change in angle of incidence)…

    Willam- your comment brought to mind an analogy with culturally-inherited perceptions of ease or work depending upon perceptions of characters walking left to right or right to left- with those who normally read left to right, a drawing, for example, of a figure walking left to right looks more comfortable as compared to the exact same image reversed but now traveling “opposite to” the same viewer’s reading direction…

The site is undergoing maintenance presently. Commenting has been disabled. Please check back later!