Prospective Coding and the Hippocampus

I wrote before about how there has been a bit of a debate about whether the hippocampus is involved in encoding spatial maps or is involved more generally in relational memory. Well, the argument for general relational memory just got a big boost. Johnson and Redish published a paper in the Journal of Neuroscience showing that rats mentally project forward to parts of a maze they haven't visited yet. It is solid evidence for prospective coding in the hippocampus.

Just to recap that a little bit, for many years we have known that the hippocampus is involved in encoding new memories because patients -- like patient HM -- who lost their hippocampi could not form new memories. We thought that the manner in which the hippocampus encoded new memories was by attaching them -- sort of like mental post-it notes -- to parts of a spatial map that was formed in the hippocampus. The reason for this fixation with spatial maps was because if you record the activity in cells in the hippocampus (usually in a rat), you find that the cells have place fields. A cell with a place field is a cell that fires in a certain part of space such as a certain part of a maze.

It turns out that hippocampal cells can encode a variety of other things besides space. Hippocampal maps are very plastic, and they change around -- called remapping -- for strange things like whether the animal is hungry or thirsty. (Reviewed here. Full disclosure: this review was written by three people in my lab.)

The observation that hippocampal neurons can encode a variety of other non-spatial attributes has led some people to wonder whether the hippocampus is involved more generally in relational memory. Particularly, it may be that the hippocampus encodes information in such a way as to enable prospective thinking.

Prospective thinking works like this.

Say A is the network state and A is related to A'. In a general structure for relating elements, A' doesn't just have to be something now. A' can be anything including a prediction about future events or a reconstruction of past events. For example, if you see a bowling ball rolling along a shelf, you can predict what can happen to that bowling ball when it reaches the end of the shelf. This is because you relate that present state to your previous experiences with future states. (Just to be clear, this is different from the prediction that the bowling ball will continue along in a straight course on the shelf or that it is a solid object. That involves the visual system.)

The interesting part about the hippocampus being involved in general relational memory is that it means that the hippocampus is capable of what I would call projection -- projecting the likely future events from present states. Most people in the field call this prospective memory.

Anyway, that is all fine speculation, but what evidence is there for it. Well, the first piece of an emerging body of evidence is that humans with lesioned hippocampi have deficits in prospective memory.

The paper I want to talk about is another piece of evidence. Johnson and Redish recorded the activity in hippocampal cells (in layer CA3 for the initiated) in rats when they were put into a maze.

The maze looks something like this. (From Figure 1 of the paper.)

The rat runs around in a figure-8. It runs down the center corridor, and when it reaches the top it has to make a choice. One of the paths will be rewarded and the other will not. The rat wants to make the right choice because if it chooses wrong then it has to run all the way through the maze again to get another chance at reward.

Now, normally when you do experiments like this you find that the hippocampal neurons show place fields meaning that certain cells only fire when the rat is a particular point on the maze. And they do find this result.

At the choice point of the maze, however, the recorded hippocampal neurons do something very odd. What the rat will do is look down the maze in the direction of each of its two choices. When the rat looks down in one direction place cells that correspond to position down that path will activate. When the rat looks down the other direction place cells corresponding to position in that part of the maze will activate. The important part is though that the rat has not reached that point in the maze yet; the choice point causes a dissociation between the location of the animal and the firing of the place cells. More interestingly, it is as if the rat's brain is simulating what is going to happen if the rat runs in that direction.

These sweeps -- as the authors call them -- happen incredibly fast: around 150 msecs. Below depicts what they data looks like this. (From Figure 7 of the paper. Click to enlarge.)

What you are looking at is an pictorialized overhead view of a rat coming to a choice point in the maze. The rat is indicated by the white circle. The colors indicate the firing rates for hippocampal neurons with place fields on the maze with reddish parts indicating high firing rate. The figure shows time slices of the firing rates of variety of neurons at the choice point. What you see is that as time passes, the focus of the reddish parts -- corresponding to high firing rate -- moves first down one choice arm and then down the other. This means that the hippocampal neurons with place fields associated with each of those choice arms are firing. But they are not firing when the rat is on that part of the maze; rather, they are firing at the choice point when (apparently) the rat is thinking about being at that point in the maze.

I know that these pictures are hard for people to read, so the authors have some videos some of which I would like to show. (The video is from supplementary data 2.)

The video shows basically the same thing as the pictures, although now you can see it in time. See how when the rat is running up the center of the maze -- the part where it doesn't need to make choices -- the firing is always on top of the rat. This indicates that the hippocampal neurons with place fields on the maze only fire when the rat is in that place on the maze -- under normal circumstances. This changes at the choice point. Here the activity sweeps to place fields down either arm. (If you are wondering why it happens so fast, the computation for deciding which arm to go down also happens fast.)

There are two complexities worth mentioning about this paper. The first is an interesting observation. Say the rat decides to go down the wrong arm. At some point it will realize that it screwed up and want to turn around. When it does this -- if the researchers allow it -- you can see that the activity sweeps up the opposite arm again. This confirms their result that the animal is using this activity to think ahead.

Another interesting aspect is that the size of the activation during a sweep does not correlate with whether the rat decides to go down that arm or not. The authors explain this by saying that the reward-assignment -- whether or not that prospective course is likely to yield reward -- is happening in a different part of the brain. (They suggest several candidates among which is the prefrontal cortex, which is what I study.)

Why is this paper so cool?

Think about it this way. You live in NY. It is Saturday night, and you would like to go out downtown to meet some friends. There is this girl/guy that you would like to meet there, but they are only staying a certain amount of time...yada yada yada. The moral is you have to get there ASAP. How are you going to get there? Well you could take a cab, but traffic is murder this time of night. You could take the subway, but in your experience you have to wait for the subway to come on Saturday evenings because they are scheduled pretty sparsely. You could take the bus, but aside from the traffic problem only crazy people ride the bus at night.

The mental calculation that you are performing is exactly like the rat on the maze. You are mentally walking yourself through the motions of getting downtown and making calculations of how long it will take.

The authors of the paper have a much more elegant explanation of their work, so I will leave you with that:

The cognitive map was originally proposed to explain the flexible spatial navigation abilities observed in animals. Several researchers have shown that behavior aligns to the goal locations expected given an active place field distribution, whereas other researchers have shown goal-sensitivity during navigation processes. These previous results suggest that how the animal's current position is represented contributes to navigation, but the mechanism by which the hippocampal representation contributes to navigation remains an open question. The results presented here show that when animals pause during behavior, the hippocampal representation becomes transiently nonlocal. At high-cost choice points and at the correction of errors, the hippocampal representation sweeps forward along the potential paths available to the animal. These transient nonlocal signals could provide a potential substrate for the prediction of the consequences of decisions and the planning of paths to goals.

Nonlocal forward representations are not sufficient for the consideration of future possibilities. Such consideration processes would also require mechanisms for evaluation of nonlocal representations as well as mechanisms for flexible translation into behavior. Previous models of goal-directed navigation have suggested that goal-directed navigation arises from a planning-capable system based on the consideration of possibilities. These models suggest that some structure is providing the animal with a prediction of the consequences of its actions, from which an evaluation of the goal can be reached, and a decision made....This may explain why we have not observed a correlation between the directions reconstructed and the choices made by the animal on the cued-choice task: the hippocampus may only be providing the prediction component; evaluation of the value of that prediction and the making of the decision may happen downstream of the hippocampal prediction process. (Emphasis mine. Citations removed.)

The complete citation for this paper:

Adam Johnson, and A. David Redish. "Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point." J. Neurosci. 27: 12176-12189; doi:10.1523/JNEUROSCI.3761-07.2007

For further reading, check out this new coverage of the article. Also, these are not the only guys who have found this result. Similar results were obtained in this paper.