Pavlovian-Instrumental Transfer in Humans

In behavioral neuroscience, we use a lot of animal models. We assume that these animal models have features that are the same or similar to features of humans. However, it is always reassuring when someone gets around to proving that this assumption is accurate.

Talmi et al., publishing in the Journal of Neuroscience, show that a well-documented type of learning called Pavlovian-Instrumental Transfer (PIT) occurs in humans under identical experimental conditions to those we use to test animals.

Background

In order to understand this paper, I need to define some terms that you come across in studying conditioning. Conditioning describes a learning process where the likelihood of a particular response changes in relation to a particular stimulus. There are many types of conditioning of which I want to talk about two.

• Pavlovian conditioning -- In Pavlovian conditioning (also called classical or respondent conditioning) a conditioned stimulus (CS) is related to an unconditioned stimulus (US) that elicits a particular response. This is a type of conditioning that was discovered by Pavlov in his experiments with dogs. It works like this: say every time I show a dog food he starts to salivate in anticipation of receiving the food. In this case, seeing the food constitutes the US and the salivation is the response associated with that US. Now what I do is every time I show a dog the food, I ring a bell. The bell ringing is the CS. What happens after repeated presentations of showing food and ringing the bell? Now the dog begins to salivate whenever you ring the bell, even if you do not show the animal food. Thus, the ringing of the bell (CS) has become associated with showing food (US), and now both are sufficient to trigger salivation (the response).
• Instrumental conditioning -- In instrumental conditioning (also called operant conditioning) the likelihood of a particular response is increased by the animal receiving reward only when it performs that response. I think this is the more intuitive of these first two types of conditioning. Say I put a rat in a chamber where every time he presses a lever down, he receives a treat. What will happen? The rat will increase the numbers of times that he pushes down the lever because he wants to get more treats. In this case, the animal has associated the performance of a response with a particular reward.

Here is an important point to remember: the key difference between Pavlovian and instrumental conditioning is that in instrumental conditioning the animal has to do something to achieve the reward. In Pavlovian conditioning, the animal is basically just sitting there. This is because in many cases the response in Pavlovian conditioning is involuntary. Instrumental conditioning deals with voluntary responses; Pavlovian deals with involuntary responses.

There are lots of variations on these two types of conditioning. One is an interesting effect called Pavlovian-instrumental transfer (PIT). In PIT, you train an animal that it will receive a reward in two ways. The first way is when the reward just falls from the sky. This is done in parallel to displaying some stimulus -- like a tone or a flashing light, and it will result in the animal associating that stimulus with the receipt of reward. Therefore, the first represents Pavlovian conditioning. Then you also train the animal that reward can be achieved a second way, by pushing down a lever or something like that. In this second way, the animal has to do something to get the reward, so this is an example of instrumental conditioning. An important part is that you need to make the rewards the same for both types of conditioning. (Sometimes...see the caveat below.)

Here is where the experiment starts. Now you take the rewards away for both types of conditioning. You put the animal in the testing area where they have access to the lever, and you present the stimulus. Remember that the animal should be pressing the lever because he assumes that pushing the lever will result in reward. The stimulus is also related to reward.

When you present the stimulus, and you ask does the rate of lever pressing increase? The answer is yes it does, and that this is a case of PIT. What happened is that the stimulus and the lever press being associated with reward has increased the motivational significance of lever pressing. The animal expects greater reward, and so it presses the lever more regularly. The Pavlovian conditioning has transferred motivational significance onto the instrumental conditioning.

(Caveats: I am eliding a couple complexities here. One, there is general and specific PIT. In some cases the stimuli can increase instrumental responding to get any reward, not just the one associated with the stimulus. This is called general PIT. In other cases, it will only encourage instrumental responding to get the same reward -- i.e. specific PIT. Also, PIT -- as I understand it -- is only active for habitual rather than newly learned behaviors.)

Why would we care about PIT? It seems like an obscure type of learning.

Well, that isn't entirely true. PIT features highly in some of our theories of motivation and addiction. Let me give you an example from drug addiction: relapse. Say a drug addict is trying to get clean. This person has been thoroughly instrumentally conditioned that going to their drug dealer will result in happy things happening to them -- i.e. a fix -- even if bad things will probably happen later on. But they have also been subject to Pavlovian conditioning to associate particular contexts with getting drugs. Say there a particular street where they regularly purchase drugs or a group of friends with whom they regularly do drugs. These stimuli -- in this case, more like context -- increase the likelihood that the drug addict will seek out the drug dealer. The stimuli increase the likelihood of that response. This is an example of PIT.

Talmi et al.

Or at least that is the theory. It turns out that existing literature for human's regarding PIT has differed from animal literature in a couple ways. First, when testing the ability of Pavlovian stimuli to increase instrumental responding, other researchers have relied on Pavlovian conditioning that occurred prior to the experiment. Second, the ideal experiment -- one comparable to the literature in animals -- measures instrumental responding under what we call extinction -- when no more reward is being offered. Previous studies have not tested human responding under conditions of extinction; therefore, they may not be comparable.

Talmi et al. attempt to remedy these deficiencies by performing a PIT experiment on humans that mirrors as closely as possible those in animal studies. To wit:

Here we adapted a standard animal PIT paradigm for use with humans in the context of functional magnetic resonance imaging (fMRI). Participants were first taught two separate relationships: (1) a purely predictive (i.e., pavlovian) association between a combined auditory and visual stimulus (CS+) and monetary reward; and (2) a contingent instrumental association between squeezing a handgrip and the delivery of money. We then monitored the frequency with which subjects squeezed the handgrip in extinction (i.e., without any money being delivered) in the presence of either CS+ or other (control) stimuli. We predicted that the frequency of handgrips would increase in the presence of the CS+ relative to the other stimuli and sought to study the association between this effect and the blood oxygenation level-dependent (BOLD) signal in regions associated with PIT in animals, specifically the amygdala and NAcc [Ed. nucleus accumbens].

The idea here is to test A) whether the CS being present (CS+) increases the frequency of instrumental responding (the handgrip) and B) whether the same brain regions that we know are required for PIT in animals are also activated during PIT in humans. (Ideally, we would like to show that such regions are required in humans as well, but lesion studies in humans tend to be frowned upon by institutional ethics committees.)

The authors had the following key findings:

• First, when the CS was present, the participants used the handgrip more. This happens under conditions of extinction meaning that when reward was terminated, the participants were more likely to persevere in using the handgrip when the CS was present than if it was not. This is an effect directly comparable to that observed in animal studies. Humans clearly show PIT.
• Second, when the subjects were imaged during the performance of the task, the areas that are required for PIT in animals -- namely the nucleus accumbens and the amygdala -- are activated in humans. They are also more active when the CS is present than when it is not.

The Figure below illustrates the second finding. This figure shows the activation in the human brain during PIT. The bar graph shows a comparison in the activity in these regions when CS was present vs. when the CS was not present. (From Figure 6 in the paper. The labels for the regions in green I added. Can someone who is better with imaging confirm that those are actually the amygdala and nucleus accumbens?)

Here is just a strange side note. It is a reasonable question to ask in this study: do the people know that they are responding more because of the CS? Are you aware of PIT while it is happening? This is something that you definitely can't ask animals, but you can ask humans. Here is what the authors had to say about whether the subjects were aware of this effect:

Participants were classified as being "aware" or "unaware" of the PIT effect according to their response to the following debriefing question: "Sometimes there were sounds playing with the grip task and images of different colors presented in the background. How did that affect you, if at all?" To be classified as aware, participants had to indicate that the stimuli influenced their behavior in the predicted way (e.g., "In the high pitch tone [this participant's CS+], you knew you will win money, it was easier to grip, you were faster, it affected you"). Responses to the debriefing question were scored for exploratory purposes, although this assessment of postexperimental awareness of PIT cannot determine a causal role for awareness at the time of test. Nine participants (56%) were classified as aware. Aware participants exhibited larger PIT than unaware participants (r = 0.62, p = 0.01). The difference between grip frequency in the presence of the CS+ and CS- was significant for aware participants (t(8) = 3.19, p < 0.05) but not for unaware participants (t(6) = 0.0).(Emphasis mine.)

Interestingly enough, even if you were aware of the effect it still happened. In fact, the effect was more powerful when you were aware of it than when you were not. Fascinating. (They qualify this finding by noting that the participants were asked afterward whether they were aware. Thus, you can't really say whether they were aware during the study.)

How is this paper important?

In science we often draw elaborate theoretical models based on data from studies in animals. We would like to pretend that these models apply easily to humans, and in most cases they do. However, this is not always the case, and it is particularly liable to be wrong when we are trying to model quintessentially human behaviors or diseases using animal models. For example, how does one model depression in a rat?

The point is that experimental verifications of this type are critical. Science works best when it goes regularly to the well of experiment, not when it builds huge theoretical castles in the sky on a limited basis of data. PIT is important because many of our models of motivation are predicated on findings from PIT experiments. I am happy to see that studies like this get done and that we weren't wrong about PIT.

Talmi, D., Seymour, B., Dayan, P., Dolan, R.J. (2008). Human Pavlovian Instrumental Transfer. Journal of Neuroscience, 28(2), 360-368. DOI: 10.1523/JNEUROSCI.4028-07.2008