Temporal discounting is our tendency to want things now rather than later. In order to encourage us to save money, banks have to offer us a reward in the form of an interest rate. In order to delay gratification, we have to be convinced that the reward in the future is going to be sufficiently large to compensate us for going without right now.
When economists talk about temporal discounting, they talk about it in terms of what is called the discount rate. The discount rate is the percentage of money that you would have to be offered after a time period to convince you to save. Alternatively it could also be viewed as how much less — by percentage — your money is worth to you in say a year than it is right now. The discount rate is related to the interest rate.
In a lot of economic models, temporal discounting is measured in terms of a exponential function. You take the present reward — money or whatever — and multiply it by the discount rate raised to the power of the time over which that reward is discounted. We know, however, from behavioral economics and neuroscience that people don’t have an exponential discounting function.
Actually, we know that the amount something is worth to you declines precipitously in the near future and declines more slowly into the far future. This makes sense right: the difference between 365 days and 366 days is not psychologically significant whereas now and an hour from now matters a lot. Because of this observation, we say that the discount rate is declining over time and that temporal discounting is a hyperbolic function.
(How do you test this? To do a temporal discounting experiment, we give people a forced choice. You can have $10 now or $11 in ten minutes. We then vary the reward or the time of the waiting period until equal proportions of individuals will take either offer. If equal numbers of people would take $10 now and $15 in an hour, we would say that those two figures are equally rewarding. Taking into account the time, we can then compute a temporal discounting function as the value that would be required to equal $10 right now at any point in the future.)
There has been a lot of interest in the neurological correlates of temporal discounting. We know that brain activity in regions associated with reward declines offer time with temporal discounting. We also know that this decline follows a hyperbolic function. (For more on this, read this post by Chris Chatham.)
But fMRI studies are sometimes difficult to interpret, right? What would be great is if we could confirm these studies by doing neuronal recordings in a primate model of temporal discounting. If we could show that reward-related neurons show declining activity with temporal discounting, we could validate the fMRI data. Furthermore, we could determine whether these neurons have a hyperbolic or exponential decline in activity.
Here is where, Kim et al. publishing in Neuron come in. Kim et al. do neurons recordings in the dorsolateral prefrontal cortex (dlPFC) in monkeys and show that these neurons show the features of temporal discounting.
The task that Kim et al. use works something like this. A monkey has to fixate on spot in a center of a screen with its eyes. After a delay period, two colored dots come up on either side of the fixation point. One colored dot (green) is associated with a small reward right now. The other colored dot (red) is associated with a larger reward that is delayed. The animal has a sense of how long the delay is going to be because the red dot is surrounded by smaller blue dots that indicate the number of seconds the animal will have to wait. (We are talking seconds because monkeys are impatient creatures. Temporal discounting functions are generally pretty steep for other species. Even with humans, they are pretty steep. Think of a toddler…)
After a delay period still fixating on the center, the animal will choose one of the colored dots and receive its reward. This delay period is critical because it when the animal is choosing which dot it will pick. It is also when the recording data that I am going to discuss was taken. (Note how this experiment is very similar to the force-choice task I discussed above.) The duration of the waiting time to receive the reward is varied until the monkey picks both dots with equal frequencies.
Rather than bore you with huge quantities of data (most of which is presented in a very technical manner), I am going to reproduce and explain one panel from one figure. Below is the firing rate of one neuron that shows the temporal discounting effect during the delay period. (Figure 3C)
The top left shows the average firing rate for that neuron through all the trials vs. time. The gray area is during the period of the delay when the animal is deciding which way to go. The red trace is when the animal chooses the normal reward when it is presented on the left. The orange trace is when the animal chooses the temporally discounted reward when it is presented on the left. See how the activity declines. The dark blue trace is when the animal chooses the normal reward on the right. The light blue trace is when the animal chooses the temporally discounted reward on the right. See again how the activity declines with temporal discounting. The top right shows this same data as spike averages during the whole waiting period. Dark line is normal reward; light line is temporally discounted.
The bottom row shows the correlation between the temporally discounted value of the reward and the firing rate of the neuron. (Just look at the black line. Ignore the rest.) DV is the discounted value of reward (mL of fruit juice). L and R indicate whether the response was to the left or right. Note how if the reward is less because of discounting, the firing rate is also less — but only if the response was on the left.
Now that I have confused you all totally, there is an important thing to note. Why did the authors break down the responses into whether the dot was on the left and right? This is because one of the things they noticed in their data is that prefrontal neurons encoding temporal discounting have a spatial preference. You may only see the effect for responses in one direction. This is not a surprising finding. Prefrontal neurons often show direction preferences, and this is probably because they are involved in encoding action selection.
I am not going to show anymore of their data, but I would like to summarize their other interesting findings.
- 1) They found that screwing with the quantity of the reward and the delay to receive it have equivalent effects on neuronal firing. If you decrease the value of the reward, the neuron fires less. If you increase the waiting time, the neuron fires less. This is exactly what you would expect if the neurons are encoding the temporally discounted magnitude of reward.
- 2) They also found that the during the delay period, activity selective to temporal discounting emerges before activity-related to the animal’s actual choice. This is interesting if you think that the neurons encoding temporal discounting are determining the animal’s choice.
- 3) They found that the animal’s behavior and the activity followed a hyperbolic rather than an exponential discounting function.
This is some very compelling work that I think really illuminates this field.
I do have some caveats, though. First, where they were doing neuronal recordings is an issue. Studies in humans have identified temporal discounting activity in the brain, but not in the dorsolateral prefrontal cortex where these guys were recording. That may have to do with the resolution of fMRI studies in humans, but that is an issue they need to resolve. Second, recording studies of this nature are difficult to interpret without lesion studies. The purpose of a lesion study is to demonstrate the function of a specific brain region. You ablate that region and see what the animal can’t do. We can say that these neurons change their activity with temporal discounting. What we can’t say is that their function is to do so. In order to say that, we would need to do a lesion study and show that the temporal discounting behavior is not present in the lesioned animals. Such a study has not to my knowledge been done.
All in all, very interesting work in a very interesting field!
Post script: I found this sentence in the Discussion particularly amusing. It shows that animals are particularly impatient:
The specific value of the discount factor obtained in the present study ranged from 0.12 to 0.46 s^-1, indicating that the subjective value of reward was diminished by half when the reward was delayed by 2.2 to 8.3 s. Although this indicates that the monkeys tested in the present study were quite impatient, this is comparable to the values observed in pigeons (0.3 to 2.24 s^-1; Mazur, 2000) and rodents (0.07 to 0.36 s^-1; Richards et al., 1997).
KIM, S., HWANG, J., LEE, D. (2008). Prefrontal Coding of Temporally Discounted Values during Intertemporal Choice. Neuron, 59(1), 161-172. DOI: 10.1016/j.neuron.2008.05.010
UPDATE: Made some spelling and grammar corrections. Sorry. I wrote the original post very early in the morning.