Temporal discounting is our tendency to want things now rather than later. In order to encourage us to save money, banks have to offer us a reward in the form of an interest rate. In order to delay gratification, we have to be convinced that the reward in the future is going to be sufficiently large to compensate us for going without right now.
When economists talk about temporal discounting, they talk about it in terms of what is called the discount rate. The discount rate is the percentage of money that you would have to be offered after a time period to convince you to save. Alternatively it could also be viewed as how much less -- by percentage -- your money is worth to you in say a year than it is right now. The discount rate is related to the interest rate.
In a lot of economic models, temporal discounting is measured in terms of a exponential function. You take the present reward -- money or whatever -- and multiply it by the discount rate raised to the power of the time over which that reward is discounted. We know, however, from behavioral economics and neuroscience that people don't have an exponential discounting function.
Actually, we know that the amount something is worth to you declines precipitously in the near future and declines more slowly into the far future. This makes sense right: the difference between 365 days and 366 days is not psychologically significant whereas now and an hour from now matters a lot. Because of this observation, we say that the discount rate is declining over time and that temporal discounting is a hyperbolic function.
(How do you test this? To do a temporal discounting experiment, we give people a forced choice. You can have $10 now or $11 in ten minutes. We then vary the reward or the time of the waiting period until equal proportions of individuals will take either offer. If equal numbers of people would take $10 now and $15 in an hour, we would say that those two figures are equally rewarding. Taking into account the time, we can then compute a temporal discounting function as the value that would be required to equal $10 right now at any point in the future.)
There has been a lot of interest in the neurological correlates of temporal discounting. We know that brain activity in regions associated with reward declines offer time with temporal discounting. We also know that this decline follows a hyperbolic function. (For more on this, read this post by Chris Chatham.)
But fMRI studies are sometimes difficult to interpret, right? What would be great is if we could confirm these studies by doing neuronal recordings in a primate model of temporal discounting. If we could show that reward-related neurons show declining activity with temporal discounting, we could validate the fMRI data. Furthermore, we could determine whether these neurons have a hyperbolic or exponential decline in activity.
Here is where, Kim et al. publishing in Neuron come in. Kim et al. do neurons recordings in the dorsolateral prefrontal cortex (dlPFC) in monkeys and show that these neurons show the features of temporal discounting.
The task that Kim et al. use works something like this. A monkey has to fixate on spot in a center of a screen with its eyes. After a delay period, two colored dots come up on either side of the fixation point. One colored dot (green) is associated with a small reward right now. The other colored dot (red) is associated with a larger reward that is delayed. The animal has a sense of how long the delay is going to be because the red dot is surrounded by smaller blue dots that indicate the number of seconds the animal will have to wait. (We are talking seconds because monkeys are impatient creatures. Temporal discounting functions are generally pretty steep for other species. Even with humans, they are pretty steep. Think of a toddler...)
After a delay period still fixating on the center, the animal will choose one of the colored dots and receive its reward. This delay period is critical because it when the animal is choosing which dot it will pick. It is also when the recording data that I am going to discuss was taken. (Note how this experiment is very similar to the force-choice task I discussed above.) The duration of the waiting time to receive the reward is varied until the monkey picks both dots with equal frequencies.
Rather than bore you with huge quantities of data (most of which is presented in a very technical manner), I am going to reproduce and explain one panel from one figure. Below is the firing rate of one neuron that shows the temporal discounting effect during the delay period. (Figure 3C)
The top left shows the average firing rate for that neuron through all the trials vs. time. The gray area is during the period of the delay when the animal is deciding which way to go. The red trace is when the animal chooses the normal reward when it is presented on the left. The orange trace is when the animal chooses the temporally discounted reward when it is presented on the left. See how the activity declines. The dark blue trace is when the animal chooses the normal reward on the right. The light blue trace is when the animal chooses the temporally discounted reward on the right. See again how the activity declines with temporal discounting. The top right shows this same data as spike averages during the whole waiting period. Dark line is normal reward; light line is temporally discounted.
The bottom row shows the correlation between the temporally discounted value of the reward and the firing rate of the neuron. (Just look at the black line. Ignore the rest.) DV is the discounted value of reward (mL of fruit juice). L and R indicate whether the response was to the left or right. Note how if the reward is less because of discounting, the firing rate is also less -- but only if the response was on the left.
Now that I have confused you all totally, there is an important thing to note. Why did the authors break down the responses into whether the dot was on the left and right? This is because one of the things they noticed in their data is that prefrontal neurons encoding temporal discounting have a spatial preference. You may only see the effect for responses in one direction. This is not a surprising finding. Prefrontal neurons often show direction preferences, and this is probably because they are involved in encoding action selection.
I am not going to show anymore of their data, but I would like to summarize their other interesting findings.
- 1) They found that screwing with the quantity of the reward and the delay to receive it have equivalent effects on neuronal firing. If you decrease the value of the reward, the neuron fires less. If you increase the waiting time, the neuron fires less. This is exactly what you would expect if the neurons are encoding the temporally discounted magnitude of reward.
- 2) They also found that the during the delay period, activity selective to temporal discounting emerges before activity-related to the animal's actual choice. This is interesting if you think that the neurons encoding temporal discounting are determining the animal's choice.
- 3) They found that the animal's behavior and the activity followed a hyperbolic rather than an exponential discounting function.
This is some very compelling work that I think really illuminates this field.
I do have some caveats, though. First, where they were doing neuronal recordings is an issue. Studies in humans have identified temporal discounting activity in the brain, but not in the dorsolateral prefrontal cortex where these guys were recording. That may have to do with the resolution of fMRI studies in humans, but that is an issue they need to resolve. Second, recording studies of this nature are difficult to interpret without lesion studies. The purpose of a lesion study is to demonstrate the function of a specific brain region. You ablate that region and see what the animal can't do. We can say that these neurons change their activity with temporal discounting. What we can't say is that their function is to do so. In order to say that, we would need to do a lesion study and show that the temporal discounting behavior is not present in the lesioned animals. Such a study has not to my knowledge been done.
All in all, very interesting work in a very interesting field!
Post script: I found this sentence in the Discussion particularly amusing. It shows that animals are particularly impatient:
The specific value of the discount factor obtained in the present study ranged from 0.12 to 0.46 s^-1, indicating that the subjective value of reward was diminished by half when the reward was delayed by 2.2 to 8.3 s. Although this indicates that the monkeys tested in the present study were quite impatient, this is comparable to the values observed in pigeons (0.3 to 2.24 s^-1; Mazur, 2000) and rodents (0.07 to 0.36 s^-1; Richards et al., 1997).
KIM, S., HWANG, J., LEE, D. (2008). Prefrontal Coding of Temporally Discounted Values during Intertemporal Choice. Neuron, 59(1), 161-172. DOI: 10.1016/j.neuron.2008.05.010
UPDATE: Made some spelling and grammar corrections. Sorry. I wrote the original post very early in the morning.
- Log in to post comments
That the function is hyperbolic rather than exponential is intriguing, and I'm very curious: what is the actual function? A comparison between this and a "real" exponential return on money could be an economic irrationality that some tactics have already grown up to exploit, and I'd like to know what the function is to see if I can spot some.
Quick edit: I see that the study mentions a discount factor of 0.12 per second.... but that's exponential unless I've misinterpreted it (value assigned = nominal value * (1-.12)^t, t in seconds). So... huh?
In order to encourage us to save money, banks have to offer us a reward in the form of an interest rate.
What a stupid (mis)characterization. The banks "reward" us for giving them our money, which they then put to use -- they don't stick it in a lock box, and they aren't rewarding us for developing good habits, as if they were our parents.
William,
Here are the forms of the functions (from the paper):
Exponential
DV(A,D)=Aexp(−kD)
Hyperbolic
DV(A,D)=A/(1+kD)
where DV is the discounted value, A is the value of reward and D is the delay. k (or Kappa) is the discounting factor. Note how in both there is a discounting factor k. It just means something different. (The factor referred to in the quote is the hyperbolic discount factor.)
truth machine,
The purpose of my statement was in no way to suggest altruism or paternalism on the part of banks. I was just saying that psychologically we are only attracted to save when banks make it worth our while.
Well, the exponential model in econometrics doesn't pretend to account for psychological or neurocognitive factors but is based simply on the way that interest is defined (with continuous compounding for analytical convenience). So it is useful as a model of a real phenomenon, albeit a wholly artificial one.
The results here, with their rather different context, are fascinating. It's hard -- at least for someone who has ostensibly escaped from finance to science -- not to wonder how this might interact with the structure of financial instruments to produce market effects. Attempts to analyse things like the term structure of interest rates invariably assume a 'rational' motivation on the part of investors. The introduction of some innate non-rational behaviour could have interesting consequences. But I fear I'm far too lazy to try to work through the maths and find out...
I got it! Primate discounting is the reverse of simple interest!
See, exponential value discounting is balanced against continuously compounded interest. If a bank wants A in cash from you now, they promise to pay A*exp(kD) to you at any indefinite point D later. If the account calculated interest simply, though, they would be paying you A*(1+kD).
The value the primate brain assigns to a future reward is the value which, if you had it now and waited through the delay time, accruing simple interest on your possession, would result in the actual value to be earned then.
Let me think. What does this imply for economic decision-making? Suppose, given my assets and personality, I discount future cash at a rate of k per year. I have a certain amount of cash right now. If a bank offers a continuously compounded interest rate of k, then at any given future time D the value I would be able to withdraw is A*exp(kD), and which I discount internally by (1+kD), meaning I value my future withdrawal at A*(exp(kD)/(1+kD)) -- and this factor is always bigger than 1, and grows with D. If the bank offers any positive rate m less than k, then my primate brain tells me that over the short time my money is more valuable with me than with them; but if I leave it long enough, exp(mD)/(1+kD) will eventually be greater than 1, and at this point, say D_t, the withdrawal will have been worth my time.
Now, D_t grows as m decreases. So... what are the economic ramifications of that?
Nice study, but I am interested to know that how one can address the issue of temporal discounting when you have a competition between a number of brains that are not willing or able to communicate with each other. In a recent game-theoretic analysis I showed that if the quantity of the reward passes a threshold of seven to one, then the reasonable player will be better off by choosing patience strategy regardless of which options the other player will choose:
http://www.vahidthinktank.com/articles/26.htm
I wonder if you can test such competition based games by fMRI studies on humans and other animals.
This is a familiar pairing.
Exponential decline: Gaussian/Poisson (scale-limited)
Hyperbolic decline: Power-law (scale-free)
The economic dichotomy between brain and financial system (William) is certainly intriguing given the recent financial woes. Certainly the application of linear mind models to higher order world realities is often a source of conflict. This deserves some thought.