Tonic Dopamine and Response Variability

In a 2006 Psychopharmacology article, Niv et al. suggest that while transient dopamine release is frequently modeled computationally (as encoding reward-prediction error, for example, or as gating information into working memory) the role of more constant dopamine release is not. In the neuroscience literature, these two patterns of release are known as "phasic" and "tonic," respectively.

The authors argue that current models of dopamine release have three major shortcomings: first, they do not explicitly address the effect of dopamine manipulation on response latency or "vigor"; second, some effects of dopamine manipulation are immediate, whereas computational models tend to assume that dopamine's effects deal with learning over the longer term; third, they do not convincingly explain the effects of differences in tonic dopamine and how that may interact with phasic DA release.

To address these issues, Niv et al. introduce a new component to computational models of dopamine: the average rate of reward. The authors suggest that this value is encoded by tonic dopamine levels (possibly in the nucleus accumbens), on the basis of the following evidence:

1) A mathematical reinforcement learning model of free-operant conditioning reproduces several findings from the behavioral literature, including the enhanced responsivity that results from hunger (modeled as higher tonic dopamine levels, because each food pellet is worth more) or dopamine agonists.

2) Depletion of dopamine from this model differentially slows responding to high ratio relative to low ratio reinforcement, just as in behaving rats. Similarly, dopamine depletion in the model may exaggerate perseveration.

The authors offer several predictions, including that tonic dopamine is higher in deprived than sated motivational states. This view of tonic dopamine's function contrasts with others in the literature, which will be reviewed in future posts.

More like this

How do the symptoms of ADHD relate to the circuitry underlying executive function and working memory? An in-press article at Neuropsychopharmacology investigates the roles of dopamine and norepinephrine in ADHD, with evidence from both behavioral and simulated experiments. This post will make…
Whereas yesteryear's artificial neural networks models were focused on achieving basic biological plausibility, today's cutting edge networks are modeling cognitive phenomena at the level of neurotransmitters. In a great example of this development, McClure, Gilzenrat & Cohen have an article…
Dopamine is probably the most studied neurotransmitter, and yet the neuroscience literature contains a huge variety of perspectives on its functional role. This post summarizes a systems-level perspective on the function of dopamine that has motivated several successful drug studies and informed…
Very early in the history of artificial intelligence research, it was apparent that cognitive agents needed to be able to maximize reward by changing their behavior. But this leads to a "credit-assignment" problem: how does the agent know which of its actions led to the reward? An early solution…