In a 2006 Psychopharmacology article, Niv et al. suggest that while transient dopamine release is frequently modeled computationally (as encoding reward-prediction error, for example, or as gating information into working memory) the role of more constant dopamine release is not. In the neuroscience literature, these two patterns of release are known as "phasic" and "tonic," respectively.
The authors argue that current models of dopamine release have three major shortcomings: first, they do not explicitly address the effect of dopamine manipulation on response latency or "vigor"; second, some effects of dopamine manipulation are immediate, whereas computational models tend to assume that dopamine's effects deal with learning over the longer term; third, they do not convincingly explain the effects of differences in tonic dopamine and how that may interact with phasic DA release.
To address these issues, Niv et al. introduce a new component to computational models of dopamine: the average rate of reward. The authors suggest that this value is encoded by tonic dopamine levels (possibly in the nucleus accumbens), on the basis of the following evidence:
1) A mathematical reinforcement learning model of free-operant conditioning reproduces several findings from the behavioral literature, including the enhanced responsivity that results from hunger (modeled as higher tonic dopamine levels, because each food pellet is worth more) or dopamine agonists.
2) Depletion of dopamine from this model differentially slows responding to high ratio relative to low ratio reinforcement, just as in behaving rats. Similarly, dopamine depletion in the model may exaggerate perseveration.
The authors offer several predictions, including that tonic dopamine is higher in deprived than sated motivational states. This view of tonic dopamine's function contrasts with others in the literature, which will be reviewed in future posts.
- Log in to post comments