Exploration, Reinforcement, and Updating in ADHD

How do the symptoms of ADHD relate to the circuitry underlying executive function and working memory? An in-press article at Neuropsychopharmacology investigates the roles of dopamine and norepinephrine in ADHD, with evidence from both behavioral and simulated experiments. This post will make more sense if you've read my previous posts on norepinephrine and dopamine.

Frank, Santamaria, O'Reilly & Willcutt review evidence that ADHD can be characterized by low dopamine levels in the basal ganglia, resulting from abnormally high levels of dopamine transporters. Recent work suggests that ADHD is associated with both lower phasic and tonic dopamine activity, both of which are alleviated with stimulants (such as ritalin), which seem to act preferentially on subcortical (as opposed to prefrontal) targets.

The authors made several predictions, outlined below, based on previous computational models of dopamine's affects on positive reinforcement (by exciting the striatal "go" pathway, and inhibiting the striatal "nogo" pathway, leading first to disinhibition of the thalamus and ultimately to action execution), on the updating of working memory (through a similar mechanism), and on the maintenance of long-term goal information in orbito-frontal cortex.

First, Frank et al. predicted that individual differences in these abilities should be correlated, as should medication-related improvements to each ability.

Second, individual differences in RT variability and "erratic behavior" should be related to noradrenaline release (as suggested here, but not necessarily by other models of task-switching), and thus dissociable from individual differences in - and medication-induced changes to - DA release.

To test these hypotheses, the authors administered two tasks to three different groups: healthy adults, unmedicated adults with ADHD, and those same adults on NA/DA combined agonists. One of these tasks, called probabilistic selection, reveals whether a given subject is more sensitive to positive or negative feedback. The second task, called the distractor version of expectancy AX-CPT, reveals the selectivity of working memory updating processes.

The results showed that relative to healthy controls, unmedicated adults with ADHD were impaired at both positive and negative reinforcement learning; however, medication selectively improved positive reinforcement learning. (The authors note that similar selective improvements in positive reinforcement also results from medicating Parkinson's patients and healthy subjects with DA agonists). This finding is consistent with predictions that higher levels of dopamine should improve learning of positive contingencies, but not necessarily negative ones.

Adults with ADHD were more likely to erratically switch choices in the probabilistic selection task; likewise, they also showed greater RT variability than controls on this task. Individual differences in these measures correlated with each other, but only among those with ADHD. This is consistent with claims that NA release is disturbed in ADHD, and results in performance variability (although the fact that medication significantly reduced RT variability but not the erratic switching patterns is only partially consistent with this claim).

Finally, neither switching nor RT variability were correlated with ability to learn from positive reinforcement (as measured by performance on one trial type in probabilistic selection, anyway), supporting the authors' claim that variability and positive reinforcement dissociably index the effects of noradrenaline and dopamine, respectively.

In the working memory task, the performance of unmedicated adults with ADHD suggests that they were not correctly updating memory with context information; medication changed this pattern, such that they began to look more like controls (actually showing more AY errors and less AX & BX errors, for those who are familiar with the task.)

Frank et al. conclude that the behavioral results broadly support computational models of dopamine, in which higher DA levels are important for both learning positive reinforcement and for correctly updating working memory. Similarly, high levels of noradrenaline release relate to errate behavior both in computational models and adults with ADHD, a disorder associated with NA abnormality.

Finally, Frank et al. also suggest that the infamous response inhibition deficits of ADHD may actually result from cerebellar malfunction (and associated impairments in precise timing of motor coordination), from exacerbated RT variability (resulting from NA abnormality), or from both cerebellar and NA dysfunction, rather than from DA specifically (although they do not rule out that possibility).

This perspective contrasts with this model which suggests that response variability may be due to tonic dopamine.

Related Posts:
Dopamine For Dummies
Exploration & Exploitation Balanced by Norepinephrine & Dopamine
Tonic Dopamine and Response Variability