Correcting an Error about Dopamine Signaling

I caught this article in O magazine by fellow ScienceBlogger, Rebecca Skloot of Culture Dish. The article isn't bad. It is about why people have trouble overcoming unproductive habits like trouble exercising.  But I want to correct something she says that is inaccurate.

Dopamine has a primary role in the signaling reward, and this is a point made in her article. But she also says:

Dopamine teaches your brain what you want, then drives you to get it, regardless of what's good for you. It does this in two steps. First you experience something that gives you pleasure (say, McDonald's french fries), which causes a dopamine surge. Some of that dopamine travels to the area of your brain where memories are formed and creates a memory connecting those fries with getting a reward. At that point, in sciencespeak, the fries have become "salient." And when you're exposed to something that's salient, you may think, "That's bad for me, I shouldn't," but your brain registers, "Dopamine jackpot!"

Which is where step two comes in: On top of creating
memories, dopamine controls the areas of the brain responsible for
desire, decision making, and motivation. So once fries become salient,
the next time you see or smell them, your brain releases a surge of
dopamine that drives you to get more fries. When you succeed, your
brain produces more dopamine, which reinforces the memory that made
fries salient in the first place, etching it further into your brain.
It's a never-ending cycle: The more you do something that's rewarding,
the more dopamine makes sure you do it again.
This is precisely how
habits form. Eventually, if the fries become salient enough, your brain
will release dopamine and push you to get fries anytime you see the
colors yellow and red, even if you're nowhere near McDonald's.

And this is true for any behavior that results in a reward: Orgasms cause dopamine surges. So does hitting the jackpot when you gamble, winning a race, acing a test, doing cocaine or methamphetamines, smoking, drinking. (Emphasis mine.)

This description of the action of dopamine is wrong for two reasons.

1)
A nitpicky point: How the reward system works in drug users and how it
works for say shopping or french fries are two very different things. 
(I wrote about how sugar is not quite like a drug here.)  There is a world of difference between cocaine and methamphetamines and acing a test. 

Think
of it this way.  Many drugs -- not all drugs...some work in different
ways -- function to short-circuit the reward system and create the
perception of reward in the absence of any tangible rewarding stimuli. 
Thus, french fries activate the reward system because they taste good. 
Heroin activates the reward system by directly floating into the brain
and activating it.  This would explain why heroin is more rewarding (at
least initially) than french fries. 

I think putting all of
those things in one sentence as if they were equivalent rewards
dangerously confuses the complexity of their different functions.

2) This notion of cascading increases of dopamine release as the result of a reward stimuli is just flat out wrong.  That is not how dopamine works at all.

Dopamine is released by what neuroscientists call reward error.  Reward error is a difference between what you the reward you expected to receive and the reward you actually received. 
Dopamine is only released in the presence of unpredicted rewards, not
repeated rewards.  Further, the dopamine release is lessened if an
expected reward is omitted. 

Basically if I give you a
cookie that you didn't expect one morning, you are like "Nice!  Free
cookie."  You release dopamine.  In contrast, if I give you a cookie
every morning for several weeks, this cookie has become a predicted
reward; hence there is no further release of dopamine.  (Why this is I
will explain in a second.)  Dopamine release would change if you
expected to get a cookie every morning and I didn't give you one.  This
would be an omitted expected reward, and it would cause a reduction in
dopamine release.

The reason that dopamine functions in this way
is because it trains your behavioral responses.  This is a point that
Skloot does make in her article.  The dopamine is released in areas of
the brain such as the striatum and frontal cortex that function in
decision making.  The computation that these regions make is that
whatever the activity was when the dopamine was released, well, let's
do more of that because it was rewarding.  Whatever you are doing when
the dopamine was released, let's do that again. 

However,
it is important to note that once this behavior has been consolidated,
there is no more need for dopamine release.  To use a metaphor, think
of dopamine as the means of digging yourself into a behavioral rut. 
After the digging is done, you are in the rut.  No more digging is
required. 

In addition, Skloot makes light of the
significance of predictors of reward, so-called "salient" cues.  It is
true that you can condition dopamine release if you show someone a
predictor of reward.  The example she uses is being in a supermarket
making you think of junk food.  The supermarket is not in and of itself
rewarding; it is just associated with junk food.  Therefore, the cue --
the supermarket -- can ellicit dopamine release.  However, cues
associated with dopamine release also have a similar period of
attenuation:  after a while the cue does not cause dopamine release
either.

If you don't believe me about this or would like further reading, I urge to read this paper by Wolfram Schultz who is one of the experts in this field.  To summarize what I have argued, here is a quote from his paper:

Dopamine
neurons report rewards relative to their prediction rather than
signaling primary rewards unconditionally...The dopamine response is
positive (activation) when primary rewards occur without being
predicted. The response is nil when rewards occur as predicted. The
response is negative (depression) when predicted rewards are omitted.
Thus dopamine neurons report primary rewards according to the
difference between the occurrence and the prediction of reward, which
can be termed an error in the prediction of reward

What
makes you repeat behaviors is not repeated dopamine release.  It is the
changed activity in the brain regions that facilitate decision making
as a consequence of dopamine action. 




Getting
back to the focus of Skloot's article -- which I didn't think was
half-bad minus that error -- why do we have trouble changing our bad
habits?  The problem is not that dopamine is being constantly
released.  Rather, it is a problem of getting past the behavioral
inertia of the bad habit and replacing it with a good habit.  Later in
her article, she mentions research about habitual behavior, and how
repitition can consolidate these habits.  She suggests that if you did
something for three weeks -- such as exercise, you can make a habit of
it.  

This is accurate.  Old habits can be overcome.  What must
often be replaced are the cues for bad habits.  This is why we tell
former drug-users to avoid their old friends and sometimes move to a
different state.  What also must happen is that you have to suffer
through a couple weeks of not being happy. 

But take heart, the new habits -- as a consequence of dopamine action -- will eventually take hold.

UPDATE:  For those of you who do not usually read the comments, I urge you to in this case.  I think that I may have misinterpreted Skloot's meaning with respect to that part of her article, and if that is true I sincerely apologize.   

Further, there is an interesting discussion about competing hypothesis concerning dopamine signaling in the brain -- and there are some. 

I made an editorial decision to focus on the reward error hypothesis because it possesses the most empirical support, but there are competing theories.  It looks like both Skloot and I both made editorial decisions with respect of much complexity to include -- which is fine so long as you make clear that the subject you are dealing with is a complicated one.  

Categories

More like this

Very early in the history of artificial intelligence research, it was apparent that cognitive agents needed to be able to maximize reward by changing their behavior. But this leads to a "credit-assignment" problem: how does the agent know which of its actions led to the reward? An early solution…
Right about now, many people are starting to fail at their New Year's resolutions. It happens every year: You were good for a couple weeks, but right about now you're starting to slip with a few cookies, some skipped days at the gym ... Why? Because humans aren't very good at changing, which…
Natalie Angiers profiles dopamine, which isn't just about rewards: In the communal imagination, dopamine is about rewards, and feeling good, and wanting to feel good again, and if you don't watch out, you'll be hooked, a slave to the pleasure lines cruising through your brain. Hey, why do you think…
I finally got the internet setup in my new apartment - I won't bore you with my customer service complaints - and I've never been so delighted to waste time on the web. At first, my information vacation was lovely, charming, an experiment in vintage living. It was like traveling back in time to…

To be honest, I wasn't going to talk about that. I have read the paper, and I found it deeply intriguing.

Just to get your take:

I find their arguments about timing compelling, but what about a dopamine system that is retrospective with respect to reward? There could be preconscious visual processing that identifies a certain signature of reward laden objects from past experience, much like the amygdala identifies things like snakes -- even before conscious recognition.

It could also be possible that there is a dual process issue. DA neurons could encode BOTH reward prediction errors and some sort of spatial novelty identifying process. This process could be weighted to identify novel items that may represent possible rewards.

I absolutely agree with them that experiments to disentangle the spatial and unpredicted aspects of the task need to be done and have not been done. I just think there may be some middle ground that incorporates major elements of reward prediction theory while incorporating more complicated time aspects.

I think it deserves mentioning that your correction of dopamine's prescribed function is also a bit of an over simplification. It's my experience that in the field, there is still active debate about dopamine (though perhaps some people have louder voices). There is the popular notion that dopamine signals "pleasure" or reward, which we see talked about a lot in the media. There is also the prediction error hypothesis, which you mention, that has gotten large exposure. A third perspective is sort of talked about in the article you're correcting, that dopamine serves to assign salience to stimuli, and so, in part directs motivated behaviors. This idea basically says that dopamine is responsible for producing the "wanting" that leads people to seek out rewarding stimuli (an oversimplification, definitely). This seems to be looked at in terms of addiction and feeding behavior, primarily. Maybe these ideas are really just different versions of a basic idea centered around reward, but I think it's useful to consider that the issue is by no means completely resolved.

It's obvious that dopamine serves many functions that are relevant for behavior, given that it is involved with multiple pathways - reward, decision making, motor output, etc. Maybe the problem is that people (scientists too) want a clean, simple theory that explains everything, but that's not the reality.

I agree with Ben that the debate on Dopamine is not settled yet and that the Incentive Salience theory of dopamine , to me, appears to be one of the leading candidates that may prove to be correct. That said, some factual corrections in the original Skloot article were definitely due- like that dopamine surge does not keep increasing with repeated exposure to the same, predictable stimulus etc.; so nice article, only perhaps the expectation is was that you would have also referred to the other contending theory of dopamine viz Incentive Salience and not considered the debate as being settled.

Well said, Jake! I particularly like

Think of it this way. Many drugs -- not all drugs...some work in different ways -- function to short-circuit the reward system and create the perception of reward in the absence of any tangible rewarding stimuli.

That "absence of tanglible rewarding stimuli" is something that people often forget. Your dopamine explanations, they are pretty. I think I will be referring to you a lot.

Thanks for taking this on, Jake. I think you may have misunderstood the sentence that forms the basis for much of your criticism. In saying, "When you succeed, your brain produces more dopamine," I didn't mean that there's a "cascading increase" as you say above (I agree, if I'd said that, it would have been wrong) ... I meant it produces more as in another shot of it, not more as in an increased amount from the last time.

And yes, the dopamine debate is a big one with many complexities ... I could have written thousands of words on the various theories out there about what dopamine does and how it works. But when writing for the general public, we have to boil things down.

Rebecca,

If I indeed misunderstood what you meant by that sentence, then it is my problem not yours. The more I think about it, this may have been an issue of simplifying complex subjects for the general public.

In the same sense that you elided a little over reward error as the driver of dopamine release, I elided over the considerable debate about dopamine.

I think the reason I chose to write this is that I am concerned with the presentation of dopamine in the public square. I have read a considerable number of news articles lately describing dopamine as the "reward molecule" -- as if it were some sort of cosmic joy button indicating how happy you are. I think that these articles miss the point: that dopamine is a learning signal that changes activity in other systems, and that it is these systems that guide decision making.

To the others,

I did make an editorial decision with respect to this post in favor of the much more accepted reward error hypothesis over the others. I do indeed realize that there is controversy on the subject, but considering that the reward error hypothesis has a great deal of empirical evidence while the others have less, I thought it was a good place to start.

Thanks for the update above clarifying that this was a misunderstanding of what I'd written, not a factual error. FWIW: I find a lot of the articles portraying dopamine as a pleasure button to be problematic too ...

It's also relevant to note that this article you're responding to was written about two and a half years ago as was heavily fact checked by scientists. But the dopamine debate has proceeded along rapidly, and plenty of new theories have arisen since it was published.

Great article, very helpful clarifying the confusing basics of domaine mechanisms. The qualifications and disagreements are helpful for understanding as well, from my perspective. Thanks!