An emptiness comes from this combination of over-the-top nonnatural sources of reward and the inevitability of habituation; this is because unnaturally strong explosions of synthetic experience and sensation and pleasure evoke unnaturally strong degrees of habituation.90 This has two consequences. First, soon we barely notice the fleeting whispers of pleasure caused by leaves in autumn, or by the lingering glance of the right person, or by the promise of reward following a difficult, worthy task. And the other consequence is that we eventually habituate to even those artificial deluges of intensity. If we were designed by engineers, as we consumed more, we’d desire less. But our frequent human tragedy is that the more we consume, the hungrier we get. More and faster and stronger. What was an unexpected pleasure yesterday is what we feel entitled to today, and what won’t be enough tomorrow.
The Anticipation of Reward
Thus, dopamine is about invidious, rapidly habituating reward. But dopamine is more interesting than that. Back to our well-trained monkey working for a reward. A light comes on in his room, signaling the start of a reward trial. He goes over to the lever, presses ten times, and gets the raisin reward; this has happened often enough that there’s only a small increase in dopamine with each raisin.
However, importantly, lots of dopamine is released when the light first comes on, signaling the start of the reward trial, before the monkey starts lever pressing.
Visit bit.ly/2ovJngg for a larger version of this graph.
In other words, once reward contingencies are learned, dopamine is less about reward than about its anticipation. Similarly, work by my Stanford colleague Brian Knutson has shown dopamine pathway activation in people in anticipation of a monetary reward.91 Dopamine is about mastery and expectation and confidence. It’s “I know how things work; this is going to be great.” In other words, the pleasure is in the anticipation of reward, and the reward itself is nearly an afterthought (unless, of course, the reward fails to arrive, in which case it’s the most important thing in the world). If you know your appetite will be sated, pleasure is more about the appetite than about the sating.* This is hugely important.
Anticipation requires learning.92 Learn Warren G. Harding’s middle name, and synapses in the hippocampus become more excitable. Learn that when the light comes on it’s reward time, and it’s hippocampal amygdaloid and frontal cortical neurons projecting to dopamine neurons that become more excitable.
This explains context-dependent craving in addiction.93 Suppose an alcoholic has been clean and sober for years. Return him to where the alcohol consumption used to occur (e.g., that rundown street corner, that fancy men’s club), and those potentiated synapses, those cues that were learned to be associated with alcohol, come roaring back into action, dopamine surges with anticipation, and the craving inundates.
Can a reliable cue of an impending reward eventually become rewarding itself? This has been shown by Huda Akil of the University of Michigan. A light in the left side of a rat’s cage signals that lever pressing will produce a reward from a food chute on the right side. Remarkably, rats eventually will work for the chance to hang around on the left side of the cage, just because it feels so nice to be there. The signal has gained the dopaminergic power of what is being signaled. Similarly, rats will work to be exposed to a cue that signals that some kind of reward is likely, without knowing what or when. This is what fetishes are, in both the anthropological and sexual sense.94
Schultz’s group has shown that the magnitude of an anticipatory dopamine rise reflects two variables. First is the size of the anticipated reward. A monkey has learned that a light means that ten lever presses earns one unit of reward, while a tone means ten presses earns ten units. And soon a tone provokes more anticipatory dopamine than does a light. It’s “This is going to be great” versus “This is going to be great.”
The second variable is extraordinary. The rule is that the light comes on, you press the lever, you get the reward. Now things change. Light comes on, press the lever, get the reward . . . only 50 percent of the time. Remarkably, once that new scenario is learned, far more dopamine is released. Why? Because nothing fuels dopamine release like the “maybe” of intermittent reinforcement.95
This additional dopamine is released at a distinctive time. The light comes on in the 50 percent scenario, producing the usual anticipatory dopamine rise before the lever pressing starts. Back in the predictable days when lever pressing always earned a reward, once the pressing was finished, dopamine levels remained low until the reward arrived, followed by a little dopamine blip. But in this 50 percent scenario, once the pressing is finished, dopamine levels start rising, driven by the uncertainty of “maybe yes, maybe no.”
Visit bit.ly/2o3Zvcq for a larger version of this graph.
Modify things further; reward now occurs 25 or 75 percent of the time. A shift from 50 to 25 percent and a shift from 50 to 75 percent are exactly opposite, in terms of the likelihood of reward, and work from Knutson’s group shows that the greater the probability of reward, the more activation in the medial PFC.96 But switches from 50 to 25 percent and from 50 to 75 percent both reduce the magnitude of uncertainty. And the secondary rise of dopamine for a 25 or 75 percent likelihood of reward is smaller than for 50 percent. Thus, anticipatory dopamine release peaks with the greatest uncertainty as to whether a reward will occur.* Interestingly, in circumstances of uncertainty, enhanced anticipatory dopamine release is mostly in the mesocortical rather than mesolimbic pathway, implying that uncertainty is a more cognitively complex state than is anticipation of predictable reward.
None of this is news to the honorary psychologists running Las Vegas. Logically, gambling shouldn’t evoke much anticipatory dopamine, given the astronomical odds against winning. But the behavioral engineering—the 24-7 activity and lack of time cues, the cheap alcohol pickling frontocortical judgment, the manipulations to make you feel like today is your lucky day—distorts and shifts the perception of the odds into a range where dopamine pours out and, oh, why not, let’s try again.
The interaction between “maybe” and the propensity for addictive gambling is seen in a study of “near misses”—when two out of three reels line up in a slot machine. In control subjects there was minimal dopaminergic activation after misses of any sort; among pathological gamblers, a near miss activated the dopamine system like crazy. Another study concerned two betting situations with identical probabilities of reward but different levels of information about reward contingencies. The circumstance with less information (i.e., that was more about ambiguity than risk) activated the amygdala and silenced dopaminergic signaling; what is perceived to be well-calibrated risk is addictive, while ambiguity is just agitating.97
Pursuit
So dopamine is more about anticipation of reward than about reward itself. Time for one more piece of the picture. Consider that monkey trained to respond to the light cue with lever pressing, and out comes the reward; as we now know, once that relationship is established, most dopamine release is anticipatory, occurring right after the cue.
What happens if the post–light cue release of dopamine doesn’t occur?98 Crucially, the monkey doesn’t press the lever. Similarly, if you destroy its accumbens, a rat makes impulsive choices, instead of holding out for a delayed larger reward. Conversely, back to the monkey—if instead of flashing the light cue you electrically stimulate the tegmentum to release dopamine, the monkey presses the lever. Dopamine is not just about reward anticipation; it fuels the goal-directed behavior needed to gain that reward; dopamine “binds” the value of a reward to the resulting work. It’s about the motivation arising from those dopaminergic projections to the PFC that is needed to do the harder thing (i.e., to work).
In other words, dopamine is not about the happiness of reward. It’s about the happiness of pursuit of reward that has a decent chance of occurring.*99
This is central to understanding the nature of
motivation, as well as its failures (e.g., during depression, where there is inhibition of dopamine signaling thanks to stress, or in anxiety, where such inhibition is caused by projections from the amygdala).100 It also tells us about the source of the frontocortical power behind willpower. In a task where one chooses between an immediate and a (larger) delayed reward, contemplating the immediate reward activates limbic targets of dopamine (i.e., the mesolimbic pathway), whereas contemplating the delayed reward activates frontocortical targets (i.e., the mesocortical pathway). The greater the activation of the latter, the more likely there’ll be gratification postponement.
These studies involved scenarios of a short burst of work soon followed by reward.101 What about when the work required is prolonged, and reward is substantially delayed? In that scenario there is a secondary rise of dopamine, a gradual increase that fuels the sustained work; the extent of the dopamine ramp-up is a function of the length of the delay and the anticipated size of the reward:
Visit bit.ly/2ngTC7V for a larger version of this graph.
This reveals how dopamine fuels delayed gratification. If waiting X amount of time for a reward has value Z; waiting 2X should logically have value ½Z; instead we “temporally discount”—the value is smaller, e.g., ¼Z. We don’t like waiting.
Dopamine and the frontal cortex are in the thick of this phenomenon. Discounting curves—a value of ¼Z instead of ½Z—are coded in the accumbens, while dlPFC and vmPFC neurons code for time delay.102
This generates some complex interactions. For example, activate the vmPFC or inactivate the dlPFC, and short-term reward becomes more alluring. And a cool neuroimaging study of Knutson’s gives insight into impatient people with steep temporal discounting curves; their accumbens, in effect, underestimates the magnitude of the delayed reward, and their dlPFC overestimates the length of the delay.103
Collectively these studies show that our dopaminergic system, frontal cortex, amygdala, insula, and other members of the chorus code for differing aspects of reward magnitude, delay, and probability with varying degrees of accuracy, all influencing whether we manage to do the harder, more correct thing.104
Individual differences among people in the capacity for gratification postponement arise from variation in the volume of these individual neural voices.105 For example, there are abnormalities in dopamine response profiles during temporal discounting tasks in people with the maladaptive impulsiveness of attention-deficit/hyperactivity disorder (ADHD). Similarly, addictive drugs bias the dopamine system toward impulsiveness.
Phew. One more complication: These studies of temporal discounting typically involve delays on the order of seconds. Though the dopamine system is similar across numerous species, humans do something utterly novel: we delay gratification for insanely long times. No warthog restricts calories to look good in a bathing suit next summer. No gerbil works hard at school to get good SAT scores to get into a good college to get into a good grad school to get a good job to get into a good nursing home. We do something even beyond this unprecedented gratification delay: we use the dopaminergic power of the happiness of pursuit to motivate us to work for rewards that come after we are dead—depending on your culture, this can be knowing that your nation is closer to winning a war because you’ve sacrificed yourself in battle, that your kids will inherit money because of your financial sacrifices, or that you will spend eternity in paradise. It is extraordinary neural circuitry that bucks temporal discounting enough to allow (some of) us to care about the temperature of the planet that our great-grandchildren will inherit. Basically, it’s unknown how we humans do this. We may merely be a type of animal, mammal, primate, and ape, but we’re a profoundly unique one.
A Final Small Topic: Serotonin
This lengthy section has concerned dopamine, but an additional neurotransmitter, serotonin, plays a clear role in some behaviors that concern us.
Starting with a 1979 study, low levels of serotonin in the brain were shown to be associated with elevated levels of human aggression, with end points ranging from psychological measures of hostility to overt violence.106 A similar serotonin/aggression relationship was observed in other mammals and, remarkably, even crickets, mollusks, and crustaceans.
As work continued, an important qualifier emerged. Low serotonin didn’t predict premeditated, instrumental violence. It predicted impulsive aggression, as well as cognitive impulsivity (e.g., steep temporal discounting or trouble inhibiting a habitual response). Other studies linked low serotonin to impulsive suicide (independent of severity of the associated psychiatric illness).107
Moreover, in both animals and humans pharmacologically decreasing serotonin signaling increases behavioral and cognitive impulsivity (e.g., impulsively torpedoing a stable, cooperative relationship with a player in an economic game).108 Importantly, while increasing serotonin signaling did not lessen impulsiveness in normal subjects, it did in subjects prone toward impulsivity, such as adolescents with conduct disorder.
How does serotonin do this? Nearly all serotonin is synthesized in one brain region,* which projects to the usual suspects—the tegmentum, accumbens, PFC, and amygdala, where serotonin enhances dopamine’s effects on goal-directed behavior.109
This is as dependable a finding as you get in this business.110 Until we get to chapter 8 and look at genes related to serotonin, at which point everything becomes a completely contradictory mess. Just as a hint of what’s to come, one gene variant has even been referred to, straight faced, by some scientists as the “warrior gene,” and its presence has been used successfully in some courtrooms to lessen sentences for impulsive murders.
CONCLUSIONS
This completes our introduction to the nervous system and its role in pro- and antisocial behaviors. It was organized around three themes: the hub of fear, aggression, and arousal centered in the amygdala; the hub of reward, anticipation, and motivation of the dopaminergic system; and the hub of frontal cortical regulation and restraint of behavior. Additional brain regions and neurotransmitters will be introduced in subsequent chapters. Amid this mountain of information, be assured that the key brain regions, circuits, and neurotransmitters will become familiar as the book progresses.
Hang on. So what does this all mean? It’s useful to start with three things that this information doesn’t mean:
First, there’s the lure of needing neurobiology to confirm the obvious. Someone claims that, for example, their crappy, violent neighborhood leaves them so anxious that they can’t function effectively. Toss them in a brain scanner and flash pictures of various neighborhoods; when their own appears, the amygdala explodes into activity. “Ah,” it is tempting to conclude, “we’ve now proven that the person really does feel frightened.”
It shouldn’t require neuroscience to validate someone’s internal state. An example of this fallacy was reports of atrophy of the hippocampus in combat vets suffering from PTSD; this was in accord with basic research (including from my lab) showing that stress can damage the hippocampus. The hippocampal atrophy in PTSD got a lot of play in Washington, helping to convince skeptics that PTSD is an organic disorder rather than neurotic malingering. It struck me that if it took brain scans to convince legislators that there’s something tragically, organically damaged in combat vets with PTSD, then these legislators have some neurological problems of their own. Yet it required precisely this to “prove” to many that PTSD was an organic brain disorder.
The notion that “if a neuroscientist can demonstrate it, we know that the person’s problem is for real” has a corollary—the fancier the neurobiology utilized, the more reliable the verification. That’s simply not true; for example, a good neuropsychologist can discern more of what’s happening to someone with subtle but pervasive memory problems than can a gazillion-dollar brain scanner.
It shouldn’t take neuroscience to “prove” what we think and feel.
There’s been a proliferation of �
�neuro-” fields. Some, like neuroendocrinology and neuroimmunology, are stodgy old institutions by now. Others are relatively new—neuroeconomics, neuromarketing, neuroethics, and, I kid you not, neuroliterature and neuroexistentialism. In other words, a hegemonic neuroscientist might conclude that their field explains everything. And with that comes the danger, raised by the New Yorker writer Adam Gopnik under the sardonic banner of “neuroskepticism,” that explaining everything leads to forgiving everything.111 This premise is at the heart of debates in the new field of “neurolaw.” In chapter 16 I will argue that it is wrong to think that understanding must lead to forgiveness—mainly because I think that a term like “forgiveness,” and others related to criminal justice (e.g., “evil,” “soul,” “volition,” and “blame”), are incompatible with science and should be discarded.
Finally, there is the danger of thinking that neuroscience supports a tacit sort of dualism. A guy does something impulsive and awful, and neuroimaging reveals that, unexpectedly, he’s missing all his PFC neurons. There’s a dualist temptation now to view his behavior as more “biological” or “organic” in some nebulous manner than if he had committed the same act with a normal PFC. However, the guy’s awful, impulsive act is equally “biological” with or without a PFC. The sole difference is that the workings of the PFC-less brain are easier to understand with our primitive research tools.
So What Does All of This Tell Us?
Sometimes these studies tell us what different brain regions do. They are getting fancier, telling us about circuits, thanks to the growing time resolution of neuroimaging, transitioning from “This stimulus activates brain regions A, B, C” to “This stimulus activates both A and B, and then C, and C activates only if B does”. And identifying what specific regions/circuits do gets harder as studies become subtler. Consider, for example, the fusiform face area. As discussed in the next chapter, it is a cortical region that responds to faces in humans and other primates. We primates sure are social creatures.
Behave: The Biology of Humans at Our Best and Worst Page 8
