Not Born Yesterday

by Hugo Mercier

  namely, ten cents. The ten cents answer is the perfect example

  of a System 1 answer: for the majority of people, it is the first thing

  that pops into their heads after they have read the prob lem. Yet ten

  cents cannot be correct, for then the bat would cost $1.10, and the

  two together $1.20. Most people have to rely on their System 2 to

  correct this intuitive mistake, and to reach the correct answer of

  five cents.22

  If System 1 consists of rough- and- ready mechanisms, while

  System 2 consists of slow, deliberate reflection, we might expect

  System 1 to be associated with credulity, and System 2 with criti-

  cal thinking. Psychologist Daniel Gilbert and his colleagues

  performed an ingenious series of experiments to tease out the

  role played by the two mental systems in the evaluation of com-

  municated information.23 In these experiments, participants

  were presented with a series of statements, and right after each

  statement was presented, they were told whether it was true or

  false. For instance, in one experiment, the statements were about

  words in Hopi (a Native American language), so participants

  e v o l v in g o p e n - mind e d ne s s 37

  might be told “A ghoren is a jug” and, a second later, they were

  told “true.” After all the statements had been presented, partici-

  pants were asked which had been true and which had been

  false. To test for the role played by the two systems, Gilbert and

  his colleagues intermittently interrupted System 2 pro cessing.

  System 2, being slow and effortful, is easily disrupted. In this case,

  participants simply had to press a button when they heard a tone,

  which tended to ring when the crucial information— whether

  a given statement was true or false— was being delivered.

  When it came time to recall which statements were true and

  which were false, people whose System 2 had been disrupted

  were more likely to believe the statements to be true—

  irrespective of whether they had in fact been signaled as true or

  false. The System 2 disruption had thus caused many participants

  to accept false statements as true. These experiments led Gilbert

  and his colleagues to conclude that our initial inclination is to

  accept what we are told, and that the slightest disruption to

  System 2 stops us from reconsidering this initial ac cep tance. As

  Gilbert and his colleagues put it in the title of their second ar-

  ticle on the topic: “You Can’t Not Believe Every thing You

  Read.”24 Kahneman summarized these findings as follows:

  “When System 2 is other wise engaged, we will believe almost

  anything. System 1 is gullible and biased to believe, System 2 is

  in charge of doubting and unbelieving, but System 2 is some-

  times busy, and often lazy.”25

  These results are in line with the associations observed be-

  tween a more “analytic” thinking style— that is, being more

  inclined to rely on System 2 than System 1— and the rejection

  of empirically dubious beliefs. In a widely publicized article, psy-

  chologists Will Gervais and Ara Norenzayan found that people

  with a more analytic frame of mind— people who are better at

  solving prob lems like the Bat and Ball, for instance— are more

  38 ch ap t er 3

  likely to be atheists.26 Other studies suggest that more analytically

  inclined participants are less likely to accept a variety of para-

  normal beliefs, from witchcraft to precognition.27

  I Hope You Did Not Believe Every thing You Just Read

  The association between lack of cognitive sophistication and

  gullibility, predicted by the arms race view of the evolution of

  vigilance, has been prevalent throughout history, from Greek

  phi los o phers to con temporary psychologists. Yet, as appealing

  as they might be, I believe that the arms race analogy, along with

  the association between lack of sophistication and gullibility, are

  completely mistaken, with critical consequences for who is more

  likely to accept wrong beliefs, and why.

  For a start, the arms race analogy doesn’t fit the broad pattern

  of the evolution of human communication. Arms races are char-

  acterized by the preservation of the status quo through parallel

  escalation. Rus sia and the United States acquired increasingly

  large nuclear arsenals, but neither nation gained the upper hand.

  Computer viruses haven’t been wiped out by security software,

  but the viruses haven’t taken over all computers either. Likewise,

  in the fight for resources between the mother and her fetus de-

  scribed in the previous chapter, the increasingly large deploy-

  ment of hormonal signals on both sides has practically no net

  effect.

  Human communication is, fortunately, very diff er ent from

  these examples. Here, the status quo might be the amount of in-

  formation our prehuman ancestors or, as an approximation, our

  closest living relatives, exchange. Clearly, we have ventured very,

  very far from this status quo. We send and consume orders of

  magnitude more information than any other primate, and, cru-

  cially, we are vastly more influenced by the information we re-

  e v o l v in g o p e n - mind e d ne s s 39

  ceive. The bandwidth of our communication has dramatically

  expanded. We discuss events that are distant in time and space;

  we express our deepest feelings; we even debate abstract enti-

  ties and tell stories about imaginary beings.

  For the evolution of human communication, a better anal-

  ogy than the arms race is the evolution of omnivorous diets.

  Some animals have extraordinarily specific diets. Koalas eat

  only eucalyptus leaves. Vampire bats drink only the blood of

  live mammals. Pandas eat only bamboo. These animals reject

  every thing that isn’t their food of choice. As an extreme ex-

  ample, koalas will not eat a eucalyptus leaf if it isn’t properly

  presented—if it is on a flat surface, for example, rather than

  attached to the branch of a eucalyptus tree.28 These animals

  have evolved extremely specific food choices. However, this

  strategy can backfire if they find themselves in a new environ-

  ment. Vampire bats drink only the blood of live mammals, so

  they don’t have to worry about whether their food is fresh.

  Because the prob lem of learning to avoid toxic food is not one

  they face in their natu ral environment, they have no mecha-

  nism for learning food aversion, and keep drinking food that

  they should associate with sickness.29

  By contrast with these specialists, omnivorous animals are

  both more open and more vigilant. They are more open in that

  they search for, detect, and ingest a much wider variety of foods.

  Rats or humans need more than thirty diff er ent nutrients, includ-

  ing “nine amino acids, a few fatty acids, at least ten vitamins, and

  at least thirteen minerals,”30 and none of their food sources can

  provide all of those at once. Omnivores have to be much more

  open in the range of foods they are wil ing to sample. Indeed, rats

  or humans
will try just about anything that looks edible. They

  are endowed with a suite of mechanisms that detects the vari-

  ous nutrients they need in what they ingest, and adjust their diet

  40 ch ap t er 3

  according to these needs— craving salty foods when low in so-

  dium, and so forth.31

  This openness makes omnivores fantastically adaptable.

  Humans have been able to survive on diets made up almost ex-

  clusively of milk and potatoes (early eighteenth- century Irish

  peasants) or meat and fish (the Inuit until recently). However,

  their openness also makes omnivores vulnerable. Meat can go

  bad and contain dangerous bacteria. To avoid being eaten, most

  plants are either toxic or hard to digest. As a result, omnivores

  are also much more vigilant toward their food than specialists.

  Using a variety of strategies, they learn how to avoid foods that

  are likely to have undesirable side effects. The most basic of these

  strategies is to keep track of which foods made them sick and

  avoid these foods in the future— something that, as omnivores,

  we take for granted, but that some animals, such as vampire bats,

  are unable to do. Keeping track of which food is safe to eat re-

  quires some dedicated circuitry, not general learning mecha-

  nisms. The sick animal must learn to avoid the food it ate a few

  hours ago, and not al the other stimuli— what it saw, felt, smel ed

  in between eating and getting sick.32 Omnivores, from rats to

  humans, but also caterpil ars, prefer food they have eaten when

  they were young.33 Rats and humans also pay close attention to

  what other members of their species eat and whether or not it

  makes them sick, learning by observation which foods are safe.34

  In terms of communication, the difference between humans

  and other primates is similar to the difference between special-

  ists and omnivores. Nonhuman primates mostly rely on specific

  signals. Vervet monkeys have a dedicated alarm call for aerial

  predators;35 chimpanzees smile in a way that signals submis-

  sion;36 dominant baboons grunt to show their pacific intentions

  before approaching lower- ranking individuals.37 Humans, as

  noted earlier, are communication omnivores: they can commu-

  e v o l v in g o p e n - mind e d ne s s 41

  nicate about nearly anything they can conceive of. Humans

  are thus vastly more open than other primates. Take something

  as basic as pointing. Human babies understand pointing shortly

  after they reach their first year.38 But adult chimpanzees, even in

  situations in which pointing seems obvious to us, do not get it.

  Repeated experiments have put chimpanzees in front of two

  opaque containers, one containing food, but they don’t know

  which. When an experimenter points to one of the containers,

  the chimpanzees are not more likely to pick this container than

  the other one.39 It is not for lack of intel igence: if you try to grab

  one of the containers, the chimpanzees rightly infer that it must

  be the container with the food.40 Communication is just much

  less natu ral for chimpanzees than it is for us.

  If we are vastly more open to diff er ent forms and contents of

  communication than other primates, we should also be more

  vigilant. I will explore in the next four chapters how we exert this

  vigilance. Here I want to focus on the overall organ ization of our

  mechanisms of open vigilance. This organ ization is critical for

  understanding what happens when some of these mechanisms

  are impaired: Do such impairments make us more or less likely

  to accept misleading information?

  According to the arms race theory, we have evolved from a

  situation of extreme openness, of general gullibility, toward a

  state of increasingly sophisticated vigilance made pos si ble by our

  more recently developed cognitive machinery. If this machinery

  were removed, the theory goes, we would revert to our previous

  state of gullibility and be more likely to accept any message, how-

  ever stupid or harmful.

  The analogy with the evolution of omnivorous diets suggests

  that the reverse is the case. We have evolved from a situation of

  extreme conservatism, a situation in which we let only a re-

  stricted set of signals affect us, toward a situation in which we

  42 ch ap t er 3

  are more vigilant but also more open to diff er ent forms and con-

  tents of communication. This organ ization, in which increased

  sophistication goes with increased openness, makes for much

  more robust overall functioning. In the arms race view, disrup-

  tion of the more sophisticated mechanisms makes us credulous

  and vulnerable. By contrast, a model in which openness and vigi-

  lance evolve hand in hand is not so fragile. If more recent mech-

  anisms are disrupted, we revert to older mechanisms, making us

  less vigilant— but also much less open. If our more recent and

  sophisticated cognitive machinery is disrupted, we revert to our

  conservative core, becoming more stubborn rather than more

  gullible.41

  Brainwashing Does Not Wash

  What about the evidence that supports the association between

  lack of sophistication and gullibility and, indirectly, the arms race

  view of the evolution of vigilance? What about brainwashing and

  subliminal influence, for a start? If disrupting our higher cogni-

  tive abilities, or bypassing them altogether, were an effective

  means of influence, then both brainwashing and subliminal stim-

  uli should leave us helpless, gullibly accepting the virtues of

  communism and thirsting for Coca- Cola. In fact, both persua-

  sion techniques are staggeringly in effec tive.

  The brainwashing scare started when twenty- three American

  POWs defected to China after the Korean War. This is already

  a rather pitiful success rate: twenty- three converts out of forty-

  four hundred captive soldiers, or half a percent. But in fact the

  number of genuine converts was likely zero. The soldiers who

  defected were afraid of what awaited them in the United States. To

  gain some favors in the camps, they had col aborated with their

  Chinese captors—or at least had not shown as much defiance

  e v o l v in g o p e n - mind e d ne s s 43

  toward the captors as their fellow prisoners. As a result, those

  POWs could expect to be court- martialed upon their return.

  Indeed, among the POWs who had returned to the United States,

  one had been sentenced to ten years in jail, while prosecutors

  sought the death penalty for another. Compared with that,

  being feted as a convert to the Chinese system did not seem so bad,

  even if it meant paying lip ser vice to communist doctrine— a

  doctrine they likely barely grasped in any case.42 More recently,

  methods derived from brainwashing, such as the “enhanced

  interrogation techniques” that rely on physical constraints,

  sleep deprivation, and other attempts at numbing the suspects’

  minds, have been used by U.S. forces in their “war on terror.”

  Like brainwashing, these techniques have been shown t
o be

  much less effective than softer methods that make full use of the

  suspects’ higher cognition— methods in which the interrogator

  builds trust and engages the subjects in discussion.43

  Similarly, the fear of subliminal influence and unconscious

  mind control was nothing but an unfounded scare. The early ex-

  periments demonstrating the power of subliminal stimuli were

  simply made up: no one had displayed a subliminal “drink Coke”

  ad in a movie theater.44 A wealth of subsequent (real) experi-

  ments have failed to show that subliminal stimuli exert any

  meaningful influence on our be hav ior.45 Seeing the message

  “drink Coke” flashed on a screen does not make us more likely

  to drink Coca- Cola. Listening to self- esteem tapes in our sleep

  does not boost our self- esteem. If some experiments suggest that

  stimuli can influence us without our being aware of it, the influ-

  ence is at best minute— for instance, making someone who is

  already thirsty drink a little bit more water.46

  What about the experiments conducted by Gilbert and his

  colleagues? They did show that some statements (such as “A

  ghoren is a jug”) are spontaneously accepted and need some

  44 ch ap t er 3

  effort to be rejected. But does that mean that System 1 accepts

  “every thing we read,” as Gilbert put it? Not at all. If participants

  have some background knowledge related to the statement, this

  background knowledge directs their initial reaction. For instance,

  people’s initial reaction to statements such as “Soft soap is edi-

  ble” is rejection.47 The statements don’t even have to be obviously

  false to be intuitively disbelieved. They simply have to have some

  relevance if they are false. It is not very helpful to know that, in

  Hopi, it is not true that “a ghoren is a jug.” By contrast, if you

  learn that the statement “John is a liberal” is false, it tells you

  something useful about John. When exposed to statements such

  as “John is a liberal,” people’s intuitive reaction is to adopt a stance

  of doubt rather than ac cep tance.48 Far from being “gullible and

  biased to believe,”49 System 1 is, if anything, biased to reject any

  message incompatible with our background belief, but also am-

  biguous messages or messages coming from untrustworthy

  sources.50 This includes many messages that happen to be true.