by Hugo Mercier
namely, ten cents. The ten cents answer is the perfect example
of a System 1 answer: for the majority of people, it is the first thing
that pops into their heads after they have read the prob lem. Yet ten
cents cannot be correct, for then the bat would cost $1.10, and the
two together $1.20. Most people have to rely on their System 2 to
correct this intuitive mistake, and to reach the correct answer of
five cents.22
If System 1 consists of rough- and- ready mechanisms, while
System 2 consists of slow, deliberate reflection, we might expect
System 1 to be associated with credulity, and System 2 with criti-
cal thinking. Psychologist Daniel Gilbert and his colleagues
performed an ingenious series of experiments to tease out the
role played by the two mental systems in the evaluation of com-
municated information.23 In these experiments, participants
were presented with a series of statements, and right after each
statement was presented, they were told whether it was true or
false. For instance, in one experiment, the statements were about
words in Hopi (a Native American language), so participants
e v o l v in g o p e n - mind e d ne s s 37
might be told “A ghoren is a jug” and, a second later, they were
told “true.” After all the statements had been presented, partici-
pants were asked which had been true and which had been
false. To test for the role played by the two systems, Gilbert and
his colleagues intermittently interrupted System 2 pro cessing.
System 2, being slow and effortful, is easily disrupted. In this case,
participants simply had to press a button when they heard a tone,
which tended to ring when the crucial information— whether
a given statement was true or false— was being delivered.
When it came time to recall which statements were true and
which were false, people whose System 2 had been disrupted
were more likely to believe the statements to be true—
irrespective of whether they had in fact been signaled as true or
false. The System 2 disruption had thus caused many participants
to accept false statements as true. These experiments led Gilbert
and his colleagues to conclude that our initial inclination is to
accept what we are told, and that the slightest disruption to
System 2 stops us from reconsidering this initial ac cep tance. As
Gilbert and his colleagues put it in the title of their second ar-
ticle on the topic: “You Can’t Not Believe Every thing You
Read.”24 Kahneman summarized these findings as follows:
“When System 2 is other wise engaged, we will believe almost
anything. System 1 is gullible and biased to believe, System 2 is
in charge of doubting and unbelieving, but System 2 is some-
times busy, and often lazy.”25
These results are in line with the associations observed be-
tween a more “analytic” thinking style— that is, being more
inclined to rely on System 2 than System 1— and the rejection
of empirically dubious beliefs. In a widely publicized article, psy-
chologists Will Gervais and Ara Norenzayan found that people
with a more analytic frame of mind— people who are better at
solving prob lems like the Bat and Ball, for instance— are more
38 ch ap t er 3
likely to be atheists.26 Other studies suggest that more analytically
inclined participants are less likely to accept a variety of para-
normal beliefs, from witchcraft to precognition.27
I Hope You Did Not Believe Every thing You Just Read
The association between lack of cognitive sophistication and
gullibility, predicted by the arms race view of the evolution of
vigilance, has been prevalent throughout history, from Greek
phi los o phers to con temporary psychologists. Yet, as appealing
as they might be, I believe that the arms race analogy, along with
the association between lack of sophistication and gullibility, are
completely mistaken, with critical consequences for who is more
likely to accept wrong beliefs, and why.
For a start, the arms race analogy doesn’t fit the broad pattern
of the evolution of human communication. Arms races are char-
acterized by the preservation of the status quo through parallel
escalation. Rus sia and the United States acquired increasingly
large nuclear arsenals, but neither nation gained the upper hand.
Computer viruses haven’t been wiped out by security software,
but the viruses haven’t taken over all computers either. Likewise,
in the fight for resources between the mother and her fetus de-
scribed in the previous chapter, the increasingly large deploy-
ment of hormonal signals on both sides has practically no net
effect.
Human communication is, fortunately, very diff er ent from
these examples. Here, the status quo might be the amount of in-
formation our prehuman ancestors or, as an approximation, our
closest living relatives, exchange. Clearly, we have ventured very,
very far from this status quo. We send and consume orders of
magnitude more information than any other primate, and, cru-
cially, we are vastly more influenced by the information we re-
e v o l v in g o p e n - mind e d ne s s 39
ceive. The bandwidth of our communication has dramatically
expanded. We discuss events that are distant in time and space;
we express our deepest feelings; we even debate abstract enti-
ties and tell stories about imaginary beings.
For the evolution of human communication, a better anal-
ogy than the arms race is the evolution of omnivorous diets.
Some animals have extraordinarily specific diets. Koalas eat
only eucalyptus leaves. Vampire bats drink only the blood of
live mammals. Pandas eat only bamboo. These animals reject
every thing that isn’t their food of choice. As an extreme ex-
ample, koalas will not eat a eucalyptus leaf if it isn’t properly
presented—if it is on a flat surface, for example, rather than
attached to the branch of a eucalyptus tree.28 These animals
have evolved extremely specific food choices. However, this
strategy can backfire if they find themselves in a new environ-
ment. Vampire bats drink only the blood of live mammals, so
they don’t have to worry about whether their food is fresh.
Because the prob lem of learning to avoid toxic food is not one
they face in their natu ral environment, they have no mecha-
nism for learning food aversion, and keep drinking food that
they should associate with sickness.29
By contrast with these specialists, omnivorous animals are
both more open and more vigilant. They are more open in that
they search for, detect, and ingest a much wider variety of foods.
Rats or humans need more than thirty diff er ent nutrients, includ-
ing “nine amino acids, a few fatty acids, at least ten vitamins, and
at least thirteen minerals,”30 and none of their food sources can
provide all of those at once. Omnivores have to be much more
open in the range of foods they are wil ing to sample. Indeed, rats
or humans
will try just about anything that looks edible. They
are endowed with a suite of mechanisms that detects the vari-
ous nutrients they need in what they ingest, and adjust their diet
40 ch ap t er 3
according to these needs— craving salty foods when low in so-
dium, and so forth.31
This openness makes omnivores fantastically adaptable.
Humans have been able to survive on diets made up almost ex-
clusively of milk and potatoes (early eighteenth- century Irish
peasants) or meat and fish (the Inuit until recently). However,
their openness also makes omnivores vulnerable. Meat can go
bad and contain dangerous bacteria. To avoid being eaten, most
plants are either toxic or hard to digest. As a result, omnivores
are also much more vigilant toward their food than specialists.
Using a variety of strategies, they learn how to avoid foods that
are likely to have undesirable side effects. The most basic of these
strategies is to keep track of which foods made them sick and
avoid these foods in the future— something that, as omnivores,
we take for granted, but that some animals, such as vampire bats,
are unable to do. Keeping track of which food is safe to eat re-
quires some dedicated circuitry, not general learning mecha-
nisms. The sick animal must learn to avoid the food it ate a few
hours ago, and not al the other stimuli— what it saw, felt, smel ed
in between eating and getting sick.32 Omnivores, from rats to
humans, but also caterpil ars, prefer food they have eaten when
they were young.33 Rats and humans also pay close attention to
what other members of their species eat and whether or not it
makes them sick, learning by observation which foods are safe.34
In terms of communication, the difference between humans
and other primates is similar to the difference between special-
ists and omnivores. Nonhuman primates mostly rely on specific
signals. Vervet monkeys have a dedicated alarm call for aerial
predators;35 chimpanzees smile in a way that signals submis-
sion;36 dominant baboons grunt to show their pacific intentions
before approaching lower- ranking individuals.37 Humans, as
noted earlier, are communication omnivores: they can commu-
e v o l v in g o p e n - mind e d ne s s 41
nicate about nearly anything they can conceive of. Humans
are thus vastly more open than other primates. Take something
as basic as pointing. Human babies understand pointing shortly
after they reach their first year.38 But adult chimpanzees, even in
situations in which pointing seems obvious to us, do not get it.
Repeated experiments have put chimpanzees in front of two
opaque containers, one containing food, but they don’t know
which. When an experimenter points to one of the containers,
the chimpanzees are not more likely to pick this container than
the other one.39 It is not for lack of intel igence: if you try to grab
one of the containers, the chimpanzees rightly infer that it must
be the container with the food.40 Communication is just much
less natu ral for chimpanzees than it is for us.
If we are vastly more open to diff er ent forms and contents of
communication than other primates, we should also be more
vigilant. I will explore in the next four chapters how we exert this
vigilance. Here I want to focus on the overall organ ization of our
mechanisms of open vigilance. This organ ization is critical for
understanding what happens when some of these mechanisms
are impaired: Do such impairments make us more or less likely
to accept misleading information?
According to the arms race theory, we have evolved from a
situation of extreme openness, of general gullibility, toward a
state of increasingly sophisticated vigilance made pos si ble by our
more recently developed cognitive machinery. If this machinery
were removed, the theory goes, we would revert to our previous
state of gullibility and be more likely to accept any message, how-
ever stupid or harmful.
The analogy with the evolution of omnivorous diets suggests
that the reverse is the case. We have evolved from a situation of
extreme conservatism, a situation in which we let only a re-
stricted set of signals affect us, toward a situation in which we
42 ch ap t er 3
are more vigilant but also more open to diff er ent forms and con-
tents of communication. This organ ization, in which increased
sophistication goes with increased openness, makes for much
more robust overall functioning. In the arms race view, disrup-
tion of the more sophisticated mechanisms makes us credulous
and vulnerable. By contrast, a model in which openness and vigi-
lance evolve hand in hand is not so fragile. If more recent mech-
anisms are disrupted, we revert to older mechanisms, making us
less vigilant— but also much less open. If our more recent and
sophisticated cognitive machinery is disrupted, we revert to our
conservative core, becoming more stubborn rather than more
gullible.41
Brainwashing Does Not Wash
What about the evidence that supports the association between
lack of sophistication and gullibility and, indirectly, the arms race
view of the evolution of vigilance? What about brainwashing and
subliminal influence, for a start? If disrupting our higher cogni-
tive abilities, or bypassing them altogether, were an effective
means of influence, then both brainwashing and subliminal stim-
uli should leave us helpless, gullibly accepting the virtues of
communism and thirsting for Coca- Cola. In fact, both persua-
sion techniques are staggeringly in effec tive.
The brainwashing scare started when twenty- three American
POWs defected to China after the Korean War. This is already
a rather pitiful success rate: twenty- three converts out of forty-
four hundred captive soldiers, or half a percent. But in fact the
number of genuine converts was likely zero. The soldiers who
defected were afraid of what awaited them in the United States. To
gain some favors in the camps, they had col aborated with their
Chinese captors—or at least had not shown as much defiance
e v o l v in g o p e n - mind e d ne s s 43
toward the captors as their fellow prisoners. As a result, those
POWs could expect to be court- martialed upon their return.
Indeed, among the POWs who had returned to the United States,
one had been sentenced to ten years in jail, while prosecutors
sought the death penalty for another. Compared with that,
being feted as a convert to the Chinese system did not seem so bad,
even if it meant paying lip ser vice to communist doctrine— a
doctrine they likely barely grasped in any case.42 More recently,
methods derived from brainwashing, such as the “enhanced
interrogation techniques” that rely on physical constraints,
sleep deprivation, and other attempts at numbing the suspects’
minds, have been used by U.S. forces in their “war on terror.”
Like brainwashing, these techniques have been shown t
o be
much less effective than softer methods that make full use of the
suspects’ higher cognition— methods in which the interrogator
builds trust and engages the subjects in discussion.43
Similarly, the fear of subliminal influence and unconscious
mind control was nothing but an unfounded scare. The early ex-
periments demonstrating the power of subliminal stimuli were
simply made up: no one had displayed a subliminal “drink Coke”
ad in a movie theater.44 A wealth of subsequent (real) experi-
ments have failed to show that subliminal stimuli exert any
meaningful influence on our be hav ior.45 Seeing the message
“drink Coke” flashed on a screen does not make us more likely
to drink Coca- Cola. Listening to self- esteem tapes in our sleep
does not boost our self- esteem. If some experiments suggest that
stimuli can influence us without our being aware of it, the influ-
ence is at best minute— for instance, making someone who is
already thirsty drink a little bit more water.46
What about the experiments conducted by Gilbert and his
colleagues? They did show that some statements (such as “A
ghoren is a jug”) are spontaneously accepted and need some
44 ch ap t er 3
effort to be rejected. But does that mean that System 1 accepts
“every thing we read,” as Gilbert put it? Not at all. If participants
have some background knowledge related to the statement, this
background knowledge directs their initial reaction. For instance,
people’s initial reaction to statements such as “Soft soap is edi-
ble” is rejection.47 The statements don’t even have to be obviously
false to be intuitively disbelieved. They simply have to have some
relevance if they are false. It is not very helpful to know that, in
Hopi, it is not true that “a ghoren is a jug.” By contrast, if you
learn that the statement “John is a liberal” is false, it tells you
something useful about John. When exposed to statements such
as “John is a liberal,” people’s intuitive reaction is to adopt a stance
of doubt rather than ac cep tance.48 Far from being “gullible and
biased to believe,”49 System 1 is, if anything, biased to reject any
message incompatible with our background belief, but also am-
biguous messages or messages coming from untrustworthy
sources.50 This includes many messages that happen to be true.
