Rationality- From AI to Zombies

by Eliezer Yudkowsky

  COLONEL TODD: What about David Chalmers? Shouldn’t he be here?

  GENERAL FRED: Chalmers . . . was one of the first victims.

  COLONEL TODD: Oh no.

  (Cut to the INTERIOR of a cell, completely walled in by reinforced glass, where DAVID CHALMERS paces back and forth.)

  DOCTOR: David! David Chalmers! Can you hear me?

  CHALMERS: Yes.

  NURSE: It’s no use, doctor.

  CHALMERS: I’m perfectly fine. I’ve been introspecting on my consciousness, and I can’t detect any difference. I know I would be expected to say that, but—

  The DOCTOR turns away from the glass screen in horror.

  DOCTOR: His words, they . . . they don’t mean anything.

  CHALMERS: This is a grotesque distortion of my philosophical views. This sort of thing can’t actually happen!

  DOCTOR: Why not?

  NURSE: Yes, why not?

  CHALMERS: Because—

  (Cut to two POLICE OFFICERS, guarding a dirt road leading up to the imposing steel gate of a gigantic concrete complex. On their uniforms, a badge reads BRIDGING LAW ENFORCEMENT AGENCY.)

  OFFICER 1: You’ve got to watch out for those clever bastards. They look like humans. They can talk like humans. They’re identical to humans on the atomic level. But they’re not human.

  OFFICER 2: Scumbags.

  The huge noise of a throbbing engine echoes over the hills. Up rides the MAN on a white motorcycle. The MAN is wearing black sunglasses and a black leather business suit with a black leather tie and silver metal boots. His white beard flows in the wind. He pulls to a halt in front of the gate.

  The OFFICERS bustle up to the motorcycle.

  OFFICER 1: State your business here.

  MAN: Is this where you’re keeping David Chalmers?

  OFFICER 2: What’s it to you? You a friend of his?

  MAN: Can’t say I am. But even zombies have rights.

  OFFICER 1: All right, buddy, let’s see your qualia.

  MAN: I don’t have any.

  OFFICER 2 suddenly pulls a gun, keeping it trained on the MAN.

  OFFICER 2: Aha! A zombie!

  OFFICER 1: No, zombies claim to have qualia.

  OFFICER 2: So he’s an ordinary human?

  OFFICER 1: No, they also claim to have qualia.

  The OFFICERS look at the MAN, who waits calmly.

  OFFICER 2: Um . . .

  OFFICER 1: Who are you?

  MAN: I’m Daniel Dennett, bitches.

  Seemingly from nowhere, DENNETT pulls a sword and slices OFFICER 2’s gun in half with a steely noise. OFFICER 1 begins to reach for his own gun, but DENNETT is suddenly standing behind OFFICER 1 and chops with a fist, striking the junction of OFFICER 1’s shoulder and neck. OFFICER 1 drops to the ground.

  OFFICER 2 steps back, horrified.

  OFFICER 2: That’s not possible! How’d you do that?

  DENNETT: I am one with my body.

  DENNETT drops OFFICER 2 with another blow, and strides toward the gate. He looks up at the imposing concrete complex, and grips his sword tighter.

  DENNETT (quietly to himself): There is a spoon.

  (Cut back to GENERAL FRED and the other military officials.)

  GENERAL FRED: I’ve just received the reports. We’ve lost Detroit.

  CAPTAIN MUDD: I don’t want to be the one to say “Good riddance,” but—

  GENERAL FRED: Australia has been . . . reduced to atoms.

  COLONEL TODD: The epiphenomenal virus is spreading faster. Civilization itself threatens to dissolve into total normality. We could be looking at the middle of humanity.

  CAPTAIN MUDD: Can we negotiate with the zombies?

  GENERAL FRED: We’ve sent them messages. They sent only a single reply.

  CAPTAIN MUDD: Which was . . . ?

  GENERAL FRED: It’s on its way now.

  An orderly brings in an envelope, and hands it to GENERAL FRED.

  GENERAL FRED opens the envelope, takes out a single sheet of paper, and reads it.

  Silence envelops the room.

  CAPTAIN MUDD: What’s it say?

  GENERAL FRED: It says . . . that we’re the ones with the virus.

  (A silence falls.)

  COLONEL TODD raises his hands and stares at them.

  COLONEL TODD: My God, it’s true. It’s true. I . . .

  (A tear rolls down COLONEL TODD’s cheek.)

  COLONEL TODD: I don’t feel anything.

  The screen goes black.

  The sound goes silent.

  The movie continues exactly as before.

  *

  227

  Excluding the Supernatural

  Occasionally, you hear someone claiming that creationism should not be taught in schools, especially not as a competing hypothesis to evolution, because creationism is a priori and automatically excluded from scientific consideration, in that it invokes the “supernatural.”

  So . . . is the idea here, that creationism could be true, but even if it were true, you wouldn’t be allowed to teach it in science class, because science is only about “natural” things?

  It seems clear enough that this notion stems from the desire to avoid a confrontation between science and religion. You don’t want to come right out and say that science doesn’t teach Religious Claim X because X has been tested by the scientific method and found false. So instead, you can . . . um . . . claim that science is excluding hypothesis X a priori. That way you don’t have to discuss how experiment has falsified X a posteriori.

  Of course this plays right into the creationist claim that Intelligent Design isn’t getting a fair shake from science—that science has prejudged the issue in favor of atheism, regardless of the evidence. If science excluded Intelligent Design a priori, this would be a justified complaint!

  But let’s back up a moment. The one comes to you and says: “Intelligent Design is excluded from being science a priori, because it is ‘supernatural,’ and science only deals in ‘natural’ explanations.”

  What exactly do they mean, “supernatural”? Is any explanation invented by someone with the last name “Cohen” a supernatural one? If we’re going to summarily kick a set of hypotheses out of science, what is it that we’re supposed to exclude?

  By far the best definition I’ve ever heard of the supernatural is Richard Carrier’s: A “supernatural” explanation appeals to ontologically basic mental things, mental entities that cannot be reduced to nonmental entities.

  This is the difference, for example, between saying that water rolls downhill because it wants to be lower, and setting forth differential equations that claim to describe only motions, not desires. It’s the difference between saying that a tree puts forth leaves because of a tree spirit, versus examining plant biochemistry. Cognitive science takes the fight against supernaturalism into the realm of the mind.

  Why is this an excellent definition of the supernatural? I refer you to Richard Carrier for the full argument. But consider: Suppose that you discover what seems to be a spirit, inhabiting a tree—a dryad who can materialize outside or inside the tree, who speaks in English about the need to protect her tree, et cetera. And then suppose that we turn a microscope on this tree spirit, and she turns out to be made of parts—not inherently spiritual and ineffable parts, like fabric of desireness and cloth of belief, but rather the same sort of parts as quarks and electrons, parts whose behavior is defined in motions rather than minds. Wouldn’t the dryad immediately be demoted to the dull catalogue of common things?

  But if we accept Richard Carrier’s definition of the supernatural, then a dilemma arises: we want to give religious claims a fair shake, but it seems that we have very good grounds for excluding supernatural explanations a priori.

  I mean, what would the universe look like if reductionism were false?

  I previously defined the reductionist thesis as follows: human minds create multi-level models of reality in which high-level patterns and low-level patterns are separately and
explicitly represented. A physicist knows Newton’s equation for gravity, Einstein’s equation for gravity, and the derivation of the former as a low-speed approximation of the latter. But these three separate mental representations are only a convenience of human cognition. It is not that reality itself has an Einstein equation that governs at high speeds, a Newton equation that governs at low speeds, and a “bridging law” that smooths the interface. Reality itself has only a single level, Einsteinian gravity. It is only the Mind Projection Fallacy that makes some people talk as if the higher levels could have a separate existence—different levels of organization can have separate representations in human maps, but the territory itself is a single unified low-level mathematical object.

  Suppose this were wrong.

  Suppose that the Mind Projection Fallacy was not a fallacy, but simply true.

  Suppose that a 747 had a fundamental physical existence apart from the quarks making up the 747.

  What experimental observations would you expect to make, if you found yourself in such a universe?

  If you can’t come up with a good answer to that, it’s not observation that’s ruling out “non-reductionist” beliefs, but a priori logical incoherence. If you can’t say what predictions the “non-reductionist” model makes, how can you say that experimental evidence rules it out?

  My thesis is that non-reductionism is a confusion; and once you realize that an idea is a confusion, it becomes a tad difficult to envision what the universe would look like if the confusion were true. Maybe I’ve got some multi-level model of the world, and the multi-level model has a one-to-one direct correspondence with the causal elements of the physics? But once all the rules are specified, why wouldn’t the model just flatten out into yet another list of fundamental things and their interactions? Does everything I can see in the model, like a 747 or a human mind, have to become a separate real thing? But what if I see a pattern in that new supersystem?

  Supernaturalism is a special case of non-reductionism, where it is not 747s that are irreducible, but just (some) mental things. Religion is a special case of supernaturalism, where the irreducible mental things are God(s) and souls; and perhaps also sins, angels, karma, etc.

  If I propose the existence of a powerful entity with the ability to survey and alter each element of our observed universe, but with the entity reducible to nonmental parts that interact with the elements of our universe in a lawful way; if I propose that this entity wants certain particular things, but “wants” using a brain composed of particles and fields; then this is not yet a religion, just a naturalistic hypothesis about a naturalistic Matrix. If tomorrow the clouds parted and a vast glowing amorphous figure thundered forth the above description of reality, then this would not imply that the figure was necessarily honest; but I would show the movies in a science class, and I would try to derive testable predictions from the theory.

  Conversely, religions have ignored the discovery of that ancient bodiless thing: omnipresent in the working of Nature and immanent in every falling leaf; vast as a planet’s surface and billions of years old; itself unmade and arising from the structure of physics; designing without brain to shape all life on Earth and the minds of humanity. Natural selection, when Darwin proposed it, was not hailed as the long-awaited Creator: It wasn’t fundamentally mental.

  But now we get to the dilemma: if the staid conventional normal boring understanding of physics and the brain is correct, there’s no way in principle that a human being can concretely envision, and derive testable experimental predictions about, an alternate universe in which things are irreducibly mental. Because if the boring old normal model is correct, your brain is made of quarks, and so your brain will only be able to envision and concretely predict things that can predicted by quarks. You will only ever be able to construct models made of interacting simple things.

  People who live in reductionist universes cannot concretely envision non-reductionist universes. They can pronounce the syllables “non-reductionist” but they can’t imagine it.

  The basic error of anthropomorphism, and the reason why supernatural explanations sound much simpler than they really are, is your brain using itself as an opaque black box to predict other things labeled “mindful.” Because you already have big, complicated webs of neural circuitry that implement your “wanting” things, it seems like you can easily describe water that “wants” to flow downhill—the one word “want” acts as a lever to set your own complicated wanting-machinery in motion.

  Or you imagine that God likes beautiful things, and therefore made the flowers. Your own “beauty” circuitry determines what is “beautiful” and “not beautiful.” But you don’t know the diagram of your own synapses. You can’t describe a nonmental system that computes the same label for what is “beautiful” or “not beautiful”—can’t write a computer program that predicts your own labelings. But this is just a defect of knowledge on your part; it doesn’t mean that the brain has no explanation.

  If the “boring view” of reality is correct, then you can never predict anything irreducible because you are reducible. You can never get Bayesian confirmation for a hypothesis of irreducibility, because any prediction you can make is, therefore, something that could also be predicted by a reducible thing, namely your brain.

  Some boxes you really can’t think outside. If our universe really is Turing computable, we will never be able to concretely envision anything that isn’t Turing-computable—no matter how many levels of halting oracle hierarchy our mathematicians can talk about, we won’t be able to predict what a halting oracle would actually say, in such fashion as to experimentally discriminate it from merely computable reasoning.

  Of course, that’s all assuming the “boring view” is correct. To the extent that you believe evolution is true, you should not expect to encounter strong evidence against evolution. To the extent you believe reductionism is true, you should expect non-reductionist hypotheses to be incoherent as well as wrong. To the extent you believe supernaturalism is false, you should expect it to be inconceivable as well.

  If, on the other hand, a supernatural hypothesis turns out to be true, then presumably you will also discover that it is not inconceivable.

  So let us bring this back full circle to the matter of Intelligent Design:

  Should ID be excluded a priori from experimental falsification and science classrooms, because, by invoking the supernatural, it has placed itself outside of natural philosophy?

  I answer: “Of course not.” The irreducibility of the intelligent designer is not an indispensable part of the ID hypothesis. For every irreducible God that can be proposed by the IDers, there exists a corresponding reducible alien that behaves in accordance with the same predictions—since the IDers themselves are reducible. To the extent I believe reductionism is in fact correct, which is a rather strong extent, I must expect to discover reducible formulations of all supposedly supernatural predictive models.

  If we’re going over the archeological records to test the assertion that Jehovah parted the Red Sea out of an explicit desire to display its superhuman power, then it makes little difference whether Jehovah is ontologically basic, or an alien with nanotech, or a Dark Lord of the Matrix. You do some archeology, find no skeletal remnants or armor at the Red Sea site, and indeed find records that Egypt ruled much of Canaan at the time. So you stamp the historical record in the Bible “disproven” and carry on. The hypothesis is coherent, falsifiable and wrong.

  Likewise with the evidence from biology that foxes are designed to chase rabbits, rabbits are designed to evade foxes, and neither is designed “to carry on their species” or “protect the harmony of Nature”; likewise with the retina being designed backwards with the light-sensitive parts at the bottom; and so on through a thousand other items of evidence for splintered, immoral, incompetent design. The Jehovah model of our alien god is coherent, falsifiable, and wrong—coherent, that is, so long as you don’t care whether Jehovah is ontologically basic or just
an alien.

  Just convert the supernatural hypothesis into the corresponding natural hypothesis. Just make the same predictions the same way, without asserting any mental things to be ontologically basic. Consult your brain’s black box if necessary to make predictions—say, if you want to talk about an “angry god” without building a full-fledged angry AI to label behaviors as angry or not angry. So you derive the predictions, or look up the predictions made by ancient theologians without advance knowledge of our experimental results. If experiment conflicts with those predictions, then it is fair to speak of the religious claim having been scientifically refuted. It was given its just chance at confirmation; it is being excluded a posteriori, not a priori.

  Ultimately, reductionism is just disbelief in fundamentally complicated things. If “fundamentally complicated” sounds like an oxymoron . . . well, that’s why I think that the doctrine of non-reductionism is a confusion, rather than a way that things could be, but aren’t. You would be wise to be wary, if you find yourself supposing such things.

  But the ultimate rule of science is to look and see. If ever a God appeared to thunder upon the mountains, it would be something that people looked at and saw.

  Corollary: Any supposed designer of Artificial General Intelligence who talks about religious beliefs in respectful tones is clearly not an expert on reducing mental things to nonmental things; and indeed knows so very little of the uttermost basics, as for it to be scarcely plausible that they could be expert at the art; unless their idiot savancy is complete. Or, of course, if they’re outright lying. We’re not talking about a subtle mistake.

  *

  228

  Psychic Powers

  In the last essay, I wrote:

  If the “boring view” of reality is correct, then you can never predict anything irreducible because you are reducible. You can never get Bayesian confirmation for a hypothesis of irreducibility, because any prediction you can make is, therefore, something that could also be predicted by a reducible thing, namely your brain.