Rationality- From AI to Zombies

by Eliezer Yudkowsky

  If you’re thinking about a world that could arise in a lawful way, but whose probability is a quadrillion to one, and something very pleasant or very awful is happening in this world . . . well, it does probably exist, if it is lawful. But you should try to release one quadrillionth as many neurotransmitters, in your reward centers or your aversive centers, so that you can weigh that world appropriately in your decisions. If you don’t think you can do that . . . don’t bother thinking about it.

  Otherwise you might as well go out and buy a lottery ticket using a quantum random number, a strategy that is guaranteed to result in a very tiny mega-win.

  Or here’s another way of thinking about it: Are you considering expending some mental energy on a world whose frequency in your future is less than a trillionth? Then go get a 10-sided die from your local gaming store, and, before you begin thinking about that strange world, start rolling the die. If the die comes up 9 twelve times in a row, then you can think about that world. Otherwise don’t waste your time; thought-time is a resource to be expended wisely.

  You can roll the dice as many times as you like, but you can’t think about the world until 9 comes up twelve times in a row. Then you can think about it for a minute. After that you have to start rolling the die again.

  This may help you to appreciate the concept of “trillion to one” on a more visceral level.

  If at any point you catch yourself thinking that quantum physics might have some kind of strange, abnormal implication for everyday life—then you should probably stop right there.

  Oh, there are a few implications of many-worlds for ethics. Average utilitarianism suddenly looks a lot more attractive—you don’t need to worry about creating as many people as possible, because there are already plenty of people exploring person-space. You just want the average quality of life to be as high as possible, in the future worlds that are your responsibility.

  And you should always take joy in discovery, as long as you personally don’t know a thing. It is meaningless to talk of being the “first” or the “only” person to know a thing, when everything knowable is known within worlds that are in neither your past nor your future, and are neither before or after you.

  But, by and large, it all adds up to normality. If your understanding of many-worlds is the tiniest bit shaky, and you are contemplating whether to believe some strange proposition, or feel some strange emotion, or plan some strange strategy, then I can give you very simple advice: Don’t.

  The quantum universe is not a strange place into which you have been thrust. It is the way things have always been.

  *

  1. Greg Egan, Quarantine (London: Legend Press, 1992).

  2. Robert S. Boynton, “The Birth of an Idea: A Profile of Frank Sulloway,” The New Yorker (October 1999).

  238

  Quantum Non-Realism

  Does the moon exist when no one is looking at it?

  —Albert Einstein, asked of Niels Bohr

  Suppose you were just starting to work out a theory of quantum mechanics.

  You begin to encounter experiments that deliver different results depending on how closely you observe them. You dig underneath the reality you know, and find an extremely precise mathematical description that only gives you the relative frequency of outcomes; worse, it’s made of complex numbers. Things behave like particles on Monday and waves on Tuesday.

  The correct answer is not available to you as a hypothesis, because it will not be invented for another thirty years.

  In a mess like that, what’s the best you could do?

  The best you can do is the strict “shut up and calculate” interpretation of quantum mechanics. You’ll go on trying to develop new theories, because doing your best doesn’t mean giving up. But we’ve specified that the correct answer won’t be available for thirty years, and that means none of the new theories will really be any good. Doing the best you could theoretically do would mean that you recognized that, even as you looked for ways to test the hypotheses.

  The best you could theoretically do would not include saying anything like, “The wavefunction only gives us probabilities, not certainties.” That, in retrospect, was jumping to a conclusion; the wavefunction gives us a certainty of many worlds existing. So that part about the wavefunction being only a probability was not-quite-right. You calculated, but failed to shut up.

  If you do the best that you can do without the correct answer being available, then, when you hear about decoherence, it will turn out that you have not said anything incompatible with decoherence. Decoherence is not ruled out by the data and the calculations. So if you refuse to affirm, as positive knowledge, any proposition which was not forced by the data and the calculations, the calculations will not force you to say anything incompatible with decoherence. So too with whatever the correct theory may be, if it is not decoherence. If you go astray, it must be from your own impulses.

  But it is hard for human beings to shut up and calculate—really shut up and calculate. There is an overwhelming tendency to treat our ignorance as if it were positive knowledge.

  I don’t know if any conversations like this ever really took place, but this is how ignorance becomes knowledge:

  GALLANT: “Shut up and calculate.”

  GOOFUS: “Why?”

  GALLANT: “Because I don’t know what these equations mean, just that they seem to work.”

  —five minutes later—

  GOOFUS: “Shut up and calculate.”

  STUDENT: “Why?”

  GOOFUS: “Because these equations don’t mean anything, they just work.”

  STUDENT: “Really? How do you know?”

  GOOFUS: “Gallant told me.”

  A similar transformation occurs in the leap from:

  GALLANT: “When my calculations show an amplitude of (-1∕3)i for this photon to get absorbed, my experiments showed that the photon was absorbed around 107 times out of 1,000, which is a good fit to 1/9, the square of the modulus. There’s clearly some kind of connection between the experimental statistics and the squared modulus of the amplitude, but I don’t know what.”

  GOOFUS: “The probability amplitude doesn’t say where the electron is, but where it might be. The squared modulus is the probability that reality will turn out that way. Reality itself is inherently nondeterministic.”

  And again:

  GALLANT: “Once I measure something and get an experimental result, I do my future calculations using only the amplitude whose squared modulus went into calculating the frequency of that experimental result. Only this rule makes my further calculations correspond to observed frequencies.”

  GOOFUS: “Since the amplitude is the probability, once you know the experimental result, the probability of everything else becomes zero!”

  The whole slip from:

  The square of this “amplitude” stuff corresponds tightly to our experimentally observed frequencies

  to

  The amplitude is the probability of getting the measurement

  to

  Well, obviously, once you know you didn’t get a measurement, its probability becomes zero

  has got to be one of the most embarrassing wrong turns in the history of science.

  If you take all this literally, it becomes the consciousness-causes-collapse interpretation of quantum mechanics. These days, just about nobody will confess to actually believing in the consciousness-causes-collapse interpretation of quantum mechanics—

  But the physics textbooks are still written this way! People say they don’t believe it, but they talk as if knowledge is responsible for removing incompatible “probability” amplitudes.

  Yet as implausible as I find consciousness-causes-collapse, it at least gives us a picture of reality. Sure, it’s an informal picture. Sure, it gives mental properties ontologically basic status. You can’t calculate when an “experimental observation” occurs or what people “know,” you just know when certain probabilities are obviously zero. And this “just know
ing” just happens to fit your experimental results, whatever they are—

  —but at least consciousness-causes-collapse purports to tell us how the universe works. The amplitudes are real, the collapse is real, the consciousness is real.

  Contrast to this argument schema:

  STUDENT: “Wait, you’re saying that this amplitude disappears as soon as the measurement tells me it’s not true?”

  GOOFUS: “No, no! It doesn’t literally disappear. The equations don’t mean anything—they just give good predictions.”

  STUDENT: “But then what does happen?”

  GOOFUS: (Whorble. Hiss.) “Never ask that question.”

  STUDENT: “And what about the part where we measure this photon’s polarization over here, and a light-year away, the entangled photon’s probability of being polarized up-down changes from 50% to 25%?”

  GOOFUS: “Yes, what about it?”

  STUDENT: “Doesn’t that violate Special Relativity?”

  GOOFUS: “No, because you’re just finding out the other photon’s polarization. Remember, the amplitudes aren’t real.”

  STUDENT: “But Bell’s Theorem shows there’s no possible local hidden variable that could describe the other photon’s polarization before we measure it—”

  GOOFUS: “Exactly! It’s meaningless to talk about the photon’s polarization before we measure it.”

  STUDENT: “But the probability suddenly changes—”

  GOOFUS: “It’s meaningless to talk about it before we measure it!”

  What does Goofus even mean, here? Never mind the plausibility of his words; what sort of state of reality would correspond to his words being true?

  What way could reality be, that would make it meaningless to talk about Special Relativity being violated, because the property being influenced didn’t exist, even though you could calculate the changes to it?

  But you know what? Forget that. I want to know the answer to an even more important question:

  Where is Goofus getting all this stuff?

  Let’s suppose that you take the Schrödinger equation, and assert, as a positive fact:

  This equation generates good predictions, but it doesn’t mean anything!

  Really? How do you know?

  I sometimes go around saying that the fundamental question of rationality is Why do you believe what you believe?

  You say the Schrödinger equation “doesn’t mean anything.” How did this item of definite knowledge end up in your possession, if it is not simply ignorance misinterpreted as knowledge?

  Was there some experiment that told you? I am open to the idea that experiments can tell us things that seem philosophically impossible. But in this case I should like to see the decisive data. Was there a point where you carefully set up an experimental apparatus, and worked out what you should expect to see if (1) the Schrödinger equation was meaningful or (2) the Schrödinger equation was meaningless; and then you got result (2)?

  GALLANT: “If I measure the 90° polarization of a photon, and then measure the 45° polarization, and then measure 90° again, my experimental history shows that in 100 trials a photon was absorbed 47 times and transmitted 53 times.”

  GOOFUS: “The 90° polarization and 45° polarization are incompatible properties; they can’t both exist at the same time, and if you measure one, it is meaningless to talk about the other.”

  How do you know?

  How did you acquire that piece of knowledge, Goofus? I know where Gallant got his—but where did yours come from?

  My attitude toward questions of existence and meaning was nicely illustrated in a discussion of the current state of evidence for whether the universe is spatially finite or spatially infinite, in which James D. Miller chided Robin Hanson:

  Robin, you are suffering from overconfidence bias in assuming that the universe exists. Surely there is some chance that the universe is of size zero.

  To which I replied:

  James, if the universe doesn’t exist, it would still be nice to know whether it’s an infinite or a finite universe that doesn’t exist.

  Ha! You think pulling that old “universe doesn’t exist” trick will stop me? It won’t even slow me down!

  It’s not that I’m ruling out the possibility that the universe doesn’t exist. It’s just that, even if nothing exists, I still want to understand the nothing as best I can. My curiosity doesn’t suddenly go away just because there’s no reality, you know!

  The nature of “reality” is something about which I’m still confused, which leaves open the possibility that there isn’t any such thing. But Egan’s Law still applies: “It all adds up to normality.” Apples didn’t stop falling when Einstein disproved Newton’s theory of gravity.

  Sure, when the dust settles, it could turn out that apples don’t exist, Earth doesn’t exist, reality doesn’t exist. But the nonexistent apples will still fall toward the nonexistent ground at a meaningless rate of 9.8 m/s2.

  You say the universe doesn’t exist? Fine, suppose I believe that—though it’s not clear what I’m supposed to believe, aside from repeating the words.

  Now, what happens if I press this button?

  In The Simple Truth, I said:

  Frankly, I’m not entirely sure myself where this “reality” business comes from. I can’t create my own reality in the lab, so I must not understand it yet. But occasionally I believe strongly that something is going to happen, and then something else happens instead . . . So I need different names for the thingies that determine my predictions and the thingy that determines my experimental results. I call the former thingies “belief,” and the latter thingy “reality.”

  You want to say that the quantum-mechanical equations are “not real”? I’ll be charitable, and suppose this means something. What might it mean?

  Maybe it means the equations which determine my predictions are substantially different from the thingy that determines my experimental results. Then what does determine my experimental results? If you tell me “nothing,” I would like to know what sort of “nothing” it is, and why this “nothing” exhibits such apparent regularity in determining e.g. my experimental measurements of the mass of an electron.

  I don’t take well to people who tell me to stop asking questions. If you tell me something is definitely positively meaningless, I want to know exactly what you mean by that, and how you came to know. Otherwise you have not given me an answer, only told me to stop asking the question.

  The Simple Truth describes the life of a shepherd and apprentice who have discovered how to count sheep by tossing pebbles into buckets, when they are visited by a delegate from the court who wants to know how the “magic pebbles” work. The shepherd tries to explain, “An empty bucket is magical if and only if the pastures are empty of sheep,” but is soon overtaken by the excited discussions of the apprentice and the delegate as to how the magic might get into the pebbles.

  Here we have quantum equations that deliver excellent experimental predictions. What exactly does it mean for them to be “meaningless”? Is it like a bucket of pebbles that works for counting sheep, but doesn’t have any magic?

  Back before Bell’s Theorem ruled out local hidden variables, it seemed possible that (as Einstein thought) there was some more complete description of reality which we didn’t have, and the quantum theory summarized incomplete knowledge of this more complete description. The laws we’d learned would turn out to be like the laws of statistical mechanics: quantitative statements of uncertainty. This would hardly make the equations “meaningless”; partial knowledge is the meaning of probability.

  But Bell’s Theorem makes it much less plausible that the quantum equations are partial knowledge of something deterministic, the way that statistical mechanics over classical physics is partial knowledge of something deterministic. And even so, the quantum equations would not be “meaningless” as that phrase is usually taken; they would be “statistical,” “approximate,” “partial information,” or at worst “w
rong.”

  Here we have equations that give us excellent predictions. You say they are “meaningless.” I ask what it is that determines my experimental results, then. You cannot answer. Fine, then how do you justify ruling out the possibility that the quantum equations give such excellent predictions because they are, oh, say, meaningful?

  I don’t mean to trivialize questions of reality or meaning. But to call something “meaningless” and say that the argument is now resolved, finished, over, done with, you must have a theory of exactly how it is meaningless. And when the answer is given, the question should seem no longer mysterious.

  As you may recall from Semantic Stopsigns, there are words and phrases which are not so much answers to questions, as cognitive traffic signals which indicate you should stop asking questions. “Why does anything exist in the first place? God!” is the classical example, but there are others, such as “Élan vital!”

  Tell people to “shut up and calculate” because you don’t know what the calculations mean, and inside of five years, “Shut up!” will be masquerading as a positive theory of quantum mechanics.

  I have the highest respect for any historical physicists who even came close to actually shutting up and calculating, who were genuinely conservative in assessing what they did and didn’t know. This is the best they could possibly do without actually being Hugh Everett, and I award them fifty rationality points. My scorn is reserved for those who interpreted “We don’t know why it works” as the positive knowledge that the equations were definitely not real.

  I mean, if that trick worked, it would be too good to confine to one subfield. Why shouldn’t physicists use the “not real” loophole outside of quantum mechanics?

  “Hey, doesn’t your new ‘yarn theory’ violate Special Relativity?”

  “Nah, the equations are meaningless. Say, doesn’t your model of ‘chaotic evil inflation’ violate CPT symmetry?”