Rationality- From AI to Zombies

Home > Science > Rationality- From AI to Zombies > Page 75
Rationality- From AI to Zombies Page 75

by Eliezer Yudkowsky


  Lavoisier also knew how to separate gases, and discovered that a burning candle diminished the amount of one kind of gas, vital air, and produced another gas, fixed air. Today we would call them oxygen and carbon dioxide. When the vital air was exhausted, the fire went out. One might guess, perhaps, that combustion transformed vital air into fixed air and fuel to ash, and that the ability of this transformation to continue was limited by the amount of vital air available.

  Lavoisier’s proposal directly contradicted the then-current phlogiston theory. That alone would have been shocking enough, but it also turned out . . .

  To appreciate what comes next, you must put yourself into an eighteenth-century frame of mind. Forget the discovery of DNA, which occurred only in 1953. Unlearn the cell theory of biology, which was formulated in 1839. Imagine looking at your hand, flexing your fingers . . . and having absolutely no idea how it worked. The anatomy of muscle and bone was known, but no one had any notion of “what makes it go”—why a muscle moves and flexes, while clay molded into a similar shape just sits there. Imagine your own body being composed of mysterious, incomprehensible gloop. And then, imagine discovering . . .

  . . . that humans, in the course of breathing, consumed vital air and breathed out fixed air. People also ran on combustion! Lavoisier measured the amount of heat that animals (and Lavoisier’s assistant, Seguin) produced when exercising, the amount of vital air consumed, and the fixed air breathed out. When animals produced more heat, they consumed more vital air and exhaled more fixed air. People, like fire, consumed fuel and oxygen; people, like fire, produced heat and carbon dioxide. Deprive people of oxygen, or fuel, and the light goes out.

  Matches catch fire because of phosphorus—“safety matches” have phosphorus on the ignition strip; strike-anywhere matches have phosphorus in the match heads. Phosphorus is highly reactive; pure phosphorus glows in the dark and may spontaneously combust. (Henning Brand, who purified phosphorus in 1669, announced that he had discovered Elemental Fire.) Phosphorus is thus also well-suited to its role in adenosine triphosphate, ATP, your body’s chief method of storing chemical energy. ATP is sometimes called the “molecular currency.” It invigorates your muscles and charges up your neurons. Almost every metabolic reaction in biology relies on ATP, and therefore on the chemical properties of phosphorus.

  If a match stops working, so do you. You can’t change just one thing.

  The surface-level rules, “Matches catch fire when struck,” and “Humans need air to breathe,” are not obviously connected. It took centuries to discover the connection, and even then, it still seems like some distant fact learned in school, relevant only to a few specialists. It is all too easy to imagine a world where one surface rule holds, and the other doesn’t; to suppress our credence in one belief, but not the other. But that is imagination, not reality. If your map breaks into four pieces for easy storage, it doesn’t mean the territory is also broken into disconnected parts. Our minds store different surface-level rules in different compartments, but this does not reflect any division in the laws that govern Nature.

  We can take the lesson further. Phosphorus derives its behavior from even deeper laws, electrodynamics and chromodynamics. “Phosphorus” is merely our word for electrons and quarks arranged a certain way. You cannot change the chemical properties of phosphorus without changing the laws governing electrons and quarks.

  If you stepped into a world where matches failed to strike, you would cease to exist as organized matter.

  Reality is laced together a lot more tightly than humans might like to believe.

  *

  1. Lyon Sprague de Camp and Fletcher Pratt, The Incomplete Enchanter (New York: Henry Holt & Company, 1941).

  182

  Universal Law

  Antoine-Laurent de Lavoisier discovered that breathing (respiration) and fire (combustion) operated on the same principle. It was one of the most startling unifications in the history of science, for it brought together the mundane realm of matter and the sacred realm of life, which humans had divided into separate magisteria.

  The first great simplification was that of Isaac Newton, who unified the course of the planets with the trajectory of a falling apple. The shock of this discovery was greater by far than Lavoisier’s. It wasn’t just that Newton had dared to unify the Earthly realm of base matter with the obviously different and sacred celestial realm, once thought to be the abode of the gods. Newton’s discovery gave rise to the notion of a universal law, one that is the same everywhere and everywhen, with literally zero exceptions.

  Human beings live in a world of surface phenomena, and surface phenomena are divided into leaky categories with plenty of exceptions. A tiger does not behave like a buffalo. Most buffalo have four legs, but perhaps this one has three. Why would anyone think there would be laws that hold everywhere? It’s just so obviously untrue.

  The only time when it seems like we would want a law to hold everywhere is when we are talking about moral laws—tribal rules of behavior. Some tribe members may try to take more than their fair share of the buffalo meat—perhaps coming up with some clever excuse—so in the case of moral laws we do seem to have an instinct to universality. Yes, the rule about dividing the meat evenly applies to you, right now, whether you like it or not. But even here there are exceptions. If—for some bizarre reason—a more powerful tribe threatened to spear all of you unless Bob received twice as much meat on just this one occasion, you’d give Bob twice as much meat. The idea of a rule with literally no exceptions seems insanely rigid, the product of closed-minded thinking by fanatics so in the grip of their one big idea that they can’t see the richness and complexity of the real universe.

  This is the customary accusation made against scientists—the professional students of the richness and complexity of the real universe. Because when you actually look at the universe, it turns out to be, by human standards, insanely rigid in applying its rules. As far as we know, there has been not one single violation of Conservation of Momentum from the uttermost dawn of time up until now.

  Sometimes—very rarely—we observe an apparent violation of our models of the fundamental laws. Though our scientific models may last for a generation or two, they are not stable over the course of centuries . . . but do not fancy that this makes the universe itself whimsical. That is mixing up the map with the territory. For when the dust subsides and the old theory is overthrown, it turns out that the universe always was acting according to the new generalization we have discovered, which once again is absolutely universal as far as humanity’s knowledge extends. When it was discovered that Newtonian gravitation was a special case of General Relativity, it was seen that General Relativity had been governing the orbit of Mercury for decades before any human being knew about it; and it would later become apparent that General Relativity had been governing the collapse of stars for billions of years before humanity. It is only our model that was mistaken—the Law itself was always absolutely constant—or so our new model tells us.

  I may repose only 80% confidence that the lightspeed limit will last out the next hundred thousand years, but this does not mean that I think the lightspeed limit holds only 80% of the time, with occasional exceptions. The proposition to which I assign 80% probability is that the lightspeed law is absolutely inviolable throughout the entirety of space and time.

  One of the reasons the ancient Greeks didn’t discover science is that they didn’t realize you could generalize from experiments. The Greek philosophers were interested in “normal” phenomena. If you set up a contrived experiment, you would probably get a “monstrous” result, one that had no implications for how things really worked.

  So that is how humans tend to dream, before they learn better; but what of the universe’s own quiet dreams that it dreamed to itself before ever it dreamed of humans? If you would learn to think like reality, then here is the Tao:

  Since the beginning

  not one unusual thing

  has ever happened
.

  *

  183

  Is Reality Ugly?

  Consider the cubes, {1, 8, 27, 64, 125, . . . }. Their first differences {7, 19, 37, 61, . . . } might at first seem to lack an obvious pattern, but taking the second differences {12, 18, 24, . . . } takes you down to the simply related level. Taking the third differences {6, 6, . . . } brings us to the perfectly stable level, where chaos dissolves into order.

  But this is a handpicked example. Perhaps the “messy real world” lacks the beauty of these abstract mathematical objects? Perhaps it would be more appropriate to talk about neuroscience or gene expression networks?

  Abstract math, being constructed solely in imagination, arises from simple foundations—a small set of initial axioms—and is a closed system; conditions that might seem unnaturally conducive to neatness.

  Which is to say: In pure math, you don’t have to worry about a tiger leaping out of the bushes and eating Pascal’s Triangle.

  So is the real world uglier than mathematics?

  Strange that people ask this. I mean, the question might have been sensible two and a half millennia ago . . . Back when the Greek philosophers were debating what this “real world” thingy might be made of, there were many positions. Heraclitus said, “All is fire.” Thales said, “All is water.” Pythagoras said, “All is number.”

  Score:

  Heraclitus: 0

  Thales: 0

  Pythagoras: 1

  Beneath the complex forms and shapes of the surface world, there is a simple level, an exact and stable level, whose laws we name “physics.” This discovery, the Great Surprise, has already taken place at our point in human history—but it does not do to forget that it was surprising. Once upon a time, people went in search of underlying beauty, with no guarantee of finding it; and once upon a time, they found it; and now it is a known thing, and taken for granted.

  Then why can’t we predict the location of every tiger in the bushes as easily as we predict the sixth cube?

  I count three sources of uncertainty even within worlds of pure math—two obvious sources, and one not so obvious.

  The first source of uncertainty is that even a creature of pure math, living embedded in a world of pure math, may not know the math. Humans walked the Earth long before Galileo/Newton/Einstein discovered the law of gravity that prevents us from being flung off into space. You can be governed by stable fundamental rules without knowing them. There is no law of physics which says that laws of physics must be explicitly represented, as knowledge, in brains that run under them.

  We do not yet have the Theory of Everything. Our best current theories are things of math, but they are not perfectly integrated with each other. The most probable explanation is that—as has previously proved to be the case—we are seeing surface manifestations of deeper math. So by far the best guess is that reality is made of math; but we do not fully know which math, yet.

  But physicists have to construct huge particle accelerators to distinguish between theories—to manifest their remaining uncertainty in any visible fashion. That physicists must go to such lengths to be unsure, suggests that this is not the source of our uncertainty about stock prices.

  The second obvious source of uncertainty is that even when you know all the relevant laws of physics, you may not have enough computing power to extrapolate them. We know every fundamental physical law that is relevant to a chain of amino acids folding itself into a protein. But we still can’t predict the shape of the protein from the amino acids. Some tiny little 5-nanometer molecule that folds in a microsecond is too much information for current computers to handle (never mind tigers and stock prices). Our frontier efforts in protein folding use clever approximations, rather than the underlying Schrödinger equation. When it comes to describing a 5-nanometer object using really basic physics, over quarks—well, you don’t even bother trying.

  We have to use instruments like X-ray crystallography and NMR to discover the shapes of proteins that are fully determined by physics we know and a DNA sequence we know. We are not logically omniscient; we cannot see all the implications of our thoughts; we do not know what we believe.

  The third source of uncertainty is the most difficult to understand, and Nick Bostrom has written a book about it. Suppose that the sequence {1, 8, 27, 64, 125, . . . } exists; suppose that this is a fact. And suppose that atop each cube is a little person—one person per cube—and suppose that this is also a fact.

  If you stand on the outside and take a global perspective—looking down from above at the sequence of cubes and the little people perched on top—then these two facts say everything there is to know about the sequence and the people.

  But if you are one of the little people perched atop a cube, and you know these two facts, there is still a third piece of information you need to make predictions: “Which cube am I standing on?”

  You expect to find yourself standing on a cube; you do not expect to find yourself standing on the number 7. Your anticipations are definitely constrained by your knowledge of the basic physics; your beliefs are falsifiable. But you still have to look down to find out whether you’re standing on 1,728 or 5,177,717. If you can do fast mental arithmetic, then seeing that the first two digits of a four-digit cube are 17__ will be sufficient to guess that the last digits are 2 and 8. Otherwise you may have to look to discover the 2 and 8 as well.

  To figure out what the night sky should look like, it’s not enough to know the laws of physics. It’s not even enough to have logical omniscience over their consequences. You have to know where you are in the universe. You have to know that you’re looking up at the night sky from Earth. The information required is not just the information to locate Earth in the visible universe, but in the entire universe, including all the parts that our telescopes can’t see because they are too distant, and different inflationary universes, and alternate Everett branches.

  It’s a good bet that “uncertainty about initial conditions at the boundary” is really indexical uncertainty. But if not, it’s empirical uncertainty, uncertainty about how the universe is from a global perspective, which puts it in the same class as uncertainty about fundamental laws.

  Wherever our best guess is that the “real world” has an irretrievably messy component, it is because of the second and third sources of uncertainty—logical uncertainty and indexical uncertainty.

  Ignorance of fundamental laws does not tell you that a messy-looking pattern really is messy. It might just be that you haven’t figured out the order yet.

  But when it comes to messy gene expression networks, we’ve already found the hidden beauty—the stable level of underlying physics. Because we’ve already found the master order, we can guess that we won’t find any additional secret patterns that will make biology as easy as a sequence of cubes. Knowing the rules of the game, we know that the game is hard. We don’t have enough computing power to do protein chemistry from physics (the second source of uncertainty) and evolutionary pathways may have gone different ways on different planets (the third source of uncertainty). New discoveries in basic physics won’t help us here.

  If you were an ancient Greek staring at the raw data from a biology experiment, you would be much wiser to look for some hidden structure of Pythagorean elegance, all the proteins lining up in a perfect icosahedron. But in biology we already know where the Pythagorean elegance is, and we know it’s too far down to help us overcome our indexical and logical uncertainty.

  Similarly, we can be confident that no one will ever be able to predict the results of certain quantum experiments, only because our fundamental theory tells us quite definitely that different versions of us will see different results. If your knowledge of fundamental laws tells you that there’s a sequence of cubes, and that there’s one little person standing on top of each cube, and that the little people are all alike except for being on different cubes, and that you are one of these little people, then you know that you have no way of deducing which cube you’re o
n except by looking.

  The best current knowledge says that the “real world” is a perfectly regular, deterministic, and very large mathematical object which is highly expensive to simulate. So “real life” is less like predicting the next cube in a sequence of cubes, and more like knowing that lots of little people are standing on top of cubes, but not knowing who you personally are, and also not being very good at mental arithmetic. Our knowledge of the rules does constrain our anticipations, quite a bit, but not perfectly.

  There, now doesn’t that sound like real life?

  But uncertainty exists in the map, not in the territory. If we are ignorant of a phenomenon, that is a fact about our state of mind, not a fact about the phenomenon itself. Empirical uncertainty, logical uncertainty, and indexical uncertainty are just names for our own bewilderment. The best current guess is that the world is math and the math is perfectly regular. The messiness is only in the eye of the beholder.

  Even the huge morass of the blogosphere is embedded in this perfect physics, which is ultimately as orderly as {1, 8, 27, 64, 125, . . . }.

  So the Internet is not a big muck . . . it’s a series of cubes.

  *

  184

  Beautiful Probability

  Should we expect rationality to be, on some level, simple? Should we search and hope for underlying beauty in the arts of belief and choice?

  Let me introduce this issue by borrowing a complaint of the late great Bayesian Master, E. T. Jaynes:1

  Two medical researchers use the same treatment independently, in different hospitals. Neither would stoop to falsifying the data, but one had decided beforehand that because of finite resources he would stop after treating n = 100 patients, however many cures were observed by then. The other had staked his reputation on the efficacy of the treatment, and decided he would not stop until he had data indicating a rate of cures definitely greater than 60%, however many patients that might require. But in fact, both stopped with exactly the same data: n = 100 [patients], r = 70 [cures]. Should we then draw different conclusions from their experiments?” [Presumably the two control groups also had equal results.]

 

‹ Prev