The Black Swan

by Nassim Nicholas Taleb

  Poincaré’s reasoning was simple: as you project into the future you may need an increasing amount of precision about the dynamics of the process that you are modeling, since your error rate grows very rapidly. The problem is that near precision is not possible since the degradation of your forecast compounds abruptly—you would eventually need to figure out the past with infinite precision. Poincaré showed this in a very simple case, famously known as the “three body problem.” If you have only two planets in a solar-style system, with nothing else affecting their course, then you may be able to indefinitely predict the behavior of these planets, no sweat. But add a third body, say a comet, ever so small, between the planets. Initially the third body will cause no drift, no impact; later, with time, its effects on the two other bodies may become explosive. Small differences in where this tiny body is located will eventually dictate the future of the behemoth planets.

  FIGURE 2: PRECISION AND FORECASTING

  One of the readers of a draft of this book, David Cowan, gracefully drew this picture of scattering, which shows how, at the second bounce, variations in the initial conditions can lead to extremely divergent results. As the initial imprecision in the angle is multiplied, every additional bounce will be further magnified. This causes a severe multiplicative effect where the error grows out disproportionately.

  Explosive forecasting difficulty comes from complicating the mechanics, ever so slightly. Our world, unfortunately, is far more complicated than the three body problem; it contains far more than three objects. We are dealing with what is now called a dynamical system—and the world, we will see, is a little too much of a dynamical system.

  Think of the difficulty in forecasting in terms of branches growing out of a tree; at every fork we have a multiplication of new branches. To see how our intuitions about these nonlinear multiplicative effects are rather weak, consider this story about the chessboard. The inventor of the chessboard requested the following compensation: one grain of rice for the first square, two for the second, four for the third, eight, then sixteen, and so on, doubling every time, sixty-four times. The king granted this request, thinking that the inventor was asking for a pittance—but he soon realized that he was outsmarted. The amount of rice exceeded all possible grain reserves!

  This multiplicative difficulty leading to the need for greater and greater precision in assumptions can be illustrated with the following simple exercise concerning the prediction of the movements of billiard balls on a table. I use the example as computed by the mathematician Michael Berry. If you know a set of basic parameters concerning the ball at rest, can compute the resistance of the table (quite elementary), and can gauge the strength of the impact, then it is rather easy to predict what would happen at the first hit. The second impact becomes more complicated, but possible; you need to be more careful about your knowledge of the initial states, and more precision is called for. The problem is that to correctly compute the ninth impact, you need to take into account the gravitational pull of someone standing next to the table (modestly, Berry’s computations use a weight of less than 150 pounds). And to compute the fifty-sixth impact, every single elementary particle of the universe needs to be present in your assumptions! An electron at the edge of the universe, separated from us by 10 billion light-years, must figure in the calculations, since it exerts a meaningful effect on the outcome. Now, consider the additional burden of having to incorporate predictions about where these variables will be in the future. Forecasting the motion of a billiard ball on a pool table requires knowledge of the dynamics of the entire universe, down to every single atom! We can easily predict the movements of large objects like planets (though not too far into the future), but the smaller entities can be difficult to figure out—and there are so many more of them.

  Note that this billiard-ball story assumes a plain and simple world; it does not even take into account these crazy social matters possibly endowed with free will. Billiard balls do not have a mind of their own. Nor does our example take into account relativity and quantum effects. Nor did we use the notion (often invoked by phonies) called the “uncertainty principle.” We are not concerned with the limitations of the precision in measurements done at the subatomic level. We are just dealing with billiard balls!

  In a dynamical system, where you are considering more than a ball on its own, where trajectories in a way depend on one another, the ability to project into the future is not just reduced, but is subjected to a fundamental limitation. Poincaré proposed that we can only work with qualitative matters—some property of systems can be discussed, but not computed. You can think rigorously, but you cannot use numbers. Poincaré even invented a field for this, analysis in situ, now part of topology. Prediction and forecasting are a more complicated business than is commonly accepted, but it takes someone who knows mathematics to understand that. To accept it takes both understanding and courage.

  In the 1960s the MIT meteorologist Edward Lorenz rediscovered Poincaré’s results on his own—once again, by accident. He was producing a computer model of weather dynamics, and he ran a simulation that projected a weather system a few days ahead. Later he tried to repeat the same simulation with the exact same model and what he thought were the same input parameters, but he got wildly different results. He initially attributed these differences to a computer bug or a calculation error. Computers then were heavier and slower machines that bore no resemblance to what we have today, so users were severely constrained by time. Lorenz subsequently realized that the consequential divergence in his results arose not from error, but from a small rounding in the input parameters. This became known as the butterfly effect, since a butterfly moving its wings in India could cause a hurricane in New York, two years later. Lorenz’s findings generated interest in the field of chaos theory.

  Naturally researchers found predecessors to Lorenz’s discovery, not only in the work of Poincaré, but also in that of the insightful and intuitive Jacques Hadamard, who thought of the same point around 1898, and then went on to live for almost seven more decades—he died at the age of ninety-eight.*

  They Still Ignore Hayek

  Popper and Poincaré’s findings limit our ability to see into the future, making it a very complicated reflection of the past—if it is a reflection of the past at all. A potent application in the social world comes from a friend of Sir Karl, the intuitive economist Friedrich Hayek. Hayek is one of the rare celebrated members of his “profession” (along with J. M. Keynes and G.L.S. Shackle) to focus on true uncertainty, on the limitations of knowledge, on the unread books in Eco’s library.

  In 1974 he received the Bank of Sweden Prize in Economic Sciences in Memory of Alfred Nobel, but if you read his acceptance speech you will be in for a bit of a surprise. It was eloquently called “The Pretense of Knowledge,” and he mostly railed about other economists and about the idea of the planner. He argued against the use of the tools of hard science in the social ones, and depressingly, right before the big boom for these methods in economics. Subsequently, the prevalent use of complicated equations made the environment for true empirical thinkers worse than it was before Hayek wrote his speech. Every year a paper or a book appears, bemoaning the fate of economics and complaining about its attempts to ape physics. The latest I’ve seen is about how economists should shoot for the role of lowly philosophers rather than that of high priests. Yet, in one ear and out the other.

  For Hayek, a true forecast is done organically by a system, not by fiat. One single institution, say, the central planner, cannot aggregate knowledge; many important pieces of information will be missing. But society as a whole will be able to integrate into its functioning these multiple pieces of information. Society as a whole thinks outside the box. Hayek attacked socialism and managed economies as a product of what I have called nerd knowledge, or Platonicity—owing to the growth of scientific knowledge, we overestimate our ability to understand the subtle changes that constitute the world, and what weight needs to be imparted t
o each such change. He aptly called this “scientism.”

  This disease is severely ingrained in our institutions. It is why I fear governments and large corporations—it is hard to distinguish between them. Governments make forecasts; companies produce projections; every year various forecasters project the level of mortgage rates and the stock market at the end of the following year. Corporations survive not because they have made good forecasts, but because, like the CEOs visiting Wharton I mentioned earlier, they may have been the lucky ones. And, like a restaurant owner, they may be hurting themselves, not us—perhaps helping us and subsidizing our consumption by giving us goods in the process, like cheap telephone calls to the rest of the world funded by the overinvestment during the dotcom era. We consumers can let them forecast all they want if that’s what is necessary for them to get into business. Let them go hang themselves if they wish.

  As a matter of fact, as I mentioned in Chapter 8, we New Yorkers are all benefiting from the quixotic overconfidence of corporations and restaurant entrepreneurs. This is the benefit of capitalism that people discuss the least.

  But corporations can go bust as often as they like, thus subsidizing us consumers by transferring their wealth into our pockets—the more bankruptcies, the better it is for us—unless they are “too big to fail” and require subsidies, which is an argument in favor of letting companies go bust early. Government is a more serious business and we need to make sure we do not pay the price for its folly. As individuals we should love free markets because operators in them can be as incompetent as they wish.

  The only criticism one might have of Hayek is that he makes a hard and qualitative distinction between social sciences and physics. He shows that the methods of physics do not translate to its social science siblings, and he blames the engineering-oriented mentality for this. But he was writing at a time when physics, the queen of science, seemed to zoom in our world. It turns out that even the natural sciences are far more complicated than that. He was right about the social sciences, he is certainly right in trusting hard scientists more than social theorizers, but what he said about the weaknesses of social knowledge applies to all knowledge. All knowledge.

  Why? Because of the confirmation problem, one can argue that we know very little about our natural world; we advertise the read books and forget about the unread ones. Physics has been successful, but it is a narrow field of hard science in which we have been successful, and people tend to generalize that success to all science. It would be preferable if we were better at understanding cancer or the (highly nonlinear) weather than the origin of the universe.

  How Not to Be a Nerd

  Let us dig deeper into the problem of knowledge and continue the comparison of Fat Tony and Dr. John in Chapter 9. Do nerds tunnel, meaning, do they focus on crisp categories and miss sources of uncertainty? Remember from the Prologue my presentation of Platonification as a top-down focus on a world composed of these crisp categories.*

  Think of a bookworm picking up a new language. He will learn, say, Serbo-Croatian or !Kung by reading a grammar book cover to cover, and memorizing the rules. He will have the impression that some higher grammatical authority set the linguistic regulations so that nonlearned ordinary people could subsequently speak the language. In reality, languages grow organically; grammar is something people without anything more exciting to do in their lives codify into a book. While the scholastic-minded will memorize declensions, the a-Platonic nonnerd will acquire, say, Serbo-Croatian by picking up potential girlfriends in bars on the outskirts of Sarajevo, or talking to cabdrivers, then fitting (if needed) grammatical rules to the knowledge he already possesses.

  Consider again the central planner. As with language, there is no grammatical authority codifying social and economic events; but try to convince a bureaucrat or social scientist that the world might not want to follow his “scientific” equations. In fact, thinkers of the Austrian school, to which Hayek belonged, used the designations tacit or implicit precisely for that part of knowledge that cannot be written down, but that we should avoid repressing. They made the distinction we saw earlier between “know-how” and “know-what”—the latter being more elusive and more prone to nerdification.

  To clarify, Platonic is top-down, formulaic, closed-minded, self-serving, and commoditized; a-Platonic is bottom-up, open-minded, skeptical, and empirical.

  The reason for my singling out the great Plato becomes apparent with the following example of the master’s thinking: Plato believed that we should use both hands with equal dexterity. It would not “make sense” otherwise. He considered favoring one limb over the other a deformation caused by the “folly of mothers and nurses.” Asymmetry bothered him, and he projected his ideas of elegance onto reality. We had to wait until Louis Pasteur to figure out that chemical molecules were either left- or right-handed and that this mattered considerably.

  One can find similar ideas among several disconnected branches of thinking. The earliest were (as usual) the empirics, whose bottom-up, theory-free, “evidence-based” medical approach was mostly associated with Philnus of Cos, Serapion of Alexandria, and Glaucias of Tarentum, later made skeptical by Menodotus of Nicomedia, and currently well-known by its vocal practitioner, our friend the great skeptical philosopher Sextus Empiricus. Sextus who, we saw earlier, was perhaps the first to discuss the Black Swan. The empirics practiced the “medical art” without relying on reasoning; they wanted to benefit from chance observations by making guesses, and experimented and tinkered until they found something that worked. They did minimal theorizing.

  Their methods are being revived today as evidence-based medicine, after two millennia of persuasion. Consider that before we knew of bacteria, and their role in diseases, doctors rejected the practice of hand washing because it made no sense to them, despite the evidence of a meaningful decrease in hospital deaths. Ignaz Semmelweis, the mid-nineteenth-century doctor who promoted the idea of hand washing, wasn’t vindicated until decades after his death. Similarly it may not “make sense” that acupuncture works, but if pushing a needle in someone’s toe systematically produces relief from pain (in properly conducted empirical tests), then it could be that there are functions too complicated for us to understand, so let’s go with it for now while keeping our minds open.

  Academic Libertarianism

  To borrow from Warren Buffett, don’t ask the barber if you need a haircut—and don’t ask an academic if what he does is relevant. So I’ll end this discussion of Hayek’s libertarianism with the following observation. As I’ve said, the problem with organized knowledge is that there is an occasional divergence of interests between academic guilds and knowledge itself. So I cannot for the life of me understand why today’s libertarians do not go after tenured faculty (except perhaps because many libertarians are academics). We saw that companies can go bust, while governments remain. But while governments remain, civil servants can be demoted and congressmen and senators can be eventually voted out of office. In academia a tenured faculty is permanent—the business of knowledge has permanent “owners.” Simply, the charlatan is more the product of control than the result of freedom and lack of structure.

  Prediction and Free Will

  If you know all possible conditions of a physical system you can, in theory (though not, as we saw, in practice), project its behavior into the future. But this only concerns inanimate objects. We hit a stumbling block when social matters are involved. It is another matter to project a future when humans are involved, if you consider them living beings and endowed with free will.

  If I can predict all of your actions, under given circumstances, then you may not be as free as you think you are. You are an automaton responding to environmental stimuli. You are a slave of destiny. And the illusion of free will could be reduced to an equation that describes the result of interactions among molecules. It would be like studying the mechanics of a clock: a genius with extensive knowledge of the initial conditions and the causal chains would be
able to extend his knowledge to the future of your actions. Wouldn’t that be stifling?

  However, if you believe in free will you can’t truly believe in social science and economic projection. You cannot predict how people will act. Except, of course, if there is a trick, and that trick is the cord on which neoclassical economics is suspended. You simply assume that individuals will be rational in the future and thus act predictably. There is a strong link between rationality, predictability, and mathematical tractability. A rational individual will perform a unique set of actions in specified circumstances. There is one and only one answer to the question of how “rational” people satisfying their best interests would act. Rational actors must be coherent: they cannot prefer apples to oranges, oranges to pears, then pears to apples. If they did, then it would be difficult to generalize their behavior. It would also be difficult to project their behavior in time.

  In orthodox economics, rationality became a straitjacket. Platonified economists ignored the fact that people might prefer to do something other than maximize their economic interests. This led to mathematical techniques such as “maximization,” or “optimization,” on which Paul Samuelson built much of his work. Optimization consists in finding the mathematically optimal policy that an economic agent could pursue. For instance, what is the “optimal” quantity you should allocate to stocks? It involves complicated mathematics and thus raises a barrier to entry by non-mathematically trained scholars. I would not be the first to say that this optimization set back social science by reducing it from the intellectual and reflective discipline that it was becoming to an attempt at an “exact science.” By “exact science,” I mean a second-rate engineering problem for those who want to pretend that they are in the physics department—so-called physics envy. In other words, an intellectual fraud.