The Black Swan

by Nassim Nicholas Taleb

  Effectively, if free markets have been successful, it is precisely because they allow the trial-and-error process I call “stochastic tinkering” on the part of competing individual operators who fall for the narrative fallacy—but are effectively collectively partaking of a grand project. We are increasingly learning to practice stochastic tinkering without knowing it—thanks to overconfident entrepreneurs, naïve investors, greedy investment bankers, and aggressive venture capitalists brought together by the free-market system. The next chapter shows why I am optimistic that the academy is losing its power and ability to put knowledge in straitjackets and that more out-of-the-box knowledge will be generated Wiki-style.

  In the end we are being driven by history, all the while thinking that we are doing the driving.

  I’ll sum up this long section on prediction by stating that we can easily narrow down the reasons we can’t figure out what’s going on. There are: a) epistemic arrogance and our corresponding future blindness; b) the Platonic notion of categories, or how people are fooled by reductions, particularly if they have an academic degree in an expert-free discipline; and, finally c) flawed tools of inference, particularly the Black Swan–free tools from Mediocristan.

  In the next section we will go deeper, much deeper, into these tools from Mediocristan, into the “plumbing,” so to speak. Some readers may see it as an appendix; others may consider it the heart of the book.

  * This chapter provides a general conclusion for those who by now say, “Taleb, I get the point, but what should I do?” My answer is that if you got the point, you are pretty much there. But here is a nudge.

  * Dan Gilbert showed in a famous paper, “How Mental Systems Believe,” that we are not natural skeptics and that not believing required an expenditure of mental effort.

  * Make sure that you have plenty of these small bets; avoid being blinded by the vividness of one single Black Swan. Have as many of these small bets as you can conceivably have. Even venture capital firms fall for the narrative fallacy with a few stories that “make sense” to them; they do not have as many bets as they should. If venture capital firms are profitable, it is not because of the stories they have in their heads, but because they are exposed to unplanned rare events.

  * There is a finer epistemological point. Remember that in a virtuous Black Swan business, what the past did not reveal is almost certainly going to be good for you. When you look at past biotech revenues, you do not see the superblockbuster in them, and owing to the potential for a cure for cancer (or headaches, or baldness, or bad sense of humor, etc.), there is a small probability that the sales in that industry may turn out to be monstrous, far larger than might be expected. On the other hand, consider negative Black Swan businesses. The track record you see is likely to overestimate the properties. Recall the 1982 blowup of banks: they appeared to the naïve observer to be more profitable than they seemed. Insurance companies are of two kinds: the regular diversifiable kind that belongs to Mediocristan (say, life insurance) and the more critical and explosive Black Swan–prone risks that are usually sold to reinsurers. According to the data, reinsurers have lost money on underwriting over the past couple of decades, but, unlike bankers, they are introspective enough to know that it actually could have been far worse, because the past twenty years did not have a big catastrophe, and all you need is one of those per century to kiss the business good-bye. Many finance academics doing “valuation” on insurance seem to have missed the point.

  It’s time to deal in some depth with four final items that bear on our Black Swan.

  Primo, I have said earlier that the world is moving deeper into Extremistan, that it is less and less governed by Mediocristan—in fact, this idea is more subtle than that. I will show how and present the various ideas we have about the formation of inequality. Secondo, I have been describing the Gaussian bell curve as a contagious and severe delusion, and it is time to get into that point in some depth. Terso, I will present what I call Mandelbrotian, or fractal, randomness. Remember that for an event to be a Black Swan, it does not just have to be rare, or just wild; it has to be unexpected, has to lie outside our tunnel of possibilities. You must be a sucker for it. As it happens, many rare events can yield their structure to us: it is not easy to compute their probability, but it is easy to get a general idea about the possibility of their occurrence. We can turn these Black Swans into Gray Swans, so to speak, reducing their surprise effect. A person aware of the possibility of such events can come to belong to the non-sucker variety.

  Finally, I will present the ideas of those philosophers who focus on phony uncertainty. I organized this book in such a way that the more technical (though nonessential) sections are here; these can be skipped without any loss to the thoughtful reader, particularly Chapters 15, 17, and the second half of Chapter 16. I will alert the reader with footnotes. The reader less interested in the mechanics of deviations can then directly proceed to Part 4.

  Chapter Fourteen

  FROM MEDIOCRISTAN TO EXTREMISTAN, AND BACK

  I prefer Horowitz—How to fall from favor—The long tail—Get ready for some surprises—It’s not just money

  Let us see how an increasingly man-made planet can evolve away from mild into wild randomness. First, I describe how we get to Extremistan. Then, I will take a look at its evolution.

  The World Is Unfair

  Is the world that unfair? I have spent my entire life studying randomness, practicing randomness, hating randomness. The more that time passes, the worse things seem to me, the more scared I get, the more disgusted I am with Mother Nature. The more I think about my subject, the more I see evidence that the world we have in our minds is different from the one playing outside. Every morning the world appears to me more random than it did the day before, and humans seem to be even more fooled by it than they were the previous day. It is becoming unbearable. I find writing these lines painful; I find the world revolting.

  Two “soft” scientists propose intuitive models for the development of this inequity: one is a mainstream economist, the other a sociologist. Both simplify a little too much. I will present their ideas because they are easy to understand, not because of the scientific quality of their insights or any consequences in their discoveries; then I will show the story as seen from the vantage point of the natural scientists.

  Let me start with the economist Sherwin Rosen. In the early eighties, he wrote papers about “the economics of superstars.” In one of the papers he conveyed his sense of outrage that a basketball player could earn $1.2 million a year, or a television celebrity could make $2 million. To get an idea of how this concentration is increasing—i.e., of how we are moving away from Mediocristan—consider that television celebrities and sports stars (even in Europe) get contracts today, only two decades later, worth in the hundreds of millions of dollars! The extreme is about (so far) twenty times higher than it was two decades ago!

  According to Rosen, this inequality comes from a tournament effect: someone who is marginally “better” can easily win the entire pot, leaving the others with nothing. Using an argument from Chapter 3, people prefer to pay $10.99 for a recording featuring Horowitz to $9.99 for a struggling pianist. Would you rather read Kundera for $13.99 or some unknown author for $1? So it looks like a tournament, where the winner grabs the whole thing—and he does not have to win by much.

  But the role of luck is missing in Rosen’s beautiful argument. The problem here is the notion of “better,” this focus on skills as leading to success. Random outcomes, or an arbitrary situation, can also explain success, and provide the initial push that leads to a winner-take-all result. A person can get slightly ahead for entirely random reasons; because we like to imitate one another, we will flock to him. The world of contagion is so underestimated!

  As I am writing these lines I am using a Macintosh, by Apple, after years of using Microsoft-based products. The Apple technology is vastly better, yet the inferior software won the day. How? Luck.

  The
Matthew Effect

  More than a decade before Rosen, the sociologist of science Robert K. Merton presented his idea of the Matthew effect, by which people take from the poor to give to the rich.* He looked at the performance of scientists and showed how an initial advantage follows someone through life. Consider the following process.

  Let’s say someone writes an academic paper quoting fifty people who have worked on the subject and provided background materials for his study; assume, for the sake of simplicity, that all fifty are of equal merit. Another researcher working on the exact same subject will randomly cite three of those fifty in his bibliography. Merton showed that many academics cite references without having read the original work; rather, they’ll read a paper and draw their own citations from among its sources. So a third researcher reading the second article selects three of the previously referenced authors for his citations. These three authors will receive cumulatively more and more attention as their names become associated more tightly with the subject at hand. The difference between the winning three and the other members of the original cohort is mostly luck: they were initially chosen not for their greater skill, but simply for the way their names appeared in the prior bibliography. Thanks to their reputations, these successful academics will go on writing papers and their work will be easily accepted for publication. Academic success is partly (but significantly) a lottery.*

  It is easy to test the effect of reputation. One way would be to find papers that were written by famous scientists, had their authors’ identities changed by mistake, and got rejected. You could verify how many of these rejections were subsequently overturned after the true identities of the authors were established. Note that scholars are judged mostly on how many times their work is referenced in other people’s work, and thus cliques of people who quote one another are formed (it’s an “I quote you, you quote me” type of business).

  Eventually, authors who are not often cited will drop out of the game by, say, going to work for the government (if they are of a gentle nature), or for the Mafia, or for a Wall Street firm (if they have a high level of hormones). Those who got a good push in the beginning of their scholarly careers will keep getting persistent cumulative advantages throughout life. It is easier for the rich to get richer, for the famous to become more famous.

  In sociology, Matthew effects bear the less literary name “cumulative advantage.” This theory can easily apply to companies, businessmen, actors, writers, and anyone else who benefits from past success. If you get published in The New Yorker because the color of your letterhead attracted the attention of the editor, who was daydreaming of daisies, the resultant reward can follow you for life. More significantly, it will follow others for life. Failure is also cumulative; losers are likely to also lose in the future, even if we don’t take into account the mechanism of demoralization that might exacerbate it and cause additional failure.

  Note that art, because of its dependence on word of mouth, is extremely prone to these cumulative-advantage effects. I mentioned clustering in Chapter 1, and how journalism helps perpetuate these clusters. Our opinions about artistic merit are the result of arbitrary contagion even more than our political ideas are. One person writes a book review; another person reads it and writes a commentary that uses the same arguments. Soon you have several hundred reviews that actually sum up in their contents to no more than two or three because there is so much overlap. For an anecdotal example read Fire the Bastards!, whose author, Jack Green, goes systematically through the reviews of William Gaddis’s novel The Recognitions. Green shows clearly how book reviewers anchor on other reviews and reveals powerful mutual influence, even in their wording. This phenomenon is reminiscent of the herding of financial analysts I discussed in Chapter 10.

  The advent of the modern media has accelerated these cumulative advantages. The sociologist Pierre Bourdieu noted a link between the increased concentration of success and the globalization of culture and economic life. But I am not trying to play sociologist here, only show that unpredictable elements can play a role in social outcomes.

  Merton’s cumulative-advantage idea has a more general precursor, “preferential attachment,” which, reversing the chronology (though not the logic), I will present next. Merton was interested in the social aspect of knowledge, not in the dynamics of social randomness, so his studies were derived separately from research on the dynamics of randomness in more mathematical sciences.

  Lingua Franca

  The theory of preferential attachment is ubiquitous in its applications: it can explain why city size is from Extremistan, why vocabulary is concentrated among a small number of words, or why bacteria populations can vary hugely in size.

  The scientists J. C. Willis and G. U. Yule published a landmark paper in Nature in 1922 called “Some Statistics of Evolution and Geographical Distribution in Plants and Animals, and Their Significance.” Willis and Yule noted the presence in biology of the so-called power laws, atractable versions of the scalable randomness that I discussed in Chapter 3. These power laws (on which more technical information in the following chapters) had been noticed earlier by Vilfredo Pareto, who found that they applied to the distribution of income. Later, Yule presented a simple model showing how power laws can be generated. His point was as follows: Let’s say species split in two at some constant rate, so that new species arise. The richer in species a genus is, the richer it will tend to get, with the same logic as the Mathew effect. Note the following caveat: in Yule’s model the species never die out.

  During the 1940s, a Harvard linguist, George Zipf, examined the properties of language and came up with an empirical regularity now known as Zipf’s law, which, of course, is not a law (and if it were, it would not be Zipf’s). It is just another way to think about the process of inequality. The mechanisms he described were as follows: the more you use a word, the less effortful you will find it to use that word again, so you borrow words from your private dictionary in proportion to their past use. This explains why out of the sixty thousand main words in English, only a few hundred constitute the bulk of what is used in writings, and even fewer appear regularly in conversation. Likewise, the more people aggregate in a particular city, the more likely a stranger will be to pick that city as his destination. The big get bigger and the small stay small, or get relatively smaller.

  A great illustration of preferential attachment can be seen in the mushrooming use of English as a lingua franca—though not for its intrinsic qualities, but because people need to use one single language, or stick to one as much as possible, when they are having a conversation. So whatever language appears to have the upper hand will suddenly draw people in droves; its usage will spread like an epidemic, and other languages will be rapidly dislodged. I am often amazed to listen to conversations between people from two neighboring countries, say, between a Turk and an Iranian, or a Lebanese and a Cypriot, communicating in bad English, moving their hands for emphasis, searching for these words that come out of their throats at the cost of great physical effort. Even members of the Swiss Army use English (not French) as a lingua franca (it would be fun to listen). Consider that a very small minority of Americans of northern European descent is from England; traditionally the preponderant ethnic groups are of German, Irish, Dutch, French, and other northern European extraction. Yet because all these groups now use English as their main tongue, they have to study the roots of their adoptive tongue and develop a cultural association with parts of a particular wet island, along with its history, its traditions, and its customs!

  Ideas and Contagions

  The same model can be used for the contagions and concentration of ideas. But there are some restrictions on the nature of epidemics I must discuss here. Ideas do not spread without some form of structure. Recall the discussion in Chapter 4 about how we come prepared to make inferences. Just as we tend to generalize some matters but not others, so there seem to be “basins of attraction” directing us to certain beliefs. Some ideas will
prove contagious, but not others; some forms of superstitions will spread, but not others; some types of religious beliefs will dominate, but not others. The anthropologist, cognitive scientist, and philosopher Dan Sperber has proposed the following idea on the epidemiology of representations. What people call “memes,” ideas that spread and that compete with one another using people as carriers, are not truly like genes. Ideas spread because, alas, they have for carriers self-serving agents who are interested in them, and interested in distorting them in the replication process. You do not make a cake for the sake of merely replicating a recipe—you try to make your own cake, using ideas from others to improve it. We humans are not photocopiers. So contagious mental categories must be those in which we are prepared to believe, perhaps even programmed to believe. To be contagious, a mental category must agree with our nature.

  NOBODY IS SAFE IN EXTREMISTAN

  There is something extremely naïve about all these models of the dynamics of concentration I’ve presented so far, particularly the socioeconomic ones. For instance, although Merton’s idea includes luck, it misses an additional layer of randomness. In all these models the winner stays a winner. Now, a loser might always remain a loser, but a winner could be unseated by someone new popping up out of nowhere. Nobody is safe.

  Preferential-attachment theories are intuitively appealing, but they do not account for the possibility of being supplanted by newcomers—what every schoolchild knows as the decline of civilizations. Consider the logic of cities: How did Rome, with a population of 1.2 million in the first century A.D., end up with a population of twelve thousand in the third? How did Baltimore, once a principal American city, become a relic? And how did Philadelphia come to be overshadowed by New York?