Fooled by Randomness

by Nassim Nicholas Taleb

  By some viciousness of the structure of randomness, a profitable person like John, someone who is a pure loser in the long run and correspondingly unfit for survival, presents a high degree of eligibility in the short run and has the propensity to multiply his genes. Recall the hormonal effect on posture and its signaling effect to other potential mates. His success (or pseudosuccess owing to its fragility) will show in his features as a beacon. An innocent potential mate will be fooled into thinking (unconditionally) that he has a superior genetic makeup, until the following rare event. Solon seems to have gotten the point; but try to explain the problem to a naive business Darwinist—or your rich neighbor across the street.

  Six

  •

  SKEWNESS AND ASYMMETRY

  We introduce the concept of skewness: Why the terms “bull” and “bear” have limited meaning outside of zoology. A vicious child wrecks the structure of randomness. An introduction to the problem of epistemic opacity. The penultimate step before the problem of induction.

  THE MEDIAN IS NOT THE MESSAGE

  The essayist and scientist Steven Jay Gould (who, for a while, was my role model), was once diagnosed when he was in his forties with a deadly form of cancer of the lining of the stomach. The first piece of information he received about his odds of making it was that the median survival for the ailment is approximately eight months; information he felt akin to Isaiah’s injunction to King Hezekiah to put his house in order in preparation for death.

  Now, a medical diagnosis, particularly one of such severity, can motivate people to do intensive research, particularly those prolific writers like Gould who needed more time with us to complete a few book projects. The further research by Gould uncovered a very different story from the information he had initially been given; mainly that the expected (i.e., average) survival was considerably higher than eight months. It came to his notice that expected and median do not mean the same thing at all. Median means roughly that 50% of the people die before eight months and 50% survive longer than eight months. But those who survive would live considerably longer, generally going about life just like a regular person and fulfilling the average 73.4 or so years predicted by insurance mortality tables.

  There is asymmetry. Those who die do so very early in the game, while those who live go on living very long. Whenever there is asymmetry in outcomes, the average survival has nothing to do with the median survival. This prompted Gould, who thus undderstood the hard way the concept of skewness, to write his heartfelt piece “The Median Is Not the Message.” His point is that the concept of median used in medical research does not characterize a probability distribution.

  I will simplify Gould’s point by introducing the concept of mean (also called average or expectation) as follows, by using a less morbid example, that of gambling. I will give an example of both asymmetric odds and asymmetric outcomes to explain the point. Asymmetric odds means that probabilities are not 50% for each event, but that the probability on one side is higher than the probability on the other. Asymmetric outcomes mean that the payoffs are not equal.

  Assume I engage in a gambling strategy that has 999 chances in 1,000 of making $1 (event A) and 1 chance in 1,000 of losing $10,000 (event B), as in Table 6.1. My expectation is a loss of close to $9 (obtained by multiplying the probabilities by the corresponding outcomes). The frequency or probability of the loss, in and by itself, is totally irrelevant; it needs to be judged in connection with the magnitude of the outcome. Here A is far more likely than B. Odds are that we would make money by betting for event A, but it is not a good idea to do so.

  Table 6.1

  Event Probability Outcome Expectation

  A 999/1000 $1 $.999

  B 1/1000 -$10,000 -$10

  Total -$9.001

  This point is rather common and simple; it is understood by anyone making a simple bet. Yet I had to struggle all my life with people in the financial markets who do not seem to internalize it. I am not talking of novices; I am talking of people with advanced degrees (albeit MBAs) who cannot come to grips with the difference.

  How could people miss such a point? Why do they confuse probability and expectation, that is, probability and probability times the payoff? Mainly because much of people’s schooling comes from examples in symmetric environments, like a coin toss, where such a difference does not matter. In fact, the so-called bell curve that seems to have found universal use in society is entirely symmetric. More on that later.

  BULL AND BEAR ZOOLOGY

  The general press floods us with concepts like bullish and bearish which refer to the effect of higher (bullish) or lower (bearish) prices in the financial markets. But also we hear people saying “I am bullish on Johnny” or “I am bearish on that guy Nassim in the back who seems incomprehensible to me,” to denote the belief in the likelihood of someone’s rise in life. I have to say that bullish or bearish are often hollow words with no application in a world of randomness—particularly if such a world, like ours, presents asymmetric outcomes.

  When I was in the employment of the New York office of a large investment house, I was subjected on occasions to the harrying weekly “discussion meeting,” which gathered most professionals of the New York trading room. I do not conceal that I was not fond of such gatherings, and not only because they cut into my gym time. While the meetings included traders, that is, people who are judged on their numerical performance, it was mostly a forum for salespeople (people capable of charming customers), and the category of entertainers called Wall Street “economists” or “strategists,” who make pronouncements on the fate of the markets, but do not engage in any form of risk taking, thus having their success dependent on rhetoric rather than actually testable facts. During the discussion, people were supposed to present their opinions on the state of the world. To me, the meeting was pure intellectual pollution. Everyone had a story, a theory, and insights that they wanted others to share. I resent the person who, without having done much homework in libraries, thinks that he is onto something rather original and insightful on a given subject matter (and I respect people with scientific minds, like my friend Stan Jonas, who feel compelled to spend their nights reading wholesale on a subject matter, trying to figure out what was done on the subject by others before emitting an opinion—would the reader listen to the opinion of a doctor who does not read medical papers?).

  I have to confess that my optimal strategy (to soothe my boredom and allergy to confident platitudes) was to speak as much as I could, while totally avoiding listening to other people’s replies by trying to solve equations in my head. Speaking too much would help me clarify my mind, and, with a little bit of luck, I would not be “invited” back (that is, forced to attend) the following week.

  I was once asked in one of those meetings to express my views on the stock market. I stated, not without a modicum of pomp, that I believed that the market would go slightly up over the next week with a high probability. How high? “About 70%.” Clearly, that was a very strong opinion. But then someone interjected,“But, Nassim, you just boasted being short a very large quantity of SP500 futures, making a bet that the market would go down. What made you change your mind?” “I did not change my mind! I have a lot of faith in my bet! [Audience laughing.] As a matter of fact I now feel like selling even more!”The other employees in the room seemed utterly confused. “Are you bullish or are you bearish?” I was asked by the strategist. I replied that I could not understand the words bullish and bearish outside of their purely zoological consideration. Just as with events A and B in the preceding example, my opinion was that the market was more likely to go up (“I would be bullish”), but that it was preferable to short it (“I would be bearish”), because, in the event of its going down, it could go down a lot. Suddenly, the few traders in the room understood my opinion and started voicing similar opinions. And I was not forced to come back to the following discussion.

  Let us assume that the reader shared my opinion, that the market over the n
ext week had a 70% probability of going up and 30% probability of going down. However, let us say that it would go up by 1% on average, while it could go down by an average of 10%. What would the reader do? Is the reader bullish or bearish?

  Table 6.2

  Event Probability Outcome Expectation

  Market goes up 70% Up 1% 0.7

  Market goes down 30% Down 10% -3.00

  Total -2.3

  Accordingly, bullish or bearish are terms used by people who do not engage in practicing uncertainty, like the television commentators, or those who have no experience in handling risk. Alas, investors and businesses are not paid in probabilities; they are paid in dollars. Accordingly, it is not how likely an event is to happen that matters, it is how much is made when it happens that should be the consideration. How frequent the profit is irrelevant; it is the magnitude of the outcome that counts. It is a pure accounting fact that, aside from the commentators, very few people take home a check linked to how often they are right or wrong. What they get is a profit or loss. As to the commentators, their success is linked to how often they are right or wrong. This category includes the “chief strategists” of major investment banks the public can see on TV, who are nothing better than entertainers. They are famous, seem reasoned in their speech, plow you with numbers, but, functionally, they are there to entertain—for their predictions to have any validity they would need a statistical testing framework. Their frame is not the result of some elaborate test but rather the result of their presentation skills.

  An Arrogant Twenty-nine-year-old Son

  Outside of the need for entertainment in these shallow meetings I have resisted voicing a “market call” as a trader, which caused some personal strain with some of my friends and relatives. One day a friend of my father—of the rich and confident variety—called me during his New York visit (to set the elements of pecking order straight, he hinted right away during the call that he came by Concorde, with some derogatory comment on the comfort of such method of transportation). He wanted to pick my brain on the state of a collection of financial markets. I truly had no opinion, nor had made the effort to formulate any, nor was I remotely interested in markets. The gentleman kept plowing me with questions on the state of economies, on the European central banks; these were precise questions no doubt aiming to compare my opinion to that of some other “expert” handling his account at one of the large New York investment firms. I neither concealed that I had no clue, nor did I seem sorry about it. I was not interested in markets (“yes, I am a trader”) and did not make predictions, period. I went on to explain to him some of my ideas on the structure of randomness and the verifiability of market calls but he wanted a more precise statement of what the European bond markets would do by the Christmas season.

  He came away under the impression that I was pulling his leg; it almost damaged the relationship between my father and his rich and confident friend. For the gentleman called him with the following grievance: “When I ask a lawyer a legal question, he answers me with courtesy and precision. When I ask a doctor a medical question, he gives me his opinion. No specialist ever gives me disrespect. Your insolent and conceited twenty-nine-year-old son is playing prima donna and refuses to answer me about the direction of the market!”

  Rare Events

  The best description of my lifelong business in the market is “skewed bets,” that is, I try to benefit from rare events, events that do not tend to repeat themselves frequently, but, accordingly, present a large payoff when they occur. I try to make money infrequently, as infrequently as possible, simply because I believe that rare events are not fairly valued, and that the rarer the event, the more undervalued it will be in price. In addition to my own empiricism, I think that the counterintuitive aspect of the trade (and the fact that our emotional wiring does not accommodate it) gives me some form of advantage.

  Why are these events poorly valued? Because of a psychological bias; people who surrounded me in my career were too focused on memorizing section 2 of The Wall Street Journal during their train ride to reflect properly on the attributes of random events. Or perhaps they watched too many gurus on television. Or perhaps they spent too much time upgrading their PalmPilot. Even some experienced trading veterans do not seem to get the point that frequencies do not matter. Jim Rogers, a “legendary” investor, made the following statement:

  I don’t buy options. Buying options is another way to go to the poorhouse. Someone did a study for the SEC and discovered that 90 percent of all options expire as losses. Well, I figured out that if 90 percent of all long option positions lost money, that meant that 90 percent of all short option positions make money. If I want to use options to be bearish, I sell calls.

  Visibly, the statistic that 90% of all option positions lost money is meaningless, (i.e., the frequency) if we do not take into account how much money is made on average during the remaining 10%. If we make 50 times our bet on average when the option is in the money, then I can safely make the statement that buying options is another way to go to the palazzo rather than the poorhouse. Mr. Jim Rogers seems to have gone very far in life for someone who does not distinguish between probability and expectation (strangely, he was the partner of George Soros, a complex man who thrived on rare events—more on him later).

  One such rare event is the stock market crash of 1987, which made me as a trader and allowed me the luxury of becoming involved in all manner of scholarship. Nero of the smaller house in Chapter 1 aims to get out of harm’s way by avoiding exposure to rare events—a mostly defensive approach. I am far more aggressive than Nero and go one step further; I have organized my career and business in such a way as to be able to benefit from them. In other words, I aim at profiting from the rare event, with my asymmetric bets.

  Symmetry and Science

  In most disciplines, such asymmetry does not matter. In an academic pass/fail environment, where the cumulative grade does not matter, only frequency matters. Outside of that it is the magnitude that counts. Unfortunately, the techniques used in economics are often imported from other areas—financial economics is still a young discipline (it is certainly not yet a “science”). People in most fields outside of it do not have problems eliminating extreme values from their sample, when the difference in payoff between different outcomes is not significant, which is generally the case in education and medicine. A professor who computes the average of his students’ grades removes the highest and lowest observations, which he would call outliers, and takes the average of the remaining ones, without this being an unsound practice. A casual weather forecaster does the same with extreme temperatures—an unusual occurrence might be deemed to skew the overall result (though we will see that this may turn out to be a mistake when it comes to forecasting future properties of the ice cap). So people in finance borrow the technique and ignore infrequent events, not noticing that the effect of a rare event can bankrupt a company.

  Many scientists in the physical world are also subject to such foolishness, misreading statistics. One flagrant example is in the global-warming debate. Many scientists failed to notice it in its early stages as they removed from their sample the spikes in temperature, under the belief that these were not likely to recur. It may be a good idea to take out the extremes when computing the average temperatures for vacation scheduling. But it does not work when we study the physical properties of the weather—particularly when one cares about a cumulative effect. These scientists initially ignored the fact that these spikes, although rare, had the effect of adding disproportionately to the cumulative melting of the ice cap. Just as in finance, an event, although rare, that brings large consequences cannot just be ignored.

  ALMOST EVERYBODY IS ABOVE AVERAGE

  Jim Rogers is not the only person committing such traditional fallacy of mistaking mean and median. In all fairness to him, some people who think for a living, such as the star philosopher Robert Nozik, have committed versions of the same mistake (Nozik, besides, was an admirable
and incisive thinker; before his premature death he was perhaps the most respected American philosopher of his generation). In his book The Nature of Rationality he gets, as is typical with philosophers, into amateur evolutionary arguments and writes the following: “Since not more than 50 percent of the individuals can be wealthier than average.” Of course, more than 50% of individuals can be wealthier than average. Consider that you have a very small number of very poor people and the rest clustering around the middle class. The mean will be lower than the median. Take a population of 10 people, 9 having a net worth of $30,000 and 1 having a net worth of $1,000. The average net worth is $27,100 and 9 out of 10 people will have above average wealth.

  Figure 6.1 shows a series of points starting with an initial level W0 and ending at the period concerned Wt. It can also be seen as the performance, hypothetical or realized, of your favorite trading strategy, the track record of an investment manager, the price of a foot of average Palazzo real estate in Renaissance Florence, the price series of the Mongolian stock market, or the difference between the U.S. and Mongolian stock markets. It is composed of a given number of sequential observations W1, W2, etc., ordered in such a way that the one to the right comes after the one to the left.

  Figure 6.1 A Primer on Time Series

  If we were dealing with a deterministic world—that is, a world stripped of randomness (the right-column world in Table P.1), and we knew with certainty that it was the case, things would be rather easy. The pattern of the series would reveal considerable and predictive information. You could tell with precision what would happen one day ahead, one year ahead, and perhaps even a decade ahead. We would not even need a statistician; a second-rate engineer would do. He does not even need to be armed with a modern degree; someone with nineteenth-century training under Laplace would be able to solve the equations, called differential equations, or, equivalently, equations of motion—since we are studying the dynamics of an entity whose position depends on time.