Fooled by Randomness

by Nassim Nicholas Taleb

  PHILOSTRATUS IN MONTE CARLO:

  ON THE DIFFERENCE BETWEEN NOISE

  AND INFORMATION

  The wise man listens to meaning; the fool only gets the noise. The modern Greek poet C. P. Cavafy wrote a piece in 1915 after Philostratus’ adage “For the gods perceive things in the future, ordinary people things in the present, but the wise perceive things about to happen.” Cavafy wrote:

  In their intense meditation the hidden sound of things approaching reaches them and they listen reverently while in the street outside the people hear nothing at all.

  I thought hard and long on how to explain with as little mathematics as possible the difference between noise and meaning, and how to show why the time scale is important in judging a historical event. The Monte Carlo simulator can provide us with such an intuition. We will start with an example borrowed from the investment world, as it can be explained rather easily, but the concept can be used in any application.

  Let us manufacture a happily retired dentist, living in a pleasant, sunny town. We know a priori that he is an excellent investor, and that he will be expected to earn a return of 15% in excess of Treasury bills, with a 10% error rate per annum (what we call volatility). It means that out of 100 sample paths, we expect close to 68 of them to fall within a band of plus and minus 10% around the 15% excess return, i.e., between 5% and 25% (to be technical; the bell-shaped normal distribution has 68% of all observations falling between -1 and 1 standard deviations). It also means that 95 sample paths would fall between -5% and 35%.

  Clearly, we are dealing with a very optimistic situation. The dentist builds for himself a nice trading desk in his attic, aiming to spend every business day there watching the market, while sipping decaffeinated cappuccino. He has an adventurous temperament, so he finds this activity more attractive than drilling the teeth of reluctant little old Park Avenue ladies.

  He subscribes to a Web-based service that supplies him with continuous prices, now to be obtained for a fraction of what he pays for his coffee. He puts his inventory of securities in his spreadsheet and can thus instantaneously monitor the value of his speculative portfolio. We are living in the era of connectivity.

  A 15% return with a 10% volatility (or uncertainty) per annum translates into a 93% probability of success in any given year. But seen at a narrow time scale, this translates into a mere 50.02% probability of success over any given second as shown in Table 3.1. Over the very narrow time increment, the observation will reveal close to nothing. Yet the dentist’s heart will not tell him that. Being emotional, he feels a pang with every loss, as it shows in red on his screen. He feels some pleasure when the performance is positive, but not in equivalent amount as the pain experienced when the performance is negative.

  Table 3.1 Probability of success at different scales

  Scale Probability

  1 year 93%

  1 quarter 77%

  1 month 67%

  1 day 54%

  1 hour 51.3%

  1 minute 50.17%

  1 second 50.02%

  At the end of every day the dentist will be emotionally drained. A minute-by-minute examination of his performance means that each day (assuming eight hours per day) he will have 241 pleasurable minutes against 239 unpleasurable ones. These amount to 60,688 and 60,271, respectively, per year. Now realize that if the unpleasurable minute is worse in reverse pleasure than the pleasurable minute is in pleasure terms, then the dentist incurs a large deficit when examining his performance at a high frequency.

  Consider the situation where the dentist examines his portfolio only upon receiving the monthly account from the brokerage house. As 67% of his months will be positive, he incurs only four pangs of pain per annum and eight uplifting experiences. This is the same dentist following the same strategy. Now consider the dentist looking at his performance only every year. Over the next 20 years that he is expected to live, he will experience 19 pleasant surprises for every unpleasant one!

  This scaling property of randomness is generally misundderstood, even by professionals. I have seen Ph.D.s argue over a performance observed in a narrow time scale (meaningless by any standard). Before additional dumping on the journalist, more observations seem in order.

  Viewing it from another angle, if we take the ratio of noise to what we call nonnoise (i.e., left column/right column), which we have the privilege here of examining quantitatively, then we have the following. Over one year we observe roughly 0.7 parts noise for every one part performance. Over one month, we observe roughly 2.32 parts noise for every one part performance. Over one hour, 30 parts noise for every one part performance, and over one second, 1,796 parts noise for every one part performance.

  A few conclusions:

  1. Over a short time increment, one observes the variability of the portfolio, not the returns. In other words, one sees the variance, little else. I always remind myself that what one observes is at best a combination of variance and returns, not just returns (but my emotions do not care about what I tell myself).

  2. Our emotions are not designed to understand the point. The dentist did better when he dealt with monthly statements rather than more frequent ones. Perhaps it would be even better for him if he limited himself to yearly statements. (If you think that you can control your emotions, think that some people also believe that they can control their heartbeat or hair growth.)

  3. When I see an investor monitoring his portfolio with live prices on his cellular telephone or his handheld, I smile and smile.

  Finally, I reckon that I am not immune to such an emotional defect. But I deal with it by having no access to information, except in rare circumstances. Again, I prefer to read poetry. If an event is important enough, it will find its way to my ears. I will return to this point in time.

  The same methodology can explain why the news (the high scale) is full of noise and why history (the low scale) is largely stripped of it (though fraught with interpretation problems). This explains why I prefer not to read the newspaper (outside of the obituary), why I never chitchat about markets, and, when in a trading room, I frequent the mathematicians and the secretaries, not the traders. It explains why it is better to read The New Yorker on Mondays than The Wall Street Journal every morning (from the standpoint of frequency, aside from the massive gap in intellectual class between the two publications).

  Finally, this explains why people who look too closely at randomness burn out, their emotions drained by the series of pangs they experience. Regardless of what people claim, a negative pang is not offset by a positive one (some psychologists estimate the negative effect for an average loss to be up to 2.5 the magnitude of a positive one); it will lead to an emotional deficit.

  Now that you know that the high-frequency dentist has more exposure to both stress and positive pangs, and that these do not cancel out, consider that people in lab coats have examined some scary properties of this type of negative pangs on the neural system (the usual expected effect: high blood pressure; the less expected: chronic stress leads to memory loss, lessening of brain plasticity, and brain damage). To my knowledge there are no studies investigating the exact properties of trader’s burnout, but a daily exposure to such high degrees of randomness without much control will have physiological effects on humans (nobody studied the effect of such exposure on the risk of cancer). What economists did not understand for a long time about positive and negative kicks is that both their biology and their intensity are different. Consider that they are mediated in different parts of the brain—and that the degree of rationality in decisions made subsequent to a gain is extremely different from the one after a loss.

  Note also that the implication that wealth does not count so much into one’s well-being as the route one uses to get to it.

  Some so-called wise and rational persons often blame me for “ignoring” possible valuable information in the daily newspaper and refusing to discount the details of the noise as “short-term events.” Some of my employers have bl
amed me for living on a different planet.

  My problem is that I am not rational and I am extremely prone to drown in randomness and to incur emotional torture. I am aware of my need to ruminate on park benches and in cafés away from information, but I can only do so if I am somewhat deprived of it. My sole advantage in life is that I know some of my weaknesses, mostly that I am incapable of taming my emotions facing news and incapable of seeing a performance with a clear head. Silence is far better. More on that in Part III.

  Four

  •

  RANDOMNESS, NONSENSE, AND THE SCIENTIFIC INTELLECTUAL

  On extending the Monte Carlo generator to produce artificial thinking and compare it with rigorous nonrandom constructs. The science wars enter the business world. Why the aesthete in me loves to be fooled by randomness.

  RANDOMNESS AND THE VERB

  Our Monte Carlo engine can take us into more literary territory. Increasingly, a distinction is being made between the scientific intellectual and the literary intellectual—culminating with what is called the “science wars,” plotting factions of literate nonscientists against no less literate scientists. The distinction between the two approaches originated in Vienna in the 1930s, with a collection of physicists who decided that the large gains in science were becoming significant enough to make claims on the field known to belong to the humanities. In their view, literary thinking could conceal plenty of well-sounding non-sense. They wanted to strip thinking from rhetoric (except in literature and poetry where it properly belonged).

  The way they introduced rigor into intellectual life is by declaring that a statement could fall only into two categories: deductive, like “2 +2 =4,” i.e., incontrovertibly flowing from a precisely defined axiomatic framework (here the rules of arithmetic), or inductive, i.e., verifiable in some manner (experience, statistics, etc.), like “it rains in Spain” or “New Yorkers are generally rude.” Anything else was plain unadulterated hogwash (music could be a far better replacement to metaphysics). Needless to say that inductive statements may turn out to be difficult, even impossible, to verify, as we will see with the black swan problem—and empiricism can be worse than any other form of hogwash when it gives someone confidence (it will take me a few chapters to drill the point). However, it was a good start to make intellectuals responsible for providing some form of evidence for their statements. This Vienna Circle was at the origin of the development of the ideas of Popper, Wittgenstein (in his later phase), Carnap, and flocks of others. Whatever merit their original ideas may have, the impact on both philosophy and the practice of science has been significant. Some of their impact on nonphilosophical intellectual life is starting to develop, albeit considerably more slowly.

  One conceivable way to discriminate between a scientific intellectual and a literary intellectual is by considering that a scientific intellectual can usually recognize the writing of another but that the literary intellectual would not be able to tell the difference between lines jotted down by a scientist and those by a glib nonscientist. This is even more apparent when the literary intellectual starts using scientific buzzwords, like “uncertainty principle,” “Gödel’s theorem,” “parallel universe,” or “relativity,” either out of context or, as often, in exact opposition to the scientific meaning. I suggest reading the hilarious Fashionable Nonsense by Alan Sokal for an illustration of such practice (I was laughing so loudly and so frequently while reading it on a plane that other passengers kept whispering things about me). By dumping the kitchen sink of scientific references in a paper, one can make another literary intellectual believe that one’s material has the stamp of science. Clearly, to a scientist, science lies in the rigor of the inference, not in random references to such grandiose concepts as general relativity or quantum indeterminacy. Such rigor can be spelled out in plain English. Science is method and rigor; it can be identified in the simplest of prose writing. For instance, what struck me while reading Richard Dawkins’ Selfish Gene is that, although the text does not exhibit a single equation, it seems as if it were translated from the language of mathematics. Yet it is artistic prose.

  Reverse Turing Test

  Randomness can be of considerable help with the matter. For there is another, far more entertaining way to make the distinction between the babbler and the thinker. You can sometimes replicate something that can be mistaken for a literary discourse with a Monte Carlo generator but it is not possible randomly to construct a scientific one. Rhetoric can be constructed randomly, but not genuine scientific knowledge. This is the application of Turing’s test of artificial intelligence, except in reverse. What is the Turing test? The brilliant British mathematician, eccentric, and computer pioneer Alan Turing came up with the following test: A computer can be said to be intelligent if it can (on average) fool a human into mistaking it for another human. The converse should be true. A human can be said to be unintelligent if we can replicate his speech by a computer, which we know is unintelligent, and fool a human into believing that it was written by a human. Can one produce a piece of work that can be largely mistaken for Derrida entirely randomly?

  The answer seems to be yes. Aside from the hoax by Alan Sokal (the same of the hilarious book a few lines ago), who managed to produce nonsense and get it published by some prominent journal, there are Monte Carlo generators designed to structure such texts and write entire papers. Fed with “postmodernist” texts, they can randomize phrases under a method called recursive grammar, and produce grammatically sound but entirely meaningless sentences that sound like Jacques Derrida, Camille Paglia, and such a crowd. Owing to the fuzziness of his thought, the literary intellectual can be fooled by randomness.

  At the Monash University program in Australia featuring the Dada Engine built by Andrew C. Bulhak, I toyed with the engine and generated a few papers containing the following sentences:

  However, the main theme of the works of Rushdie is not theory, as the dialectic paradigm of reality suggests, but pretheory. The premise of the neosemanticist paradigm of discourse implies that sexual identity, ironically, has significance.

  Many narratives concerning the role of the writer as observer may be revealed. It could be said that if cultural narrative holds, we have to choose between the dialectic paradigm of narrative and neoconceptual Marxism. Sartre’s analysis of cultural narrative holds that society, paradoxically, has objective value.

  Thus, the premise of the neodialectic paradigm of expression implies that consciousness may be used to reinforce hierarchy, but only if reality is distinct from consciousness; if that is not the case, we can assume that language has intrinsic meaning.

  Some business speeches belong to this category in their own right, except that they are less elegant and draw on a different type of vocabulary than the literary ones. We can randomly construct a speech imitating that of your chief executive officer to ensure whether what he is saying has value, or if it is merely dressed-up nonsense from someone who was lucky to be put there. How? You select randomly five phrases below, then connect them by adding the minimum required to construct a grammatically sound speech.

  We look after our customer’s interests / the road ahead / our assets are our people / creation of shareholder value / our vision / our expertise lies in / we provide interactive solutions / we position ourselves in this market / how to serve our customers better / short-term pain for long-term gain / we will be rewarded in the long run / we play from our strength and improve our weaknesses / courage and determination will prevail / we are committed to innovation and technology / a happy employee is a productive employee / commitment to excellence / strategic plan / our work ethics.

  If this bears too close a resemblance to the speech you just heard from the boss of your company, then I suggest looking for a new job.

  The Father of All Pseudothinkers

  It is hard to resist discussion of artificial history without a comment on the father of all pseudothinkers, Hegel. Hegel writes a jargon that is meaningless outside of a chic
Left Bank Parisian café or the humanities department of some university extremely well insulated from the real world. I suggest this passage from the German “philosopher” (this passage was detected, translated, and reviled by Karl Popper):

  Sound is the change in the specific condition of segregation of the material parts, and in the negation of this condition; merely an abstract or an ideal ideality, as it were, of that specification. But this change, accordingly, is itself immediately the negation of the material specific subsistence; which is, therefore, real ideality of specific gravity and cohesion, i.e.—heat. The heating up of sounding bodies, just as of beaten and or rubbed ones, is the appearance of heat, originating conceptually together with sound.

  Even a Monte Carlo engine could not sound as random as the great philosophical master thinker (it would take plenty of sample runs to get the mixture of “heat” and “sound.” People call that philosophy and frequently finance it with taxpayer subsidies! Now consider that Hegelian thinking is generally linked to a “scientific” approach to history; it has produced such results as Marxist regimes and even a branch called “neo-Hegelian” thinking. These “thinkers” should be given an undergraduate-level class on statistical sampling theory prior to their release into the open world.

  MONTE CARLO POETRY

  There are instances where I like to be fooled by randomness. My allergy to nonsense and verbiage dissipates when it comes to art and poetry. On the one hand, I try to define myself and behave officially as a no-nonsense hyperrealist ferreting out the role of chance; on the other, I have no qualms indulging in all manner of personal superstitions. Where do I draw the line? The answer is aesthetics. Some aesthetic forms appeal to something in our biology, whether or not they originate in random associations or plain hallucination. Something in our human genes is deeply moved by the fuzziness and ambiguity of language; then why fight it?