One of my colleagues, a man with prestigious degrees, grew to believe in such a virtual world to the point of losing all sense of reality. Whether the modicum of common sense left in him might have rapidly vanished under the mounds of simulations, or whether he might have had none to engage in such pursuit, I cannot tell. By closely watching him I learned that what natural skepticism he may have had vanished under the weight of data—for he was extremely skeptical, but in the wrong area. Ah, Hume!
A More Unsettling Extension
Historically, medicine has operated by trial and error—in other words, statistically. We know by now that there can be entirely fortuitous connections between symptoms and treatment, and that some medications succeed in medical trials for mere random reasons. I cannot claim expertise in medicine, but have been a steady reader of a segment of the medical literature over the past half decade, long enough to be concerned with the standards, as we will see in the next chapter. Medical researchers are rarely statisticians; statisticians are rarely medical researchers. Many medical researchers are not even remotely aware of this data mining bias. True, it may only play a small role, but it is certainly present. One recent medical study links cigarette smoking to a reduction in breast cancer, thus conflicting with all previous studies. Logic would indicate that the result may be suspicious, the result of mere coincidence.
The Earnings Season: Fooled by the Results
Wall Street analysts, in general, are trained to find the accounting tricks that companies use to hide their earnings. They tend to (occasionally) beat the companies at that game. But they are neither trained to reflect nor to deal with randomness (nor to understand the limitations of their methods by introspecting—stock analysts have both a worse record and higher idea of their past performance than weather forecasters). When a company shows an increase in earnings once, it draws no immediate attention. Twice, and the name starts showing up on computer screens. Three times, and the company will merit some buy recommendation.
Just as with the track record problem, consider a cohort of 10,000 companies that are assumed on average to barely return the risk-free rate (i.e., Treasury bonds). They engage in all forms of volatile business. At the end of the first year, we will have 5,000 “star” companies showing an increase in profits (assuming no inflation), and 5,000 “dogs.” After three years, we will have 1,250 “stars.” The stock review committee at the investment house will give your broker their names as “strong buys.” He will leave a voice message that he has a hot recommendation that necessitates immediate action. You will be e-mailed a long list of names. You will buy one or two of them. Meanwhile, the manager in charge of your 401(k) retirement plan will be acquiring the entire list.
We can apply the reasoning to the selection of investment categories—as if they were the managers in the example above. Assume you are standing in 1900 with hundreds of investments to look at. There are the stock markets of Argentina, Imperial Russia, the United Kingdom, Unified Germany, and plenty of others to consider. A rational person would have bought not just the emerging country of the United States, but those of Russia and Argentina as well. The rest of the story is well-known; while many of the stock markets like those of the United Kingdom and the United States fared extremely well, the investor in Imperial Russia would have no better than medium-quality wallpaper in his hands. The countries that fared well are not a large segment of the initial cohort; randomness would be expected to allow a few investment classes to fare extremely well. I wonder if those “experts” who make foolish (and self-serving) statements like “markets will always go up in any twenty-year period” are aware of this problem.
COMPARATIVE LUCK
A far more acute problem relates to the outperformance, or the comparison, between two or more persons or entities. While we are certainly fooled by randomness when it comes to a single times series, the foolishness is compounded when it comes to the comparison between, say, two people, or a person and a benchmark. Why? Because both are random. Let us do the following simple thought experiment. Take two individuals, say, a person and his brother-in-law, launched through life. Assume equal odds for each of good and bad luck. Outcomes: lucky-lucky (no difference between them), unlucky-unlucky (again, no difference), lucky-unlucky (a large difference between them), unlucky-lucky (again, a large difference).
I recently attended for the first time a conference of investment managers where I sat listening to a very dull presenter comparing traders. His profession is to select fund managers and package them together for investors, something called “funds of funds” and I was listening to him as he was pouring out numbers on the screen. The first revelation was that I suddenly recognized the speaker, a former colleague biologically transformed by the passage of time. He used to be crisp, energetic, and nice; he became dull, portly, and inordinately comfortable with success. (He was not rich when I knew him—can people react to money in different ways? Do some take themselves seriously while others do not?) The second revelation was that while I suspected that he was fooled by randomness, the extent had to be far greater than one could imagine, particularly with the survivorship bias. A back of the envelope calculation showed that at least 97% of what he was discussing was just noise. The fact that he was comparing performances made the matter far worse.
Cancer Cures
When I return home from an Asian or European trip, my jet lag often causes me to rise at a very early hour. Occasionally, though very rarely, I switch on the TV set searching for market information. What strikes me in these morning explorations is the abundance of claims by the alternative medicine vendors of the curing power of their products. These no doubt are caused by the lower advertising rates at that time. To prove their claim, they present the convincing testimonial of someone who was cured thanks to their methods. For instance, I once saw a former throat cancer patient explaining how he was saved by a combination of vitamins for sale for the exceptionally low price of $14.95—in all likelihood he was sincere (although of course compensated for his account, perhaps with a lifetime supply of such medicine). In spite of our advances, people still believe in the existence of links between disease and cure based on such information, and there is no scientific evidence that can convince them more potently than a sincere and emotional testimonial. Such testimonial does not always come from the regular guy; statements by Nobel Prize winners (in the wrong discipline) could easily suffice. Linus Pauling, a Nobel Prize winner in chemistry, was said to believe in vitamin C’s medicinal properties, himself ingesting massive daily doses. With his bully pulpit, he contributed to the common belief in vitamin C’s curative properties. Many medical studies, unable to replicate Pauling’s claims, fell on deaf ears as it was difficult to undo the testimonial by a “Nobel Prize winner,” even if he was not qualified to discuss matters related to medicine.
Many of these claims have been harmless outside of the financial profits for these charlatans—but many cancer patients may have replaced the more scientifically investigated therapies, in favor of these methods, and died as a result of their neglecting more orthodox cures (again, the nonscientific methods are gathered under what is called “alternative medicine,” that is, unproven therapies, and the medical community has difficulties convincing the press that there is only one medicine and that alternative medicine is not medicine). The reader might wonder about my claims that the user of these products could be sincere, without it meaning that he was cured by the illusory treatment. The reason is something called “spontaneous remission,” in which a very small minority of cancer patients, for reasons that remain entirely speculative, wipe out cancer cells and recover “miraculously.” Some switch causes the patient’s immune system to eradicate all cancer cells from the body. These people would have been equally cured by drinking a glass of Vermont spring water or chewing on dried beef as they were by taking these beautifully wrapped pills. Finally, these spontaneous remissions might not be so spontaneous; they might, at the bottom, have a cause that we are not yet
sophisticated enough to detect.
The late astronomer Carl Sagan, a devoted promoter of scientific thinking and an obsessive enemy of nonscience, examined the cures from cancer that resulted from a visit to Lourdes in France, where people were healed by simple contact with the holy waters, and found out the interesting fact that, of the total cancer patients who visited the place, the cure rate was, if anything, lower than the statistical one for spontaneous remissions. It was lower than the average for those who did not go to Lourdes! Should a statistician infer here that cancer patients’ odds of surviving deteriorates after a visit to Lourdes?
Professor Pearson Goes to Monte Carlo (Literally):
Randomness Does Not Look Random!
At the beginning of the twentieth century, as we were starting to develop techniques to deal with the notion of random outcomes, several methods were designed to detect anomalies. Professor Karl Pearson (father of Egon Pearson of Neyman-Pearson fame, familiar to every person who sat in a statistics 101 class) devised the first test of nonrandomness (it was in reality a test of deviation from normality, which, for all intents and purposes, was the same thing). He examined millions of runs of what was called a Monte Carlo (the old name for a roulette wheel) during the month of July 1902. He discovered that, with a high degree of statistical significance (with an error of less than one to a billion), the runs were not purely random. What! The roulette wheel was not random! Professor Pearson was greatly surprised at the discovery. But this result in itself tells us nothing; we know that there is no such thing as a pure random draw, for the outcome of the draw depends on the quality of the equipment. With enough minutiae one would be able to uncover the nonrandomness somewhere (e.g., the wheel itself may not have been perfectly balanced or perhaps the spinning ball was not completely spherical). Philosophers of statistics call this the reference case problem to explain that there is no true attainable randomness in practice, only in theory. Besides, a manager would question whether such nonrandomness can lead to any meaningful, profitable rules. If I need to gamble $1 on 10,000 runs and expect to make $1 for my efforts, then I would do much better in the part-time employment of a janitorial agency.
But the result bears another suspicious element. Of more practical relevance here is the following severe problem about nonrandomness. Even the fathers of statistical science forgot that a random series of runs need not exhibit a pattern to look random; as a matter of fact, data that is perfectly patternless would be extremely suspicious and appear to be man-made. A single random run is bound to exhibit some pattern—if one looks hard enough. Note that Professor Pearson was among the first scholars who were interested in creating artificial random data generators, tables one could use as inputs for various scientific and engineering simulations (the precursors of our Monte Carlo simulator). The problem is that they did not want these tables to exhibit any form of regularity. Yet real randomness does not look random!
I would further illustrate the point with the study of a phenomenon well-known as cancer clusters. Consider a square with 16 random darts hitting it with equal probability of being at any place in the square. If we divide the square into 16 smaller squares, it is expected that each smaller square will contain one dart on average—but only on average. There is a very small probability of having exactly 16 darts in 16 different squares. The average grid will have more than one dart in a few squares, and no dart at all in many squares. It will be an exceptionally rare incident that no (cancer) cluster would show on the grid. Now, transpose our grid with the darts in it to overlay a map of any region. Some newspaper will declare that one of the areas (the one with more than the average of darts) harbors radiation that causes cancer, prompting lawyers to start soliciting the patients.
The Dog That Did Not Bark: On Biases
in Scientific Knowledge
By the same argument, science is marred by a pernicious survivorship bias, affecting the way research gets published. In a way that is similar to journalism, research that yields no result does not make it to print. That may seem sensible, as newspapers do not have to have a screaming headline saying that nothing new is taking place (though the Bible was smart enough to declare ein chadash tachat hashemesh—“nothing new under the sun,” providing the information that things just do recur). The problem is that a finding of absence and an absence of findings get mixed together. There may be great information in the fact that nothing took place. As Sherlock Holmes noted in the Silver Blaze case—the curious thing was that the dog did not bark. More problematic, there are plenty of scientific results that are left out of publications because they are not statistically significant, but nevertheless provide information.
I HAVE NO CONCLUSION
I am frequently asked the question “When is it truly not luck?” There are professions in randomness for which performance is low in luck, like casinos, which manage to tame randomness. In finance? Perhaps. All traders are not speculative traders: There exists a segment called market makers whose job is to derive, like bookmakers, or even like store owners, an income against a transaction. If they speculate, their dependence on the risks of such speculation remains too small compared to their overall volume. They buy at a price and sell to the public at a more favorable one, performing large numbers of transactions. Such income provides them some insulation from randomness. Such category includes floor traders on the exchanges, bank traders who “trade against order flow,” money changers in the souks of the Levant. The skills involved are sometimes rare to find: Fast thinking, alertness, a high level of energy, an ability to guess from the voice of the seller her level of nervousness; those who have them make a long career (that is, perhaps a decade).They never make it big, as their income is constrained by the number of customers, but they do well probabilistically. They are, in a way, the dentists of the profession.
Outside of this very specialized bookmaker-style profession, to be honest, I am unable to answer the question of who’s lucky or unlucky. I can tell that person A seems less lucky than person B, but the confidence in such knowledge can be so weak as to be meaningless. I prefer to remain a skeptic. People frequently misinterpret my opinion. I never said that every rich man is an idiot and every unsuccessful person unlucky, only that in absence of much additional information it is preferable to reserve one’s judgment. It is safer.
Ten
•
LOSER TAKES ALL—ON THE NONLINEARITIES OF LIFE
The nonlinear viciousness of life. Moving to Bel Air and acquiring the vices of the rich and famous. Why Microsoft’s Bill Gates may not be the best in his business (but please do not inform him of such a fact). Depriving donkeys of food.
Next I put the platitude life is unfair under some examination, but from a new angle. The twist: Life is unfair in a nonlinear way. This chapter is about how a small advantage in life can translate into a highly disproportionate payoff, or, more viciously, how no advantage at all, but a very, very small help from randomness, can lead to a bonanza.
THE SANDPILE EFFECT
First we define nonlinearity. There are many ways to present it, but one of the most popular ones in science is what is called the sand-pile effect, which I can illustrate as follows. I am currently sitting on a beach in Copacabana, in Rio de Janeiro, attempting to do nothing strenuous, away from anything to read and write (unsuccessfully, of course, as I am mentally writing these lines). I am playing with plastic beach toys borrowed from a child, trying to build an edifice—modestly but doggedly attempting to emulate the Tower of Babel. I continuously add sand to the top, slowly raising the entire structure. My Babylonian relatives thought they could thus reach the heavens. I have more humble designs—to test how high I can go before it topples. I keep adding sand, testing to see how the structure will ultimately collapse. Unused to seeing adults build sandcastles, a child looks at me with amazement.
In time—and much to the onlooking child’s delight—my castle inevitably topples to rejoin the rest of the sand on the beach. It could be said that the l
ast grain of sand is responsible for the destruction of the entire structure. What we are witnessing here is a nonlinear effect resulting from a linear force exerted on an object. A very small additional input, here the grain of sand, caused a disproportionate result, namely the destruction of my starter Tower of Babel. Popular wisdom has integrated many such phenomena, as witnessed by such expressions as “the straw that broke the camel’s back” or “the drop that caused the water to spill.”
These nonlinear dynamics have a bookstore name, “chaos theory,” which is a misnomer because it has nothing to do with chaos. Chaos theory concerns itself primarily with functions in which a small input can lead to a disproportionate response. Population models, for instance, can lead to a path of explosive growth, or extinction of a species, depending on a very small difference in the population at a starting point in time. Another popular scientific analogy is the weather, where it has been shown that a simple butterfly fluttering its wings in India can cause a hurricane in New York. But the classics have their share to offer as well: Pascal (he of the wager in Chapter 7) said that if Cleopatra’s nose had been slightly shorter, the world’s fate would have changed. Cleopatra had comely features dominated by a thin and elongated nose that made Julius Caesar and his successor, Marc Antony, fall for her (here the intellectual snob in me cannot resist dissenting against conventional wisdom; Plutarch claimed that it was Cleopatra’s skills in conversation, rather than her good looks, that caused the maddening infatuation of the shakers and movers of her day; I truly believe it).
Enter Randomness
Things can become more interesting when randomness enters the game. Imagine a waiting room full of actors queuing for an audition. The number of actors who will win is clearly small, and they are the ones generally observed by the public as representative of the profession, as we saw in our discussion on survivorship bias. The winners would move into Bel Air, feel pressure to acquire some basic training in the consumption of luxury goods, and, perhaps owing to the dissolute and unrhythmic lifestyle, flirt with substance abuse. As to the others (the great majority), we can imagine their fate; a lifetime of serving foamed caffe latte at the neighboring Starbucks, fighting the biological clock between auditions.
Fooled by Randomness Page 19
