How Not to Be Wrong : The Power of Mathematical Thinking (9780698163843)

by Ellenberg, Jordan

  UNCORRELATED DOESN’T MEAN UNRELATED

  When two variables are correlated, we’ve seen that they’re somehow related to each other. So what if they’re not? Does that mean the variables are completely unrelated, neither one affecting the other? Far from it. Galton’s notion of correlation is limited in a very important way: it detects linear relations between variables, where an increase in one variable tends to coincide with a proportionally large increase (or decrease) in the other. But just as not all curves are lines, not all relationships are linear relationships.

  Take this one:

  You’re looking at a picture I made of a political survey taken by Public Policy Polling on December 15, 2011; there are one thousand dots, each representing a voter who responded to a twenty-three-question poll. The position of a point on the left-right axis represents, well, left and right: people who said they supported President Obama, approved of the Democratic Party, and opposed the Tea Party tend to be on the left-hand side, while those who favored the GOP, disliked Harry Reid, and believed there is a “War on Christmas” are over on the right. The vertical axis stands roughly for “informedness”—voters toward the bottom of the graph tended to answer “don’t know” to more insidery questions like “Do you approve or disapprove of the job [Senate Minority Leader] Mitch McConnell is doing?” and to express little or no excitement about the 2012 presidential election.

  One can check that the variables measured by two axes are uncorrelated,* just as eyeballing the graph suggests; it doesn’t look like the points tend to be farther left or right as you move up the page. But that doesn’t mean that the two variables aren’t related to each other. In fact, the relation is quite clear from the picture. The plot is “heart-shaped,” with a lobe on either side and a point at the bottom. As the voters get more informed, they don’t get more Democratic or more Republican, but they do get more polarized: lefties go farther left, right-wingers get farther right, and the sparsely populated space in the middle gets even sparser. In the lower half of the graph, the less-informed voters tend to adopt a more centrist stance. The graph reflects a sobering social fact, which is by now commonplace in the political science literature. Undecided voters, by and large, aren’t undecided because they’re carefully weighing the merits of each candidate, unprejudiced by political dogma. They’re undecided because they’re barely paying attention.

  A mathematical tool, like any scientific instrument, detects some kinds of phenomena but not others; a correlation computation can’t see the heart-shapedness (cardiomorphism?) of this scatterplot any more than your camera can detect gamma rays. Keep this in mind when you’re told that two phenomena in nature or society were found to be uncorrelated. It doesn’t mean there’s no relationship, only that there’s no relationship of the sort that correlation is designed to detect.

  SIXTEEN

  DOES LUNG CANCER MAKE YOU SMOKE CIGARETTES?

  And what about when two variables are correlated? What does that really mean?

  To make this simple, let’s start with the simplest kind of variable, a binary variable with only two possible values. Oftentimes a binary variable is the answer to a yes-or-no question: “Are you married?” “Do you smoke?” “Are you now, or have you ever been, a member of the Communist Party?”

  When you’re comparing two binary variables, correlation takes on a particularly simple form. To say that marital status and smoking status are negatively correlated, for example, is simply to say that married people are less likely than the average person to smoke. Or, to put it another way, smokers are less likely than the average person to be married. It’s worth taking a moment to persuade yourself that those two things are indeed the same! The first statement can be written as an inequality

  married smokers / all married people < all smokers / all people

  and the second as

  married smokers / all smokers < all married people / all people

  If you multiply both sides of each inequality by the common denominator (all people) × (all smokers) you can see that the two statements are different ways of saying the same thing:

  (married smokers) × (all people) < (all smokers) × (all married people)

  In the same way, if smoking and marriage were positively correlated, it would mean that married people were more likely than average to smoke and smokers more likely than average to be married.

  One problem presents itself immediately. Surely the chance is very small that the proportion of smokers among married people is exactly the same as the proportion of smokers in the whole population. So, absent a crazy coincidence, marriage and smoking will be correlated, either positively or negatively. And so will sexual orientation and smoking, U.S. citizenship and smoking, first-initial-in-the-last-half-of-the-alphabet and smoking, and so on. Everything will be correlated with smoking, in one direction or the other. It’s the same issue we encountered in chapter 7; the null hypothesis, strictly speaking, is just about always false.

  To throw up our hands and say, “Everything is correlated with everything else!” would be fairly uninformative. So we don’t report on all of these correlations. When you read a report that one thing is correlated with another, you’re implicitly being told that the correlation is “strong enough” to be worth reporting—usually because it passed a test of statistical significance. As we’ve seen, the statistical significance test brings with it many dangers, but it is, at least, a signal that makes a statistician sit up, take notice, and say, “Something must be going on.”

  But what? Here we come to the really sticky part. Marriage is negatively correlated with smoking; that’s a fact. A typical way to express that fact is to say

  “If you’re a smoker, you’re less likely to be married.”

  But one small change makes the meaning very different:

  “If you were a smoker, you’d be less likely to be married.”

  It seems strange that changing the sentence from the indicative to the subjunctive mood can change what it says so drastically. But the first sentence is merely a statement about what is the case. The second concerns a much more delicate question: What would be the case if we changed something about the world? The first sentence expresses a correlation; the second suggests a causation. As we’ve already mentioned, the two are not the same. That smokers are less frequently married than others doesn’t mean that quitting smoking will summon up your future spouse. The mathematical account of correlation has been pretty much fixed in place since the work of Galton and Pearson a century ago. Putting the idea of causation on a firm mathematical footing has been much more elusive.*

  There’s something slippery about our understanding of correlation and causation. Your intuition tends to grasp it quite firmly in some circumstances but lose its grip in others. When we say that HDL is correlated with a lower risk of heart attack, we’re making a factual statement: “If you’ve got a higher level of HDL cholesterol, you’re less likely to have a heart attack.” It’s hard not to think that the HDL is doing something—that the molecules in question are literally causing your cardiovascular health to improve, say, by “scrubbing” lipidic cruft off your arterial walls. If that were so—if the mere presence of a lot of HDL were working to your benefit—then it would be reasonable to expect any HDL-increasing intervention to reduce your risk of heart attack.

  But it might be that HDL and heart attack are correlated for a different reason; say, that some other factor, one we haven’t measured, tends both to increase HDL and decrease the risk of cardiovascular events. If that’s the case, an HDL-increasing drug might or might not prevent heart attack; if the drug affects HDL by way of the mystery factor, it’ll probably help your heart, but if it boosts HDL in some other way, all bets are off. That’s the situation with Tim and Sara. Their financial success is correlated, but it’s not because Tim’s fund is causing Sara’s to take off, or the reverse. It’s because there’s a mystery factor, the Honda stock, th
at affects both Tim and Sara. Clinical researchers call this the surrogate endpoint problem. It’s time consuming and expensive to check whether a drug improves average life span, because in order to record someone’s life span you have to wait for them to die. HDL level is the surrogate endpoint, the easy-to-check biomarker that’s supposed to stand in for “long life with no heart attack.” But the correlation between HDL and absence of heart attack might not indicate any causal link.

  Teasing apart correlations that come from causal relationships from those that don’t is a maddeningly hard problem, even in cases you might think of as obvious, like the relation between smoking and lung cancer. At the turn of the twentieth century, lung cancer was an extremely rare disease. But by 1947, the disease accounted for nearly a fifth of cancer deaths among British men, killing fifteen times as many people as it had a few decades earlier. At first, many researchers thought that lung cancer was simply being diagnosed more effectively than before, but it soon became clear that the increase in cases was too big and too fast to be accounted for by any such effect. Lung cancer really was on the rise. But no one was sure what to blame. Maybe it was smoke from factories, maybe increased levels of car exhaust, or maybe some substance not even thought of as a pollutant. Or maybe it was cigarette smoking, whose popularity had exploded during the same period.

  By the early 1950s, large studies in England and America had shown a powerful association between cigarette smoking and lung cancer. Among nonsmokers, lung cancer was still a rare disease, but for smokers, the risk was spectacularly higher. A famous paper of Doll and Hill from 1950 found that among 649 male lung cancer patients in twenty London hospitals, only two were nonsmokers. That’s not as impressive as it sounds by modern standards; in midcentury London, smoking was an extremely popular habit, and nonsmokers were much rarer than they are now. Even so, in a population of 649 male patients admitted for complaints other than lung cancer, twenty-seven were nonsmokers, a lot more than two. What’s more, the association got stronger as smoking got heavier. Of the lung cancer patients, 168 went through more than twenty-five cigarettes a day, while only eighty-four men hospitalized for some other condition smoked that much.

  Doll and Hill’s data showed that lung cancer and smoking were correlated; their relation was not one of strict determination (some heavy smokers don’t get lung cancer, while some nonsmokers do), but neither were the two phenomena independent. Their relation lay in that fuzzy, intermediate zone that Galton and Pearson had been the first to map.

  The mere assertion of correlation is very different from an explanation. Doll and Hill’s study doesn’t show that smoking causes cancer; as they write, “The association would occur if carcinoma of the lung caused people to smoke or if both attributes were end-effects of a common cause.” That lung cancer causes smoking, as they point out, is not very reasonable; a tumor can’t go back in time and give someone a pack-a-day habit. But the problem of the common cause is more troubling.

  Our old friend R. A. Fisher, the founding hero of modern statistics, was a vigorous skeptic of the tobacco-cancer link on exactly those grounds. Fisher was the natural intellectual heir to Galton and Pearson; in fact, he succeeded Pearson in 1933 as the Galton Chair of Eugenics at University College, London. (In deference to modern sensibilities, the position is now called the Galton Chair of Genetics.)

  Fisher felt it was premature even to rule out the cancer-causes-smoking theory:

  Is it possible then, that lung cancer—that is to say, the pre-cancerous condition which must exist and is known to exist for years in those who are going to show overt lung cancer—is one of the causes of smoking cigarettes? I don’t think it can be excluded. I don’t think we know enough to say that it is such a cause. But the pre-cancerous condition is one involving a certain amount of slight chronic inflammation. The causes of smoking cigarettes may be studied among your friends, to some extent, and I think you will agree that a slight cause of irritation—a slight disappointment, an unexpected delay, some sort of a mild rebuff, a frustration—are commonly accompanied by pulling out a cigarette and getting a little compensation for life’s minor ills in that way. And so, anyone suffering from a chronic inflammation in part of the body (something that does not give rise to conscious pain) is not unlikely to be associated with smoking more frequently, or smoking rather than not smoking. It is the kind of comfort that might be a real solace to anyone in the fifteen years of approaching lung cancer. And to take the poor chap’s cigarettes away from him would be rather like taking away his white stick from a blind man. It would make an already unhappy person a little more unhappy than he need be.

  One sees here both a brilliant and rigorous statistician’s demand that all possibilities receive fair consideration, and a lifelong smoker’s affection for his habit. (Some have also seen the influence of Fisher’s work as a consultant to the Tobacco Manufacturer’s Standing Committee, a British industry group; in my view, Fisher’s reluctance to assert a causal relationship was consistent with his general statistical approach.) Fisher’s suggestion that the men in Doll and Hill’s sample might have been driven to smoke by precancerous inflammation never caught on, but his argument for a common cause gained more traction. Fisher, true to his academic title, was a devoted eugenicist, who believed that genetic differences determined a healthy portion of our fate and that the better sort of people were in grave danger, in these evolutionarily forgiving times, of being outbred by their natural inferiors. From Fisher’s point of view, it was perfectly natural to imagine that a common genetic factor, as yet unmeasured, was behind both lung cancer and propensity to smoke cigarettes. That might seem rather speculative. But remember, at the time, the generation of lung cancer by smoking rested on equally mysterious grounds. No chemical component of tobacco had yet been shown to produce tumors in the lab.

  There’s an elegant way to test for genetic influence on smoking, by studying twins. Say two twin siblings “match” if either both are smokers or both are not. You might expect matching to be fairly common, since twins typically grow up in the same home, with the same parents, and in the same cultural conditions, and that’s indeed what you see. But identical twins and fraternal twins are subject to these commonalities to exactly the same degree; so if identical twins are more likely to match than fraternal twins, it’s evidence that heritable factors exert some influence on smoking. Fisher presented some small-scale results to that effect, from unpublished studies, and more recent work has borne out his intuition; smoking appears to be subject to at least some heritable effects.

  Which, of course, isn’t to say that those same genes are what give you lung cancer down the road. We know a lot more now about cancer and how tobacco brings it about. That smoking gives you cancer is no longer in serious dispute. And yet it’s hard not to be somewhat sympathetic to Fisher’s let’s-not-be-hasty approach. It’s good to be suspicious of correlations. The epidemiologist Jan Vandenbroucke wrote of Fisher’s articles on tobacco, “To my surprise, I found extremely well-written and cogent papers that might have become textbook classics for their impeccable logic and clear exposition of data and argument if only the authors had been on the right side.”

  Over the course of the 1950s, scientific opinion on the question of lung cancer and smoking steadily converged toward consensus. True, there was still no clear biological mechanism for the generation of tumors by tobacco smoke, and there was still no case for the association between smoking and cancer that didn’t rest on observed correlations. But by 1959, so many such correlations had been seen, and so many possible confounding factors ruled out, that U.S. Surgeon General Leroy E. Burney was willing to assert, “The weight of evidence at present implicates smoking as the principal factor in the increased incidence of lung cancer.” Even then, this stance was not uncontroversial. John Talbott, the editor of the Journal of the American Medical Association, fired back just weeks later in a JAMA editorial: “A number of authorities who have examined the same evidence cited b
y Dr. Burney do not agree with his conclusions. Neither the proponents nor the opponents of the smoking theory have sufficient evidence to warrant the assumption of an all-or-none authoritative position. Until definitive studies are forthcoming, the physician can fulfill his responsibility by watching the situation closely, keeping courant of the facts, and advising his patients on the basis of his appraisal of those facts.” Talbott, like Fisher before him, was accusing Burney and those who agreed with him of being, scientifically speaking, out in front of their skis.

  Just how fierce the dispute remained, even within the scientific establishment, is made clear by the remarkable work of historian of medicine Jon Harkness. His exhaustive archival research has shown that the statement signed by the surgeon general was in fact written by a large group of scientists at the Public Health Service, with Burney himself having little direct involvement. As for Talbott’s response, that too, was ghostwritten—by a rival group of PHS researchers! What looked like a tussle between government officialdom and the medical establishment was in fact a scientific in-fight projected onto a public screen.

  We know how this story ends. Burney’s successor as surgeon general, Luther Terry, convened a blue-ribbon commission on smoking and health in the early 1960s, and in January 1964, to nationwide press coverage, announced their findings in terms that made Burney look timid:

  In view of the continuing and mounting evidence from many sources, it is the judgment of the Committee that cigarette smoking contributes substantially to mortality from certain specific diseases and to the overall death rate. . . . Cigarette smoking is a health hazard of sufficient importance in the United States to warrant appropriate remedial action [boldface from the original report].

  What had changed? By 1964, the association between smoking and cancer had appeared consistently across study after study. Heavier smokers suffered more cancer than lighter smokers, and cancer was most likely at the point of contact between tobacco and human tissue; cigarette smokers got more lung cancer, pipe smokers more lip cancer. Ex-smokers were less prone to cancer than smokers who kept up the habit. All these factors combined to lead the surgeon general’s committee to the conclusion that smoking was not just correlated with lung cancer, but caused lung cancer, and that efforts to reduce tobacco consumption would be likely to lengthen American lives.