by Ben Goldacre
One of the great studies of cherry-picking in the academic literature comes from an article about Linus Pauling, the greatgrandfather of modern nutritionism, and his seminal work on vitamin C and the common cold. In 1993 Paul Knipschild, Professor of Epidemiology at the University of Maastricht, published a chapter in the mighty textbook Systematic Reviews: he had gone to the extraordinary trouble of approaching the literature as it stood when Pauling was working, and subjecting it to the same rigorous systematic review that you would find in a modern paper.
He found that while some trials did suggest that vitamin C had some benefits, Pauling had selectively quoted from the literature to prove his point. Where Pauling had referred to some trials which seriously challenged his theory, it was to dismiss them as methodologically flawed: but as a cold examination showed, so too were papers he quoted favourably in support of his own case.
In Pauling’s defence, his was an era when people knew no better, and he was probably quite unaware of what he was doing: but today cherry-picking is one of the most common dubious practices in alternative therapies, and particularly in nutrition-ism, where it seems to be accepted essentially as normal practice (it is this cherry-picking, in reality, which helps to characterise what alternative therapists conceive of rather grandly as their ‘alternative paradigm’). It happens in mainstream medicine also, but with one crucial difference: there it is recognised as a major problem, and hard work has been done to derive a solution.
That solution is a process called ‘systematic review’. Instead of just mooching around online and picking out your favourite papers to back up your prejudices and help you sell a product, in a systematic review you have an explicit search strategy for seeking out data (openly described in your paper, even including the search terms you used on databases of research papers), you tabulate the characteristics of each study you find, you measure—ideally blind to the results—the methodological quality of each one (to see how much of a ‘fair test’ it is), you compare alternatives, and then finally you give a critical, weighted summary.
This is what the Cochrane Collaboration does on all the healthcare topics that it can find. It even invites people to submit new clinical questions that need an answer. This careful sifting of information has revealed huge gaps in knowledge, it has revealed that ‘best practices’ were sometimes murderously flawed, and simply by sifting methodically through pre-existing data, it has saved more lives than you could possibly imagine. In the nineteenth century, as the public-health doctor Muir Gray has said, we made great advances through the provision of clean, clear water; in the twenty-first century we will make the same advances through clean, clear information. Systematic reviews are one of the great ideas of modern thought. They should be celebrated.
Problematising antioxidants
We have seen the kinds of errors made by those in the nutri-tionism movement as they strive to justify their more obscure and technical claims. What’s more fun is to take our new understanding and apply it to one of the key claims of the nutrition-ism movement, and indeed to a fairly widespread belief in general: the claim that you should eat more antioxidants.
As you now know, there are lots of ways of deciding whether the totality of research evidence for a given claim stacks up, and it’s rare that one single piece of information clinches it. In the case of a claim about food, for example, there are all kinds of different things we might look for: whether it is theoretically plausible, whether it is backed up by what we know from observing diets and health, whether it is supported by ‘intervention trials’ where we give one group one diet and another group a different one, and whether those trials measured real-world outcomes, like ‘death’, or a surrogate outcome, like a blood test, which is only hypothetically related to a disease.
My aim here is by no means to suggest that antioxidants are entirely irrelevant to health. If I had a T–shirt slogan for this whole book it would be: ‘I think you’ll find it’s a bit more complicated than that’. I intend, as they say, to ‘problematise’ the prevailing nutritionist view on antioxidants, which currently lags only about twenty years behind the research evidence.
From an entirely theoretical perspective, the idea that antioxidants are beneficial for health is an attractive one. When I was a medical student—not so long ago—the most popular biochemistry textbook was called Stryer. This enormous book is filled with complex interlocking flow charts of how chemicals—which is what you are made of—move through the body. It shows how different enzymes break down food into its constituent molecular elements, how these are absorbed, how they are reassembled into new larger molecules that your body needs to build muscles, retina, nerves, bone, hair, membrane, mucus, and everything else that you’re made of; how the various forms of fats are broken down, and reassembled into new forms of fat; or how different forms of molecule—sugar, fat, even alcohol—are broken down gradually, step by step, to release energy, and how that energy is transported, and how the incidental products from that process are used, or bolted onto something else to be transported in the blood, and then ditched at the kidneys, or metabolised down into further constituents, or turned into something useful elsewhere, and so on. This is one of the great miracles of life, and it is endlessly, beautifully, intricately fascinating.
Looking at these enormous, overwhelming interlocking webs, it’s hard not to be struck by the versatility of the human body, and how it can perform acts of near alchemy from so many different starting points. It would be very easy to pick one of the elements of these vast interlocking systems and become fixated on the idea that it is uniquely important. Perhaps it appears a lot on the diagram; or perhaps rarely, and seems to serve a uniquely important function in one key place. It would be easy to assume that if there was more of it around, then that function would be performed with greater efficiency.
But, as with all enormous interlocking systems—like societies, for example, or businesses—an intervention in one place can have quite unexpected consequences: there are feedback mechanisms, compensatory mechanisms. Rates of change in one localised area can be limited by quite unexpected factors that are entirely remote from what you are altering, and excesses of one thing in one place can distort the usual pathways and flows, to give counterintuitive results.
The theory underlying the view that antioxidants are good for you is the ‘free radical theory of ageing’. Free radicals are highly chemically reactive, as are many things in the body. Often this reactivity is put to very good use. For example, if you have an infection, and there are some harmful bacteria in your body, then a phagocytic cell from your immune system might come along, identify the bacteria as unwelcome, build a strong wall around as many of them as it can find, and blast them with destructive free radicals. Free radicals are basically like bleach, and this process is a lot like pouring bleach down the toilet. Once again, the human body is cleverer than anybody you know.
But free radicals in the wrong places can damage the desirable components of cells. They can damage the lining of your arteries, and they can damage DNA; and damaged DNA leads to ageing or cancer, and so on. For this reason, it has been suggested that free radicals are responsible for ageing and various diseases. This is a theory, and it may or may not be correct.
Antioxidants are compounds which can—and do—‘mop up’ these free radicals, by reacting with them. If you look at the vast, interlocking flow chart diagrams of how all the molecules in your body are metabolised from one form to the next, you can see that this is happening all over the shop.
The theory that antioxidants are protective is separate to—but builds upon—the free radical theory of disease. If free radicals are dangerous, the argument goes, and antioxidants on the big diagrams are involved in neutralising them, then eating more antioxidants should be good for you, and reverse or slow ageing, and prevent disease.
There are a number of problems with this as a theory. Firsdy, who says free radicals are always bad? If you’re going to reason just from theory, an
d from the diagrams, then you can hook all kinds of things together and make it seem as if you’re talking sense. As I said, free radicals are vital for your body to kill off bacteria in phagocytic immune cells: so should you set yourself up in business and market an antioxidant-free diet for people with bacterial infections?
Secondly, just because antioxidants are involved in doing something good, why should eating more of them necessarily make that process more efficient? I know it makes sense superficially, but so do a lot of things, and that’s what’s really interesting about science (and this story in particular): sometimes the results aren’t quite what you might expect. Perhaps an excess of antioxidants is simply excreted, or turned into something else. Perhaps it just sits there doing nothing, because it’s not needed. After all, half a tank of petrol will get you across town just as easily as a full tank. Or perhaps, if you have an unusually enormous amount of antioxidant lying around in your body doing nothing, it doesn’t just do nothing. Perhaps it does something actively harmful. That would be a turn-up for the books, wouldn’t it?
There were a couple of other reasons why the antioxidant theory seemed like a good idea twenty years ago. Firsdy, when you take a static picture of society, people who eat lots of fresh fruit and vegetables tend to live longer, and have less cancer and heart disease; and there are lots of antioxidants in fruit and vegetables (although there are lots of other things in them too, and, you might rightly assume, lots of other healthy things about the lives of people who eat lots healthy fresh fruit and vegetables, like their posh jobs, moderate alcohol intake, etc.).
Similarly, when you take a snapshot picture of the people who take antioxidant supplement pills, you will often find that they are healthier, or live longer: but again (although nutritionists are keen to ignore this fact), these are simply surveys of people who have already chosen to take vitamin pills. These are people who are more likely to care about their health, and are different from the everyday population—and perhaps from you—in lots of other ways, far beyond their vitamin pill consumption: they may take more exercise, have more social supports, smoke less, drink less, and so on.
But the early evidence in favour of antioxidants was genuinely promising, and went beyond mere observational data on nutrition and health: there were also some very seductive blood results. In 1981 Richard Peto, one of the most famous epidemiologists in the world, who shares the credit for discovering that smoking causes 95 per cent of lung cancer, published a major paper in Nature. He reviewed a number of studies which apparently showed a positive relationship between having a lot of ß-carotene onboard (this is an antioxidant available in the diet) and a reduced risk of cancer.
This evidence included ‘case-control studies’, where people with various cancers were compared against people without cancer (but matched for age, social class, gender and so on), and it was found that the cancer-free subjects had higher plasma carotene. There were also ‘prospective cohort studies’, in which people were classified by their plasma carotene level at the beginning of the study, before any of them had cancer, and then followed up for many years. These studies showed twice as much lung cancer in the group with the lowest plasma carotene, compared with those with the highest level. It looked as if having more of these antioxidants might be a very good thing.
Similar studies showed that higher plasma levels of antioxidant vitamin E were related to lower levels of heart disease. It was suggested that vitamin E status explained much of the variations in levels of ischaemic heart disease between different countries in Europe, which could not be explained by differences in plasma cholesterol or blood pressure.
But the editor of Nature was cautious. A footnote was put onto the Peto paper which read as follows:
Unwary readers (if such there are) should not take the accompanying article as a sign that the consumption of large quantities of carrots (or other dietary sources of (3-carotene) is necessarily protective against cancer.
It was a very prescient footnote indeed.
The antioxidant dream unravels
Whatever the shrill alternative therapists may say, doctors and academics have an interest in chasing hints that could bear fruit, and compelling hypotheses like these—which could save millions of lives—are not taken lightly. These studies were acted upon, with many huge trials of vitamins set up and run around the world. There’s also an important cultural context for this rush of activity which cannot be ignored: it was the tail end of the golden age of medicine. Before 1935 there weren’t too many effective treatments around: we had insulin, liver for iron deficiency anaemia, and morphine—a drug with superficial charm at least—but in many respects, doctors were fairly useless. Then suddenly, between about 1935 and 1975, science poured out a constant stream of miracles.
Almost everything we associate with modern medicine happened in that time: treatments like antibiotics, dialysis, transplants, intensive care, heart surgery, almost every drug you’ve ever heard of, and more. As well as the miracle treatments, we really were finding those simple, direct, hidden killers that the media still pine for so desperately in their headlines. Smoking, to everybody’s genuine surprise—one single risk factor—turned out to cause almost all lung cancer. And asbestos, through some genuinely brave and subversive investigative work, was shown to cause mesothelioma.
The epidemiologists of the 1980s were On a roll, and they believed that they were going to find lifestyle causes for all the major diseases of humankind. A discipline that had got cracking when John Snow took the handle off the Broad Street pump in 1854, terminating that pocket of the Soho cholera epidemic by cutting off the supply of contaminated water (it was a bit more complicated than that, but we don’t have the time here) was going to come into its own. They were going to identify more and more of these one-to-one correlations between exposures and disease, and, in their fervent imaginations, with simple interventions and cautionary advice they were going to save whole nations of people. This dream was very much not realised, as it turned out to be a bit more complicated than that.
Two large trials of antioxidants were set up after Peto’s paper (which rather gives the lie to nutritionists’ claims that vitamins are never studied because they cannot be patented: in fact there have been a great many such trials, although the food supplement industry, estimated by one report to be worth over $50 billion globally, rarely deigns to fund them). One was in Finland, where 30,000 participants at high risk of lung cancer were recruited, and randomised to receive either ß-carotene, vitamin E, or both, or neither. Not only were there more lung cancers among the people receiving the supposedly protective ß-carotene supplements, compared with placebo, but this vitamin group also had more deaths overall, from both lung cancer and heart disease.
The results of the other trial were almost worse. It was called the ‘Carotene and Retinol Efficacy Trial’, or ‘CARET’, in honour of the high p-carotene content of carrots. It’s interesting to note, while we’re here, that carrots were the source of one of the great disinformation coups of World War II, when the Germans couldn’t understand how our pilots could see their planes coming from huge distances, even in the dark. To stop them trying to work out if we’d invented anything clever like radar (which we had), the British instead started an elaborate and entirely made-up nutritionist rumour. Carotenes in carrots, they explained, are transported to the eye and converted to retinal, which is the molecule that detects light in the eye (this is basically true, and is a plausible mechanism, like those we’ve already dealt with): so, went the story, doubtless with much chortling behind their excellent RAF moustaches, we have been feeding our chaps huge plates of carrots, to jolly good effect.
Anyway. Two groups of people at high risk of lung cancer were studied: smokers, and people who had been exposed to asbestos at work. Half were given 3-carotene and vitamin A, while the other half got placebo. Eighteen thousand participants were due to be recruited throughout its course, and the intention was that they would be followed up for an average of
six years; but in fact the trial was terminated early, because it was considered unethical to continue it. Why? The people having the antioxidant tablets were 46 per cent more likely to die from lung cancer, and 17 per cent more likely to die of any cause,* than the people taking placebo pills. This is not news, hot off the presses: it happened well over a decade ago.
≡ I have deliberately expressed this risk in terms of the ‘relative risk increase’, as part of a dubious in-joke with myself. You will learn about this on page 240.
Since then the placebo-controlled trial data on antioxidant vitamin supplements has continued to give negative results. The most up-to-date Cochrane reviews of the literature pool together all the trials on the subject, after sourcing the widest possible range of data using the systematic search strategies described above (rather than ‘cherry-picking’ studies to an agenda): they assess the quality of the studies, and then put them all into one giant spreadsheet to give the most accurate possible estimate of the risks of benefits, and they show that antioxidant supplements are either ineffective, or perhaps even actively harmful.
The Cochrane review on preventing lung cancer pooled data from four trials, describing the experiences of over 100,000 participants, and found no benefit from antioxidants, and indeed an increase in risk of lung cancer in participants taking ß-carotene and retinol together. The most up-to-date systematic review and meta-analysis on the use of antioxidants to reduce heart attacks and stroke looked at vitamin E, and separately 0-carotene, in fifteen trials, and found no benefit for either. For ß-carotene, there was a small but significant increase in death.
Most recently, a Cochrane review looked at the number of deaths, from any cause, in all the placebo-controlled randomised trials on antioxidants which have ever been performed (many of which looked at quite high doses, but perfectly in line with what you can buy in health-food stores), describing the experiences of 230,000 people in total. This showed that overall, antioxidant vitamin pills do not reduce deaths, and in fact they may increase your chance of dying.
