2008 - Bad Science

by Ben Goldacre

  These are perhaps petty episodes. But for me the most concerning aspect of the way she responds to questioning of her scientific ideas is exemplified by a story from 2000, when Dr McKeith approached a retired Professor of Nutritional Medicine from the University of London. Shortly after the publication of her book Living Food for Health, John Garrow wrote an article about some of the bizarre scientific claims Dr McKeith was making, and his piece was published in a fairly obscure medical newsletter. He was struck by the strength with which she presented her credentials as a scientist (‘I continue every day to research, test and write furiously so that you may benefit…’ etc.). He has since said that he assumed—like many others—that she was a proper doctor. Sorry: a medical doctor. Sorry: a qualified, conventional medical doctor who attended an accredited medical school.

  In this book McKeith promised to explain how you can ‘boost your energy, heal your organs and cells, detoxify your body, strengthen your kidneys, improve your digestion, strengthen your immune system, reduce cholesterol and high blood pressure, break down fat, cellulose and starch, activate the enzyme energies of your body, strengthen your spleen and liver function, increase mental and physical endurance, regulate your blood sugar, and lessen hunger cravings and lose weight’.

  These are not modest goals, but her thesis was that they were all possible with a diet rich in enzymes from ‘live’ raw food—fruit, vegetables, seeds, nuts, and especially live sprouts, which are ‘the food sources of digestive enzymes’. She even offered ‘combination living food powder for clinical purposes’, in case people didn’t want to change their diet, and explained that she used this for ‘clinical trials’ with patients at her clinic.

  Garrow was sceptical of her claims. Apart from anything else, as Emeritus Professor of Human Nutrition at the University of London, he knew that human animals have their own digestive enzymes, and any plant enzyme you eat is likely to be digested like any other protein. As any Professor of Nutrition, and indeed many GCSE biology students, could tell you.

  Garrow read McKeith’s book closely, as have I. These ‘clinical trials’ seemed to be a few anecdotes about how incredibly well her patients felt after seeing her. No controls, no placebo, no attempt to quantify or measure improvements. So Garrow made a modest proposal in a fairly obscure medical newsletter. I am quoting it in its entirety, partly because it is a rather elegantly written exposition of the scientific method by an eminent academic authority on the science of nutrition, but mainly because I want you to see how politely he stated his case:

  I also am a clinical nutritionist, and I believe that many of the statements in this book are wrong. My hypothesis is that any benefits which Dr McKeith has observed in her patients who take her living food powder have nothing to do with their enzyme content. If I am correct, then patients given powder which has been heated above 118°F for twenty minutes will do just as well as patients given the active powder. This amount of heat would destroy all enzymes, but make little change to other nutrients apart from vitamin C, so both groups of patients should receive a small supplement of vitamin C (say 60mg⁄day). However, if Dr McKeith is correct, it should be easy to deduce from the boosting of energy, etc., which patients received the active powder and which the inactivated one.

  Here, then, is a testable hypothesis by which nutritional science might be advanced. I hope that Dr McKeith’s instincts, as a fellow-scientist, will impel her to accept this challenge. As a further inducement I suggest we each post, say, £1,000, with an independent stakeholder. If we carry out the test, and I am proved wrong, she will of course collect my stake, and I will publish a fulsome apology in this newsletter. If the results show that she is wrong I will donate her stake to HealthWatch [a medical campaigning group], and suggest that she should tell the 1,500 patients on her waiting list that further research has shown that the claimed benefits of her diet have not been observed under controlled conditions. We scientists have a noble tradition of formally withdrawing our publications if subsequent research shows the results are not reproducible—don’t we?

  Sadly, McKeith—who, to the best of my knowledge, despite all her claims about her extensive ‘research’, has never published in a proper ‘Pubmed-listed’ peer-reviewed academic journal—did not take up this offer to collaborate on a piece of research with a Professor of Nutrition. Instead, Garrow received a call from McKeith’s lawyer husband, Howard Magaziner, accusing him of defamation and promising legal action. Garrow, an immensely affable and relaxed old academic, shrugged this off with style.

  He told me, ‘I said, ‘Sue me.’ I’m still waiting.’ His offer of £1,000 still stands.

  But there is one vital issue we have not yet covered. Because despite the way she seems to respond to criticism or questioning of her ideas, her illegal penis pills, the unusually complicated story of her qualifications, despite her theatrical abusiveness, and the public humiliation pantomime of her shows, in which the emotionally vulnerable and obese cry on television, despite her apparently misunderstanding some of the most basic aspects of GCSE biology, despite doling out ‘scientific’ advice in a white coat, despite the dubious quality of the work she presents as somehow being of ‘academic’ standard, despite the unpleasantness of the food she endorses, there are still many who will claim: ‘You can say what you like about McKeith, but she has improved the nation’s diet.’

  This is not to be shrugged off lightly. Let me be very clear, for I will say it once again: anyone who tells you to eat more fresh fruit and vegetables is all right by me. If that was the end of it, I’d be McKeith’s biggest fan, because I’m all in favour of ‘evidence-based interventions to improve the nation’s health’, as they used to say to us in medical school.

  Let’s look at the evidence. Diet has been studied very extensively, and there are some things that we know with a fair degree of certainty: there is reasonably convincing evidence that having a diet rich in fresh fruit and vegetables, with natural sources of dietary fibre, avoiding obesity, moderating one’s intake of alcohol, cutting out cigarettes and taking physical exercise are protective against things such as cancer and heart disease.

  Nutritionists don’t stop there, because they can’t: they have to manufacture complication, to justify the existence of their profession. These new nutritionists have a major commercial problem with the evidence. There’s nothing very professional or proprietary about ‘Eat your greens,’ so they have had to push things further. But unfortunately for them, the technical, confusing, overcomplicated, tinkering interventions that they promote—the enzymes, the exotic berries—are very frequently not supported by convincing evidence.

  That’s not for lack of looking. This is not a case of the medical hegemony neglecting to address the holistic needs of the people. In many cases the research has been done, and has shown that the more specific claims of nutritionists are actually wrong. The fairy tale of antioxidants is a perfect example. Sensible dietary practices, which we all know about, still stand. But the unjustified, unnecessary overcomplication of this basic dietary advice is, to my mind, one of the greatest crimes of the nutritionist movement. As I have said, I don’t think it’s excessive to talk about consumers paralysed with confusion in supermarkets.

  But it’s just as likely that they will be paralysed with fear. They may have a bad reputation for paternalism, but it’s hard to picture any doctor in the past century using McKeith’s consultation methods as a serious tactic for inducing lifestyle change in his patients. With McKeith we see fire and brimstone hailing down until her subjects cry on national television: a chocolate gravestone with your name on it in the garden; a shouty dressing-down in public for the obese. As a posture it is as seductive as it is telegenic, it has a sense of generating movement; but if you drag yourself away from the theatricality of souped-up recipe and lifestyle shows on telly, the evidence suggests that scare campaigns may not get people changing their behaviour in the long term.

  What can you do? There’s the rub. The most important take-home
message with diet and health is that anyone who ever expresses anything with certainty is basically wrong, because the evidence for cause and effect in this area is almost always weak and circumstantial, and changing an individual person’s diet may not even be where the action is.

  What is the best evidence on the benefits of changing an individual person’s diet? There have been randomised controlled trials, for example—where you take a large group of people, change their diet, and compare their health outcomes with another group—but these have generally shown very disappointing results.

  The Multiple Risk Factor Intervention Trial was one of the largest medical research projects ever undertaken in the history of mankind, involving over 12,866 men at risk of cardiovascular events, who went through the trial over seven years. These people were subjected to a phenomenal palaver: questionnaires, twenty-four-hour dietary recall interviews, three-day food records, regular visits, and more. On top of this, there were hugely energetic interventions which were supposed to change the lives of individuals, but which by necessity required that whole families’ eating patterns were transformed: so there were weekly group information sessions for participants—and their wives—individual work, counselling, an intensive education programme, and more. The results, to everyone’s disappointment, showed no benefit over the control group (who were not told to change their diet). The Women’s Health Initiative was another huge randomised controlled trial into dietary change, and it gave similarly gave negative results. They all tend to.

  Why should this be? The reasons are fascinating, and a window into the complexities of changing health behaviour. I can only discuss a few here, but if you are genuinely interested in preventive medicine—and you can cope with uncertainty and the absence of quick-fix gimmicks—then may I recommend you pursue a career in it, because you won’t get on telly, but you will be both dealing in sense and doing good.

  The most important thing to notice is that these trials require people to turn their entire lives upside down, and for about a decade. That’s a big ask: it’s hard enough to get people signed up for participating in a seven-week trial, let alone one that lasts seven years, and this has two interesting effects. Firstly, your participants probably won’t change their diets as much as you want them to; but far from being a failing, this is actually an excellent illustration of what happens in the real world: individual people do not, in reality, change their diets at the drop of a hat, alone, as individuals, for the long term. A dietary change probably requires a change in lifestyle, shopping habits, maybe even what’s in the shops, how you use your time, it might even require that you buy some cooking equipment, how your family relates to each other, change your work style, and so on.

  Secondly, the people in your ‘control group’ will change their diets too: remember, they’ve agreed voluntarily to take part in a hugely intrusive seven-year-long project that could require massive lifestyle changes, so they may have a greater interest in health than the rest of your population. More than that, they’re also being weighed, measured, and quizzed about their diet, all at regular intervals. Diet and health are suddenly much more at the forefront of their minds. They will change too.

  This is not to rubbish the role of diet in health—I bend over backwards to find some good in these studies—but it does reflect one of the most important issues, which is that you might not start with goji berries, or vitamin pills, or magic enzyme powders, and in fact you might not even start with an individual changing their diet. Piecemeal individual life changes—which go against the grain of your own life and your environment—are hard to make, and even harder to maintain. It’s important to see the individual—and the dramatic claims of all lifestyle nutritionists, for that matter—in a wider social context.

  Reasonable benefits have been shown in intervention studies—like the North Karelia Project in Finland—where the public health gang have moved themselves in lock, stock and barrel to set about changing everything about an entire community’s behaviour, liaising with businesses to change the food in shops, modifying whole lifestyles, employing community educators and advocates, improving healthcare provision, and more, producing some benefits, if you accept that the methodology used justifies a causal inference. (It’s tricky to engineer a control group for this kind of study, so you have to make pragmatic decisions about study design, but read it online and decide for yourself: I’d call it a ‘large and promising case study’.)

  There are fairly good grounds to believe that many of these lifestyle issues are, in fact, better addressed at the societal level. One of the most significant ‘lifestyle’ causes of death and disease, after all, is social class. To take a concrete example, I rent a flat in London’s Kentish Town on my modest junior medic’s salary (don’t believe what you read in the papers about doctors’ wages). This is largely a white, working-class area, and the adult male life expectancy is about seventy years. Two miles away, in Hampstead, where the millionaire entrepreneur Dr Gillian McKeith PhD owns a large property, surrounded by other wealthy middle–class people, male life expectancy is almost eighty years. I know this because I have the Annual Public Health Report for Camden open on my kitchen table right now.

  The reason for this phenomenal disparity in life expectancy—the difference between a lengthy and rich retirement, and a very truncated one indeed—is not that the people in Hampstead are careful to eat goji berries and a handful of Brazil nuts every day, thus ensuring they’re not deficient in selenium, as per nutritionists’ advice. That’s a fantasy, and in some respects one of the most destructive features of the whole nutritionist project, graphically exemplified by McKeith: it’s a distraction from the real causes of ill health, but also—do stop me if I’m pushing this too far—in some respects, a manifesto of right-wing individualism. You are what you eat, and people die young because they deserve it. They choose death, through ignorance and laziness, but you choose life, fresh fish, olive oil, and that’s why you’re healthy. You’re going to see eighty. You deserve it. Not like them.

  Back in the real world, genuine public health interventions to address the social and lifestyle causes of disease are far less lucrative, and far less of a spectacle, than anything a Gillian McKeith—or, more importantly, a television commissioning editor—would ever dream of dipping into. What prime-time TV series looks at food deserts created by giant supermarket chains, the very companies with which these stellar media nutritionists so often have their lucrative commercial contracts? Who puts the issue of social inequality driving health inequality onto our screens? Where’s the human interest in prohibiting the promotion of bad foods, facilitating access to healthier foods by means of taxation, or maintaining a clear labelling system?

  Where is the spectacle in ‘enabling environments’ that naturally promote exercise, or urban planning that prioritises cyclists, pedestrians and public transport over the car? Or in reducing the ever-increasing inequality between senior executive and shop-floor pay? When did you ever hear about elegant ideas like ‘walking school buses’, or were stories about their benefits crowded out by the latest urgent front-page food fad news?

  I don’t expect Dr Gillian McKeith, or anyone in the media, to address a single one of these issues, and neither do you: because if we are honest with ourselves we understand that these programmes are only partly about food, and much more about salacious and prurient voyeurism, tears, viewing figures and vaudeville.

  Dr McKeith puts a cabbie straight

  Here is my favourite Dr McKeith story, and it comes from her own book, Living Food for Health. She is in a cab, and the driver, Harry, has spotted her. He tries to spark up a friendly conversation by suggesting that fish contains more omega oils than flax. Dr McKeith disputes this: ‘Flax seeds contain far greater levels of the healthy oils (omega-3 and omega-6) in a properly balanced and assimilable form.’ When Harry disagrees, she replies: ‘What do you mean, you disagree? Have you spent years conducting clinical research, working with patients, lecturing, teaching, stud
ying the omega oils in flax, obtaining worldwide data, compiling one of the largest private health libraries on the planet, and writing extensively on the topic? Are you a scientist, a biochemist, a botanist, or have you spent a lifetime studying food and biochemistry as I have done? Where is your scientific authority?’ Harry responds that his wife is a doctor, a gynaecologist. ‘Is she a food specialist or nutritional biochemist as well?’ demands Dr McKeith. ‘Um, ah, well, no, but she is a doctor.’

  I am not a food specialist, nor am I a nutritional biochemist. In fact, as you know, I claim no special expertise whatsoever: I hope I can read and critically appraise medical academic literature—something common to all recent medical graduates—and I apply this pedestrian skill to the millionaire businesspeople who drive our culture’s understanding of science.

  Flax seeds contain large amounts of fibre (along with oestro-genic compounds), so they’re not very ‘assimilable’, as Dr McKeith asserts, unless you crush them, in which case they taste foul. They’re sold as a laxative in doses of 15g, and you will need a lot of the stuff, partly because there’s also a problem with the form of omega oils in them: flax contains a short-chain plant form, and these must be converted in your body to the long-chain animal forms which may be beneficial (called DHA and EPA). When you account for the poor conversion in the body, then flax seeds and fish contain rouglily the same amounts of omega oil.

  We must also remember that we live not in a laboratory, but in the real world. It’s very easy to eat 100g of mackerel—if this were a completely different kind of book I’d be giving you my kedgeree recipe right now—whereas I would suggest that it’s slightly trickier to get a tablespoon of flax seed into you. Parsley, similarly, is a rich source of vitamin C, but you’re not going to eat an orange-sized lump of the stuff. As for Dr McKeith’s further claim that flax is ‘properly balanced’, I don’t know if she means spiritually or biologically, but fish is much higher in omega-3, which most people would say is better.