2008 - Bad Science

by Ben Goldacre

  But there is another, more subtle benefit to be had from editorial coverage of your pseudo-medical product: the claims that can be made in advertising and on packaging for food supplements and ‘borderline medical products’ are tightly regulated, but there are no such regulations covering the claims made by journalists.

  This rather clever division of labour is one of the more interesting features of the alternative therapy industry. Take a moment to think of all the things that you have come to believe are true, or at least have heard claimed regularly, about different supplements: glucosamine can treat arthritis; antioxidants prevent cancer and heart disease; omega-3 fish oils improve intelligence. These claims are now common currency, a part of our culture as much as ‘dock leaves for stinging nettles’; but you will rarely, if ever, see them being made explicitly on packaging or in advertising material.

  Once you realise this, it makes the colour supplements a marginally more interesting read: the alternative therapy columnist will make a dramatic and scientifically untenable claim for glucosamine, stating that it will improve the joint pain of a reader who has written in; the pill company, meanwhile, will have a full-page advertisement for glucosamine, which merely states the dose and possibly makes a bland claim at the level of basic biology, rather than about clinical efficacy: ‘Glucosamine is a known chemical constituent of cartilage.’

  Sometimes the overlap is close enough to be amusing. Some examples are predictable. The vitamin pill magnate Patrick Holford, for example, makes sweeping and dramatic claims for all kinds of supplements in his ‘Optimum Nutrition’ books; yet these same claims are not to be found on the labels of his own-brand ‘Optimum Nutrition’ range of vitamin pills (which do feature, however, a photograph of his face).

  Alternative health columnist Susan Clark—who has argued, among other things, that water has calories—is another shining example of this fine line that journalists can sometimes tread. She had a column in the Sunday Times, Grazia and the Observer for several years. In the course of these columns she recommended one company’s products, Victoria Health’, with notable frequency: once a month, regular as clockwork, by my reckoning. The papers deny that there was any impropriety, as does she, and I have no reason to doubt this. But she had done paid work for the company in the past, and she has now left her newspaper jobs to take up a full-time position at Victoria Health, writing their in-house magazine. (It’s a scene that is eerily reminiscent of the well-known free flow of employees in America between the pharmaceutical industry regulator and the boards of various pharmaceutical companies: in fact, at the risk of hammering a point home too hard, you will have noticed by now that I am telling the story of all the pill industries, using examples from mainstream media, and you couldn’t put a cigarette paper between them.)

  The Royal Pharmaceutical Society was expressing concern at these covert marketing strategies in the mainstream pharmaceutical industry as long ago as 1991: ‘Barred from labelling products with detailed medicinal claims unless they submit to the licensing procedure,’ it said, ‘manufacturers and marketing companies are resorting to methods such as celebrity endorsements, free pseudomedical product literature, and press campaigns that have resulted in uncritically promotional features in large-circulation newspapers and magazines.’

  Access to the unpoliced world of the media is well recognised as a major market advantage for Equazen, and it is an advantage that they press home hard. In the press release announcing the company’s acquisition by the pharmaceutical company Galenica, they declared: ‘Coverage of research showing the benefits of our Eye Q has appeared numerous times on national television and radio…that is widely credited with being instrumental in the significant growth of the UK omega-3 sector since 2003.’ To be honest, I would prefer to see a clearly labelled ‘nonsense box’ on all packaging and advertising, in which alternative therapy producers can freely make any claims they want to, instead of this misleading editorial coverage, because adverts are at least clearly labelled as such.

  The wheels of time

  Of course, these Durham trials are not the first time the world has seen such an extraordinary effort to promote one food supplement’s powers through media stories on inaccessible research. David Horrobin was a 1980s pharmaceutical multi-millionaire—one of the richest men in Britain—and his Efamol food supplements empire (built, like Equazen’s, on ‘essential fatty acids’) was worth an eye-watering £550 million at its peak. The efforts of his company went far further than anything one could find in the world of Equazen and Durham Council.

  In 1984, staff at Horrobin’s US distributors were found guilty in court of misbranding their food supplement as a drug; they were circumventing Food and Drug Administration regulations which forbade them from making unfounded claims for their supplement pills in advertising by engineering media coverage that treated them as if they had proven medical benefits. In the court case, paperwork was produced as evidence in which Horrobin explicitly said things like: ‘Obviously you could not advertise [evening primrose oil] for these purposes but equally obviously there are ways of getting the information across…’ Company memos described elaborate promotional schemes: planting articles on their research in the media, deploying researchers to make claims on their behalf, using radio phone-ins and the like.

  In 2003 Horrobin’s researcher Dr Goran Jamal was found guilty by the GMC of fraudulently concocting research data on trials which he had performed for Horrobin. He had been promised 0.5 per cent of the product’s profits should it come to market (Horrobin was not responsible, but this is a fairly unusual payment arrangement which would rather dangle temptation in front of your eyes).

  As with the fish-oil pills, Horrobin’s products were always in the news, but it was difficult to get hold of the research data. In 1989 he published a famous meta-analysis of trials in a dermatology journal which found that his lead product, evening primrose oil, was effective in eczema. This meta-analysis excluded the one available large published trial (which was negative), but included the two oldest studies, and seven small positive studies sponsored by his own company (these were still unavailable at the last review I could find, in 2003).

  In 1990 two academics had their review of the data binned by the journal after Horrobin’s lawyers got involved. In 1995 the Department of Health commissioned a meta-analysis from a renowned epidemiologist. This included ten unpublished studies held by the company which was marketing evening primrose oil. The ensuing scene was only fully described by Professor Hywel Williams a decade later in an editorial for the British Medical Journal. The company erupted over a leak, and the Department of Health forced all authors and referees to sign written statements to reassure it. The academics were not allowed to publish their report. Alternative therapy, the people’s medicine!

  It has since been shown, following a wider but undisclosed review, that evening primrose oil is not effective in eczema, and it has lost its medicines licence. The case is still cited by leading figures in evidence-based medicine, such as Sir Iain Chalmers, founder of the Cochrane Collaboration, as an example of a pharmaceutical company refusing to release information on clinical trials to academics wishing to examine their claims.

  David Horrobin, I feel duty bound to mention, is the father of the founding director of Equazen, Cathra Kelliher, nee Horrobin, and her husband and co-director Adam Kelliher specifically cited him in interviews as a major influence on his business practices. I am not suggesting that their business practices are the same, but in my view the parallels—with inaccessible data, and research results that wind up being presented directly to the media—are striking.

  In 2007 the GCSE results of the children in the Durham fish-oil year came in. This was an area of failing schools, receiving a huge amount of extra effort and input of all forms. The preceding year, with no fish oil, the results—the number of kids getting five GCSE grades A* to C—had improved by 5.5 per cent.

  ≡ ‘Nutritionist’, ‘nutrition therapist’, ‘nutritional th
erapy consultant’ and the many variations on this theme are not protected terms, unlike ‘nurse’, ‘dietitian’ or ‘physiotherapist’, so anyone can use them. Just to be clear, I’ll say it again: anyone can declare themselves to be a nutritionist. After reading this book, you will know more about the appraisal of evidence than most, so in the manner of Spartacus I suggest you call yourself one too; and academics working in the field of nutrition will have to move on, because the word doesn’t belong to them any more.

  After the fish-oil intervention the rate of improvement deteriorated notably, giving only a 3.5 per cent improvement. This was against a backdrop of a 2 per cent increase in GCSE scores nationally. You would have expected an improvement from a failing region whose schools were receiving a large amount of extra assistance and investment, and you might also remember, as we said, that GCSE results improve nationally every year. If anything, the pills seem to have been associated with a slowing of improvements.

  Fish oils, meanwhile, are now the most popular food supplement product in the UK, with annual sales for that single product worth over £110 million a year. And the Kellihers recently sold Equazen to a major pharmaceutical corporation for an undisclosed sum. If you think I have been overly critical, I would invite you to notice that they win.

  9 Professor Patrick Holford

  Where do all these ideas about pills, nutritionists and fad diets come from? How are they generated, and propagated? While Gillian McKeith leads the theatrical battalions, Patrick Holford is a very different animal: he is the academic linchpin at the centre of the British nutritionism movement, and the founder of its most important educational establishment, the ‘Institute for Optimum Nutrition’. This organisation has trained most of the people who describe themselves as ‘nutrition therapists’ in the UK.*

  ≡ ‘Nutritionist’, ‘nutrition therapist’, ‘nutritional therapy consultant’ and the many variations on this theme are not protected terms, unlike ‘nurse’, ‘dietitian’ or ‘physiotherapist’ so anyone can use them. Just to be clear, I’ll say it again: anyone can declare themselves to be a nutritionist. After reading this book, you will know more about the appraisal of evidence than most, so in the manner of Spartacus I suggest you call yourself one too; and academics working in the field of nutrition will have to move on, because the word doesn’t belong to them any more.

  Holford is, in many respects, the originator of their ideas, and the inspiration for their business practices.

  Praise is heaped upon him in newspapers, where he is presented as an academic expert. His books are best-sellers, and he has written or collaborated on around forty. They have been translated into twenty languages, and have sold over a million copies worldwide, to practitioners and the public. Some of his earlier works are charmingly fey, with one featuring a Blue Peter-ish ‘dowsing kit’ to help you diagnose nutritional deficiencies. The modern ones are drenched in scientific detail, and stylistically they exemplify what you might call ‘referenciness’: they have those nice little superscript numbers in the text, and lots of academic citations at the back.

  Holford markets himself vigorously as a man of science, and he has recently been awarded a visiting professorship at the University of Teesside (on which more later). At various times he’s had his own slot on daytime television, and hardly a week goes by without him appearing somewhere to talk about a recommendation, his latest ‘experiment’, or a ‘study’: one school experiment (with no control group) has been uncritically covered in two separate, dedicated programmes on Tonight with Trevor MacDonald, ITV’s peak-hour investigative slot, and that sits alongside his other appearances on This Morning, BBC Breakfast, Horizon, BBC News, GMTV, London Tonight, Sky News, CBS News in America, The Late Late Show in Ireland, and many more. According to the British media establishment, Professor Patrick Holford is one of our leading public intellectuals: not a vitamin-pill salesman working in the $50-billion food-supplement industry—a fact which is very rarely mentioned, if ever—but an inspiring academic, embodying a diligent and visionary approach to scientific evidence. Let us see what calibre of work is required for journalists to accord you this level of authority in front of the nation.

  AIDS, cancer and vitamin pills

  I first became fully aware of Holford in a bookshop in Wales. It was a family holiday, I had nothing to write about, and it was New Year. Like a lifesaver, here was a copy of his New Optimum Nutrition Bible, the 500,000-copy best-seller. I seized it hungrily, and looked up the big killers. First I found a section heading which explains that ‘people who take vitamin C live four times longer with cancer’. Excellent stuff.

  I looked up AIDS (this is what I call ‘the AIDS test’). Here is what I found on page 208: ‘AZT, the first prescribable anti-HIV drug, is potentially harmful, and proving less effective than vitamin C Now, AIDS and cancer are very serious issues indeed. When you read a dramatic claim like Holford’s, you might assume it’s based on some kind of study, perhaps where people with AIDS were given vitamin C. There’s a little superscript ‘23’, referring you to a paper by someone called Jariwalla. With bated breath I grabbed a copy of this paper online.

  The first thing I noticed was that this paper does not mention ‘AZT’. It does not compare AZT with vitamin C. Nor does it involve any human beings: it’s a laboratory study, looking at some cells in a dish. Some vitamin C was squirted onto these cells, and a few complicated things were measured, like ‘giant cell syncytia formation’, which changed when there was lots of vitamin C swimming around. All well and good, but this laboratory-bench finding very clearly does not support the rather dramatic assertion that ‘AZT, the first prescribable anti-HIV drug, is potentially harmful, and proving less effective than vitamin C In fact, it seems this is yet another example of that credulous extrapolation from preliminary laboratory data to clinical claim in real human beings that we have come to recognise as a hallmark of the ‘nutritionist’.

  But it gets more interesting. I casually pointed all this out in a newspaper article, and Dr Raxit Jariwalla himself appeared, writing a letter to defend his research paper against the accusation that it was ‘bad science’. This, to me, raised a fascinating question, and one which is at the core of this issue of ‘referenci-ness’. Jariwalla’s paper was a perfectly good one, and I have never said otherwise. It measured some complicated changes at a basic biological level in some cells in a dish on a lab bench, when they had lots of vitamin C squirted onto them. The methods and results were impeccably well described by Dr Jariwalla. I have no reason to doubt his clear description of what he did.

  But the flaw comes in the interpretation. If Holford had said: ‘Dr Raxit Jariwalla found that if you squirt vitamin C onto cells in a dish on a lab bench it seems to change the activity of some of their components,’ and referenced the Jariwalla paper, that would have been fine. He didn’t. He wrote: ‘AZT, the first prescribable anti-HIV drug, is potentially harmful, and proving less effective than vitamin C The scientific research is one thing. What you claim it shows—your interpretation—is entirely separate. Holford’s was preposterous over-extrapolation.

  I would have thought this was the point at which many people might have said: ‘Yes, in retrospect, that was perhaps a little foolishly phrased.’ But Professor Holford took a different tack. He has claimed that I had quoted him out of context (I did not: you can view the full page from his book online). He has claimed that he has corrected his book (you can read about this in a note at the back of the book you are holding). He has thrown around repeated accusations that I have only criticised him on this point because I am a paw a of big pharmaceutical corporations (I am not; in fact, bizarrely, I am one of their most vicious critics). Crucially, he suggested that I had focused on a trivial, isolated error.

  A vaguely systematic review

  The joy of a book is that you have plenty of space to play with. I have here my copy of The New Optimum Nutrition Bible. It’s ‘the book you have to read if you care about your health’, according to the Sunday Ti
mes quote on the front cover. ‘Invaluable’, said the Independent on Sunday, and so on. I have decided to check every single reference, like a crazed stalker, and I will now dedicate the entire second half of this book to producing an annotated edition of Holford’s weighty tome.

  Only kidding.

  There are 558 pages oi plausible technical jargon in Holford’s book, with complicated advice on what foods to eat, and which kinds of pills you should buy (in the ‘resources’ section it turns out that his own range of pills are ‘the best’). For our sanity I have restricted our examination to one key section: the chapter where he explains why you should take supplements. Before we begin, we must be very clear: I am only interested in Professor Holford because he teaches the nutritionists who treat the nation, and because he has been given a professorship at Teesside University, with plans for him to teach students and supervise research. If Professor Patrick Holford is a man of science, and an academic, then we should treat him as one, with a scrupulously straight bat.

  So, turning to Chapter 12, page 97 (I’m working from the ‘completely revised and updated’ 2004 edition, reprinted in 2007, if you’d like to follow the working at home), we can begin. You’ll see that Holford is explaining the need to eat pills. This might be an apposite moment to mention that Professor Patrick Holford currently has his own range of best-selling pills, at least twenty different varieties, all featuring a photograph of his smiling face on the label. This range is available through the pill company BioCare, and his previous range, which you will see in older books, was sold by Higher Nature.*