I Think You'll Find It's a Bit More Complicated Than That

by Ben Goldacre

  Journalists and religious commentators are already writing lengthy moral screeds on the implications of this case for our treatment of people in a coma. That seems premature. Mr Houben’s typing may well be genuine, and therefore atypical: nobody can have a meaningful opinion, because newspapers are no place to communicate breakthroughs which are incompatible with large swathes of current knowledge, and based on what seems to be weak and even contradictory evidence.

  Now that the amazing case of Mr Houben’s facilitated communication has been made the subject of a huge media sensation around the world, and extensive ethical speculation, I think we can all look forward to seeing it formally assessed and presented in an academic paper by his doctor, Professor Steven Laureys of Belgium’s Coma Science Group. I’ve made a note in my diary for this date next year. Just to check.

  Prof Laureys never published such a paper. In 2010, after international criticism, he allowed facilitated communication to be tested more rigorously with Mr Houben. It didn’t work. This was barely reported.

  BAD JOURNALISM

  Asking for It

  Guardian, 4 July 2009

  There’s nothing like science for giving that objective, white-coat-flavoured legitimacy to your prejudices, so it must have been a great day for Telegraph readers when they came across the headline ‘Women who dress provocatively more likely to be raped, claim scientists’. Ah, scientists. ‘Women who drink alcohol, wear short skirts and are outgoing are more likely to be raped, claim scientists at the University of Leicester.’ Well there you go. Oddly, though, the title of the press release for the same research was ‘Promiscuous men more likely to rape’.

  Normally we berate journalists for rewriting press releases. Had the Telegraph found some news?

  I rang Sophia Shaw at the University of Leicester. She was surprised to have been presented as an expert scientist on the pages of the Daily Telegraph; because Sophia is an MSc student, and this is her dissertation project. It’s also not finished. ‘We are intending on getting it published, but my findings are very preliminary.’ She was discussing her dissertation at an academic conference, when the British Psychological Society’s PR team picked it up, and put out a press release. We will discuss that later.

  But first, the science. Shaw spoke to about a hundred men, presented them with various situations around being with a woman, and asked them when they would call it a night, in order to explore men’s attitudes towards coercing women into sex. ‘I’m very aware that there are limitations to my study. It’s self-report data about sensitive issues, so that’s got its flaws; participants were answering when sober, and so on.’

  But more than that, she told me, every single one of the first four statements made by the Telegraph is a flat, factually incorrect misrepresentation of her findings.

  Women who drink alcohol, wear short skirts and are outgoing are more likely to be raped? ‘We found no evidence that women who are more outgoing are more likely to be raped. This is completely inaccurate. We found no difference whatsoever. The alcohol thing is also completely wrong: if anything, we found that men reported they were willing to go further with women who are completely sober.’

  And what about the Telegraph’s next claim: its reach for objective distance, and its assertion that it’s not just judgemental newspapers, but also scientists who claim that provocatively dressed women are more likely to be raped?

  ‘We have found at the minute that people will go slightly further with women who are provocatively dressed, but this result is not statistically significant. Basically, you can’t say that’s an effect, it could easily be the play of chance. I told the journalist it isn’t one of our main findings, you can’t say that. It’s not significant, which is why we’re not reporting it in our main analysis.’

  So if the Telegraph is throwing blame around with rape, who do we blame for this story, and what do we do about it? On the one hand, Sophia Shaw is not very impressed with the newspaper: ‘When I saw the article my heart completely sank, and it made me really angry, given how sensitive this subject is. To be making claims like the Telegraph did, in my name, places all the blame on women, which is not what we were doing at all. I just felt really angry about how wrong they’d got this study.’ Since I started sniffing around, and Sophia complained, the Telegraph has quietly changed the online copy of the article, although there has been no formal correction, and in any case, it remains inaccurate.

  But there is a second, less obvious problem. Repeatedly, unpublished work – often of a highly speculative and eye-catching nature – is shepherded into newspapers by the press officers of the British Psychological Society and other organisations. A rash of news coverage and popular speculation ensues, in a situation where nobody can read the academic work. I could only get to the reality of what was measured, and how, by personally tracking down and speaking to an MSc student on the phone about her dissertation. In any situation this would be ridiculous, but in a sensitive area like rape, you’d hope that PR staff could temper their desperate hunger for coverage, and wait for the finished paper.

  Jab ‘as Deadly as the Cancer’

  Guardian, 10 October 2009

  Last month I had a debate at the Royal Institution with Lord Drayson, the Science Minister, in which he argued that I was too harsh on British science coverage, which is the best in the world. During this event our chairman (excellently, Simon Mayo) pulled out a health front page from the Express, and asked what we thought about it. I said the article might well be accurate, but it’s also quite likely to be a work of fantasy. As a serious matter of public health, I would urge people to be extremely sceptical about health information on the front page of the Express. Lord Drayson thought this was cynical and unfair. He warmly encouraged us to trust this newspaper.

  Here’s the latest front-page story from the same paper: ‘Jab “as deadly as the cancer” ’. ‘Cervical drug expert hits out as new doubts raised over death of teenager’, said the subheading, although no such new doubts were raised in the article. Here is one whole paragraph from that story. Almost every single assertion it makes is false.

  The cervical cancer vaccine may be riskier and more deadly than the cancer it is designed to prevent, a leading expert who developed the drug has warned. She also claimed the jab would do nothing to reduce the rates of cervical cancer in the UK. Speaking exclusively to the Sunday Express, Dr Diane Harper, who was involved in the clinical trials of the controversial drug Cervarix, said the jab was being ‘over-marketed’ and parents should be properly warned about the potential side effects.

  The story seemed unlikely for three reasons. Firstly, Professor Harper is not a known member of the anti-vaccination community, which is vanishingly small. Secondly, it was on the front page of the Sunday Express, which is always cause for concern. Lastly, it was by specialist health journalist Lucy Johnston, whose previous work includes ‘Doctor’s MMR fears’, ‘Exclusive: Experts Cast Doubt on Claim for “Wonder” Cancer Jabs’, ‘Children “Used as Guinea Pigs For Vaccines” ’, ‘Dangers of MMR Jab “Covered Up” ’, ‘Teenage Girls Sue Over Cancer Jab’, ‘Jab Makers Linked to Vaccine Programme’, and so many more, including the memorable front-page exclusive: ‘Suicides “Linked to Phone Masts”’.

  I contacted Professor Harper. To avoid any doubt, I will explain her position on this issue using only her own words: ‘I did not say that Cervarix was as deadly as cervical cancer. I did not say that Cervarix could be riskier or more deadly than cervical cancer. I did not say that Cervarix was controversial, I stated that Cervarix is not a “controversial drug”. I did not “hit out” – I was contacted by the press for facts. And this was not an exclusive interview.’

  Professor Harper did not ‘develop Cervarix’, as the Sunday Express said, but she did work on some important trials of Gardasil, and also Cervarix. ‘Gardasil is not a “sister vaccine”, as the Express said, it is a different compound. I do not know of the side effects of Cervarix as it is not available in the US.’ Furthermor
e, she did not say that Cervarix was being overmarketed. ‘I did say that Merck was egregiously overmarketing Gardasil in the US – but Gardasil and Cervarix are not the same vaccines.’

  And here is the tragedy. Academics are often independently-minded about the interventions they work on, and Professor Harper – who worked on Gardasil – is critical of Gardasil. More specifically, she is critical of how it is being marketed.

  Briefly, her view (which was already published a long time ago) is that we do not yet know how long the protection from these vaccines will last, and that this will therefore affect the cost-benefit calculations. She is concerned that aggressive advertising aimed directly at the public (which is not permitted in Europe, with good reason) may lead people to falsely believe they are immune to HPV, which causes cervical cancer, and so neglect other precautions. Lastly, she suspects from modelling data that, for the specific and restricted group of women who are punctilious about attending every single one of their cervical-cancer screening appointments, vaccination may have little impact on their risk of death from cancer; but she also says that even this group will still benefit from the reduction in reproductive problems caused by treating precancerous changes in cervical cells, and from avoiding the unpleasantness of screening and treatment.

  The article has now disappeared from the Express website, and Professor Harper has complained to the PCC. ‘I fully support the HPV vaccines,’ she says. ‘I believe that in general they are safe in most women. I told the Express all of this.’

  Her criticisms of some aspects of cervical-cancer vaccination are nuanced and valuable: but they don’t fit into the black-and-white hysteria of British news media. It would be nice if we could have a serious public discussion about the relative risks and merits of different vaccine options. Sadly, with this kind of ugly reporting, scientists around the world may learn that such a discussion is not currently possible in the UK media. That is the greatest tragedy.

  Health Warning: Exercise Makes You Fat

  Guardian, 29 August 2009

  Why would you listen to a government health message, or your GP practice nurse, when the Sunday Telegraph has much more exciting news? ‘Health Warning: Exercise Makes You Fat’ is the kind of full-width headline you want to see across a broadsheet page: it’s affirmative, it’s reassuring, and it gives you clear permission to sit on your arse all day. ‘Re-programming body fat is the key to weight loss, not working out.’ Praise be. ‘Is it possible that all that exercise is doing nothing to make us slimmer?’ Please, let the answer be yes.

  The Telegraph produced three lines of research for this claim. Firstly, more people are spending more money on more exercise than before, but there is also more obesity around in the UK than before: explain that with your science. Then there was some speculative laboratory research about interfering with brown fat in animal models, using stem cells and things: interesting to read, but very far from the headline claim.

  To properly examine whether exercise really will make you fat, the paper described two trials.

  The first one, I can tell you right now, is cherry-picked. The Cochrane Library is a non-profit collaboration of academics who produce unbiased, systematic reviews of the medical literature, and they have a systematic review of all the forty-three trials that have been done on exercise for weight loss. This produces clear evidence that exercise is beneficial, albeit more modestly than you’d hope. ‘Exercise plus diet’ was compared with ‘diet alone’ in fourteen trials: both groups lost weight, but 1.1 kg more in the exercise group. High-intensity exercise was compared with low-intensity in four trials: high-intensity exercise came out better in all of them, with extra weight loss of 1.5 kg. There are also improvements in blood pressure, cholesterol, blood sugars, sense of well-being, and so on.

  The Telegraph quoted one trial from Dr Timothy Church of Louisiana University, which compared three different levels of exercise with a personal trainer in overweight people. There were no significant differences between the weight lost in any of the groups, including the ‘control’ group, who were not given a personal trainer at all. So while it is true that exercise did indeed have no benefit, in the one single trial the Telegraph quoted, it has also ignored the vast, overwhelming majority of published literature examining the same question. Dr Church speculates that the explanation for his finding is that people who exercised more also ate more. Dr Church is speculating in order to explain the odd results of his one single trial. I think the most helpful suggestion he could make here would be: ‘Our unusual results are probably a fluke, because almost all other trials using similar methods found a completely different answer.’

  Then there is the Telegraph’s second trial. ‘Another study due to be published next month in the Journal of Public Health Nutrition by researchers at the University of Leeds draws similar conclusions. Professor John Blundell and his colleagues found that people asked to do supervised exercise to lose weight also increased the amount they ate and reduced their intake of fruit and vegetables.’

  I have this trial in front of me. It’s simply not true that participants increased their food intake. A tiny proportion did (15 per cent), but that’s hardly an issue, because what the Telegraph doesn’t tell you – bizarrely – is that overall, participants doing supervised exercise in this trial lost more weight. Much more weight: 3.2 kg more, on average, over just twelve weeks.

  Prof Blundell says: ‘The Telegraph article was a complete distortion of the facts of our investigation, which showed that exercise is very effective for weight loss. They completely reversed the outcome of our study.’

  You might well view my work as pointless: like Sisyphus in an anorak, fighting my way up a greasy waterslide, defeated by the torrent of sewage, desperately trying to scratch one grumpy correction into yesterday’s chip wrapper. But journalism like this is a genuine public health problem. Research has repeatedly shown that people change their health behaviour in response to what they read in the media, and just this month, the World Cancer Research Fund commissioned a survey from YouGov: a proper survey, in a representative sample, from a reputable data collector, where anyone is allowed to see the questions and the results.

  Half of all respondents said they thought scientists and doctors were constantly changing their minds about healthy-living advice, although in reality healthy-living advice hasn’t changed at all for at least a decade (don’t smoke, do some exercise, eat more fruit and veg). And a quarter of all respondents said that because scientists keep changing their minds, you might as well eat whatever you want, because it won’t make any difference anyway.

  Have another pastry and put the telly on.

  The Caveat in Paragraph Number 19

  Guardian, 16 October 2010

  You will be familiar with the Daily Mail’s ongoing project to divide all the inanimate objects in the world into the ones that either cause or prevent cancer. It’s hardly worth documenting the individual cases any more: you can appreciate the phenomenon in bulk, through websites like the Daily Mail Oncological Ontology Project and Kill Or Cure, with its alphabetised list: from almonds, apples and artificial light; through horseradish, hot drinks and housework; to wasabi, water, watercress, and more.

  But occasionally one story pops up to illustrate a wider issue, and ‘Strict diet two days a week “cuts risk of breast cancer by 40 per cent” ’ is a good example. It goes on: ‘A strict diet for two days a week consisting solely of vegetables, fruit, milk and a mug of Bovril could prevent breast cancer, scientists say.’

  Now, if you read the academic paper which this news article is describing, from the International Journal of Obesity: it’s not a study of breast cancer, and it does not find that the risk of cancer is reduced by 40 per cent. The press release wasn’t exactly a masterpiece of clarity either, but in any case, the study doesn’t even measure breast cancer as an outcome, at all.

  If I was to leave it there, the journalist would correctly complain: because after all the grand and misleading claims, firstl
y, briefly, in the body of the piece, it does mention that the outcome is not cancer, but some hormones related to cancer (with no explanation of how tenuous that relationship is). Then, finally, at the very bottom of the piece, comes the reality. Although it’s not spoken in the authoritative third person of the paper itself, it’s there, in a quote, at paragraph number 19:

  But Dr Julie Sharp, senior science information manager at Cancer Research UK, said: ‘This study is not about breast cancer, it’s a study showing how different diet patterns affect weight loss and it’s misleading to draw any conclusions about breast cancer from this research.’

  The late caveat, torpedoing the central premise of a news piece, is a common strategy in many newspapers. But what use is this information, at the end of a long article, in paragraph number 19?

  The way people read newspapers has been studied widely, using eye-tracking technology. It’s through this that we discover, for example, that when presented with a full-length photograph of a man, men are more likely to look at the penis area than women.

  Most of this research is more interested in adverts than news, because research in all fields is driven by money (top left of the page is best, apparently): but there is plenty of other useful stuff, much of it by the Poynter Institute.

  They did an early study in 1990, with predictable findings: photos attract attention; eyes travel from the dominant photo to the biggest headline, then teasers, and finally text; text is read the least, headlines the most; and so on.

  But their most recent project was far bigger: they took a representative sample of 582 people from four cities in the US, and invited them to read a newspaper and a website as they normally would, wearing the eye-tracking equipment, over five days in 2006, for fifteen minutes each. This yielded a dataset of more than 102,000 eye stops.