I Think You'll Find It's a Bit More Complicated Than That

by Ben Goldacre

  Finally, they probed these mechanisms even further. Thirty-one students were asked to bring their lucky charm. It was either taken away or not, and they were given an anagram task. Before starting, they were asked to set a goal: what percentage of all the hidden words did they think they could find? Then they began. As expected, participants who had their lucky charm in the room performed better, and reported a higher degree of ‘self-efficacy’ as before. But more than that, people who had their lucky charm in the room set higher goals, and also persisted longer in working on the task.

  So there you go. Almost everyone has some kind of superstition (mine is that I should mention I noticed this study through my friends Vaughan Bell and Ed Yong on Twitter). What’s interesting is that superstition works, because it improves confidence, lets you set higher goals, and encourages you to work harder. In a lab. In one experiment. In a field riven with publication bias. You now know everything you need to decide if this applies to your life.

  Evidence-Based Smear Campaigns

  Guardian, 1 May 2010

  Elections are a time for smearing. But do smears work, and if so, what’s the best way to combat them? A new experiment published this month in the journal Political Behavior tries to examine the impact of corrections. The findings are disturbing: far from changing people’s minds, if you are deeply entrenched in your views, a correction will only reinforce them.

  The first experiment used articles claiming that Iraq had weapons of mass destruction immediately before the US invasion. One hundred and thirty participants were asked to read a mock news article, attributed to Associated Press, reporting on a Bush campaign stop in Pennsylvania during October 2004. The article describes Bush’s appearance as ‘a rousing, no-retreat defense of the Iraq war’, and gives genuine Bush quotes about WMD: ‘There was a risk, a real risk, that Saddam Hussein would pass weapons or materials or information to terrorist networks, and in the world after September the 11th … that was a risk we could not afford to take.’ And so on.

  The 130 participants were then randomly assigned to one of two conditions. For half of them, the article stopped there. For the other half, the article continues, and includes a correction: it discusses the release of the Duelfer Report, which documented the lack of Iraqi WMD stockpiles – and the lack of an active production programme – immediately prior to the US invasion.

  After reading the article, subjects were asked to state whether they agreed with the following statement: ‘Immediately before the US invasion, Iraq had an active weapons-of-mass-destruction program, the ability to produce these weapons, and large stockpiles of WMD, but Saddam Hussein was able to hide or destroy these weapons right before US forces arrived.’ Their responses were measured on a five-point scale ranging from ‘strongly disagree’ to ‘strongly agree’.

  As you would expect, those who self-identified as conservatives were more likely to agree with the statement. Separately, meanwhile, more knowledgeable participants (independently of political persuasion) were less likely to agree. But then the researchers looked at the effect of whether you were also given the correct information at the end of the article, and this is where things get interesting. They had expected that the correction would be less effective for more conservative participants, and this was true, up to a point. For very liberal participants the correction worked as expected, making them more likely to disagree with the statement that Iraq had WMD, when compared with those who were also very liberal but who received no correction. For those who described themselves as left of centre or centrist, the correction had no effect either way.

  But for people who placed themselves ideologically to the right of centre, the correction wasn’t just ineffective, it actively backfired: conservatives who received a correction telling them that Iraq did not have WMD were more likely to believe that Iraq had WMD, when compared with those who were given no correction at all. You might have expected people simply to dismiss a correction that was incongruous with their pre-existing view, or to regard it as having no credibility: in fact, it seems such information actively reinforced their false beliefs.

  Maybe the cognitive effort of mounting a defence against the incongruous new facts entrenches you even further. Maybe you feel marginalised and motivated to dig in your heels. Who knows? But these experiments were then repeated, in various permutations, on the issue of tax cuts (or rather, the idea that tax cuts had increased national productivity so much that tax revenue increased overall) and stem-cell research. All the studies found exactly the same thing: if the original dodgy fact fits with your prejudices, a correction only reinforces these even more. If your goal is to move opinion, this depressing finding suggests that smears work; and what’s more, corrections don’t challenge them much, because for people who already disagree with you, it only make them disagree even more.

  Why Cigarette Packs Matter

  Guardian, 12 March 2011

  This week our government committed itself to the removal, albeit slowly, of cigarette displays in shops. But plain packaging on cigarettes has been delayed for further consultation.

  The Unite union is unimpressed. It represents 6,000 people in tobacco production and distribution, and put out a statement: ‘Switching to plain packaging will make it easier to sell illicit and unregulated products, especially to young people’. This ‘may increase long-term health problems’. Tory MP Philip Davies said: ‘Plain packaging for cigarettes would be gesture politics … it would have no basis in evidence.’

  Everyone is entitled to their own opinions, but not their own facts. Cigarette packaging has been used for brand building and sales expansion, and that is bad enough; but it has also been used for many decades to sell the crucial lie that cigarettes which are ‘light’, ‘mild’, ‘silver’ and the rest are somehow ‘safer’.

  This is one of the most important con tricks of all time, because people base real-world decisions on it, even though we have known for several decades that low-tar cigarettes are no safer than normal cigarettes. Manufacturers’ gimmicks, like the holes on the filter by your fingers, confuse laboratory smoking machines, but not people. Smokers who switch to lower-tar brands compensate with larger, faster, deeper inhalations, and by smoking more cigarettes. The collected data from a million people shows that those who smoke low-tar and ‘ultra-light’ cigarettes get lung cancer at the same rate as people who smoke ‘normal’ cigarettes. They are also, paradoxically, less likely to give up smoking.

  So the ‘light’, ‘pale’ and ‘mild’ packaging sells a lie. But do people know this? In data from two population-based surveys, a third of smokers believed incorrectly that ‘light’ cigarettes reduce health risks, and were less addictive (it’s 71 per cent in China). A random telephone digit survey of 2,120 smokers found they believed, on average, that ‘ultra lights’ convey a 33 per cent reduction in risk. A postal survey of five hundred smokers found a quarter believed ‘light’ cigarettes are safer. A school-based questionnaire of 267 adolescents found, once again, as you’d expect, that they incorrectly believed ‘light’ cigarettes to be healthier and less addictive.

  Where do all these incorrect beliefs come from? Careful manipulation by the tobacco companies, as you can see for yourself, in their internal documents available for free online. They aimed to deter quitters, and ‘mild’ products – which were made to seem safer and less addictive – were the perfect vehicle.

  But over fifty countries, including the UK, have now banned a few magic words like ‘light’ and ‘mild’. So is that enough? No. A survey of 15,000 people in four countries found that after the ban there was a brief dip in false beliefs in the UK, but by 2005 we bounced back to having the same false beliefs about ‘safer-looking’ brands as the US.

  This is because brand packaging continues to peddle these lies. A street-interception survey from 2009 of three hundred smokers and three hundred non-smokers found that people think packages with ‘smooth’ and ‘silver’ in the names are safer, and that cigarettes in packaging with ligh
ter colours, and a picture of a filter, were also safer.

  Of course tobacco companies know this. As Philip Morris said in its internal document ‘Marketing New Products in a Restrictive Environment’: ‘Lower delivery products tend to be featured in blue packs. Indeed, as one moves down the delivery sector, then the closer to white a pack tends to become. This is because white is generally held to convey a clean healthy association.’

  If you’re in doubt about the impact this branding can have, ‘brand imagery’ studies show that when participants smoke the exact same cigarettes presented in lighter-coloured packs, or in packs with ‘mild’ in the name, they rate the smoke as lighter and less harsh, simply through the power of suggestion. These illusory perceptions of mildness, of course, further reinforce the false belief that the cigarettes are healthier.

  But these aren’t the only reasons why banning a few words from packaging isn’t enough. A study on six hundred adolescents, for example, found that plain packages increase the noticeability, recall and credibility of warning labels.

  There’s no real doubt that the extended, complex, interlocking branding and packaging machinations of cigarette companies play a major role in misleading smokers about the risks. They downplay the harms of smoking, one of the biggest killers in the world, and sadly nothing from Unite – for shame – or some Tory MP will change that.

  If you don’t care about this evidence, or you think jobs are more important than people killed by cigarettes, or you think libertarian principles are more important than both, then that’s a different matter. But if you say the evidence doesn’t show evidence of harm from branded packaging, you are simply wrong.

  All Bow Before the Mighty Power of the Nocebo Effect

  Guardian, 28 November 2009

  I’m fine with people wasting their money on sugar pills, but I have higher expectations of government bodies. The Medicines and Healthcare Regulatory Authority has decided to let homeopathy manufacturers make medical claims on their sugar pill bottles, without any evidence of efficacy, and the government funds homeopathy on the NHS. This week the Parliamentary Science and Technology Select Committee looked into the evidence behind these decisions.

  There was much cheap comedy – as you’d expect from a government inquiry into an industry based on magical beliefs. But the best moment was when Dr Peter Fisher from the Royal London Homeopathic Hospital (funded by the NHS) explained that homeopathic sugar pills have physical side effects, so they must be powerful. Many raised their eyebrows; but interestingly, the homeopath is correct. People can experience side effects when they receive pills that contain no medicine at all.

  A paper published in the journal Pain next month looks at this very issue. Its authors found every single placebo-controlled trial ever conducted on a migraine drug, and looked at the side effects reported by the people in the control group, who received a dummy ‘placebo’ sugar pill instead of the real drug. Not only were these side effects common, they were also similar to the specific side effects you’d have expected if you’d received real drug in the trial: so patients getting placebo instead of anticonvulsants, for example, reported memory difficulties, sleepiness and loss of appetite, which are typical side effects of anticonvulsants; while patients getting placebo instead of painkillers got digestive problems, which themselves are commonly caused by painkillers.

  This is nothing new. A study in 2006 sat seventy-five people in front of a rotating drum to make them feel nauseous, and gave them a placebo sugar pill. Twenty-five were told it was a drug that would make the nausea worse: their nausea became worse, and they also exhibited more gastric tachyarrhythmia, the abnormal stomach activity that frequently accompanies nausea.

  A paper in 2004 took six hundred patients from three different specialist drug-allergy clinics, split them into two groups at random, and handed them either the drug that was causing their adverse reactions, or a dummy pill with no ingredients: 27 per cent of the patients experienced side effects such as itching, malaise and headache from the placebo dummy pill.

  And a classic paper from 1987 looked at the impact of listing side effects on the consent form which all patients sign before accepting treatment in a randomised trial. This was a large placebo-controlled trial comparing aspirin against placebo, conducted in three different centres. In two of them, the consent form contained a statement outlining various gastrointestinal side effects, and in these centres there was a sixfold increase in the number of people reporting such symptoms, and many more people dropping out of the trial, compared with the one centre that did not list such side effects on the form.

  This is the amazing world of the nocebo effect, the evil twin of the placebo effect, where negative expectations can induce unpleasant symptoms in the absence of a physical cause.

  Sadly, though, it doesn’t help homeopaths: in 2003 Professor Edzard Ernst conducted a systematic review of every single homeopathy trial that reported side effects. This found, in total, fifty episodes of side effects in patients treated with placebo and sixty-three in patients treated with homeopathically diluted remedies, with no statistically significant difference in the rates of side effects between the two groups.

  Quacks like to present themselves as holistic, but in reality this research into the placebo effect and the nocebo effect suggests quite the opposite. The world of the homeopath is reductionist, one-dimensional, and built on the power of the pill: it cannot accommodate the fascinating reality of connections between mind and body, which have been revealed in these experiments, and many more. The next time you find yourself trapped, at dinner, next to some bore who’s decided they have secret mystical healing powers – while they earnestly explain how their crass efforts at selling sugar pills represent a meaningful political stand against the crimes of big pharma – just think: some lucky person, somewhere in the world, is sat next to a nocebo researcher.

  So Brilliantly You’ve Presented a Really Transgressive Case Through the Mainstream Media

  Guardian, 5 December 2009

  Here is a mystery. Rom Houben, a Belgian man, was diagnosed as being in a coma for twenty-three years, and he has now made a partial recovery. This has been demonstrated with a series of recently developed brain-scanning techniques (whose predictive value is not entirely known, but they are promising), and he is also opening his eyes. But the story in the media this week goes further than that: it is also claimed that he was conscious all along, but simply unable to move, a well-documented phenomenon called ‘locked-in syndrome’. It has been reported as a news story around the world, from the Sun, Sky news, the BBC, the Guardian (in four separate pieces) and the Telegraph, through to CNN, Der Spiegel, Australian TV news, and hundreds more.

  But there is a problem. Mr Houben has described his horrifying experience of being locked in using something called ‘facilitated communication’: someone holds his finger; they can sense where his hand wants to go on a screen; and by moving with him, they help him type. If you watch the TV footage, it all happens very fast, too.

  What is known about facilitated communication? Many researchers have compared it to ouija boards, which is an attempt, I think, to be fair. Some facilitators may well believe that they are guided by an external force; but there have been several large reviews of research into this technique, and overall, it’s not good.

  The practice was popular in the 1980s and 1990s, and used mostly in severe autism, so that is where much of the work is found. You might feel this is not entirely applicable to someone with locked-in syndrome; equally, you wouldn’t ignore it. A lengthy research review on educational interventions in autism commissioned by the Department for Education and Employment in 1998 found that in FC ‘almost all scientifically controlled studies showed that the facilitator was the author of the communication’. This finding was so clear that they concluded further research would be hard to justify.

  An academic review in 2001 looked at all the more recent studies, updating two earlier reviews with negative conclusions from 1995, and f
ound that overall, again, the claims made for FC are unsubstantiated.

  If you prefer authorities to studies, the National Autistic Society says that five major US professional bodies now formally oppose the use of FC, including the American Academy of Child and Adolescent Psychiatry, the American Speech-Language-Hearing Association, and the American Association on Mental Retardation. The American Psychological Association issued a position paper on FC in 1994 (at the height of its popularity) saying ‘studies have repeatedly demonstrated that facilitated communication is not a scientifically valid technique’, and calling it ‘a controversial and unproved communicative procedure with no scientifically demonstrated support for its efficacy’.

  My concern about this is pretty simple. If you watch the video of Mr Houben’s facilitated communication in action – and I encourage you to do so – you will see the facilitator looking at the screen and the keyboard, moving Mr Houben’s finger at remarkably high speed to type out a message, while both of Mr Houben’s eyes are closed, with his head slumped sideways across his chair.

  Perhaps this was due to bad video editing. It has also been reported that the facilitated communicator was able to correctly identify objects shown only to Mr Houben in private, although that is a less taxing task than the very rapid one-fingered typing shown on TV. But all of these claims can only be assessed in the context of the overwhelmingly negative research on FC.