I Think You'll Find It's a Bit More Complicated Than That
Page 33
Did New Scientist finally give it up? No. ‘New Scientist can still provide no definitive proof of Wightman’s claims, but looks forward to a return visit when the complete ChatNannies software is available for testing.’ Please. Did it ask Wightman about his claim to have a seven-figure offer from an American corporation which had ‘full independent testing performed on the AI and are confident of its validity and effecacy [sic]’? He was, apparently, quite capable of giving them a proper demonstration. Did it quiz Wightman on his previous false claims about writing software, or any of the other issues Bad Science raised? No. To those of us brought up loving the great institution of New Scientist this is – as Tibor Fischer said in that famous book review – a bit like bouncing out of the classroom at breaktime, only to catch your favourite uncle masturbating in the school playground.
BOOKENDS
Be Very Afraid: The Bad Science Manifesto
Guardian, 3 April 2003
It was the MMR story that finally made me crack. My friends had always seemed perfectly rational: now, suddenly, they were swallowing media hysteria hook, line and sinker. All sensible scientific evidence was twisted to promote fear and panic. I tried to reason with them, but they turned upon me: I was another scientist trying to kill their baby.
Many of these people were hardline extremists, humanities graduates, who treated my reasoned arguments about evidence as if I was some religious zealot, a purveyor of scientism, a fool to be pitied. The time had clearly come to mount a massive counter-attack.
Science, you see, is the optimum belief system: because we have the error bar, the greatest invention of mankind, a pictorial representation of the glorious, undogmatic uncertainty in our results, which science is happy to confront and work with. Show me a politician’s speech, or a religious text, or a news article, with an error bar next to it.
And so I give you my taxonomy of bad science, the things that make me the maddest. First, of course, we shall take on duff reporting: ill-informed, credulous journalists, taking their favourite loonies far too seriously, or misrepresenting good science, for the sake of a headline. They are the first against the wall.
Next we’ll move on to the quacks: the creationists, the new-age healers, the fad diets. They’re sad and they’re lonely. I know that. But still they must learn. Advertisers, with their wily ways, and their preposterous diagrams of molecules in little white coats: I’ll pull the trigger.
And the same goes for the quantum spin on government science. I’m watching you all.
And finally, let us not forget the strays, the good scientists who have passed to the dark side. Was it those shares in that drug company, or the lust for fame and glory? Bad scientists, your days are numbered.
If you are a purveyor of bad science, be afraid. If you are on the side of light and good, be vigilant; and for the love of Karl Popper, email me every last instance you find of this evil. Only by working joyously together can we free this beautiful, complex world from such a vile scourge.
What Eight Years of Writing the Bad Science Column Has Taught Me
Guardian, 4 November 2011
I’ve got to go and finish a book: I’ll be back in six months, but in case it kills me, here’s what I’ve learned in eight years of writing this column.
Alternative therapists don’t kill many people, but they do make a great teaching tool for the basics of evidence-based medicine, because their efforts to distort science are so extreme. When they pervert the activities of people who should know better – medicines regulators, or universities – it throws sharp relief onto the role of science and evidence in culture. Characters from this community who wonder why people keep writing about them should look at their libel cases and their awesomely bad behaviour under fire. You are a comedy factory. Don’t go changing.
Next: the real story of how the world works is much weirder than anything a quack can make up. The placebo effect is maddening, the nocebo effect more so, but the research on how we make decisions, and are misled by heuristics and mental shortcuts, is the wildest of all. Knowing about these belief-hacks gives you thrills, and power.
Pharmaceutical companies can behave dismally. Most important, they still won’t publish all the results of all the clinical trials conducted on humans. This is indefensible, and because we tolerate it, we don’t know the true effect sizes of the medicines that we give. This absurd situation mocks the whole of medicine: we need legislation to fix it, and popular movements to drive that. I’ll join yours.
Journalists can mislead the public about the answers of evidence-based medicine, which is bad. But they also mislead us on the methods and techniques. We live in a new era of doctors and patients – at our best – making decisions together. For that collaboration to work, everyone needs to understand how we know if something is good for us, or bad for us. The basics of evidence-based medicine, of trials, meta-analyses, cohort studies and the like should be taught in schools and waiting rooms. It’s interesting, but it’s also life and death: people care about it.
Politicians misuse evidence, and distort it to shameful degrees. But more than that, there are endless cases where we could do randomised trials on policies – old and new – to find out if they achieve the outcomes they’re aiming for. There is no honourable excuse for failing to use the fairest tests we can design.
Real scientists can behave as badly as anyone else. Science isn’t about authority, or white coats, it’s about following a method. That method is built on core principles: precision and transparency; being clear about your methods; being honest about your results; and drawing a clear line between the results, on the one hand, and your judgement calls about how those results support a hypothesis. Anyone blurring these lines is iffy.
Conflict-of-interest stories – where someone has a vested interest in the results of their study – are important, because they tell you when there’s a risk that something’s wrong in a piece of science. But this is only motive: the gruesome, fascinating mechanism of a crime against science – the methodological flaws – that’s where the action is. People who don’t really understand science can only critique it in terms of motive. Let them have that; we’ll do the details.
Last, nerds are more powerful than we know. Changing mainstream media will be hard, but you can help create parallel options. More academics should blog, post videos, post audio, post lectures, offer articles, and more. You’ll enjoy it: I’ve had threats and blackmail, abuse, smears and formal complaints with forged documentation.
But it’s worth it, for one simple reason: pulling bad science apart is the best teaching gimmick I know for explaining how good science works. I’m not a policeman, and I’ve never set out to produce a long list of what’s right and what’s wrong. For me, things have to be interestingly wrong, and the methods are all that matter.
So keep the nonsense coming, I’ll see you next year for more, and if you miss me, I’ll be procrastinating at badscience.net, and @bengoldacre on Twitter.
I haven’t yet gone back to a weekly column. The things I write take a huge amount of time: not just for the research, but also from chasing down endless blind alleys to find that one shining gem of bad behaviour that can illustrate an interesting bit of science.
To reassure you, I haven’t been smoking dope on the sofa. Over the past three years I’ve written Bad Pharma, and pushed hard on the policy fallout that it triggered (which you can read about in the updated second edition). I’ve fallen back in love with seeing patients; dived headlong into teaching; had exciting, busy day jobs; done hundreds of talks; put fingers into pies; and fallen into lobbying and policy work on the changes I’ve advocated for (which you can read about in the last book and, especially, the next one). Lastly, I’ve become obsessed with the technical fun of shaping a big argument over a book rather than 700 words: there are two of these to come very soon. Also, we’ve had babies.
I am extremely optimistic about the growing role of science in society. There is still an endless stream of nonsense
in mainstream media, politics, and medicine: but the last ten years have seen a spectacular flourishing of science, in live shows, and most importantly online. Rebuttals, fact-checks and evidence can be thrown onto Twitter or blogs within minutes of a dubious claim being made, and anyone can get access to good quality information, wherever they’re motivated.
If you are a purveyor of bad science: the mountain of bullshit is slowly shifting beneath your feet. You should be worried.
And if you are a nerd: keep talking, stay vigilant, and be proud.
This is our time.
Footnotes
Nullius in Verba
1 This column uses the example of some work by the Guardian’s health correspondent to illustrate the importance of transparency about research methods, as well as results. This is a growing issue, as raw data and the tools for analysis have become more accessible (excitingly) – and more widely used – outside of traditional academia. It is the only column by me that the Guardian has ever declined to publish.
How Myths Are Made
1 See here.
Publish or Be Damned
1 Where there are updates, occasionally, throughout the book, they are after the piece in italics.
Pink, Pink, Pink, Pink. Pink Moan
1 Unless you have an Athens login, you are not allowed to read what the researchers actually said, instead of what the media said they said. Because although they are publicly funded academics at the University of Newcastle, and although this work has been publicised in every major mainstream media outlet in Britain and the US, and although the journal is edited by academics you fund, and paid for by subscriptions from university libraries … the actual academic article is behind a paywall, with a payment model geared towards institutions, rather than interested individuals. Bad luck you. I guess you have to rely on journalists.
‘Hello Madam, Would You Like Your Children to Be Unemployed?’
1 See here.
Confound You!
1 In case it’s been puzzling you, epidemiologists use ‘odds’ (e.g. 366 ÷ 2300 for the top row of Table 1) rather than ‘proportions’ (which would be 366 ÷ 2666 for the top row of Table 1) because odds work more neatly when you use ‘logistic regression’, which is the more advanced technique mentioned above. If you’re interested to know more, I recommend coming to London School of Hygiene and Tropical Medicine to do our MSc in Epidemiology.
Bicycle Helmets and the Law
1 This is an editorial I wrote for the British Medical Journal with David Spiegelhalter about the complex, contradictory mess of evidence on the impact of bicycle helmets. Like most places where there’s controversy and disagreement, this is a great opportunity to walk through the benefits and shortcomings of different epidemiological techniques, from case-control studies to modelling. Epidemiology is my day job – Bad Science and Bad Pharma are both, effectively, epidemiology textbooks with bad guys – and since the techniques of epidemiology are at the core of most media stories and squabbles on health, it’s very weird that you don’t hear the word more often.
Medical Hypotheses Fails the Aids Test
1 See here.
2 See here.
Building Evidence into Education
1 Writing is just a hobby, alongside seeing patients, doing research, and – increasingly – putting these ideas into practice through campaigning and lobbying. In 2011 I co-authored a Cabinet Office White Paper explaining how randomised controlled trials can be used to improve government policy (this is the subject of my next book). And in 2012 I did an Independent External Review for the Department for Education, looking at what could be done to improve the use of evidence and data in schools. This was a dry internal report, but I was also asked to write something to explain what these changes might look like, aimed specifically at teachers, which is reproduced here. If you like it, and want to share it, there’s a PDF on the DfE website. I feel fairly hopeful, but these are long, slow cultural shifts. In June 2014 DfE advertised a public tender to assess progress towards the goals I set out in the internal report; so we shall see.
A Rock of Crack as Big as the Ritz
1 I chose titles on my blog spontaneously when I posted the pieces. ‘A Rock of Crack as Big as the Ritz’ is the title of a short story by Will Self.
Heroin on Prescription
1 There might be other explanations; I was only a medical student – BG, 2014.
The Noble and Ancient Tradition of Moron-Baiting
1From the preface to the second edition: ‘The first edition of this book prompted many curious letters from irate readers. The most violent letters came from Reichians, furious because the book considered orgonomy alongside such (to them) outlandish cults as dianetics. Dianeticians, of course, felt the same about orgonomy. I heard from homoeopaths who were insulted to find themselves in company with such frauds as osteopathy and chiropractic, and one chiropractor in Kentucky “pitied” me because I had turned my spine on God’s greatest gift to suffering humanity. Several admirers of Dr. Bates favoured me with letters so badly typed that I suspect the writers were in urgent need of strong spectacles. Oddly enough, most of these correspondents objected to one chapter only, thinking all the others excellent.’
Why Don’t Journalists Mention the Data?
1 The thirty-seventh study was released in the fortnight between the previous article and this one.
Empathy’s Failures
1 In 2014 Rolf Harris was found guilty of several sexual offences against children and jailed for over five years. I’ve left this piece in, partly as a reminder that abusers don’t always have the word ‘monster’ tattooed on their forehead. That phrasing comes from PinkZapCat, when I asked on Twitter about deleting the column with Rolf in it. I think it’s extremely wise.
MMR: The Scare Stories Are Back
1 Although it wasn’t adequately recognised at the time, I feel I should take some credit for getting ‘shit head’, ‘fuck yourself’, ‘twathead’ and ‘twat’ into a top-ten academic journal. The gauntlet is down.
Who’s the Daddy?
1 After this column, Theodore Gray bought some five-gram chunks of caesium and rubidium – much more than Brainiac didn’t use – and threw them into water in his garden. In the video, you can see some light, some pinging, and some phutting, but no explosion. Why not? Although caesium and rubidium are technically more reactive than sodium, he explains, in reality you get a bigger bang for your money from sodium: the atoms are smaller, so you get more atoms per gram, so the same-sized lump makes more hydrogen. ‘Under typical night-time escapade conditions, the larger hydrogen explosion created by sodium more than makes up for the more vigorous initial decomposition reaction of caesium. It’s a pity that Brainiac felt they needed to perpetuate a myth by faking it, when the truth is even better: common everyday sodium beats out those high-priced exotic elements.’
How I Stalked My Girlfriend
1 I still get a handful of emails every year from creepy men asking me how to do this.
Notes
HOW SCIENCE WORKS
Why Won’t Professor Susan Greenfield Publish This Theory in a Scientific Journal?
Why Won’t Professor: http://www.badscience.net/2011/11/why-wont-professor-greenfield-publish-this-theory-in-a-scientific-journal/
announced that computer games: http://www.thesun.co.uk/sol/homepage/woman/health/health/3871474/Computer-games-are-giving-kids-dementia.html
dementia in children: http://www.dailymail.co.uk/health/article-2049040/Computer-games-leave-children-dementia-warns-neurologist.html
not really what she meant: http://beefjack.com/news/the-sun-misrepresented-scientist-in-games-dementia-article/
rise in autism diagnoses: http://www.newscientist.com/article/mg21128236.400-susan-greenfield-living-online-is-changing-our-brains.html
then pulled back: http://www.dailymail.co.uk/sciencetech/article-2023535/Battle-dons-internet-link-autism-scientists-claim-PCs-shorten-attention-span.html
autism charities: http:/
/www.guardian.co.uk/society/2011/aug/06/research-autism-internet-susan-greenfield?CMP=twt_fd
Oxford professor of psychology: http://deevybee.blogspot.com/2011/08/open-letter-to-baroness-susan.html
They seem changeable: http://www.guardian.co.uk/science/the-lay-scientist/2011/aug/08/1
derided in the media as sexist: http://www.thisislondon.co.uk/standard/article-23793960-the-male-rage-that-is-a-bad-rap-for-science.do
Professor Greenfield responded: http://www.guardian.co.uk/society/2011/aug/06/research-autism-internet-susan-greenfield?CMP=twt_fd
Cherry-Picking Is Bad. At Least Warn Us When You Do It
Cherry-Picking: http://www.badscience.net/2011/09/cherry-picking-is-bad-at-least-warn-us-when-you-do-it/
Aric Sigman: http://www.guardian.co.uk/commentisfree/2011/sep/14/daycare-cortisol-levels-children?INTCMP=SRCH
Professor Dorothy Bishop: http://deevybee.blogspot.com/2011/09/how-to-become-celebrity-scientific.html
Sigman himself admits it: http://www.badscience.net/2011/09/2009/02/the-evidence-aric-sigman-ignored/
pdf on his website: http://www.aricsigman.com/IMAGES/Statement.pdf