by Ben Goldacre
Perneger et al. (1998) have noted that although the Swiss trial was small, it was similar to the initial evaluations of methadone, such as the seminal paper by Dole and Nyswander (1965), which led to its widespread use in the treatment of drug addiction. It seems likely that a contributory factor was the medical profession’s emotional and moral attitudes towards drug users.
However noble our intentions when we approach a clinical or social problem, we may often be confounded by extraneous factors and preconceptions, and fail in our objectivity. We share an obligation to submit all medical interventions to rigorous, continuous and objective reassessment. Drug addiction affects 100,000 people in Britain directly, and many more indirectly; it is responsible for an enormous drain on health-care resources, a large proportion of acquisitive crime, and the fastest-growing group of HIV infection. That we should apparently neglect our obligations in such an important field is astonishing.
LIBEL
NMT Is Suing Dr Peter Wilmshurst. So How Trustworthy Is This Company? Let’s Look at Its Website . . .
Guardian, 11 December 2010
You will hopefully remember – from the era before WikiLeaks – that US medical-device company NMT is suing NHS cardiologist Peter Wilmshurst over his comments about the conduct and results of the MIST trial, which sadly for NMT found no evidence that their device prevents migraine. The MIST trial was funded by NMT, and Wilmshurst was lead investigator until problems arose.
Wilmshurst has already paid £100,000 of his own money to defend himself, risking his house, and has spent every weekend and all his annual leave, unpaid, dealing with this, at great cost to his family. So what kind of a company is NMT Medical, which the British libel courts have allowed to hound one man for almost two years? And how trustworthy are its utterances?
Let’s go to its website and find out. On the front page you will see positive quotes from three patients prominently displayed, on a rotating banner (reload the page to see the full collection), accompanied by smiling studio photographs.
At this point we should remember that the outcome of the MIST trial – the study in question here – really was negative. It set out to see if the device permanently prevents migraine. One hundred and forty-seven patients with migraine took part. Seventy-four had the NMT STARFlex device implanted, seventy-three had a fake operation with no device implanted, and three people in each group stopped having migraines. The NMT STARFlex device made no difference at all. This is not a statement of opinion, and there are no complex statistics involved.
This might also be a good point to mention that the journal Circulation had to publish a lengthy correction for the MIST trial because the original paper failed to mention, for example, that Wilmshurst had declined to be listed as an author over concerns about how the study was conducted; that two of the devices were lost in patients’ bodies during the procedure (one embolised to the right atrium, one to the left pulmonary artery, both worrying, both were luckily able to be retrieved); and so on.
Back to NMT’s three positive case studies with their smiling studio photographs. They were all (it explains in the 2005 NMT annual report) treated with the STARFlex device in the MIST trial. Jean Richards says: ‘I feel so much better now. I don’t live in fear of a migraine coming on all the time.’ Zoe Willows says: ‘People at my new job have never known me to have a migraine. I’m a totally different person.’
There are several problems here. Firstly, two of these patients, it seems, are advertising devices they were not treated with. Jean is smiling and advertising CardioSEAL, a successor to the failed STARFlex device, although she was not treated with CardioSEAL; and Liz is advertising BioSTAR, but she was not treated with BioSTAR. I asked NMT why these patients were advertising products with which they were not treated. NMT declined to answer.
Secondly, the patients’ anecdotal experiences are entirely misleading: the MIST trial was negative (though I can find no mention of its final results anywhere on the NMT site, which is odd, because it’s the only published trial I’m aware of that tests whether NMT’s device prevents migraine).
But lastly, the protocol for the MIST trial, as is standard, states that the sponsoring company is not supposed to have access to individual patients. How did NMT get hold of these patients?
I tried to contact Dr Michael Mullen, previously of the Royal Brompton Hospital, now of University College London Hospitals, cardiologist on the MIST trial, to see if he knew how these patients hit the public domain, since the RBH website has a page – hurriedly removed since I contacted them – stating that the MIST trial results were positive (these appear to be initial results, from before the final paper was published), and also featuring the patient Zoe Willows saying ‘I’ve now been completely cured.’ Dr Mullen himself appears in a smiling studio shot on the NMT website, and in 2008 declared owning shares in NMT. I invited him to criticise NMT’s use of misleading patient anecdotes. He declined. I asked if he knew how the company got hold of the patients, or how these positive results appeared on the RBH website. He said he could not remember.
So I asked NMT. It told me that all three patients got in touch with the medical-device company themselves, spontaneously. I asked NMT if the three patients whose migraines stopped after the fake operation had also got in touch to express their gratitude, because they might be able to provide useful and less misleading anecdotes. NMT declined to answer.
I could then have asked Dr Andrew Dowson, the new lead investigator on MIST, whose licence to practise was restricted by the GMC at the time of the MIST trial, as he had been found guilty of research misconduct in an earlier clinical trial. But by then I was exhausted, and not sure it was worth it.
Meanwhile, NMT’s share price has fallen from $20 to 20 cents over four years, perhaps unsurprisingly after the negative results of the MIST trial. A judge has now insisted that the company puts £200,000 into a UK account in case it loses its libel case against Dr Wilmshurst, or the case will be struck out next month, but NMT’s solicitor argues that the company’s financial situation is ‘dire’. This suggests that even if Dr Wilmshurst successfully defends himself, he may never get his £100,000 back. I’m not convinced that a libel law which allows a company like NMT to do this to one man is in society’s best interests.
‘We Are More Possible Than You Can Powerfully Imagine’
Guardian, 29 July 2009
Today the Australian magazine Cosmos, along with a vast number of other blogs and publications, reprinted an article by Simon Singh, in slightly tweaked form, as an act of solidarity. The British Chiropractic Association has been suing Singh personally for the past fifteen months, over a piece in the Guardian in which he criticised the BCA for claiming that its members could treat children for colic, ear infections, asthma, prolonged crying, and sleeping and feeding conditions, by manipulating their spines.
The BCA maintains that the efficacy of these treatments is well documented. Singh said the claims were made without sufficient evidence, described the treatments as ‘bogus’, and criticised the BCA for ‘happily promoting’ them. At a preliminary hearing in May to decide the meaning of this article, Mr Justice Eady ruled that Singh’s wording implied that the BCA was being deliberately dishonest. Singh has repeatedly been clear that he never intended this meaning, but has been forced to defend this single utterance, out of his own pocket, at a cost that has run to six figures.
Soon we will get to the story of the backlash, but first, while you may view this as a free-speech issue, there are also some specific worries raised when people sue in medicine and science.
It is possible in healthcare to do great harm, while intending to do good, and so medicine thrives on criticism: this is how ideas improve, and therefore how lives are saved. The three most highly rated articles in the latest chart from the British Medical Journal are all highly critical of medical practice. Academic conferences are often bloodbaths. To stand in the way of ideas and practices being improved through critical appraisal is not just dangerous,
it is disrespectful to patients, and even if someone has been technically defamatory in their wording, it is plainly undesirable for all critical discourse in healthcare to be conducted in a stifling climate of fear. Neither the General Medical Council nor the British Medical Association has ever sued anyone for saying that their members are up to no good. I asked them. The idea is laughable.
But beyond whether it is right, there is the more entertaining issue of whether it was wise, and here it is hard to contain a sense of Schadenfreude as the chiropractors’ world unravels. First, there is the media exposure. This case, and the chilling effects of libel threats in science, have now been covered by The Times, the Daily Mail, the Daily Telegraph, the Independent, Nature, the Economist, Times Higher Education, the Sunday Times, Channel 4, the Financial Times, the Wall Street Journal, Private Eye, the Observer, the BBC, and an editorial in the British Medical Journal, to name just a few. This story has travelled around the world.
Most of these articles and programmes drew attention to the evidence for chiropractic’s efficacy, which is often not compelling. Some discussed chiropractic’s dubious origins: it was invented by a magnet therapist, convicted of practising medicine without a licence, who suddenly decided in 1895 that 95 per cent of all diseases are caused by displaced vertebrae, and compared himself to Christ, Mohammed and Martin Luther. Who knew?
An international petition against the BCA has been signed by professors, journalists, celebrities and more, with Ricky Gervais and Stephen Fry alongside the previous head of the Medical Research Council and the last government Chief Scientific Adviser. There have been public meetings, with stickers and badges. But it is a ragged band of science bloggers who have done the most detailed work. Fifteen months after the case began, the BCA finally released the academic evidence it was using to support its specific claims. Within twenty-four hours this was meticulously taken apart by bloggers, referencing primary research papers and looking in every corner.
Professor David Colquhoun of UCL pointed out, on infant colic, that the BCA cited weak evidence in favour of its claims, while ignoring strong evidence contradicting them. He posted the evidence and explained it. LayScience flagged up the BCA selectively quoting a Cochrane review. Every stone was turned by Quackometer, APGaylard, Gimpyblog, EvidenceMatters, Dr Petra Boynton, MinistryofTruth, Holfordwatch, legal blogger Jack of Kent, and many more. At every turn they have taken the opportunity to explain the principles of evidence-based medicine – the sin of cherry-picking results, the ways a clinical trial can be unfair by design – to an engaged lay audience, with clarity as well as swagger.
Then the formal complaints began. There have been successes with the Advertising Standards Authority, including one in which the ASA concluded that the BCA’s claims to treat colic breached the guidelines on ‘truthfulness’ and ‘substantiation’. This interested many, since treating colic was one of the claims over which the BCA sued Singh when he called it ‘bogus’.
Professional complaints followed in May, mostly about individual chiropractors’ claims. Then, in June, blogger Simon Perry found the BCA database of 1,029 members online, containing four hundred website URLs. He wrote a quick computer program to automatically identify all the chiropractors in the UK who claimed to treat colic, locate their local Trading Standards office, and report them (more than five hundred in total) automatically, followed up with printed letters.
Chiropractic is a profession regulated by the General Chiropractic Council, supervised by the Health Professional Council, which is obliged to investigate all complaints. So Perry reported over five hundred chiropractors to them, alleging that they had made claims without adequate evidence. The GCC rejected his letter, saying that it only considered individual complaints. A pile of individual complaint letters was instantly generated, printed, stuffed into envelopes, and delivered to its door. Astonishingly, ZenosBlog had done exactly the same thing. These thousand complaints are now being investigated.
You may view this as bullying individuals, and initially I had some sympathy for them. But my heart was hardened as a result of reading commentary from the chiropractic and alternative-therapy community saying that Singh must expect six-figure consequences for criticising them, and transgressing the letter of the law, even in just one article.
Some clue as to whether chiropractors feel able to defend these complaints about the evidence for their practices came a few days later. On 8 June the McTimoney Chiropractic Association sent a confidential email to its members, which has been obtained and is available in full on Quackometer. ‘If you have a website,’ this email begins, ‘take it down NOW … REMOVE all the blue MCA patient information leaflets, or any patient information leaflets of your own that state you treat whiplash, colic or other childhood problems in your clinic … IF YOU DO NOT FOLLOW THIS ADVICE, YOU MAY BE AT RISK FROM PROSECUTION. Finally, we strongly suggest you do NOT discuss this with others’ – and on this it was clear – ‘especially patients.’
The MCA says the complaints are a ‘vexatious campaign against the profession’, that it has nothing to hide, and that it believes its members have not intentionally breached any rules regarding their websites’ content. The entire MCA website disappeared on the same day, and continues to be nothing more than a holding page (it is ‘currently being updated’), but its former site, along with every single chiropractor’s website, has been archived in full online by the science blogging community, for anyone who is interested to look.
We could go on, but there are lessons from this debacle – beyond the ethical concerns over suing in the field of science and medicine – and they are clear. First, if you have more reputation and superficial plausibility than evidence to support your activities, then it may be wise to keep under the radar, rather than start expensive fights. But more interestingly than that, a ragged band of bloggers from all walks of life has, to my mind, done a better job of subjecting an entire industry’s claims to meaningful, public, scientific scrutiny than the media, the industry itself, and even its own regulator. It’s strange that this task has fallen to them, but I’m glad someone is doing it, and they do it very, very well indeed.
Science Is About Embracing Your Knockers
Guardian, 13 November 2010
If science has any credibility, it derives from transparency: when you make a claim about how something works, you provide references to experiments, which describe openly and in full what was done, in enough detail for the experiment to be replicated. You explain what was measured, and how. Then people can freely discuss what they think this all means in the real world.
Maria Hatzistefanis is a star of lifestyle pages and the owner of Rodial, a cosmetics company which sells a product called ‘Boob Job’, which it claims will give you a ‘fuller bust’, ‘increase the bust size’ and ‘plump up the décolleté area’ with ‘an instant lifting and firming effect’, and an increase of half a cup size in fifty-six days. Or rather, an increase of ‘8.4 per cent’. It’s all very precise.
Now, I’m not going to lose a great deal of sleep over anybody who buys a magic cream to make their breasts grow bigger. What worries me is that Maria Hatzistefanis’s company is now threatening a doctor with a libel case, simply for daring to voice doubts over these claims.
This is her crime. Dr Dalia Nield told the Mail it was ‘highly unlikely’ that the cream would make your breasts bigger, and questioned the amount of information provided by Rodial. ‘The manufacturers are not giving us any information on tests they have carried out. They are not telling us the exact ingredients in the product and how they increase the size of the breast.’
That’s fair. I don’t trust claims without evidence, especially not unlikely ones about a magic cream that makes your breasts expand. Maybe it does work – I don’t particularly care either way – but when I asked the company to send me any evidence it had, or any information on ingredients, it flatly refused to send me anything at all.
This is odd, since I’ve seen the letter that Rodial’s la
wyers sent, and they tell Dr Nield: ‘Our client on request would have provided all information required on clinical assessment and product ingredients.’
Apparently not.
Dr Nield went on to speculate that the gel could be ‘potentially dangerous … it may even harm the skin and the breasts – we need a full analysis’. Again, this is perfectly reasonable: anything that has real effects on the body may also have unintended side effects. That is an entirely uncontroversial statement, especially when important information is being withheld.
But then the story gets stranger. When Sense About Science, which has helped drive the campaign for libel law reform in the UK, put out a press release about Rodial threatening Dr Nield with libel, they themselves were contacted by Hegarty LLP, solicitors acting for Rodial Limited. This time they seemed to be trying to stop people from even daring to talk about the existence of their libel threat.
People often ask if there are short cuts in spotting nonsense. In reality, it’s not easy to do in a checklist, because there are so many elaborate ways to distort evidence, but for me there is one very clear risk factor. The entirety of science is built on transparency, on giving your evidence and engaging with legitimate criticism. If you hear of a company refusing to hand over the evidence it says supports its claims – whether it is a drug company, or some dismal cosmetics firm – all you know is that you are being deprived of information, and that vital parts of the picture are missing. If you hear that someone is threatening to sue their critics, again, all you know is that people will be intimidated from raising legitimate concerns, and again, you are being deprived of information.
