But 'chance' does not mean, in the ordinary meaning of these words, spread out, or shared, or messy. It does not mean what we would think of as disordered, or without the appearance of pattern. It does not, strictly speaking, even mean unpredictable, since we know such things will happen, we just don't know where or when; and chance certainly does not mean without cause. All those cancers had causes, but they are likely to have been many and various and their concentration no more than the chance result of many overlapping and unrelated causes. It is predictable that there will be cancer clusters in a country the size of the UK, we just don't know where, and though it is a surprise when they appear, so is six consecutive tails in thirty tosses. Both ought to be expected.
Following a programme in the More or Less series on BBC Radio 4 about randomness and cancer clusters, in which we nervously described to one of the Wishaw villagers active in a campaign against the phone mast how clusters can occur – a woman who herself had lived with cancer – we received an email from a ferociously angry listener. How dare we, he said, take away their hope?
Chance is heartless. 'As flies to wanton boys are we to the Gods,' said Shakespeare's King Lear, 'they kill us for their sport.' More comforting amidst hurt and distress, maybe, to find a culprit you can hope to hold accountable, or even destroy.
When the surgeon and academic Atul Gawande wrote in the late 1990s about why cancer clusters in America were seldom the real thing, he quoted the opinion of the Chief of California's Division of Environmental and Occupational Disease Control that more than half the state's 5,000 districts (2,750 in fact) had a cancer rate above average. A moment's reflection tells us that this is more or less the result we should expect: simply put, if some are below average, others must be above. With some grains spread out, others must be squeezed closer together. The health department in another state – Massachusetts – responded to between 3,000 and 4,000 cancer cluster alarms in one year alone, he said. Almost never substantiated, they nevertheless have to be investigated: the public anxiety is genuine and cannot be fobbed off, even if with the weary expectation of finding nothing. And these findings are not necessarily from reluctance by the medical or public health authorities to admit a problem. It is – usually – no conspiracy to suppress inconvenient facts, as the willingness to detect cancers in other ways shows. Unlike geographical clusters, discoveries of real occupational cancer clusters, or clusters of illness around a single life event – exposure to a drug or chemical for example – have been common and readily divulged. The inconvenience of the malign effects of asbestos and tobacco has not prevented the medical authorities from exposing them, despite the resistance of those industries.
Even apart from those hooked by conspiracy theories, we are all gifted recognisers of patterns and the moments when a pattern is interrupted. Something chimes or jars with an instinctive aesthetic for regularity, and we are inclined to look for causes or explanations. Whatever stands out from the crowd, or breaks from the line, invites us to wonder why. Some argue, plausibly, that we evolved to see a single cause even where there is none, on the basis that it is better to be safe than sorry, better to identify that pattern in the trees as a tiger, better to run – far better – than to assume that what we see is a chance effect of scattered light and shifting leaves in the breeze, creating an illusion of stripes. But this habit, ingrained, defiant, especially if indulged by a snap decision, makes us wretched natural statisticians. Most often the pattern will be a chance effect, but we will struggle to believe it. 'Where's the tiger?' we say. 'No tiger,' says the statistician, just chance, the impostor, up to her callous old mischief. In more considered moments, now we have moved on from evolution in the jungle, we should remember our experience of chance, and check the instinct born in moments in haste.
One reason for the punishing cost of medical research is that new drugs have to be trialled in ways that seek to rule out the effect of chance on whether or not they work. You might wonder how hard that can be: administer a drug to a patient and wait to see if the patient gets better, surely, is all there is to it. The joke that a bad cold treated with the latest drugs can be cured in seven days, but left to run its own course may linger for a whole week, shows us what is wrong with that. Patients improve – or not – for many reasons, at different rates. Say we put them in two groups, give one group the drug and another a placebo, and observe that the drugged group improves more. Was it the drug, or chance? Ideally we'll see a big difference between two large groups. But if the difference is small, or few people take part, it's a problem. Like rice grains or cancer cases, some patients in a medical trial will produce results that look meaningful, but are in fact quite unrelated to the influence everyone suspects – the new drug, the phone mast – and everything to do with chance.
Statistics as a discipline has made most of its progress only in the last 200 years. Perhaps the reason it took so long to get started, when science and mathematics had already achieved so much, is that it is a dry challenge to instinct. Particularly in relation to patterns, chance or coincidence, statistics can feel counter-intuitive when it frustrates a yearning for meaning.
'The tiger that isn't' makes a good standard for numbers that seem to say something important but might be random, and there are imaginary tigers in all walks of life. Our question could be this: is the tiger real? Or are we merely seeing stripes? Is this a pattern of numbers that tells us something, or a purely chance effect that bears only unsettling similarity to something real?
Like illness, events cluster too. In 2005 three passenger airliners came down in the space of a few weeks, leading to speculation of some systemic problem – 'What is causing our planes to fall?' To repeat, chance does not mean without cause – there was a cause for each of those crashes – just separate causes. What chance can do is to explain why the causes came together at the same time, why, in effect, they clustered.
Does this prove that every cluster, cancer or otherwise, is chance alone? Of course not, but we have to rule out that explanation before fastening on another. People in suits seen on the news advising disbelieving residents that their fears are unfounded might be part of a conspiracy against the public interest – it is conceivable – but let's also allow that they speak from acquaintance with what chance is capable of, and have worked honestly to tell stripes from the tiger. The difference often comes down to nothing more than size. One case of a real cluster, in High Wycombe, of a rare type of nasal cancer with a genuinely local cause, was eventually attributed to the inhalation of sawdust in the furniture industry, and had prevalence 500 times more than expected. And downwind of the Chernobyl nuclear power station, there are a large number of thyroid cancers, far more than even chance might cause.
Whenever we see patterns or clusters in numbers, whenever they seem to have order, we're quick to offer explanations. But the explanation most easily overlooked is that there is no explanation; it was just chance. That doesn't mean everything that happens is due to chance, of course. But in numbers, we need to be every bit as alert for phantom tigers as for real ones.
A shocking example comes with the two innocent doctors who felt confronted – on nothing more than the turn of a roulette wheel – with an insinuation of murder.
It has happened. Two general practitioners, summoned to a meeting, sat tense and anxious, trying to explain the unusual death rates of their patients. The meeting took place in the shadow of Dr Harold Shipman – the recently convicted Yorkshire GP who murdered probably well in excess of 200 people. An inquiry into the Shipman case, led by Dame Janet Smith, was trying to establish whether GP patient mortality could be monitored in order to spot any other GPs who gave cause for concern. These two doctors had been identified from a sample of about 1,000 as among a dozen whose patients were dying at rates as high as, or higher than Shipman's.
An initial statistical analysis had failed to find an innocent explanation. What other remained, except for quality of care – for which read the hand of the two GPs? This was wher
e Dr Mohammed Mohammed came in. He was one of those asked to investigate. 'This was not a trivial meeting', he told us. One can imagine. It was deeply uncomfortable, but necessary, he added, in order to explore any plausible alternatives. He was not about to throw at anyone a casual accusation of murder, but they had all read the newspapers. A softly spoken, conscientious man, determined to proceed by scientific method, he wanted to know before coming to a conclusion if there were any other testable hypotheses.
The statistical analysis that brought the GPs to this point had sought to sift from the data all variation due to chance or any unusual characteristics of their patients: the ordinary rise and fall in the numbers dying that would be expected anywhere, together perhaps with an unusually elderly population in the area, or some other characteristic in the case-mix that made deaths more frequent, were obvious explanations. But these were examined and found wanting. Now it was the GPs' turn.
And the reason they offered, essentially, was chance; chance missed even by a statistical attempt to spot and control for it. No, their patients were not significantly older than others, but the GPs did have, by chance, an unusually high number of nursing homes on their patch. And while age alone is often a good proxy for frailty or increased illness, a nursing home is much better. People here are most likely to be among the most frail or in poor health.
We know without doubt that there was nothing accidental about Shipman, but the variation of death rates, even variation on a par with mass murder, can have a perfectly innocent explanation, as chance ruffles the smoothness of events, people and places, changing a little here and there, bringing a million and one factors into play, suggesting vivid stripes without a tiger to be seen.
To see the problem of telling one from another, think of your signature. It varies, but not too much, and what variation there is worries no one, it is normal and innocent. Try with the other hand, though, and the degree of variation probably jumps to the point where you hope that on a cheque the bank would reject it. The task in detecting a suspect pattern is to know where the normal chance variation stops, and what is known as 'special cause' variation begins. How varied can a signature be and still be legitimate? How much variation in mortality rates can there be before we assume murder? The answer is that innocent or chance variation can exceed even the deliberate effect of Britain's most prolific serial killer.
The GPs' explanation turned out to be entirely consistent with the data, once examined closely, mortality rates at the nursing homes being perfectly in line with what would be expected – for nursing homes. They were cleared completely, but hardly happy. If the sample of GPs that identified these two was anything to go by, we would expect to find about 500 across the UK with mortality rates similarly inviting investigation. That is an awful lot of chance at work, but then, chance is like that: busy – and cunning, and with about 40,000 GPs to choose from in the UK, some would certainly be unlucky even if none was murderous.
Dame Janet Smith said in her final report from the Shipman inquiry: 'I recognise, of course, that a system of routine monitoring of mortality rates would not, on its own, provide any guarantee that patients would be protected against a homicidal doctor. I also agree with those who have emphasised the importance, if a system of routine monitoring is to be introduced, of ensuring that PCTs (Primary Care Trusts), the medical profession and the public are not lulled into a false sense of security, whereby they believe the system of monitoring will afford adequate protection for patients.'
Aware of these difficulties, she recommended GP monitoring all the same, but justified as much by the value of the insight it might offer into what works for patient care – a view shared by Dr Mohammed – as for detecting or deterring murder. Can we really not tell chance death from deliberate killing? With immense care, and where the numbers are fairly clear-cut, as in the Shipman case, we can, just about. But he was able to avoid detection for so long because the difference was not instantly obvious then and, with all our painfully learned statistical wisdom since, the chilling conclusion is that we are only a little better at spotting it now, because chance will always fog the picture. It was in February 2000, shortly after Shipman's conviction, that the then Secretary of State for Health, Alan Milburn, announced to Parliament that the department would work with the Office of National Statistics 'to find new and better ways of monitoring deaths of GPs' patients'. This promise was still unfulfilled seven years later. That is a measure of the problem – and testimony to the power of chance.
When statisticians express caution, this is often why: chance is a dogged adversary. And yet it can be overcome. Trials for a new polio vaccine in America and Canada in the 1950s had to meet persistence with persistence. At that time, among 1,000 people, the chances were that none would have polio. It always was quite rare. So let's say a vaccine comes along and it is given to 1,000 people. How do we know if it worked? Chance would probably have spared them anyway. How can we tell if it was the vaccine or chance that did it?
The answer is that you need an awful lot of people. In the trials for the Salk polio vaccine, nearly 2 million children were observed in two types of study. The number who for one reason or another did not receive the vaccine, either because they were in one of the various control groups, or simply refused, was 1,388,785. Of these, 609 were subsequently diagnosed with polio, a rate of about one case in every 2,280 children.
Among the nearly half million who were vaccinated, the rate was about one case in nearly 6,000 children. The difference was big enough, among so many people, that the research teams could be confident they had outwitted chance's ability to imitate a real effect. Though even here they looked closely to ensure there were no chance differences between the groups that might have accounted for different rates of infection. Getting the better of chance can be done, with care and determination, and often involves simply having more stamina or patience than she does.
There is more chance about than many of us think.
4
Up and Down:
A Man and His Dog
Brace yourself for a radical fact, a fact surprising to much political debate, capable of wrecking the most vaunted claims of government. Ready? Numbers go up and down.
That's it. No one has to do anything special to cause this. No policy is necessary. In the ordinary course of life, things happen. But they do not often happen with absolute regularity or in the same quantities. Measure (almost) anything you like and on some days there are more, some days fewer. The numbers rise and fall. They just do.
Of course, you knew this. You might think those in authority know it too. But do they? 'We did that!' they shout, whenever the numbers move in a favourable way, apparently oblivious to the possibility that the numbers would have gone up or down anyway.
Mistaking chance ups and downs for the results of new policies can have dire consequences: it means failure to understand what really works, spending money on things that don't, ignoring what does.
To guard against the problem, think of a man walking uphill with a dog on a long, extendable lead. You can't see the man, it's dark, but the dog's collar is fl uorescent, so you watch as it zips up and down, stops and switches. How do you know for sure if the man is heading up, down or sideways? How do you know how fast? When the dog changes direction, has the man changed too? Whenever you see numbers go up and down, always ask: is this the man, or the dog?
Hit the heights in sport or business in America and you might make the cover of those illustrious magazines, Business Week or Sports Illustrated. That is, unless you are also superstitious, when you might scramble to be overlooked.
Sudden shyness in people who bathe in celebrity demands an explanation, and so there is. These magazines (here put on your eye-patch and best piratical voice), be cursed. If you believe in such things – and plenty do – then any who dares appear on the cover invites fickle fortune to turn her wheel with a vengeance.
It is true that a surprising number of people or businesses featured this way real
ly do tend to find success suddenly more elusive: firms with a Midas touch lose it, the brightest athletic stars are eclipsed. It is known, in the now famous case of Sports Illustrated, as the SI Cover Jinx.
The magazine itself reported on this in 2002: 'Millions of superstitious readers – and many athletes – believe that an appearance on Sports Illustrated's cover is the kiss of death,' it said, finding 913 jinxes in 2,456 issues 'a demonstrable misfortune or decline in performance following a cover appearance roughly 37.2 per cent of the time.' Eddie Matthews was a major league baseball player who graced the first cover in 1954. A week later, he injured a hand and missed seven games. There was the champion skier who broke her neck, numerous winning streaks ended by humiliation to lesser rivals, and many other examples of abrupt bad luck.
Spooky? Preferring statistics to magic, we don't think so. Our scepticism rests on a simple, subversive principle: things go up and down.
Seeking explanation for change, it is easy to over-interpret ordinary ups and downs and attribute them to some special cause – such as a jinx. No magic is required, just the knowledge that a season of spectacular goals is seldom beaten next year, or that the skier who takes most risks might be closest to victory, but also to disaster. If you have been 'up' sufficient to make the cover of a celebrated magazine, it could be that you are at your peak and, from the peak, there is only down. When the dog reaches the end of the leash, it often runs back. And that, as we say, is it. The jinx is in all probability due to what stat-isticians call regression to the mean. When things have been out of the ordinary lately, the next move is most likely back to something more average or typical; after a run of very good luck, chance might be ready for revenge, whether you appear on the cover or not.
The Tiger That Isn't Page 5
