by Hannah Fry
With an algorithm like that, the category your biopsy fitted into would become far less important. You wouldn’t need to bother with why or how, you could just jump straight to the information that matters: do you need treatment or not?
The good news is, work on such an algorithm has already begun. Andy Beck, the Harvard pathologist and CEO of PathAI we met earlier, recently let his algorithm loose on a series of samples from patients in the Netherlands and found that the best predictors of patient survival were to be found not in the cancer itself but in other abnormalities in the adjacent tissue.29 This is an important development – a concrete example of the algorithms themselves driving the research forwards, proving they can find patterns that improve our powers of prediction.
And, of course, there’s now an incredible wealth of data we can draw on. Thanks to routine mammogram screening around the world, we probably have more images of breast tissue than of any other organ in the body. I’m not a pathologist, but every expert I’ve spoken to has convinced me that being able to confidently predict whether a troublesome sample will become cancerous is well within our sights. There’s a very real chance that by the time this book is published in paperback, someone somewhere will have actually made this world-changing idea a reality.
Digital diagnosis
These ideas apply well beyond breast cancer. The neural networks that Andy Beck and others are building don’t particularly care what they’re looking at. You could ask them to categorize anything: dogs, hats, cheeses. As long as you’re letting them know when they’re getting it right and wrong, they’ll learn. And now that this family of algorithms are good enough to be usable, they’re having an impact on all sorts of areas of modern medicine.
One big recent success, for instance, comes from the Google Brain team, who have built an algorithm that screens for the world’s biggest cause of preventable blindness – diabetic retinopathy. It’s a disease that affects the blood vessels in the light-sensitive areas of the eye. If you know you have it, you can be given injections to save your sight, but if it’s not caught early it can lead to irreversible blindness. In India, where access to experts capable of diagnosing the condition is limited, 45 per cent of people with diabetic retinopathy will lose some of their sight before they know they have the disease. The Google team’s algorithm, which was built in a collaboration with doctors from India, is now just as good at diagnosing the condition as a human ophthalmologist.
Similarly, there are algorithms that look for cardiovascular diseases in the heart,30 emphysema in the lungs,31 strokes in the brain32 and melanomas on the skin.33 There are even systems that diagnose polyps during a live colonoscopy in real time.
The fact is, if you can take a picture of it and stick a label on it, you can create an algorithm to find it. And you’re likely to end up with a more accurate (and possibly earlier) diagnosis than any human doctor could manage.
But what about the messier forms of medical data? Can the success of these algorithms be taken further, to something beyond these kinds of highly specialized, narrowly focused tasks? Could a machine search for meaning in your doctor’s scribbled notes, for instance? Or pick up on tiny clues in how you describe the pain you’re experiencing?
How about the ultimate healthcare science-fiction fantasy, where a machine in your doctor’s surgery would carefully listen to your symptoms and analyse your medical history? Can we dare to imagine a machine that has a mastery of every scrap of cutting-edge medical research? One that offers an accurate diagnosis and perfectly tailored treatment plan?
In short, how about something a little like IBM’s Watson?
Elementary, my dear
In 2004, Charles Lickel was tucking into a steak dinner with some colleagues in a New York restaurant. Partway through the meal, the dining area began to empty. Intrigued, Charles followed the crowd of diners and found them huddled around a television, eagerly watching the popular game show Jeopardy. The famous Jeopardy champion Ken Jennings had a chance to hold on to his record-breaking six-month winning streak and the diners didn’t want to miss it.34
Charles Lickel was the vice-president of software at IBM. For the past few years, ever since Deep Blue had beaten Garry Kasparov at chess, the IBM bosses had been nagging Charles to find a new challenge worthy of the company’s attention. As he stood in the New York restaurant, watching the diners captivated by this human Jeopardy champion, Lickel began to wonder if a machine could be designed to beat him.
It wouldn’t be easy. The machine known as ‘Watson’ that Charles imagined in that restaurant took seven long years to build. But eventually Watson would challenge Ken Jennings on a special episode of Jeopardy and convincingly defeat him at the game he’d made his own. In the process, IBM would set themselves on the path to their attempt to build the world’s first all-singing, all-dancing diagnosis machine. We’ll come back to that in a moment. But first, let me walk you through some of the key ideas behind the Jeopardy-winning machine that formed the basis of the medical diagnosis algorithms.
For those who’ve never heard of it, Jeopardy is a well-known American game show in the form of a kind of reverse general knowledge quiz: the contestants are given clues in the form of answers and have to phrase their responses in the form of questions. For instance, in the category ‘self-contradictory words’ a clue might be:
A fastener to secure something; or it could be to bend, warp & give way suddenly with heat or pressure.
The algorithmic player would have to learn to work through a few layers to get to the correct response: ‘What does “buckle” mean?’ First, Watson would need to understand language well enough to derive meaning from the question and realize that ‘fastener’, ‘secure’, ‘bend’, ‘warp’ and ‘give way suddenly’ are all separate elements of the clue. This, in itself, is an enormous challenge for an algorithm.
But that was only the first step. Next, Watson would need to hunt for potential candidates that fitted each of the clues. ‘Fastener’ might conjure up all manner of potential answers: ‘clasp’, ‘button’, ‘pin’ and ‘tie’, as well as ‘buckle’, for instance. Watson needs to consider each possibility in turn and measure how it fits with the other clues. So while you’re unlikely to find evidence of ‘pin’ being associated with the clues ‘bend’ and ‘warp’, the word ‘buckle’ certainly is, which increases Watson’s confidence in it as a possible answer. Eventually, once all of the evidence has been combined, Watson has to put its imaginary money where its metaphorical mouth is by choosing a single response.
Now, the challenge of playing Jeopardy is rather more trivial than that of diagnosing disease, but it does require some of the same logical mechanisms. Imagine you go to the doctor complaining of unintentional weight loss and stomach aches, plus a bit of heartburn for good measure. In analogy with playing Jeopardy, the challenge is to find potential diagnoses (responses) that might explain the symptoms (clues), look for further evidence on each and update the confidence in a particular answer as more information becomes available. Doctors call this differential diagnosis. Mathematicians call it Bayesian inference.fn1
Even after Watson the quiz champion had been successfully created, building Watson the medical genius was no simple task. None the less, when IBM went public with their plans to move into healthcare, they didn’t hold back from making grand promises. They told the world that Watson’s ultimate mission was to ‘eradicate cancer’,35 and hired the famous actor John Hamm to boast that it was ‘one of the most powerful tools our species has created’.
It’s certainly a vision of a medical utopia to inspire us all. Except – as you probably know already – Watson hasn’t quite lived up to the hype.
First a prestigious contract with the University of Texas M. D. Anderson Cancer Center was terminated in 2016. Rumour had it that even after they had shelled out $62 million on the technology36 and spent four years working with it, Watson still wasn’t able to do anything beyond heavily supervised pilot tests. Then, in late September 20
17, an investigation by the health news website STAT reported that Watson was ‘still struggling with the basic step of learning about different forms of cancer’.37
Ouch.
To be fair, it wasn’t all bad news. In Japan, Watson did manage to diagnose a woman with a rare form of leukaemia, when doctors hadn’t.38 And analysis by Watson led to the discovery of five genes linked to motor neurone disease, or ALS.39 But overall, IBM’s programmers haven’t quite managed to deliver on the promises of their excitable marketing department.
It’s hard not to feel sympathy for anyone trying to build this kind of machine. It’s theoretically possible to construct one that can diagnose disease (and even offer patients sensible treatment plans), and that’s an admirable goal to have. But it’s also really difficult. Much more difficult than playing Jeopardy, and much, much more difficult than recognizing cancerous cells in an image.
An all-purpose diagnostic machine might seem only a simple logical step on from those cancer-spotting image-based algorithms we met earlier, but those algorithms have a big advantage: they get to examine the actual cells that might be causing a problem. A diagnostic machine, by contrast, only gets information several steps removed from the underlying issue. Maybe a patient has pins and needles caused by a muscle spasm caused by a trapped nerve caused by too much heavy lifting. Or maybe they have blood in their stool caused by haemorrhoids caused by constipation caused by poor diet. An algorithm (or a doctor) has to take a single symptom and trace the route backwards to an accurate diagnosis. That is what Watson had to do. It is a monumentally difficult task.
And there are other problems too.
Remember that dog/wolf neural network? Training that was easy. All the programmers had to do was find a stack of photos labelled ‘dog’ or ‘wolf’ and feed them in. The dataset was simple and not ambiguous. ‘But’, as Thomas Fuchs, a computational pathologist, told MIT Technology Review, ‘in a specialized domain in medicine, you might need experts trained for decades to properly label the information you feed to the computer.’40
That might be a surmountable problem for a really focused question (such as sorting breast cancer pathology slides into ‘totally benign’ and ‘horribly malignant’). But an all-seeing diagnostic machine like Watson would need to understand virtually every possible disease. This would require an army of incredibly highly qualified human handlers prepared to feed it information about different patients and their specific characteristics for a very long time. And generally speaking, those people tend to have other stuff on their plate – like actually saving lives.
And then we come to the final problem. The most difficult of all to overcome.
The trouble with data
Tamara Mills was just a baby when her parents first noticed something wasn’t right with her breathing. By the time she was nine months old, doctors had diagnosed her with asthma – a condition that affects 5.4 million people in the UK and 25 million in the US.41 Although Tamara was younger than most sufferers, her symptoms were perfectly manageable in the early years of her life and she grew up much like any other kid with the condition, spending her childhood playing by the sea in the north of England (although always with an inhaler to hand).
When she was 8, Tamara caught a nasty bout of swine flu. It would prove to be a turning point for her health. From that point on, one chest infection followed another, and another. Sometimes, during her asthma attacks, her lips would turn blue. But no matter how often Tamara and her mother went to her doctor and the local hospital, no matter how often her parents complained that she was getting though her supply of inhalers faster than they could be prescribed,42 none of the doctors referred her to a specialist.
Her family and teachers, on the other hand, realized things were getting serious. After two near-fatal attacks that put Tamara in hospital, she was excused from PE lessons at school. When the stairs at home became too much to manage, she went to live with her grandparents in their bungalow.
On 10 April 2014, Tamara succumbed to yet another chest infection. That night her grandfather found her struggling for breath. He called an ambulance and tried his best to help her with two inhalers and an oxygen tank. Her condition continued to deteriorate. Tamara died later that night, aged just 13 years old.
Asthma is not usually a fatal condition, and yet 1,200 people across the UK die from it every year, 26 of whom are children.43 It’s estimated that two-thirds of those deaths are preventable – as was Tamara’s. But that prevention depends entirely on the warning signs being spotted and acted upon.
In the four years leading up to her final and fatal attack, Tamara visited her local doctors and hospital no fewer than 47 times. Her treatment plan clearly wasn’t working, and yet each time she saw a healthcare professional, they only treated the immediate problem. No one looked at the bigger picture. No one spotted the pattern emerging in her visits; no one noticed that her condition was steadily deteriorating; no one suggested it was time to try something new.44
There was a reason for this. Believe it or not (and anyone who lives here probably will), the UK’s National Health Service doesn’t link its healthcare records together as a matter of standard practice. If you find yourself in an NHS hospital, the doctors won’t know anything about any visits you’ve made to your local GP. Many records are still kept on paper, which means the method of sharing them between doctors hasn’t changed for decades. It’s one of the reasons why the NHS holds the dubious title of the world’s biggest purchaser of fax machines.45
Crazy as this sounds, we’re not alone. The United States has a plethora of private doctors and large hospital networks that are not connected to each other; and while other countries, such as Germany, have begun to build electronic patient records, they are a long way from being the norm around the world. For Tamara Mills, the lack of a single, connected medical history meant that it was impossible for any individual doctor to fully understand the severity of her condition. Any solution to this profound shortcoming will come sadly too late for Tamara, but it remains a huge challenge for the future of healthcare. A machine like Watson could help save any number of Tamaras, but it’ll only be able to find patterns in the data if that data is collected, collated and connected.
There is a stark contrast between the rich and detailed datasets owned by data brokers and the sparse and disconnected datasets found in healthcare. For now, medical data is a mess. Even when our detailed medical histories are stored in a single place (which they often aren’t), the data itself can take so many forms that it’s virtually impossible to connect the information in a way that’s useful to an algorithm. There are scans to consider, reports to include, charts, prescriptions, notes: the list goes on. Then you have the problem of how the written data is recorded. You need to be able to understand all the acronyms and abbreviations, decipher the handwriting, identify the possibilities for human error. And that’s before you even get to symptoms. Does this person mean ‘cold’ as in temperature? Or ‘cold’ as in cough? Is this person’s stomach ‘killing them’ literally? Or just hurting a bit? Point is, medicine is really, really complicated, and every single layer of complexity makes the data a little less penetrable for a machine.46
IBM aren’t the only blue-chip big boys to have struggled with the messy, unstructured problem of healthcare data. In 2016, DeepMind, the artificial intelligence arm of Google, signed a contract with the Royal Free NHS Trust in London. DeepMind was granted access to the medical data from three of the city’s hospitals in return for an app that could help doctors identify acute kidney injuries. The initial intention was to use clever learning algorithms to help with healthcare; but the researchers found that they had to rein in their ambitions and opt for something much simpler, because the data just wasn’t good enough for them to reach their original goals.
Beyond these purely practical challenges, DeepMind’s collaboration with the NHS raised a more controversial issue. The researchers only ever promised to alert doctors to kidney injuries, but the Royal
Free didn’t have a kidney dataset to give them. So instead DeepMind was granted access to everything on record: medical histories for some 1.6 million patients going back over a full five years.
In theory, having this incredible wealth of information could help to save innumerable lives. Acute kidney injuries kill one thousand people a month, and having data that reached so far back could potentially help DeepMind to identify important historical trends. Plus, since kidney injuries are more common among people with other diseases, a broad dataset would make it much easier to hunt for clues and connections to people’s future health.
Instead of excitement, though, news of the project was met with outrage. And not without justification. Giving DeepMind access to everything on record meant exactly that. The company was told who was admitted to hospital and when. Who came to visit patients during their stay. The results of pathology reports, of radiology exams. Who’d had abortions, who’d had depression, even who had been diagnosed with HIV. And worst of all? The patients themselves were never asked for their consent, never given an opt-out, never even told they were to be part of the study.47
It’s worth adding that Google was forbidden to use the information in any other part of its business. And – in fairness – it does have a much better track record on data security than the NHS, whose hospitals were brought to a standstill by a North Korean ransomware computer virus in 2017 because it was still running Windows XP.48 But even so, there is something rather troubling about an already incredibly powerful, world-leading technology company having access to that kind of information about you as an individual.
