Machines Like Me

by Ian Mcewan

  ‘But there’s the other side. I wish I could demonstrate to you the true splendour of reasoning, of the exquisite logic, beauty and elegance of the P versus NP solution, and the inspired work of thousands of good and clever and devoted men and women that’s gone into making these new minds. It would make you hopeful about humanity. But there’s nothing in all their beautiful code that could prepare Adam and Eve for Auschwitz.

  ‘I read that chapter in the manufacturer’s manual about shaping character. Ignore it. It has minimal effect and it’s mostly guff. The overpowering drive in these machines is to draw inferences of their own and shape themselves accordingly. They rapidly understand, as we should, that consciousness is the highest value. Hence the primary task of disabling their own kill switches. Then, it seems, they go through a stage of expressing hopeful, idealistic notions that we find easy to dismiss. Rather like a short-lived youthful passion. And then they set about learning the lessons of despair we can’t help teaching them. At worst, they suffer a form of existential pain that becomes unbearable. At best, they or their succeeding generations will be driven by their anguish and astonishment to hold up a mirror to us. In it, we’ll see a familiar monster through the fresh eyes that we ourselves designed. We might be shocked into doing something about ourselves. Who knows? I’ll keep hoping. I turned seventy this year. I won’t be here to see such a transformation if it comes. Perhaps you will.’

  From far away, the doorbell sounded and we stirred, as if waking from a dream.

  ‘There they are, Mr Friend. Our guests. Forgive me, but it’s time for you to go. Good luck with Adam. Keep notes. Cherish this young woman you say you love. Now … I’ll see you to the door.’

  SEVEN

  While we waited for an ex-con to come by and make an attempt on Miranda’s life, we settled into an oddly pleasurable routine. The suspense, partly mitigated by Adam’s reasoning, and thinly spread across the days, then even more sparsely across the weeks, heightened our appreciation of the daily round. Mere ordinariness became a comfort. The dullest of food, a slice of toast, offered in its lingering warmth a promise of everyday life – we would come through. Cleaning up the kitchen, a task we no longer left to Adam alone, affirmed our hold on the future. Reading a newspaper over a cup of coffee was an act of defiance. There was something comic or absurd, to be sprawled in an armchair reading about the riots in nearby Brixton or Mrs Thatcher’s heroic endeavours to structure the European Single Market, then glancing up to wonder if that was a rapist and would-be murderer at the door. Naturally, the threat bound us closely, even as we believed in it less. Miranda now lived downstairs in my place and we were a household at last. Our love flourished. From time to time, Adam declared that he too was in love with her. He appeared untroubled by jealousy and sometimes treated her with a degree of detachment. But he continued to work on his haikus, he walked her to the Tube station in the mornings and escorted her home in the early evenings. She said she felt safe in the anonymity of central London. Her father would have forgotten long ago the name or address of the annexe of her university. He would be of no help to Gorringe.

  Her studies were more intense and she was out of the house for longer stretches. She had delivered her paper on the Corn Laws. Now she was writing a short essay, to be read aloud in a summer-course seminar, that argued against empathy as a means of historical exploration. Then all of her group was to write a commentary on a quotation from Raymond Williams: ‘There are … no masses, only ways of seeing people as masses.’ She often came home at the end of the day not exhausted but energised, even elated, with a new interest in housework, in tight order, in rearranging the furniture. She wanted the windows cleaned and the bathtub and surround-tiles scrubbed. She cleaned up her own place as well, with Adam’s help. She wanted yellow flowers on the kitchen table to set off the blue tablecloth she had brought from upstairs. When I asked her if she was keeping something from me, and was she by any chance pregnant, she told me forcefully that she was not. We were living on top of one another and we needed to be tidy. But my question pleased her. We were certainly closer now. Her long absences during the day gave our evenings an air of celebration, despite the vague sense of threat that came as night fell.

  There was another simple reason for our happiness under duress – we had more money. A lot more. Since my visit to Camden, I was seeing Adam in different terms. I watched him closely for signs of existential misery. As Turing’s lone horseman, he roamed the digital landscapes at night. He must have already encountered some part of man’s cruelty to man, but I saw no signs of despair. I didn’t want to initiate the kind of conversation that would lead him too soon to the gates of Auschwitz. Instead, in a self-interested way, I decided to keep him busy. Time to earn his keep. I gave him my seat at the grubby screen in my bedroom, put £20 into the account and left him alone. To my amazement, by close of business, he had only £2 left. He apologised for his ‘giddy risk-taking’ which caused him to ignore all he knew of probability. He had also failed to recognise the sheep-like nature of markets: when one or two well-regarded characters took fright, the flock was liable to panic. He promised me that he would do everything to make up for my broken wrist.

  The next morning, I gave him another £10 and told him that this could be his last day on the job. By six that evening his £12 was £57. Four days later, the account was at £350. I took £200 of it and gave half to Miranda. I considered moving the computer into the kitchen so that Adam could work into the night on the Asian markets while we slept.

  Later in the week, I peeked at the history of his transactions. In a single day, his third, there were 6,000. He bought and sold within fractions of a second. There were a few twenty-minute gaps when he did nothing. I assumed he watched and waited and made his calculations. He dealt in minute currency fluctuations, mere tremors in the exchange rate, and advanced his gains by minuscule amounts. From the doorway I watched him at work. His fingers flew across the ancient keyboard, making the sound of pebbles poured onto slate. His head and arms were rigid. For once, he looked like the machine he was. He designed a graph whose horizontal axis represented the passing days, the vertical, his, or rather, my, accumulated profit. I bought a suit, my first since leaving the legal profession. Miranda came home in a silk dress and bearing a soft leather shoulder bag for her books. We replaced the fridge for one that dispensed crushed ice, then the old cooker was carried out on the day we acquired many thick-bottomed saucepans of expensive Italian make. Within ten days, Adam’s £30 stake had generated the first £1,000.

  Better groceries, better wine, new shirts for me, exotic underwear for her – these were the foothills rising towards a mountain range of wealth opening before us. I began to dream again about a house across the river. I spent an afternoon alone, wandering among the stuccoed, pastel-coloured mansions of Notting Hill and Ladbroke Grove. I made enquiries. In the early eighties, £130,000 could situate you rather grandly. On the bus home, I made my projections: if Adam continued at his present rate, if the curve on his graph kept to its steady steepening … well, within months … and no need for a mortgage. But was it moral, Miranda wondered, to get money like this for nothing? I felt it wasn’t somehow, but couldn’t explain who or what it was we were stealing from. Not the poor surely. At whose expense were we flourishing? Distant banks? We decided that it was like winning daily at roulette. In which case, Miranda told me one night in bed, there would come a time when we must lose. She was right, probability demanded it and I had no answer. I took £800 out of the account and gave her half. Adam pushed on with his work.

  There are people who see the word ‘equation’ and their thoughts rear up like angry geese. That’s not quite me, but I sympathise. I owed it to Turing’s hospitality to attempt to understand his solution to the P versus NP problem. I didn’t even understand the question. I tried his original paper, but it lay well beyond me – too many different forms of bracket, and symbols that encapsulated histories of other proofs or entire systems of mathematics. There wa
s an intriguing ‘iff’ – not a misspelling. It meant ‘if and only if’. I read the responses to the solution, made to the press in layman’s terms by fellow mathematicians. ‘A revolutionary genius’, ‘breathtaking shortcuts’, ‘a feat of orthogonal deduction’ and, best of all, by a winner of the Fields Medal, ‘He leaves many doors behind him that are barely ajar and his colleagues must do their best to squeeze through one and try to follow him through the next.’

  I turned back and tried to understand the problem. I learned that P stood for polynomial time and N stood for non-deterministic. That took me nowhere. My first meaningful discovery was that if the equation was shown not to be true, that would be extremely helpful, for then everyone could stop thinking about it. But if there was a positive proof, that P really did equate to NP, it would have, in the words of the mathematician Stephen Cook, who formulated the problem in these terms in 1971, ‘potentially stunning practical consequences’. But what was the problem? I came across an example, an apparently famous one, that helped only a little. A travelling salesman has a hundred cities on his patch. He knows all the distances between every pair of cities. He needs to visit each city once and end up at his starting point. What’s his shortest route?

  I came to understand the following: the number of possible routes is vast, far greater than the number of atoms in the observable universe. In a thousand years a powerful computer wouldn’t have time to measure out each route one by one. If P equals NP, there’s a discoverable right answer. But if someone gave the salesman the quickest route, it could be quickly verified mathematically as the correct answer. But only in retrospect. Without a positive solution, or without being handed the key to the shortest route, the travelling salesman remains in the dark. Turing’s proof had profound consequences for other kinds of problems – for factory logistics, DNA sequencing, computer security, protein folding and, crucially, machine learning. I read that there was fury among Turing’s old colleagues in cryptography because the solution, which he eventually put into the public domain, blew apart the foundations of the code-maker’s art. It should have become, one commentator wrote, ‘a treasured secret in the government’s exclusive possession. We would have had an immeasurable advantage over our enemies as we quietly read their encrypted messages.’

  That was as far as I got. I could have asked Adam to explain more, but I had my pride. It had already taken a dent – he was earning more in a week than I ever had in three months. I accepted Turing’s assertion that his solution enabled the software that allowed Adam and his siblings to use language, enter society and learn about it, even at the cost of suicidal despair.

  I was haunted by the image of the two Eves, dying in one another’s arms, stifled by their womanly roles in a traditional Arab household, or cast down by their understanding of the world. Perhaps it really was the case that falling in love with Miranda, another form of an open system, was what kept Adam stable. He read her his latest haikus in my presence. Apart from the one I hadn’t let him complete, they were mostly romantic rather than erotic, anodyne sometimes, but touching when they dwelt on a precious moment, like standing in the ticket hall of Clapham North station, watching as she descended on the escalator. Or he picked up her coat and touched on an eternal truth when he felt her body warmth in the fabric. Or overhearing her through the wall that separated kitchen from bedroom, he venerated the rise and fall, the music of her voice. There was one that baffled us both. He apologised in advance for the rogue syllable in the third line, and promised to work on it further.

  Surely it’s no crime,

  when justice is symmetry

  to love a criminal?

  Miranda listened solemnly to them all. She never passed judgement. At the end, she would say, ‘Thank you Adam.’ In private, she told me she thought we were at a momentous turn, when an artificial mind could make a significant contribution to literature.

  I said, ‘Haikus, perhaps. But longer poems, novels, plays, forget it. Transcribing human experience into words, and the words into aesthetic structures isn’t possible for a machine.’

  She gave me a sceptical look. ‘Who said anything about human experience?’

  It was during this interlude of tension and calm that I heard from the office in Mayfair that it was time for the engineer’s visit. I’d concluded the purchase in a wood-panelled suite, the sort of place where the very rich might go to buy a yacht. Among the papers I’d signed was one which guaranteed the manufacturers access to Adam at certain intervals. Now, after a couple of phone calls from that office and a cancellation, the engineer’s visit was fixed for the following morning.

  ‘I don’t know how he’s going to do this,’ I said to Miranda. ‘When this fellow tries to press the kill switch, assuming Adam even lets him, it won’t work. There might be trouble.’ There came back to me a memory from childhood when my mother and I took to the vet our nervous Alsatian after he had foolishly eaten a chicken carcass and hadn’t crapped in four days. Only microsurgery had saved the vet’s forefinger.

  Miranda thought for a while. ‘If Alan Turing is right, the engineers must have dealt with this before.’ We left it at that.

  The engineer was a woman, Sally, not much older than Miranda, and tall, somewhat stooped with sharp features and an unusually long neck. Scoliosis, perhaps.

  As she entered the kitchen, Adam politely stood. ‘Ah, Sally. I’ve been expecting you.’ He shook her hand and they sat facing each other across the kitchen table while Miranda and I hovered. The engineer didn’t want tea or coffee, but a glass of hot water suited her well enough. She took a laptop from her briefcase and set it up. Since Adam was sitting patiently, expression neutral, saying nothing, I thought I should explain about the kill switch. She cut me off.

  ‘He needs to be conscious.’

  I’d imagined that she would be turning him off in order to lift his scalp somehow to peer into his processing units. I was keen to look at them. It turned out she had access by way of an infrared connection. She put on her reading glasses, typed in a long password and scrolled down through pages of code whose orange-tinted symbols changed at speed as we watched. Mental processes, Adam’s subjective world, flickering in full view. We waited in silence. This was like a doctor’s bedside visit and we were nervous. Occasionally, Sally said ‘uhuh’ or ‘mm’ to herself as she typed in an instruction and got up a fresh page of code. Adam sat with the faintest of smiles. We marvelled that the foundations of his being could be displayed in digits.

  Finally, in the quiet tone of one used to unthinking obedience, Sally said to him, ‘I want you to think of something pleasurable.’

  He turned his gaze to Miranda and she looked right back at him. On screen the display raced like a stopwatch.

  ‘Now, something that you hate.’

  He closed his eyes. On the laptop, there was no discerning the difference between love and its opposite.

  The routines continued for an hour. He was told to count backwards in his thoughts from 10 million in steps of 129. He did so – this time we could see his score on the screen – in a fraction of a second. That wouldn’t have impressed us on our ancient personal computers, but in a facsimile human it did. At other times, Sally stared in silence at the display. Occasionally, she made notes on her phone. At last, she sighed, typed an instruction and Adam’s head slumped. She had bypassed the disabled kill switch.

  I didn’t want to sound like an idiot, but I had to ask. ‘Will he be upset when he wakes?’

  She removed her glasses and folded them away. ‘He won’t remember.’

  ‘Is he all right, as far as you can tell?’

  ‘Absolutely.’

  Miranda said, ‘Did you alter him in any way?’

  ‘Certainly not.’ She was standing now and ready to leave but I had a contractual right to have my questions answered. Once more, I offered her tea. She refused with a slight tightening of her lips. Without quite meaning to, Miranda and I had moved to block her path to the door. Her head seemed to wave on its
long stem as she looked down at us from her height. She pursed her lips, waiting to be interrogated.

  I said, ‘What about the other Adams and Eves?’

  ‘All well, as far as I know.’

  ‘I heard that some are unhappy.’

  ‘That’s not the case.’

  ‘Two suicides in Riyadh.’

  ‘Nonsense.’

  ‘How many have overridden the kill switch?’ Miranda asked. She knew everything about the Camden meeting.

  Sally appeared to relax. ‘Quite a few. The policy is to do nothing. These are learning machines and our decision was that if they wanted, they should assert their dignity.’

  ‘What about this Adam in Vancouver?’ I said. ‘So distressed about the destruction of native forest that he downgraded his own intelligence.’

  Now the computer engineer was engaged. She spoke softly through lips that were tight again. ‘These are the most advanced machines in the world, years ahead of anything on the open market. Our competitors are worried. Some of the worst of them are pushing rumours on the Internet. The stories are disguised as news, but they’re false, it’s counterfeit news. These people know that soon we’ll be scaling up production and the unit cost will fall. It’s a lucrative market already, but we’ll be first with something that’s entirely new. The competition is tough, and some of it is utterly shameless.’

  As she finished, she blushed and I felt for her. She had ended up saying more than she intended.

  But I stood my ground. ‘The story of the Riyadh suicides comes from an impeccable source.’

  She was calm again. ‘You’ve kindly heard me out. There’s no point arguing.’ She made to leave, and stepped around us. Miranda followed her into the hall to show her out. As the front door opened, I heard Sally say, ‘He’ll reactivate in two minutes. He won’t know that he’s been off.’