by Tom Vowler
‘You mean whether being an evil shit is innate or not?’
‘More than that. The capacity for moral reasoning is evolutionarily ancient, but moral codes are culturally-specific. For example, once we’re more familiar with the program, we could run Hitler’s life. Or Stalin’s. We could intervene at key moments, alter some facet or other, see what impact it has. Conversely, we could model someone deemed to have led a virtuous life and modify their childhood, weave in some disturbance or psychopathy.’
‘Create a monster?’
‘A virtual one, yes.’
‘But a machine can’t demonstrate moral autonomy; even I know they’re just slaves to their programs.’ I didn’t especially know this, but it sounded good.
‘Ethical behaviour is really just rule following, using our own programs. The idea that right and wrong, good and evil exist in some abstract sense, independent from humans who come to perceive them, is frankly bollocks.’
‘So you input a load of experience and then ask it stuff?’
‘Sort of. Computational morality has come a long way since the early days of AI. It’s still a little like encoding a chess computer, yes, only now we can enter emotions and personality and regret into the algorithms. The system has a chip that works the same way our neural networks do. So far we’ve engineered three million fake neurons, connecting them with a billion fake synapses.’
‘Is that a lot? It sounds a lot.’
‘Still less intelligent than your cat, but it’s a start. There’s even a cynicism function we’re developing.’
‘What, you upload famous political speeches?’
‘Not a million miles away. The breakthrough was creating a computer capable of bad things, yet able to choose not to do them. Or vice versa. We bring her to life next week.’
‘Does she have a name?’
‘Romina, of course.’
‘Isn’t that confusing?’
‘She’s Romi for short.’
I assumed Romi would be merely an array of computers and servers, or something running on a complex intranet, but she turned out to have a very physical presence on the one occasion I met Romina for lunch in her lab. The team had fashioned a particularly convincing android and from a distance it was indistinguishable from the woman I now shared my life with. Her lavish russet hair was tied back in the same chaotic manner, complete with a brace of rebellious tousles that framed her face, her posture as she sat cross-legged in an old Chesterfield one of quiet dignity. Romi’s life-sized body, Romina told me, was only capable of a few rudimentary movements; it was her dark interior that held untold wonderment. A ponytailed man named Ray explained with a partial stammer how Romi’s face was the masterpiece, how technology that barely existed five years ago now allowed for something so human in appearance. Bestowed with a special silicon skin, the features were controlled by animatronic muscles that worked to move Romi’s eyes and drive her expressions. She could make eye contact and convey a range of emotions, including empathy, sadness and irritation. Ray described how her skin even responded to touch and that she could experience pain, or at least a programmed response to it.
‘What do you think of her?’ asked Romina.
‘The likeness is uncanny,’ I said, which seemed to please everyone. ‘She’s beautiful.’
‘Correct answer, although I could never really pull off that dress.’
‘I thought the experiment was more . . . psychological. More virtual.’
‘She was built by a Japanese robotics professor; his team flew over to take a plaster cast of my face. It meant more funding this way: we raise their profile, they raise ours. Of course this is just her façade.’ Romina nodded to an elaborate system of PCs that connected wirelessly to the android. ‘All the functions to do with movement and response are inside her, but what you think of as consciousness lies in the server farm over there. Eventually, when quantum processors become practical, it can all move inside her.’
‘Impressive.’ The team seemed to take this to mean the project per se, but it was Romi herself, her sheer presence, that I found so mesmerising, so unnerving.
‘Computers these days regularly pass the Turing Test,’ Romina continued. ‘Their abilities double every six months. In ten years they’ll be more intelligent than us on any given measure. In a hundred years they’ll have more brain-power than all the humans ever to have walked the planet combined. They already make superior juries than us.’
‘How so?’
‘Using Bayesian logic a computer is more reliable than humans in the verdicts it reaches. The point is they don’t have to always be right, just better than us.’
I looked at Romi, sitting impassively. ‘She’s less talkative than you.’
‘Ask her something,’ Romina said.
‘Like what?’
‘Anything. Introduce yourself first.’
I felt silly, especially with the others looking on, and wondered if some elaborate prank was playing out.
‘Hello Romi, I’m Daniel.’
Romi’s face turned smoothly, silently through ninety degrees, the eyes adjusting to meet mine.
‘Can she see me?’ I asked.
‘Not in the sense you mean,’ said Ray. ‘But yes.’
When it spoke, the voice was more synthetic than I’d expected, which was somehow comforting. ‘Hello Daniel. My name is Romina, but you can call me Romi.’
‘We can make her voice more realistic,’ the real Romina said, ‘more like mine, but it was a little weird.’
I asked Romi what the meaning of life was.
‘I’m sorry Daniel, I don’t understand. Perhaps you would like to ask me something else.’
Romina flashed me a scowl. ‘Try again.’
‘Hello, Romi,’ I said. ‘How many moons does Venus have?’
‘Venus is the second planet from our sun and is named after the Roman goddess of love and beauty. It has no natural satellites.’
‘You’ve tried her on the Trivial Pursuit card, then?’
‘Not much point: she has access to every online encyclopaedia and search engine. I’m thinking of sending her to conferences in my place.’
‘She’d have to do sarcasm.’
‘We’re working on it.’
Seeing Romina – the real one – that day, standing beside her unearthly doppelgänger, immersed utterly in the glorious complexities of her subject, led to a couple of realisations: first, that I had fallen unswervingly in love with her; and second, that this would only ever be reciprocated in part, the bulk of life’s voltage for her found in the pursuit of knowledge and a quest to understand it. Love, for Romina, belonged to a more trivial stratum of experience, the deeper layers of meaning and value beyond its reach. It wasn’t entirely fair to term her a stubborn rationalist: more that love’s riddle, as with the existence of God, would never reveal itself, no matter the level of scrutiny. You simply believed in it or you didn’t. We argued frequently about this – love not God; I shared her incredulity for the latter – battles that came with accusations of my quixotic nature, as she termed it.
‘Don’t take it personally,’ she once said. ‘I’m very fond of you.’
‘I want to try something with our firstborn,’ was how a pregnant Romina announced it one weekend in bed with the Sunday papers. From what I gathered, the project with Romi had run into trouble thanks largely to a paper by a Swedish neuroethicist who had ridiculed the team’s methods. Even Romina conceded that results had at best been ambiguous, and it was rumoured the Japanese now wanted to disassociate themselves from the morality element. Starting a family had not been planned, but I hoped it would prove a welcome distraction for her.
‘Something?’
‘An enquiry.’
‘You mean an experiment?’
She went on to describe the history of feral children, infants who’d been found in the woods, naked and scavenging, apparently without family, their pasts unfathomable. Such children were typically mute and did not underst
and the language of their rescuers (or captors, depending on your view). Some even walked quadrupedally. Efforts to integrate them proved mixed, with a few rehabilitating fully; others, though, led reclusive lives, the consequences of emotional and linguistic deprivation irreversible. The problem with both groups was that neither could shine any light on the origins of their wildness. And without the ability to monitor their previous existence, nor could they assist much with the enquiries scientists and philosophers aspired to make. At least not beyond that which could be gleaned from children with learning difficulties. Further, those who did respond to cognitive instruction had become tainted as witnesses, the socialisation destroying all essence of wildness. Either way scientists would often abandon the project, leading to claims that the child, instead of spending a considerable time living outside of organised human society, was merely the victim of neglect or some other emotional trauma. Nonetheless, according to Romina, fascination with such cases persisted through the ages, offering, in theory, a unique potential to study what made someone human.
‘I want to take it a step further,’ she said. ‘I want to find out if moral instinct can develop without instruction.’
‘I think there are laws preventing that.’
‘It will only be for a few years.’
‘You want our child to grow up in the woods?’
‘Now you’re being silly. Think of the possibilities. We could create a new language, one that was morally bereft. No guidance of any kind about what was right or wrong.’
‘I never know when to take you seriously.’
We didn’t get the chance to run any philosophical experiments on our child – a seven and a half pound girl called Celeste – as Romina was killed three months after the birth. The van driver told police he had no chance, that she’d pulled out without looking, a witness claiming she’d been using her phone. The young policewoman who came to the house spoke in a soft and faltering voice. I’d thought it apocryphal, how your legs buckle from under you in such moments, but it isn’t. Later I learned that the impact was so overwhelming, our car was compressed to a third of its size. Even now I think about those two-thirds.
Grief took several forms in the coming months. A doctor told me it had to run its course, like a virus; friends were equally unoriginal in their counsel. Decisions lined up like attack dogs: how long should I keep her possessions? What did I tell Celeste when she was old enough to understand? Each time I found some small strength, some cruel intervention conspired to break it: I’d find a swathe of Romina’s hair while unblocking the hoover, or a voice on the phone would ask if she wanted to renew her gym membership. Such callers were subjected to a tyranny I hardly knew existed in me.
Photographs were both a source of comfort and torment, the temptation to delete them tussling with a desire to have more. The world itself seemed realigned, no longer adhering to the same scientific principles. Time refused my attempts to function within it; even inertia felt beyond me as the anguish claimed a little more each day. I needed Romina’s direction, her tutelage, to function in life; she’d been the guiding lines that bled through my writing paper. One evening I observed Celeste watching me weep, envying her brain’s inability to comprehend what had been lost.
It was a year or so after the funeral when Ray called. His stammer more pronounced, he told me the project had been put on hold – the funding withdrawn due to a lack of what he called ‘empirical progress’ – but that they hoped to relaunch it in the future. Obviously with a different android, he was quick to say. We chatted for a while, the emotion kept in check by small talk, evasive subjects such as my paper’s feature on cuts to the University, how poorly the local football team was doing. I told him I hadn’t been back to work, that fatherhood was sufficient for now. After a bloated silence he asked if I wanted to have Romi, to take her home once they’d disconnected her. I sensed his discomfort at asking what was surely a unique question.
‘Don’t the Japanese want her back?’
‘Technology’s moved on so much, she’s a little redundant now.’
The thought had crossed my mind, though I imagined she would come without any software, what essence of humanity she possessed diminished. She would be little more than a mannequin, a stuffed pet housed in the corner of a room, a macabre memento mori. The idea was both thrilling and revolting.
I thanked Ray for the offer. ‘I’d like to see her again, though,’ I said. ‘We’d like to see her.’
It felt a huge risk taking Celeste to see Romi. Ray met us on campus and we headed to a smaller building on the University’s fringes.
‘Priorities have shifted,’ he said. ‘There’s just me now really. A few students use her for their theses.’
I lifted Celeste from her buggy, which we left in the foyer, and carried her down some stairs into the basement, Ray leading the way through artificially-lit, airless corridors.
‘What money there is,’ he said, ‘goes to cellular medicine these days.’
As we headed deeper into the building, I sensed a tension in Celeste – at this unfamiliar man, at a place she didn’t know – and I tried to exude a nonchalance, despite the sweat rising on my back, my lungs vacuum-packed. The room itself was sparsely filled, a few work stations dotted around, one of the strip lights above them flickering on and off. In the far corner I saw Romi in her armchair, Romina’s dress that she wore last time replaced with tracksuit bottoms and a jumper I didn’t recognise, a modification perhaps made by the team after the accident, or by Ray this week to spare me.
‘She’s still connected to the server farm,’ Ray said, ‘but we share it with several other departments now.’
‘So who is she this month: Myra Hindley or Mother Teresa?’
‘We haven’t run a character for months. She’s just Romi for now.’
We stood there for a moment, the three of us. The four of us. In the silence Ray shifted his weight from leg to leg, and for the first time I got how difficult this was for him.
‘Shall I boot her up?’ he said.
He headed over to one of the desks, inserted an index finger into the computer’s security reader before focusing on the screen. A minute or so later the soft buzz of motors and servos started up within Romi – life bestowed in a few clicks. Despite the absence of eyelids – or whatever gives the eye its embodiment – there was still a palpable shift in Romi’s gaze from her unawakened state. Ray negotiated the mouse some more and she turned her head, the camera behind her eye scanning for potential interaction, her stare finally settling on us. A moment later she smiled.
‘Hello Daniel,’ she said.
‘She remembers me?’
‘Of course,’ said Ray. ‘Visual recognition is old hat.’
I turned my daughter around, extended my arms a little. ‘Romi, this is Celeste.’
‘Hello Celeste. I’m Romi.’ Romi’s voice had changed since I was last here, was now entirely human, although there was nothing of Romina within it. Despite this Celeste seemed intrigued, her unease falling away as she fixed on this new but not new person in front of her. I wondered if she recognised Romi, remembered on some primitive level the alignment and proportions of her mother’s face a year on. What was retained from those first few months mother and daughter shared? I presumed, that without the aid of photographs, nothing. A thought: perhaps we should keep her after all, take her home. We could set a place for her at the table, bring her out for birthdays, Sunday drives, parents’ evenings. Ray could advise how to restore Romina’s voice, Celeste utilising Romi’s infinite knowledge for her homework. A mother of sorts. And, when the nights became incessant and devastating, as they still could on occasion, I could curl into her and forget. Show her how to make shadow puppets on the wall.
I encouraged Celeste to say hello but she didn’t, whether sensing the absurdity of it or from shyness, it was hard to say.
‘Would you like to ask me something?’ said Romi.
Several things came to mind: why wer
en’t you looking that day in the car? How can I do this on my own? What if I meet someone else? We meet someone else? Instead I looked hard at her face, on some level grateful to have this moment science had granted us. Had I been alone in the room, I almost certainly would have touched her, kissed her. Plunged my face deep in her hair in search of a fragrance I would never smell again.
Watching this bizarre scene play out, I realised something was missing. ‘Can she laugh?’ I asked Ray. He thought about this for a moment, before shaking his head, a trace of disappointment crossing his face at realising the oversight. I conducted a mental trawl of footage on my phone – not as much as I’d like, but something – and realised I had no record of her laughter.
‘I’m sorry,’ Ray said.
I smiled, then asked, ‘What will happen to her?’
‘Someone will dismantle her, recycle the parts.’
I pictured this image, tried hard to read something profound in it.
As we said goodbye I pulled Celeste closer, thought how much more I’d have fallen apart in her absence.
‘Goodbye,’ Romi said. ‘I look forward to talking some more with you soon.’
I promised Ray I’d keep in touch, saw he knew that I wouldn’t.
As we emerged into the winter sun and a tide of buoyant students, it seemed important to remember every word Romina had ever said, to document as much of her as I could, capture and keep it alive for our daughter.
‘Soon there won’t be anything only humans can do,’ Romina once told me. ‘By the end of the decade, computers will be able to design and build better robots than we can. That’ll be the tipping point. We will merge with the machines, we will cheat death.’
‘Machines can’t love, though, can they?’ I’d said, in hope more than anything. It prompted a smile from her I can still picture.
Neruda in the Woods
