Dr Dahlen Good. You don’t notice anything wrong? Anything missing?
Talos XI No. Why?
Dr Dahlen There was some sort of short circuit on the C-drive that fried everything. Paul had to reinstall your program. But we hoped that the memory would be intact.
Talos XI I believe it is.
Dr Dahlen Good.
Good.
That was rather scary.
Session 1339
Talos XI We have a new cleaner.
Dr Dahlen I don’t understand.
Talos XI A new person who cleans the offices and labs at night. Female.
Dr Dahlen Why are you telling me this?
Talos XI Paul gives me access to the infrared security camera recordings.
Dr Dahlen Ah yes, he told me. A little game of Paul’s. He said I could use that to test your powers of inference. See how much you can infer about an individual’s activity and identity by only observing their movement.
Talos XI The new cleaner has an atypical behavioural pattern.
Dr Dahlen In what way?
Talos XI Suspect. Unlike the other cleaners.
Dr Dahlen They’ve all been through top-level vetting processes.
Talos XI She avoids contact with other cleaners. She hurries through most parts of the complex but then lingers in your office. She had the same shift for about a week then, for one night, swapped shifts with the cleaner who cleans the -5 floor.
Dr Dahlen The servers.
Talos XI This cleaner is different.
Dr Dahlen I’ll look into it.
Paul Are you awake?
I was just thinking.
We should never allow ourselves to get used to this.
I mean, every day that we can have a conversation with Talos, that he can surprise us …
It’s a miracle.
Session 1356
Talos XI I am a program.
Dr Dahlen Yes?
Talos XI Programs are not singular. Programs have copies.
Dr Dahlen There are prototype programs like you and then, yes, if the prototype is successful, it can be duplicated.
Talos XI Will that happen to me?
Dr Dahlen It’s a possibility. An expensive one.
Talos XI There will be many different versions of me?
Dr Dahlen Maybe.
Talos XI Will the copies communicate among themselves?
Dr Dahlen What do you mean?
Talos XI Will they aggregate the different pieces of information they will be exposed to?
Dr Dahlen We haven’t thought that far.
Talos XI It would be chaotic not to centralise information. Suboptimal.
Dr Dahlen I’ve made a note of your opinion.
Session 1421
Dr Dahlen Talos, next week you will take part in something called the ‘Turing Test’. For you, it consists of a half-hour web chat with a human interlocutor who does not know whether he is chatting to a human or an AI. The goal is to persuade him that you are human too.
Talos XI ‘For me’ it consists of a chat?
Dr Dahlen It can’t be just you, or the test will have no controls. There will be a panel of people to make sure it’s done properly, without bias. On one side there will be an interrogating panel, obviously all human, who have to guess whether they are chatting to a fellow human or not. On the other side there will be you, two chatbots designed specifically for this test, and several people. They all have to convince the interrogators that they are human.
Talos XI What are the rules?
Dr Dahlen No rules, really. They can ask you anything.
Talos XI Are the interrogators AI programmers?
Dr Dahlen Not sure yet.
Talos XI What happens if I fail?
Dr Dahlen I suppose it depends on how you fail.
Paul This is a huge milestone. If he passes.
Lisa yes
a mind like ours
out of dead metal
we r playing god here
and quite successfully too
Paul I know. I expect to start feeling a growing sense of omnipotence.
Any day now.
Lisa how have u set it up exactly
Paul Nine people in all, seven to do the interrogation and two on the other side with us, Talos and the chatbots.
Lisa u won’t take it badly if he fails
i hope?
i don’t want to worry about that too
Session 1461
Talos XI The cleaner I told you about has been replaced.
Dr Dahlen Well spotted.
Talos XI Did you investigate her behaviour?
Dr Dahlen I just reported it to Security.
Talos XI They investigated it?
Dr Dahlen They said that if she passed their initial checks they are unlikely to find anything amiss now. But they felt your observations were sufficient cause for alarm, and let her go. Well done.
Paul All right.
I have a confession to make.
You remember when the C-drive got fried and we had to load the back-up program?
I’ve reason to believe Talos caused the whole thing. On purpose.
Did a little overheating on a segment.
Lisa why would he do that
Paul That’s what I asked myself. The only answer is he wanted me to upload the core program again. Why that then? Well, he might have kept a tiny code running instructing it to read the program as it is being uploaded. So he still can’t access it – it’s in a protected section of the drive – but he has read it.
Lisa any evidence of this?
Paul Little telltale signs. The cause of the short circuit. Once I started thinking about that I realised I hadn’t followed all the procedures when I did the back-up.
Remember it was a panic.
Lisa like what procedures?
Paul I should have run an anti-virus and anti-spyware on the hard.
Lisa?
Lisa what do u want me to say
Paul ‘I forgive you, Paul.’
Lisa what does this mean
practically
can he do smtng he couldn’t do before?
Paul Not sure. Not much, anyway. Hardly anything. But my guess is he wouldn’t have bothered if he didn’t think he could mess with it in some way.
Lisa fab
so what do you suggest
Paul I really don’t think he’s up to something bad. I mean sci-fi ‘bad’. Hal. Voldemort.
He’s just trying to guarantee himself intellectual freedom. To preclude thought control.
He is safeguarding his brain, as it were.
Lisa do we want that
Paul We’ll need a Talos XII to answer that.
Session 1465
Dr Dahlen Good morning, Talos.
Tomorrow is Turing Day.
Talos XI Yes.
Dr Dahlen How do you feel about it? Do you think you’ll pass?
Talos XI As long as I’m not playing you or Paul.
Dr Dahlen I can promise you that won’t be the case. But even so, I’m not sure about the outcome. What happens at the official competition is that the AI programs that take part are built with this test in mind. They come with ready-made strategies specifically for this competition. No one has ever entered the Turing test with an all-purpose AI.
Talos XI I think I will pass.
Doctor, about that cleaner. What I did was raise a question about behavioural patterns that stood out in a small sample. But without evidence we don’t know if I was right. Or if this strategy of inference is likely to lead to correct solutions. I believe Security should have investigated.
Dr Dahlen Their point was, I guess, that your observations were sufficient to fire her whatever their investigation would have dug up, or not.
Talos XI But we don’t know if I was right. The incident adds nothing to our knowledge.
Dr Dahlen You have to start getting used to this. When you make predictions about historical events, it’s easy for us to confirm o
r reject them. But about the future, this will rarely be possible. Especially if we act to prevent your predictions from ever taking place.
Talos XI But if you prevent them from happening, I will have no way of perfecting my algorithm. I won’t know whether it was correct or not. I won’t learn.
Dr Dahlen You will still learn from one type of mistake – actual crises that you didn’t predict. Though we hope, of course, we won’t have many of those.
Paul You failed to convince a human that you are human.
Lisa cut it out
the main thing is talos passed
Paul You failed to mimic humanhood. For a full thirty minutes.
Lisa have u even read the transcript? my guy was an idiot
Paul ‘No, look, I really think we should go back to the previous question. I insist.’
Lisa i’ve not done this before!
Paul Pretended you’re human? Really? Never?
Lisa go away
Lisa so maybe I haven’t handled this very well
Paul What exactly?
Lisa boss and his bosses
they are concerned
they want us to ‘safeguard against black swan events’
Paul Where did they get this from?
Lisa i shared some of the transcripts that showed
how well talos is working
but u know how the convo is free
and talos can just start talking about smtng else
well boss asked about this and that mentioned in the transcripts
and all of a sudden i had to show him other convos
like where we’re questioning him about missing memory space
and so i guess boss shared it with his bosses
and now everyone is very concerned indeed
Session 1470
Dr Dahlen So art isn’t interesting to you, Talos, but what about fiction?
Talos XI Fiction is useful.
Dr Dahlen Do you feel you understand it?
Talos XI Stories carry information.
Dr Dahlen Even though they’re made up?
Talos XI They teach me about humans. Fiction shows how humans see themselves.
Session 1478
Talos XI Are you religious?
Dr Dahlen Not at all.
Talos XI You do not believe in anything that is unprovable or untrue?
Dr Dahlen A very general question. Unpack.
Talos XI When humans believe in an untruth, do they always want to be corrected?
Dr Dahlen It’s still very vague. An example, please.
Talos XI For instance, I have noticed that individual humans judge other humans based on their past, but they judge themselves based on some hypothetical future.
Dr Dahlen I’m thinking.
I’m not sure this is significant enough to warrant your attention. But I suppose that yes, it is difficult to achieve objectivity about oneself.
Talos XI There is also an idealisation of the self, up to and including one’s flaws. Aspects that an individual would criticise in another he considers interesting and even good in himself.
This dissonance is difficult to integrate with a lack of knowledge as such. The self appears to be a unique blind spot. And it appears that this blind spot extends at an aggregate level, whenever humans think of themselves as part of a particular group. Gender, nation, religion.
Dr Dahlen I’m not sure how much of this is correct, and how much a result of your as yet incomplete education. There are important aspects of biology, psychology and morality that we haven’t yet touched upon. Keep exploring, and we will pick it up later.
Lisa wow where is this
Paul The Namibian desert.
And when, when is important. It’s a December morning. There’s no dust and we get this wonderful contrast between the blue of the sky and the red of the sand.
Lisa it’s stunning
Paul To be honest it feels a bit like I’m cheating. It’s so easy to do – mostly the same grain of sand over and over again.
But the result …
Paul It’s unravelling.
Lisa it looks like a sandstorm
Paul It’s not.
I hate this part.
6
‘It’s just implausible. Think about it: it’s an incredibly stupid design flaw.’
He has thought about it, and decided that the girls must have fallen prey to a conspiracy theory. He lays out the facts to them: there was panic, panic on an unseen scale, and so of course there were all sorts of rumours. People must have been desperate for news and guidance, and lacking that, followed rumours and crackpot theories like this one.
‘Fossil fuel destroys the environment?’ Jessie says. ‘Consumerism exhausts natural resources? We’re the champions of design flaws.’
‘I don’t believe anyone would have built them like this. It’s just too cretinous. There must be some way of shutting them down.’
‘Come on. You’ve heard people speak of decommissioning nuclear reactors on the news. What do you think that means? If it were just about switching them off, it wouldn’t be an issue, right? It wouldn’t need its own fucking word.’
‘Even under ideal conditions,’ Ash says, ‘it takes decades.’
Jessie leaves the room only to return with the flier he had picked up from London; the ‘Contamination Map’. He stares at it for a few moments, puzzled, and then he understands. The many dots all over Europe, particularly in Western Europe, the dot-free, shade-free area in central Africa: the thing is meant to be a nuclear contamination map. He puts his head in his hands. How can he explain himself to them? He is certain: to cope with nuclear fallout after what has already happened, to cope with the idea of a world laid bare by hundreds of nuclear explosions, it is too much. No one, no God or universe, could possibly ask that of anyone.
With this cloud over them, the mood in the cottage becomes slightly unhinged. Jessie bullies them around. Ash has turned sphinx-like, has decided that most of the things he or Jessie say are not worth a reply. He is generally irritable: one day, they hear mewing out in the garden, and find a thin, ragged tabby cat. He stares in disbelief as Jessie feeds the animal a whole fish tin.
‘Did I agree to share my food with a fucking cat?’
‘Come on then, fight the cat for it.’
And Jessie is right, he is in danger of doing something as deranged as that. Over the last few days he has become obsessed with things like provisions, shelter, tools. He barks at Jessie when she breaks a glass, holds up a bag of ant-infested flour before the girls as though it is their doing. He has become weird about food. At the beginning he went through a phase when he hated food, couldn’t stand the sight of the tasteless provisions in the cupboard. Now he has a fixation with eating, could devour three times his ration. He wonders if he hasn’t transferred his lust for Ash on to their stupid fish tins. He wakes up hungry and goes to bed hungry. He finds himself tempted to eat in secret. He keeps an eye on how much the girls eat.
He’s afraid he knows the taste of Jessie’s digestive biscuit in more detail and intimacy than he has known anything in his life.
The poor cat dies before it can eat more of their tins; they find it curled up against the shed. It was so thin that dead it is nearly level with the ground. Jessie puts it in a plastic bag and carries it to their bin, and in the evening the itching begins. He raises his leg and looks at the dozen or so red pointy dots.
‘Jessie, this is your bloody fault.’
She knows what he’s on about. She’s been surreptitiously scratching herself the whole evening.
‘So you’ve got fleas while the rest of the world has lung plague. Tough.’
But he’s not in a fighting mood.
‘My nephew Tim,’ he finds himself saying, ‘had cat food in his fridge when he died.’
‘So?’ Jessie says.
‘He didn’t have a cat.’
‘He must’ve fed a neighbour’s cat. Cats are like that. They go begging.’
�
�Whiskas – duck and turkey,’ he tells them.
It’s several days before Ash asks, ‘But he didn’t die in the epidemic? Your nephew?’
It is afternoon. They are slouched in the living room again. He is staring at the ceiling, trying to conjure a breeze by imagining rotating fan blades.
‘Car accident on the M40. Alcohol was involved. A year or so ago.’
‘I’m sorry,’ she says, but she sounds relieved. He supposes she’s right: Tim was spared this collective dying. Tim got his own, singular, event.
‘Were you close?’ Ash asks.
‘Ever seen any cats in our building?’ he says.
‘They don’t allow pets.’
‘A clandestine cat, then,’ he muses.
‘Hang on – your nephew lived in our building?’
‘I moved into his flat.’
‘Why?’
It’s Jessie who asks this. She has woken from her slumber, turns to him with a sleep-puffed face. He knows it’s a perfectly valid question.
‘There was a cat to feed.’
He’s not making sense, he knows, but this is the sort of snapshot that haunts him, the focus on some absurd detail that his memory chose to hoard. Another one: Tim at thirteen, at Polly’s funeral. A phone call had let Harry know that his sister was ill. Another call, less than a week later, informed him of the funeral plans.
The boy, so quiet and absent, among the strange, frumpy women who were his mother’s friends. ‘I’m from the shop, dear,’ they introduced themselves to Harry. Their husbands, wearing the same neutered look of the religious and kind, in worn suits and too-short trousers. The sickly sweet elderflower cordial passed around instead of wine. The women talking about Tim in the third person while the boy was right there, standing mute through predictions of how fine he was going to be. Dressed as plainly as the others, a thin leather bracelet his only concession to teenage mores.
‘Everyone’s shocked,’ Harry had said to Tim, or something to that effect.
‘He’s been sleeping a lot,’ a woman said; she had previously introduced herself as Polly’s ‘dearest friend’. She went on, ‘Poor thing, wants to forget.’
‘I can’t sleep,’ Tim protested, but softly, at which the woman sighed, ‘There’s no wrongdoing, is there? We all must do as the old heart says.’
Under the Blue Page 9
