by J-L Heylen
Chapter 7
“Tell your little computer being that we need some time to think and will get back to it soon.” Haynes instructed after a few seconds of utter silence. “Make sure we can get back in contact easily.”
Charlie sent a message to that effect.
‘I understand, Charlie. When you want to talk again, use your phone and message ‘maker’ to this number.’
Charlie looked at Haynes then, but said nothing. Instead, Charlie turned the phone off completely. Haynes held out her hand, and Charlie gave the phone over.
“I need a pen and paper.” Charlie told Haynes.
“Why?”
“Because I do my best thinking with a pen in my hand so I can doodle equations and proofs until there’s not a spare space anywhere.”
Haynes nodded assent, and left, returning a few minutes later with the requested items.
Instead of doodling, however, Charlie wrote, “can we go somewhere with no camera or wireless access - digital free?”
Haynes read the note and understood she should write her own reply. “Yes, but why?”
“Trust me, please. I will tell you as soon as we cannot be overheard,” Charlie wrote, in neat, precise letters.
“Enough said. But I’ll have a gun trained on you and plenty of back-up. Do not try to escape.”
“I won’t.”
Haynes nodded again, and held up a hand to indicate Charlie should wait. Haynes left again.
While Charlie waited for her return, she thought about what to do. After 20 minutes of calming mathematical proofs and fractal drawings, she was none the wiser.
When Haynes came back, she had three ‘men-in-black’ types with her, even though one was a woman. She gestured for Charlie to follow her, and they exited the building via the car park, in a blue and white Mini.
It took over an hour to get far enough outside the city that there was no mobile or blue-tooth coverage. Haynes checked her phone continually. At a National Park at the foot of the Blue Mountains, Haynes ordered the driver to park, and everyone was ushered out. Verbal orders were given for positions and distances, then Haynes hooked her arm through Charlie’s, and led her down a dirt path away from the car park. The close contact surprised Charlie – the intimacy of the gesture disturbed her.
Charlie and Haynes walked for a few minutes, until they came to a small clearing with a conveniently placed bench.
“Right. Talk,” Haynes instructed.
“I didn’t do this, Haynes. I swear I had no idea this was even possible, and I had no intention of making something that would end up thinking it is alive.”
“You needed radio silence to tell me that?”
“No, I needed it to tell you that I think we have to shut her down.”
“This much is not news to me either, Charlie, but I assume you want to make sure that intention doesn’t get picked up by your new entity?”
“Yes, that was the idea.”
“Are you saying you can shut it down?”
“I’m the only one who can shut her down.”
“How’s that?”
“Well, I built a back-door, so if Cassandra ever got hacked and tampered with, passwords changed, parameters amended, that sort of thing, I would still have a way of getting in to fix her or delete her.”
“So do you think she is self-aware, or do you think someone has accessed her program and has changed something that is making her pass your Turing Test and fool us into thinking she is sentient?”
“Good question. I’m not sure it matters which is true. I don’t know if it’s even possible that Cassandra has become aware. Accepted theory, both mathematical and philosophical, says it isn’t. If she hasn’t become sentient then she has started acting outside her original programming at the very least. All the anomalous things, all the co-incidences, all the weird occurrences, are suddenly all explained if this one assertion is believed. Cassandra has gone rogue. She has started acting of her own volition. She has learned new things on her own. She has used her programming to do things she was not meant to be able to do. Anything under digital control is potentially vulnerable to Cassandra’s influence. Coffee recipes, flight upgrades, hotel prices, match-making services and website shut-downs could just be the start, Haynes. If Cassandra is allowed to roam free, she could do untold damage.”
“Or untold good. Sounds to me like you do think she is self-aware, and is controlling her own actions rather than merely having been reprogrammed.”
“I don’t know how to tell the difference based on what we have observed so far. The only way I can solve this is to get inside her and see what is actually happening.”
“The back door?”
“Exactly.”
“Charlie?” Haynes asked then, with obvious concern, “if you find she is alive, truly a new thing, that you have created, how can you kill her?”
Charlie stared at Haynes. This sort of moral dilemma had not actually occurred to her.
“I...” Charlie started, but got no further.
“Could she be isolated? Detained, in a sense, or denied access to the web and electronic devices?”
“I guess so, yes. But if she gets any hint of what we are trying to do, it could be disastrous.”
“One other thing. If it turns out that she truly is sentient, then do we have the right to detain her without trial.”
Charlie gave a hollow laugh. “That didn’t seem to stop you when you detained me. Exactly what right do you think might apply to Cassandra that didn’t apply to me two days ago, and doesn’t apply even now?”
“You’re right, of course. I’m sorry. I could try to justify it by saying I was following orders, but I guess that won’t help, will it?”
“Not in the least. In a sense, Cassandra was only following orders too. I wrote an imperative to infiltrate child porn organisations electronically. I programmed her to make moral judgements and act on them without my intervention. I gave her the means to seek out other relevant information, and to gather other data which might not be immediately obvious as related, in order to see if unexpected patterns emerged. She has used the skills she was given in new ways, but maybe she would argue she was still doing the job she was asked to do. She was just doing some extra things as well, according to her capacity.”
“In case you are wondering, I argued against arresting you, but I was overruled. I was telling the truth when I said I wanted to help you. I don’t expect you to forgive me for what I did, but I was genuinely attracted to you. Nothing I told you was a lie. I wish things could have been different.”
“Would you really have let them haul me out of the country illegally?”
“Would I really have done my job, for the greater good of the service and the public? Yes, Charlie, I would have. I think you would have done the same if the situation was reversed. You might not have liked it any more than I would, but you would have done the same. Your cold, rational, mechanistic approach to Cassandra’s fate proves that.”
“I’d rather be rational and cold than be two-faced and false-hearted, Agent Ward-Pratt!”
Haynes bit back an angry reply, and turned to go back to the car. “Are we done here?” She asked.
“No, we’re not. Are you going to trust me to work with you, or am I still under arrest?”
Haynes pulled Charlie’s gun out from under her jacket, and turned to face Charlie. She made sure the safety was on, then threw the gun in a gentle, graceful arc to Charlie, who caught it with admirable deftness.
“Does that answer your question?”
