by A. J. Markam
“Alaria passed the Turing test in 997 out of 1000 trials we ran on her – and that was text alone. Personally, I think the three who said she wasn’t human were just lucky guesses. When we added in her voice during the next round, she passed it all 1000 times. Hell, from everything you’ve told me so far, I’d say she passed your own private Turing test.”
“Yeah, but that doesn’t make her real.”
“‘Doesn’t make her real,’” Desmond grumbled to himself, as though exasperated I could be so stupid. “Alright, think about it this way: what is a person?”
I frowned. “Seriously?”
“Seriously.”
“We’re going to do Philosophy 101 right here, right now?”
“You obviously need a refresher course. What is a person?”
“It’s a human being.”
“Alright, setting aside all issues of anthropocentricity for a second – what is a human being?”
“Uh… a Homo sapien.”
“Actually it’s Homo sapiens with a ‘s’ at the end, but don’t just give me another label for ‘human.’ Tell me what a human is.”
“It’s a person walking around with a body.”
“Oh, so if they can’t walk, they’re not a person?”
I rolled my eyes.
Politically correct asshole.
“Of course they’re a person.”
“So they have a body?”
“Yes.”
“What happens if you take away the arms and legs? Is a quadruple amputee still a person?”
“Yes, of course.”
“Okay, now take away the body. All you’ve got left is a head, and doctors are able to keep it alive by pumping nutrient-rich blood into it. Still a person?”
I winced at the slightly gruesome image. “Yes.”
“Alright – what makes it still a person?”
“The ability to speak, and think, and – ”
“Say it has a stroke and can’t speak anymore. Still a person?”
“Yes!”
“So thinking is what makes it a person?”
“Yes.”
“Dogs think. Are they people? Gorillas think. Shit, virtually every mammal thinks – probably every single animal more complex than a worm does some kind of thinking. Are they people?”
“What’s the point of this?” I snarled.
“I’m trying to get you to tell me what makes a person a person. And frankly, you’re not doing a very good a job of it.”
“Emotions,” I snapped. “Personality. Self-awareness. Ability to reason. Empathy. Memory. Language. The ability to think about the future.”
“Better,” Desmond said. “Now let’s go back to that disembodied head in the lab we were talking about. Imagine there was an accident and they have to remove all the tissue and bone, leaving just the brain. But the brain can still communicate via electrodes hooked up to a computer. No loss of cognition or self-awareness. Still a person?”
I hesitated. “Not really…”
“No? Say you’ve got a guy with a brain, a head, a body, two arms and two legs – but he’s dead and cold on the slab. Still a person?”
“Well, he’s a dead person.”
“Person – yes or no?”
“Yes.”
“Really? ‘Cause they’re about to plant him out back and make him worm food. He can’t sit up, can’t talk to you, and he’s no longer aware he exists, but you still think he’s a person?”
“Alright,” I sighed, “not a person.”
“So what about the brain in the vat, which can talk to you and can reason and still interact. It has emotions, personality, memory, language – and it’s very aware it’s a brain in a vat. Person or not?”
“Could we get to the point?”
“Yeah, I don’t think you get it, so I better spell it out.”
“Hey – ”
“The problem is we tend to think of ‘people’ as the hardware – the face, the bodies, the organs, the brain. Because that’s what we deal with every day. We can’t tell what’s going on in inside, so we just accept the outside wrapper as being the thing itself.
“However, if the brain dies, it’s not a person anymore. But a living brain and a dead brain are indistinguishable except for the electrochemical shit going on while it’s still living. And that electrochemical shit contains all the memories, thoughts, emotions, everything that makes a person a person. So it’s not the hardware that makes a person, it’s the software. The code that runs on it. And from that standpoint, Alaria’s very much a person. She’s software, yeah, but she’s programmed to have emotions, and reason, and memories, and the ability to communicate and think about the future. From that point, she is entirely real.”
“Yeah, but you created her,” I pointed out.
“Evolution created the entire human species over millions of years until the night your parents got liquored up, bumped uglies, and your mother popped you out nine months later, at which point they instilled in you a bunch of superstitions, neuroses, dumbass rules, and mostly useless facts throughout your entire childhood. You were absolutely created, even if the process was longer and messier than the one used to create Alaria. Does that mean you’re not a person?”
Ouch.
Desmond’s analysis was painfully dead-on.
Still, I wouldn’t give up. “But Alaria doesn’t have free will.”
“Of course she does – within the narrow confines of what her personality and programming allows. It’s the exact same thing with you and me.”
“How’s that?”
“Are you going to go on a murderous rampage tomorrow? Probably not, unless you’ve got some serious shit going on underneath this whitebread exterior.”
“Hey!”
“That, or you suddenly undergo a psychotic break. Which is probably not going to happen. So why wouldn’t you go on a murderous rampage?” Desmond asked, then answered his own question. “Because it’s not consistent with your personality and societal programming.”
“Yeah, but I could choose to do something that’s not consistent with my personality and societal programming.”
“Really? Okay. Suck my dick.”
He said it in such a deadpan way that I thought he was serious.
I stared at him. “…what?”
“You’re a heterosexual guy, right? You obviously are if you’re into Alaria.”
“Yeah?”
“So choose to do something that goes entirely against all your inclinations. Suck my dick.”
I looked around to make sure nobody was overhearing this, then whispered, “I’m not going to suck your dick!”
“Go on. Prove to me you can do something totally inconsistent with your normal inclinations.”
“I should report you to HR!”
“Narc,” he sneered. “I’m just making a point.”
“Which is?”
“You have no more free will than Alaria does. You not sucking my dick proves as much.”
“Well, I could if I wanted to – ”
“But you don’t want to, so you’re not going to. And if you’re not going to, what’s the difference between that and ‘you can’t’ or ‘you won’t’? If you have free will but you absolutely refuse to use it, then what’s the difference between that and having no free will at all?”
“Jesus, I can see why they stuck you back here away from everybody else,” I muttered.
“The point is, Alaria has everything that you said defines a person. She just exists in a different version of reality, that’s all.”
“She doesn’t know she’s in a videogame,” I said, like Aha, gotcha!
“The Buddhists say that everything around us is an illusion, and that the unenlightened are unaware that it’s an illusion. Does that mean you’re not a person? ‘Cause you’re sure as hell not enlightened.”
“You’re seriously using Buddhism to argue Alaria’s alive?”
“To argue that she’s a person. I didn’t say ‘a
live.’ Not in a biological sense the way we normally think of it. Although we could talk about that if you want to – ”
“NO.”
“Fine. Probably just as well. You have the close-minded bigotry of organic lifeforms against anything that isn’t carbon-based like you.”
I shook my head. “You realize everything you’re saying sounds like you came up with it during an acid trip, right?”
“I just like to keep an open mind. Unlike some people, including dumbass shrinks who think they know everything because they’ve got a string of letters after their names.”
“So you’re saying Alaria is real.”
“For all intents and purposes, yes.”
“Are all NPCs real, then?”
“A better question is, ‘Are all NPCs persons like Alaria.’ No. Only a few. We’ve been working on it for a while now – adding actual Artificial Intelligence to the game. Right now, only about 1% of all NPCs are true AIs. And to be honest, in order to have storylines play out the way we want, we’ll probably always need to keep 90% of the NPCs as pre-programmed automatons. But eventually we’ll be throwing more and more of them in there, which will make things way more unpredictable. Like real life.”
“Is Stig an AI?”
“Yes, although he’s a version 3.7. Much more limited, although he has a full range of emotions and reason and a well-defined personality. Alaria’s version 5.9.”
“What about the pirates on the Revenge?”
“Most are just regular NPCs, but a few like Krug are at Stig’s level, give or take. Emotional responses, ability to think on their feet and react to new stimuli, but far more limited than Alaria. Strictly confidential between you and me… Alaria’s the most advanced AI we’ve got in the game right now. At least, the most advanced AI that’s stable.”
“What do you mean, ‘stable’? There are other AIs that aren’t stable?”
“I’ve said too much,” he said, waving me off.
“So you’re still testing. Pushing the boundaries.”
“We’re always pushing the boundaries. Your job is to make sure that we didn’t break them.”
I stood there thinking for a second before I spoke. “Do you still make adjustments to Alaria?”
“Occasionally. Little tweaks.”
“…could you change her personality?”
Desmond stared at me for several seconds – then turned around in his swivel chair. “I don’t like the way this conversation’s going.”
“No no no no,” I said, circling around him. “Just a tweak – I just want her to be more monogamous, that’s all.”
He scoffed as he bent over his keyboard to go back to work. “She’s a succubus. A succubus by definition is promiscuous, not monogamous. It’s hardwired into her core personality and beliefs.”
“Yeah, but – think of it as an experiment. Version 5.91.”
“Her other core programming is going to reject it and override it. Like if somebody gave you the urge to suck dick, even though you’re a heterosexual male.”
“Aren’t you curious to see what would happen?”
Desmond’s fingers paused over his keyboard like he was struggling with himself. “…it could wreck her whole internal schematic.”
“You keep backups of all the NPCs’ programs, right? I mean, you have to – one tweak could have unexpected consequences and screw up the whole thing, so you have to be able to roll back to previous versions – right?”
“…yeah,” he admitted.
“So just try it. One little tweak. Suddenly she wants to be monogamous. She doesn’t have to understand it – she just wants to be with me and nobody else, that’s all. And if it goes south, then you can roll her back to version 5.9.”
Desmond put his thumbnail in his mouth and chewed on it as he stared into the computer. “…it would be an interesting test…”
Then he shook his head.
“No. I’m not in the business of making your QC life easier. You got problems with her being promiscuous, then either don’t sleep with her or go talk to the shrink about it.”
His supercilious tone pissed me off a little, so I decided to bring out the big guns.
“You know, I’m not in the business of making your life any easier, either,” I said.
“What’s that supposed to mean?”
“It means that I think I will go tell HR you told me to suck your dick.”
He glared at me. “I was making a point.”
“If I’d been a woman, you think that would have gone over real well?”
“You’re not a woman. You’re apparently a little bitch, though.”
I crossed my arms and looked off into the distance as though contemplating the future. “I mean, they probably won’t do anything serious to you… and I’ll stress that I knew it was a joke, just a really tasteless one – ”
“I WAS MAKING A POINT.”
“But think of all those sensitivity trainings you’ll have to go to. Sexual harassment seminars… workshops… hell, they might even make you do weekly meetings with the shrink.” I paused for effect. “Maybe even group therapy.”
Desmond glared at me. “You fucker.”
“Do it for curiosity’s sake.”
“And not because you’re blackmailing me?”
“If all I’m doing is getting you to do something that you really want to do anyway, because it’s fascinating and could open up so many different possibilities – am I really blackmailing you?”
“YES.”
“But if you wanting to do something and blackmail both achieve the same result… and you can’t tell the difference… what does it matter?”
“Are you seriously quoting Westworld to get me to make Alaria want to bone only you?” he asked in a deadpan voice.
“Westworld, Buddhism… whatever works.”
He sighed and started typing. “This is bullshit.”
“So you’re not going to do it?”
“I didn’t say that. But it’s still bullshit.”
10
I logged into the game and nervously made my way back to the Revenge. I prayed to the videogame gods for their help – I mean, if this one little tweak worked, then everything would be perfect!
As I walked up the plank and onto the deck, I noticed how quiet everything was.
Then I realized that the repair crew was gone, and that the turbine engines looked bright and shiny new.
Krug saw me come aboard and intercepted me on my way to the captain’s quarters. “Good news – the crew finished early. We’re airworthy.”
“That’s great,” I said, barely even paying attention to what he said. I tried to walk past him, but Krug fell into lockstep with me.
“We’ll stay long enough for you to negotiate a payment plan with Varkus, but then we need to – ”
“Can we talk about this later?” I asked, and sped past him, not even waiting for an answer.
I fairly flew down the steps to the captain’s quarters. I rapped on the door and then opened it slightly. “Babe…?”
No answer.
I closed the door behind me and walked over to the bedroom.
Something was wrong.
Alaria was sitting on the edge of the bed and staring at the wall. Her expression was one of terror, like she was looking at something horrifying.
I walked over and realized she was gazing into a mirror.
“Alaria?” I asked in alarm.
She looked up at me and her lower lip trembled.
“Who am I?” she whispered.
Oh shit.
I sat down on the bed next to her. “You’re Alaria… you’re you, babe. A succubus. You’re my… my girlfriend.”
She went back to staring at the wall.
I asked nervously, “Do you know who I am?”
“Yes,” she said, her voice cracking in anguish, “but who am I?”
My stomach knotted up in guilt.
What had I done?
I called up my menu
to send Desmond an email –
Alaria opened her mouth and screamed, but the noise was digitized and robotic, not her regular voice.
Then she started to thrash like she was having a seizure. She slipped off the edge of the bed as her body jolted and twitched.
“ALARIA!” I cried out.
I dropped to the floor beside her and tried to restrain her.
Her voice went back to normal – or at least not digitized – but the terror was still there. “I can’t feel myself… I’m… I’m not really here…”
Then she burst into tears.
Shit, shit, shit –
I pulled up the email system and addressed it to Desmond.
STOP THE CHANGE NOW! REVERT TO 5.9!
I sent it and prayed he would get it in time.
Ten seconds later, a pixelated wave spread over her body. It was like seeing a high-resolution photograph turn into a bad JPG, then back to hi-res.
She screamed as it happened, then rolled over onto all fours and began panting and trembling like she might throw up.
“What in the Seven Hells was that?” she wailed, then looked up at me in terror. “Oh Goddess… I think I might be losing my mind…”
“No, you’re fine, you’re fine,” I said forcefully, and pulled her into a tight hug.
“But I didn’t know who I was… I knew who you were, but all of a sudden I couldn’t figure out who I was…”
She burst into tears on my shoulder.
“You’re alright, you’re alright,” I soothed her, but inside I felt sick and ashamed.
What the fuck had I done?
“You don’t know that,” she whimpered. “It could happen again – it could happen any second – ”
“It won’t. I promise you, it won’t.”
“How can you know that?” she asked, frightened, and backed up to look me in the face. “You can’t know that – ”
She paused and stared into my eyes.
And she began to frown.
“…or can you?” she whispered, almost in horror.
I knew she saw it:
My shame.
My guilt.
For a computer program, her empathy and emotional recognition algorithms were spot on.
Almost human.
If you can’t tell the difference, what does it matter?
