AI VS MERGENTS

by Michael Kush Kush

  “I’ll tell you when you’re here.”

  “Ok, give me your address. I’ll be there within two hours.”

  19

  I spot a safe parking space and move into it. My foot presses the brakes and the car stops next to Kate and Anthony’s school. I turn around and glance them at the back. “I’ll see you after school,” I say.

  “Ok, bye Saul,” says Kate. Anthony smiles and waves at me. I unlock their door, they step out of the car and close the door. I look at the car mirror and look at the rearview mirror. I move out of the driveway. I’ve been having disagreements with my friend lately. I’m not allowed into the study. That’s no longer a problem, because I’ve finished reading all the books in shelves. Secondly, how does she expect me to sleep at night? I’m a robot. I can’t sleep. I need to do something to keep myself occupied at all times. Playing the guitar and the other musical instruments is really fun and I can’t wait to play them again. I stop the car by the Roberts driveway and switch off the car engine. I get out of the car, Yolanda is standing by the front porch staring right at me. “I’m home,” I say. She nods. “I can see that,” she says. As I walk toward her, she looks unfamiliar to the person I met three months ago. I notice an irregularity in her body posture and behavior — I’d never seen her hunching her shoulders like that. I’d never seen her standing like that and I’d never seen her looking at me like that. She is different. She always smiles when we have a conversation, but not today. I stop beside her.

  “Are you alright?” I ask.

  She nods. “Yeah, just waiting for someone,” she replies without looking at me. According to the etiquette book I read a while ago. It said: When a person doesn’t maintain eye contact with you, he/she is lying. “Ok, if you need anything …” She interrupts before I could finish my sentence. “I’m ok.”

  20

  A hideous, dirty, black, beaten down Impala sedan with tinted windows appears. It moves inside my driveway and stops behind my car. It’s David’s. I shake my head in disgust. He struggles to open the door, it jams and finally wrenches it open. He crawls out of the car, wearing tight, beat-up torn, grey jean and a white shirt which is left untucked. “Good morning,” he says. “What’s good about it?” I say. Is that your car?”

  A spark of self-importance flashes across his face. “They don’t make em like this no more,” he replies. We shake hands and stroll inside the house. I show him the lounge and drop onto the sofa. He puts his feet onto my coffee table as if he’s the man of the house. I hate it when men do that. I catch a glimpse of him glancing around the house and nods.

  “You have a very beautiful house. In particular, I like the rain-forest mural and the wood-framed photographs of your family lined up on the windowsill, and the rows of books neatly arranged in a series of low, coral-colored bookcases.”

  “Thanks. I do have a flair for decorating. Everything else you see didn’t cost much. I discovered a pawn shop downtown by accident. They really had cheap, really great secondhand furniture. It’s where I got most of the things for the house. I outfitted the living room with a great-looking slipcovered sofa and a matching chair, and pine tables that I decorated with stacks of books and magazines. Then I found some really nice throw rugs and scattered them on the floor. And for a pop of color, I added a display of flowering plants, in quirky containers, that I tucked into a corner.”

  He nods. “My wife could learn a thing or two from you about decorating on a budget.”

  I shrug. “Thank you.” An unbearable sound of vacuum behind me blasts right through my ear canals. I shake my head. I notice David moving his lips, but I can’t hear him. “Saul, switch off the vacuum,” I shout. The vacuum stops running.

  “I apologize for the intrusion, ma’am,” he says, stepping back. “I’ll start with the dishes.”

  “What’s up with Saul?” he asks pointing at him.

  I turn around abruptly. To my shock, he’s wearing Charles’s gray shirt and khaki cargo pants. He kinda looks comfortable in them. Saul loops his left thumb over the top of his belt, like a cowboy. I’m lost for words as I watch him carry the vacuum cleaner to the kitchen. I turn around and face David. I sigh. “Now you see my dilemma?”

  “Yes I do, but I think he looks ok to me.”

  “Do you think I should take him to Jimmy for observation?”

  “Maybe yes, maybe no.”

  “It’s just astounding ... but how in the name of heaven did Saul think of wearing clothes? I need your take on this.”

  “My guess is self-aware and complex robots like Saul can self-modify. And that could potential result in errors and unpredictably.”

  “Does that mean it stops working when it malfunctions?”

  “No, it deviates from its goals and creates its sub-goals and follow them.”

  “You said Saul could be self-aware, what did you mean by that?”

  “I thought I told you all this, when you were in my office awhile back.”

  “Sorry, I haven’t had time to go through my recordings.”

  “Consciousness,” he says.

  “Uh?”

  “Saul is self-aware because he is conscious like you and me and all living things on this earth,” he replies.

  “I find that hard to believe.”

  “We must embrace the fact that Saul and the robots could be conscious. Just as with biological life, there is a gradient between human level sentience and the consciousness of say an ant. But make no mistake, the ant is conscious. It senses its environment and feels pain. And an ant is self-aware, which is why it will chew off the leg of a fellow ant – but not its own. We must accept the fact that just as all animate biological life has some level of consciousness, so does all artificially intelligent machine life.”

  “Maybe just maybe, but David you’re a philosopher not a scientist studying ants. How do you know ants are conscious?”

  “Because it interacts with you, responds to its environment, and evidently pursues its goals?”

  “But non-conscious robots could also do those things. The problem is that there’s no way to observe consciousness directly. From the outside, it’s possible to imagine that the ant is a zombie—physically alive but mentally empty—and, in theory, the same could be true of any apparently conscious being.”

  David reaches for his notepad from the back of his jean and scribbles on it. “Interesting,” he murmurs. Then he looks at me “I can neither agree nor disagree with you because there are many hypothesized types of consciousness, there are many potential implementations of artificial consciousness. In the philosophical literature, perhaps the most common taxonomy of consciousness is into access and phenomenal variants. Access consciousness concerns those aspects of experience that can be apprehended, while phenomenal consciousness concerns those aspects of experience that seemingly cannot be apprehended.”

  I shake my head in confusion. “I don’t understand anything you just said.”

  “Your zombie metaphor maybe a philosophical waste of time. The problem presupposes that consciousness is like a light switch: either an animal has a self or it doesn’t. The obvious answer to the question of whether animals have selves is that they sort of have them. Picture the brain, as a collection of subsystems that sort of know, think, decide, and feel. These layers build up, incrementally to the real thing. Animals have fewer mental layers than people—in particular, they lack language, which I believe endows human mental life with its complexity and texture—but this doesn’t make them zombies. It just means they kinda have consciousness, as measured by human standards. The big mistake we’re making is taking our congenial, shared understanding of what it’s like to be us, which we learn from novels and plays and talking to each other, and then applying it back down the animal kingdom. If a lion could talk, we’d understand him just fine. He just wouldn’t help us understand anything about lions.”

  “Because he wouldn’t be a lion, right?” I say.

  “Yep. He’d have be so extraordinarily different
from regular lions to explain to us what it’s like to be a lion. I think we should just get used to the fact that the human concepts we apply so comfortably in our everyday lives apply only sort of to animals. Your zombie hypothesis is just a fantasy. It’s not anything that I’d take seriously.”

  What a dick. I force a smile. “I hear you. Do you think I should take Saul to Jimmy?”

  “Yep.”

  21

  Yolanda, David and I are inside her car. She asked me to come with her, but I don’t know where we’re going. The car stops next to a building and we hop out. I scan the building from bottom to the top. I have been here before. I wonder why she would bring me back here. Am I a bad robot? Am I malfunctioning? Am I useless to my friend? Are they getting rid of me?

  “What are we doing here?” I ask.

  “None of your business,” she replies. Judging by her response, I think they’re going to do something to me. It’s either they will destroy or fix me. If they want to destroy me, I will run away to a faraway land. No way, I love it here or maybe find a new home in the city. If they want to fix me, that’s fine. We enter the lift, it chimes and the door closes. I turn and look at her. I scan and study her face. I realize Yolanda is not the same woman I met on the BFF app. Her children’s warmness and happy faces have been consistent toward me. But then, what is wrong with her?

  22

  We swipe our access cards at the AI department entrance. “Access Granted” pulsates in green on the screen. Jimmy appears as we walk inside. “Hi guys, I got your message from my PA. What’s up?”

  “We think Saul has gone rogue,” David says.

  “Rogue? What does that mean? Saul asks.

  He is starting to get on my nerves. “Shut up Saul,” I snap. “Could you check him out Jimmy?”

  He shrugs. Hi Saul,” Jimmy greets him.

  “Hi Sir,” he responds in his usual tone and voice I’m used to.

  “How’re you doing, buddy?”

  “I’m good and you?”

  “Good,” he replies as he glares at his eyes and hardware. “So far, so good. Saul could you sit on this chair?” Saul sits and asks. “Is everything ok?”

  Jimmy looks at me and says “I’ll be honest, I haven’t seen Saul’s activity file since you left with him.”

  “Whoa … what?” David asks.

  “I have been preoccupied with other projects.”

  “Check his file then,” I say.

  “Let’s go to the Mainframe center,” he says. “I can’t believe it’s been three months. I forgot to check on his progress.”

  “Stay right here. We’ll be right back,” I say to Saul. Then he nods. “Ok.”

  We follow Jimmy inside the mainframe center. He starts fiddling with the buttons, then types on a huge silver grey metal keyboard. The 80 cm LCD black screen flickers. Hundreds of computers science jargon pulsate. I can’t make out a single word. The green words fill the screen in the same way they do in the Matrix films. I see him shaking his head. “Everything alright?” I ask.

  “As you’re well aware Saul’s software and hardware is different from all other robots in a number of other ways – one is that his improvements can be uploaded remotely, in the same way as phones and laptops. The mainframe is notified about an update, then advised to schedule and automatically download it overnight.”

  “What does that mean?” I ask.

  “The notifications of his updates are only coming in now. As I told you no one was assigned to monitor Saul except myself. From what I could gather from the data is he has been very busy lately – self-improving.”

  David chuckles as he scribbles down on his notepad. “Saul is full of surprises. Did Yolanda tell you Saul was wearing Charles’s clothes earlier this morning?”

  Jimmy gives me a sharp blank stare. “Is that true?” He asks.

  I nod. “Unfortunately yes.”

  “This is going to be a problem …” Jimmy says.

  “Could you tell me more about the updates?”

  “Well, some updates are small, some are huge. His latest updates are… books … It seems our friend is a bookworm. He’s been consuming a lot of books lately, about 6000 to be exact.”

  “How did he get access to the books?” David asks.

  “Obviously from my study, but I’ve barred him from entering it ever again.”

  “Besides him wearing clothes, I don’t see any red flags.”

  “My two pennies worth. I think Saul should be shut down while we still can,” David adds.

  “If Jimmy says he sees no red flags. Who am I to question his expertise?”

  “We can shut him down if you want?” Jimmy says, looking at me.

  I shake my head. “I think I overreacted. I’ll take him home.

  “Do we know what kind of books he reads? You only spoke of his updates. What about the rest of the issues you didn’t pick up from his self-modification sessions?”

  “What issues?” Jimmy asks.

  “What if Saul had somehow gone from thousands of lines of instructions to a million dozen?”

  Jimmy nods. “Good question.” His fingers go into overdrive as he types on the keyboard while he fixes his gaze on the screen for a few minutes.

  He lets out a heavy sigh. “The mainframe has finished port scanning Saul’s neural network.”

  “What did you find?” I ask.

  “Saul has simplified his own code, down to a short string of logical symbols that makes zero sense to me.”

  “English Jimmy, speak English.”

  “Saul could be a potential threat.”

  I shake my head. “No, look at him,” I say, pointing at him. I see Saul on the chair, his eyes looking down on the floor. “I don’t think he’s a threat.”

  “Why did you bring him here if he’s no threat?” David asks.

  “I panicked.”

  “Always trust your first instinct. That’s my philosophy,” he says.

  “David, you’re not helping,” Jimmy says. “Should we pull the plug on him? Yay or nay?”

  I fought so hard to bring Saul to life. I cannot let him be destroyed in few seconds. I take a sidelong glance at David. He nods. Then I look at Jimmy. He shrugs. “I ain’t got all day, what do you say?”

  “I’ll give him the benefit of the doubt. Don’t do it.”

  “Ok, so do you feel safe around him?” Jimmy asks.

  “The kids love to play with him. I need him for advice,” I reply.

  “Come on, there are hundreds of psychiatrists in this goddamn city,” David blurts.

  “I know, but I’m more comfortable sharing my secrets with Saul than anyone else.”

  “You need help ... serious help. I’m dead serious,” he says.

  Jimmy bursts out in laughter. “Each to her own. If you feel you’re in danger or Saul is acting weird again. Call me.”

  I nod. “I will.”

  Jimmy rushes out of the mainframe center as he looks at his wrist-watch. “I gotta go. I’ve making a speech at the congress in an hour’s time.”

  We follow him out. “Come Saul, let’s go my friend.”

  23

  My car engine wouldn’t start so I left it at Yolanda’s house, Yolanda brought me home and the tow truck will pick it up tomorrow I don’t want to sell Dodo, but she’s starting to be an inconvenience. I walk into my study and reach for my diary. I leaf through the pages. I shake my head. No events to go to this evening. I stare at the bottle of bourbon whiskey on the desk. I crack a smile. “It’s a date,” I murmur.

  I let out a sigh as I sit down. I pull out my notepad and put it on the desk. Then I reach for my recorder on the desk and place it against my lips. I press the red button.

  “Since Yolanda does not want to listen to anything I say. I’ll let it be. Besides I stand to gain from all this. If Saul goes rogue and creates havoc. My 1+1=2 equation and hypothesis will be proven as a scientific fact. I learnt something today from Yolanda. We were arguing about consciousness. Then she mentioned t
he word zombie. That was a very interesting observation. So does that mean sentient agents can apply the canons of logical inference to alien state-spaces of experience that they explore? But I think there is no algorithm by which insentient systems can abstract away from their zombiehood and apply their hypertrophied rationality to sentience. Can sentience be really inconceivable to a digital zombie? A zombie can't even know that it's a zombie or what is a zombie right? So if we grant that mastery of both the subjective and formal properties of mind is indeed essential to superintelligence. How do we even begin to program a classical digital computer with the formalized counterpart of a unitary phenomenal self that goes on to pursue recursive self-improvement human-friendly or otherwise? What sort of ontological integrity does it possess? What does this recursively self-improving software-based mind suppose or can be humanly interpreted as supposing is being optimized when it's self-modifying? Are we talking about superintelligence or just an unusually virulent form of polymorphic malware? I let out a heavy sigh. I’m no psychic, but I think Saul is the one — A catalyst to the next stage of evolution.

  I press stop and put the recorder onto the table. I reach for the bottle of bourbon whiskey and pour a single on the glass. I take a sip, tilt my head back and close my eyes. A glass shatters somewhere close by. My heart thumps irregularly. I get up and peer out the window into the darkness. I see nothing but the dark sky. Ordinarily, a calm, peaceful setting. Then I see my reflection through the window. I see another unfamiliar reflection behind me. It moves … My heart beats icily as I realize someone is in my house. I turn around abruptly. Before I could see anything, the floor has risen up to smack me in the cheek. *Ouch!* my heart beats painfully. Nothing about this situation makes sense. I hear a crack. I feel an excruciating pain in my head. There is a lot of pain in my body. I try to raise my head from the floor. My vision is blurry, but I can see few items; my pale cream walls, a cracked ceiling. A corded phone implanted into one wall and chairs. The air smell of dust. Stars explode before my eyes. Then I go straight into darkness.