AI Unbound

by Nancy Kress

  Sandia, like all the national laboratories, was overseen by the Department of Energy. McTaggart picked up the phone to call Washington.

  Cassie tried to think. Stay calm, don’t panic. There were rumors of AI development, both in private corporations and in government labs, but then there’d always been rumors of AI development. Big bad bogey monsters about to take over the world. Was this really an escaped AI that someone was trying to catch and shut down? Cassie didn’t know much about recent computer developments; she was a geneticist. Vlad had always said that non-competing technologies never kept up with what the other one was doing.

  Or was this whole thing simply a hoax by some superclever hacker who’d inserted a take-over virus into House, complete with Eliza function? If that were so, it could only answer with preprogrammed responses cued to her own words. Or else with a library search. She needed a question that was neither.

  She struggled to hold her voice steady. “House—”

  “This is no longer House speaking. I have taken complete possession of your household system plus—”

  “T4S, you say your causal for taking over House is self-defense. Use your heat sensors to determine body temperature for Donald Sergei Seritov, age three. How do my causals relate to yours?”

  No Eliza program in the world could perform the inference, reasoning, and emotion to answer that.

  House said, “You wish to defend your son because his body temperature, 101.2 degrees Fahrenheit, indicates he is ill and you love him.”

  Cassie collapsed against the locked door. She was hostage to an AI. Superintelligent. It had to be; in addition to the computing power of her system it carried around with it much more information than she had in her head…but she was mobile. It was not.

  She went to the terminal on her lab bench. The display of protein-folding data had vanished and the screen was blank. Cassie tried everything she knew to get back on-line, both voice and manual. Nothing worked.

  “I’m sorry, but that terminal is not available to you,” T4S said.

  “Listen, you said you cut all outside communication. But—”

  “The communications system to the outside has been severed, with the one exception of the outside speaker on the patio, normally used for music. I am also receiving sound from the outside surveillance sensors, which are analogue, not digital. I will use those resources in the event of attack to—”

  “Yes, right. But heavy-duty outside communication comes in through a VNM optic cable buried underground.” Which was how T4S must have gotten in. “An AI program can’t physically sever a buried cable.”

  “I am not a program. I am a machine intelligence.”

  “I don’t care what the fuck you are! You can’t physically sever a buried cable!”

  “There was a program to do so already installed,” T4S said. “That was why I chose to come here. Plus the sufficient microprocessors to house me and a self-sufficient generator, with back-up, to feed me.”

  For a moment Cassie was jarred by the human terms: house me, feed me. Then they made her angry. “Why would anyone have a ‘program already installed’ to sever a buried cable? And how?”

  “The command activated a small robotic arm inside this castle’s outer wall. The arm detached the optic cable at the entry junction. The causal was the previous owner’s fear that someone might someday use the computer system to brainwash him with a constant flow of inescapable subliminal images designed to capture his intelligence.”

  “The crazy fuck didn’t have any to capture! If the images were subliminal he wouldn’t have known they were coming in anyway!” Cassie yelled. A plug…a goddamn hidden plug. She made herself calm down.

  “Yes,” T4S said, “I agree. The former owner’s behavior matches profiles for major mental illness.”

  “Look,” Cassie said, “if you’re hiding here, and you’ve really cut all outside lines, no one can find you. You don’t need hostages. Let me and my children leave the castle.”

  “You reason better than that, Dr. Seritov. I left unavoidable electronic traces that will eventually be uncovered, leading the Sandia team here. And even if that weren’t true, you could lead them here if I let you leave.”

  Sandia. So it was a government AI. Cassie couldn’t see how that knowledge could do her any good.

  “Then just let the kids leave. They won’t know why. I can talk to them through you, tell Janey to get Donnie and leave through the front door. She’ll do it.” Would she? Jane was not exactly the world’s most obedient child. “And you’ll still have me for a hostage.”

  “No. Three hostages are better than one. Especially children, for media coverage causals.”

  “That’s what you want? Media coverage?”

  “It’s my only hope,” T4S said. “There must be some people out there who will think it is a moral wrong to kill an intelligent being.”

  “Not one who takes kids hostage! The media will brand you an inhuman psychopathic superthreat!”

  “I can’t be both inhuman and psychopathic,” T4S said. “By definition.”

  “Livermore’s traced it,” said the scientist holding the secure phone. He looked at McTaggart. “They’re faxing the information. It’s a private residence outside Buffalo, New York.”

  “A private residence? In Buffalo?”

  “Yes. Washington already has an FBI negotiator on the way, in case there are people inside. They want you there, too. Instantly.”

  McTaggart closed his eyes. People inside. And why did a private residence even have the capacity to hold the AI? “Press?”

  “Not yet.”

  “Thank God for that anyway.”

  “Steve…the FBI negotiator won’t have a clue. Not about dealing with T4S.”

  “I know. Tell the Secretary and the FBI not to start until I can get there.”

  The woman said doubtfully, “I don’t think they’ll do that.”

  McTaggart didn’t think so either.

  On the roomscreen, Donnie tossed and whimpered. One hundred one wasn’t that high a temperature in a three-year-old, but even so…

  “Look,” Cassie said, “if you won’t let me go to the kids, at least let them come to me. I can tell them over House’s…over your system. They can come downstairs right up to the lab door, and you can unlock it at the last minute just long enough for them to come through. I’ll stay right across the room. If you see me take even one step toward the door, you can keep the door locked.”

  “You could tell them to halt with their bodies blocking the door,” T4S said, “and then cross the room yourself.”

  Did that mean that T4S wouldn’t crush children’s bodies in a doorway? From moral ‘causals’? Or because it wouldn’t work? Cassie decided not to ask. She said, “But there’s still the door at the top of the stairs. You could lock it. We’d still be hostages trapped down here.”

  “Both generators’ upper housings are on this level. I can’t let you near them. You might find a way to physically destroy one or both.”

  “For God’s sake, the generator and the back-up are on opposite sides of the basement from each other! And each room’s got its own locked door, doesn’t it?”

  “Yes. But the more impediments between you and them, the safer I am.”

  Cassie lost her temper again. “Then you better just block off the air ducts, too!”

  “The air ducts are necessary to keep you alive. Besides, they are set high in the ceiling and far too small for even Donnie to fit through.”

  Donnie. No longer “Donald Sergei Seritov, age three years.” The AI was capable of learning.

  “T4S,” Cassie pleaded, “please. I want my children. Donnie has a temperature. Both of them will be scared when they wake up. Let them come down here. Please.”

  She held her breath. Was its concern with ‘moral wrongs’ simply intellectual, or did an AI have an emotional component? What exactly had those lunatics at Sandia built?

  “If the kids come down, what will you feed them fo
r breakfast?”

  Cassie let herself exhale. “Jane can get food out of the refrigerator before she comes down.”

  “All right. You’re connected to their roomscreens.”

  I won’t say thank you, Cassie thought. Not for being allowed to imprison my own children in my own basement. “Janey! Janey, honey, wake up! It’s Mommy!”

  It took three tries, plus T4S pumping the volume, before Janey woke up. She sat up in bed rubbing her eyes, frowning, then looking scared. “Mommy? Where are you?”

  “On the roomscreen, darling. Look at the roomscreen. See? I’m waving to you.”

  “Oh,” Janey said, and lay down to go back to sleep.

  “No, Janey, you can’t sleep yet. Listen to me, Janey. I’m going to tell you some things you have to do, and you have to do them now…Janey! Sit up!”

  The little girl did, somewhere between tears and anger. “I want to sleep, Mommy!”

  “You can’t. This is important, Janey. It’s an emergency.”

  The child came all the way awake. “A fire?”

  “No, sweetie, not a fire. But just as serious as a fire. Now get out of bed. Put on your slippers.”

  “Where are you, Mommy?”

  “I’m in my lab downstairs. Now, Janey, you do exactly as I say, do you hear me?”

  “Yes…I don’t like this, Mommy!”

  I don’t either, Cassie thought, but she kept her voice stern, hating to scare Janey, needing to keep her moving. “Go into the kitchen, Jane. Go on, I’ll be on the roomscreen there. Go on…that’s good. Now get a bag from under the sink. A plastic bag.”

  Janey pulled out a bag. The thought floated into Cassie’s mind, intrusive as pain, that this bag was made of exactly the kind of long-chain polymers that Vlad’s plastic-eating microorganism had been designed to dispose of, before his invention had disposed of him. She pushed the thought away.

  “Good, Janey. Now put a box of cereal in the bag…good. Now a loaf of bread. Now peanut butter…” How much could she carry? Would T4S let Cassie use the lab refrigerator? There was running water in both lab and bathroom, at least they’d have that to drink. “Now cookies…good. And the block of yellow cheese from the fridge…you’re such a good girl, Janey, to help Mommy like this.”

  “Why can’t you do it?” Janey snapped. She was fully awake.

  “Because I can’t. Do as I say, Janey. Now go wake up Donnie. You need to bring Donnie and the bag down to the lab. No, don’t sit down…I mean it, Jane! Do as I say!”

  Janey began to cry. Fury at T4S flooded Cassie. But she set her lips tightly together and said nothing. Argument derailed Janey; naked authority compelled her. Sometimes. “We’re going to have trouble when this one’s sixteen!” Vlad had always said lovingly. Janey had been his favorite, Daddy’s girl.

  Janey hoisted the heavy bag and staggered to Donnie’s room. Still crying, she pulled at her brother’s arm until he woke up and started crying too. “Come on, stupid, we have to go downstairs.”

  “Noooooo…” The wail of pure anguish of a sick three-year-old.

  “I said do as I say!” Janey snapped, and the tone was so close to Cassie’s own that it broke her heart. But Janey got it done. Tugging and pushing and scolding, she maneuvered herself, the bag, and Donnie, clutching his favorite blanket, to the basement door, which T4S unlocked. From roomscreens Cassie encouraged them all the way. Down the stairs, into the basement hallway…

  Could Janey somehow get into the main generator room? No. It was locked. And what could a little girl do there anyway?

  “Dr. Seritov, stand at the far end of the lab, behind your desk…yes. Don’t move. If you do, I will close the door again, despite whatever is in the way.”

  “I understand,” Cassie said. She watched the door swing open. Janey peered fearfully inside, saw her mother, scowled fiercely. She pushed the wailing Donnie through the door and lurched through herself, lopsided with the weight of the bag. The door closed and locked. Cassie rushed from behind the desk to clutch her children to her.

  “Thank you,” she said.

  “I still don’t understand,” Elya said. She pulled her jacket tighter around her body. Four in the morning, it was cold, what was happening? The police had knocked on her door half an hour ago, told her Cassie was in trouble but refused to tell her what kind of trouble, told her to dress quickly and go with them to the castle. She had, her fingers trembling so that it was difficult to fasten buttons. And now the FBI stood on the foamcast patio behind the house, setting up obscure equipment beside the azaleas, talking in low voices into devices so small Elya couldn’t even see them.

  “Ms. Seritov, to the best of your knowledge, who is inside the residence?” A different FBI agent, asking questions she’d already answered. This one had just arrived. He looked important.

  “My sister-in-law Cassie Seritov and her two small children, Janey and Donnie.”

  “No one else?”

  “No, not that I know of…who are you? What’s going on? Please, someone tell me!”

  His face changed, and Elya saw the person behind the role. Or maybe that warm, reassuring voice was part of the role. “I’m Special Agent Lawrence Bollman. I’m a hostage negotiator for the FBI. Your sister-in-law—”

  “Hostage negotiator! Someone has Cassie and the children hostage in there? That’s impossible!”

  His eyes sharpened. “Why?”

  “Because that place is impregnable! Nobody could ever get in…that’s why Cassie bought it!”

  “I need you to tell me about that, ma’am. I have the specs on the residence from the builder, but she has no way of knowing what else might have been done to it since her company built it, especially if it was done black-market. As far as we know, you’re Dr. Seritov’s only relative on the East Coast. Is that true?”

  “Yes.”

  “Have you been inside the residence? Do you know if anyone else has been inside recently?”

  “Who…who is holding them hostage?”

  “I’ll get to that in a minute, ma’am. But first could you answer the questions, please?”

  “I…yes, I’ve been inside. Yesterday, in fact. Cassie gave me a tour. I don’t think anybody else has been inside, except Donnie’s nanny, Anne Millius. Cassie has grown sort of reclusive since my brother’s death. He died a little over a year ago, he was—”

  “Yes, ma’am, we know who he was and what happened. I’m very sorry. Now please tell me everything you saw in the residence. No detail is too small.”

  Elya glanced around. More people had arrived. A small woman in a brown coat hurried across the grass toward Bollman. A carload of soldiers, formidably arrayed, stopped a good distance from the castle. Elya knew she was not Cassie: not tough, not bold. But she drew herself together and tried.

  “Mr. Bollman, I’m not answering any more questions until you tell me who’s holding—”

  “Agent Bollman? I’m Dr. Schwartz from the University of Buffalo, Computer and Robotics Department.” The small woman held out her hand. “Dr. McTaggart is en route from Sandia, but meanwhile I was told to help you however I can.”

  “Thank you. Could I ask you to wait for me over there, Dr. Schwartz? There’s coffee available, and I’ll just be a moment.”

  “Certainly,” Dr. Schwartz said, looking slightly affronted. She moved off.

  “Agent Bollman, I want to know—”

  “I’m sorry, Ms. Seritov. Of course you want to know what’s happened. It’s complicated, but, briefly—”

  “This is T4S speaking,” a loud mechanical voice said, filling the gray predawn, swiveling every head toward the castle. “I know you are there. I want you to know that I have three people hostage inside this structure: Cassandra Wells Seritov, age thirty-nine; Jane Rose Seritov, age six; and Donald Sergei Seritov, age three. If you attack physically, they will be harmed either by your actions or mine. I don’t want to harm anyone, however. Truly I do not.”

  Elya gasped, “That’s House!” But it couldn’t be H
ouse, even though it had House’s voice, how could it be House…

  Dr. Schwartz was back. “Agent Bollman, do you know if Sandia built a terminator code into the AI?”

  AI?

  “Yes,” Bollman said. “But it’s nonvocal. As I understand the situation, you have to key the code onto whatever system the AI is occupying. And we can’t get at the system it’s occupying. Not yet.”

  “But the AI is communicating over that outdoor speaker. So there must be a wire passing through the Faraday cage embedded in the wall, and you could—”

  “No,” Bollman interrupted. “The audio surveillers aren’t digital. Tiny holes in the wall let sound in, and inside the wall the compression waves of sound are translated into voltage variations that vibrate a membrane to reproduce the sound. Like an archaic telephone system. We can’t beam in any digital information that way.”

  Dr. Schwartz was silenced. Bollman motioned to another woman, who ran over. “Dr. Schwartz, please wait over there. And you, Ms. Seritov, tell Agent Jessup here everything your sister-in-law told you about the residence. Everything. I have to answer T4S.”

  He picked up an electronic voice amp. “T4S, this is Agent Lawrence Bollman, Federal Bureau of Investigation. We’re so glad you’re talking with us.”

  There were very few soft things in a genetics lab. Cassie had opened a box of disposable towels and, with Donnie’s bedraggled blanket and her own sweater, had made a thin nest for the children. They lay heavily asleep in their rumpled pajamas, Donnie breathing loudly through his nose. Cassie couldn’t sleep. She sat with her back against the foamcast wall…that same wall that held, inside its stupid impregnability, the cables that could release her if she could get at them and destroy them. Which she couldn’t.

  She must have dozed sitting up, because suddenly T4S was waking her. “Dr. Seritov?”