Three Laws Lethal

by David Walton

  “Yes.”

  “Tyler?”

  A guard suddenly snatched the receiver out of her hand and clapped it back into its place on the wall. “Time’s up,” she said.

  Naomi met Tyler’s eyes through the glass. He nodded and mouthed, “I’ll be here.” Naomi gave him a thumbs-up, and then allowed herself to be led away.

  “That your man?” the guard asked. “Don’t get too happy. They never last, not while you in here.”

  Naomi didn’t respond. She wiped the smile off her face and shrugged noncommittally, as if the guard was probably right.

  There was one advantage to being in jail. She almost never had to talk to anyone else. She had been terrified she would have a cellmate who would talk her ear off, but the thin, beaten-down woman who shared her imprisonment seemed just as happy for the silence as Naomi was.

  Naomi stretched out on her hard bunk and stared at the ceiling, a position she stayed in much of the time, when she wasn’t reading the books she was permitted to borrow from the prison’s library. The library was sadly short on science fiction novels, but it had one battered copy of Philip K. Dick’s Do Androids Dream of Electric Sheep? that she had read through five times now.

  At the moment, though, she didn’t need a book. She was living in her own story. When the guard left, she allowed the smile to grow again until her face hurt from it. Jane was conscious. She had developed her own self-aware perspective and was thinking and making choices for herself. There was little doubt anymore. It was a unique event in the history of the world—contact with a new and alien species—and Naomi was the one to do it.

  She might have felt scared, but she didn’t. She felt happier than she had been in a long time.

  CHAPTER 27

  The two men waiting for Brandon at his office didn’t look like government agents. They wore jeans and oxford shirts instead of black suits. Both were middle-aged and didn’t seem in the best shape. The shorter of the two, clean shaven with a craggy, battered face, stood blocking the door, his arms crossed. The other man, who was pale and wore a thin brown beard, leaned against the wall, paging through something on his phone. Brandon flexed his arms. He thought he could take both of them, if he had to.

  “Mr. Kincannon,” the shorter one said.

  Brandon nodded. “Yes, I’m Brandon Kincannon.”

  “Can you show me some ID?”

  Brandon looked back and forth between them. “You first,” he said.

  “My name is Lewis Avery,” the shorter man said, flipping open a slim wallet to show an ID card.

  Brandon studied it. It had an eagle with some embossed ribbons and an anchor, along with a three-letter acronym he didn’t recognize. “ONR? What’s that?”

  “Office of Naval Research,” Avery said. “I’m a civilian employee, and this is Greg Harrison. He’s a contractor with Lockheed Martin.”

  Harrison reached out a hand, and Brandon shook it. “Can’t say I’m pleased to meet you,” Brandon said. “I don’t like being ambushed at my office door. What’s this about?”

  “Can we step inside?”

  “No, I don’t think so. Look, I told the police everything I know.”

  “We’re not with the police. We have an opportunity for you.”

  “An opportunity.” It sounded like a threat.

  Brandon heard a door open to his left. He turned to look, just as a bald Asian man walked out of the bathroom, shaking water off of his hands. A man Brandon recognized. “Professor Lieu?”

  “Ah, Brandon. I see you’ve met my colleagues.”

  “They’re with you?”

  “Of course. I’m very impressed with what you’ve accomplished here, and so are they. We think there’s a real future for your technology, though perhaps one you had not envisioned. May we tell you about it?”

  Brandon unlocked the door, and they followed him in. The office wasn’t organized for visitors. Brandon wheeled his office chair around his desk, as well as the chairs that had been Naomi’s and Min-seo’s, and motioned for the three men to sit. Professor Lieu stretched out on one, his hands clasped behind his head. Avery sat at attention. Harrison took the middle seat and leaned forward, hands on his knees. Brandon himself leaned against his desk, crossing his legs at the ankles.

  Harrison was the first to speak. “As your professor said, we’ve been impressed by your company’s accomplishments. Specifically your driving and vehicle routing algorithms.”

  Brandon gave him a thin smile. The truth was, the company was in trouble. Yusuf had managed to get the cars running again at least. He said Naomi had swapped out the training program for her machine learning algorithm, so that its objective wasn’t driving anymore. Once Yusuf swapped it back in, the cars started up again. And Yusuf had deleted all her accesses and changed all the passwords, so she wouldn’t be able to do it again. But it didn’t matter. The damage had already been done.

  Black Knight’s fortunes rose or fell depending on how much faith people put in the safety of his cars. The news that someone had been murdered by one of his cars had already reduced that faith. He was trying to shift that fear onto Naomi and Tyler as much as possible, but any loss of faith in self-driving cars meant more people would take taxis or the subway or drive themselves. Add to that a night where all his cars had suddenly stopped for no apparent reason, and it didn’t matter what explanations he gave. Business was flagging.

  He had relied too much on Naomi. His best option now was probably to sell the company before its value plummeted too far. The thought made him furious. Everything good that life ever gave him got yanked away just when he thought it would last. It wasn’t fair.

  Harrison shifted in his chair. “Have you ever considered what other purposes your algorithms might be applied to?”

  Brandon studied him, trying to guess what he was after. These guys were from the Navy, right? “What are we talking about, self-driving boats?”

  Harrison chuckled. “Could be. The Navy’s a lot more than boats, though. We’ve got planes and submarines, too. More to the point, we deploy a lot of UAVs and UUVs—”

  “I’m sorry, what?”

  “Unmanned aerial vehicles and unmanned underwater vehicles.”

  “Drones.”

  Harrison wrinkled his nose as if the term offended him, but he didn’t object. “That’s the idea. We’ve got software that allows us to fly them during missions, and we’re shifting toward more autonomous control. We’ve even refitted some of your friend Tyler Daniels’s open source software for the purpose.”

  Brandon felt sick. “He’s not my friend. And I wouldn’t touch his software if I were you.”

  The two men glanced at each other, and Professor Lieu raised his eyebrows. Brandon realized he’d probably shouted. “Friend or not,” Harrison said, “he doesn’t have what you have. A routing solution so sophisticated it can predict the movements of all the other players in the game. That can anticipate where they’re going to be and get there first.”

  “And kill them, you mean?”

  Harrison opened his hands and spread them wide. “If that’s what the situation demands.”

  Brandon thought about it. “My software isn’t open source, though.”

  “Actually, we prefer it that way,” Avery said, stepping into the conversation for the first time. His voice was deeper, slower, with a slight Southern flavor to his vowels. Brandon got the impression that Harrison brought the technical knowledge, while Avery represented the interests of the agency. “With open source, we’re always nervous about uncontrolled modifications. Anybody can contribute to that code, so who knows what secrets might be lurking? Better to buy it from a known source. An American source.”

  Brandon nodded, but inside his mind was spinning. This might be just what he needed. A government contract could revitalize his company, help him weather the storm that was sure to come with the commercial business. The only problem was, he didn’t actually know what his algorithms could do. Yusuf was just scratching the surface. The o
nly person who really understood his software had abandoned him.

  “Let me understand you clearly,” Brandon said. “You want to hire my company to write software to command your UAVs and UUVs to operate independently. So they can make choices on their own, anticipate where the enemy’s going to be, and take them out before they know what’s coming.”

  “Right now, we’re just talking,” Avery said. “But yeah,” Harrison said. “There’s a road we have to travel to get there, but you’ve got the basic picture.”

  “These are armed vehicles?”

  “They serve a lot of different purposes. Surveillance and reconnaissance. Explosive mine countermeasures, for the UUVs. But with your software, yes, strike missions are our primary interest.”

  “But you’re not looking for joystick control. You want to set these things loose with general instructions and have them figure out the best way to satisfy their goals.”

  “In groups, if possible,” Harrison said. “Think platoon level: automated soldiers that can communicate, coordinate, and follow high-level orders. It’s not just ‘kill’, though of course that’s part of it. It’s knowing when and how to kill to meet the objectives they’ve been given.”

  Brandon picked up a pencil from his desk and started twirling it around his fingers. “I’m guessing an important part of this is going to be training them to recognize the difference between good guys and bad guys.”

  “That’s essential, yes. We don’t want our drones turning around and killing our own forces.”

  “Sounds like a complicated thing to define,” Brandon said, “but no worries. That’s what these types of algorithms do best. They’re great at classification and recognition, often better than the humans who trained them.”

  “Glad to hear it,” Harrison said. “Are you familiar with Asimov’s Three Laws of Robotics?”

  Brandon laughed. “Sure. But that’s hardly relevant here, right? You want these robots to kill people.”

  “True enough. But given that they’re going to be sophisticated and smart and trained to kill, we want to make sure we don’t lose control.”

  “You want to make sure they kill the right people,” Brandon said. He meant it sarcastically, but Harrison didn’t seem to pick up on it.

  “Exactly. Let me show you a concept I’ve been developing. I started with Asimov’s Laws and made some modifications to make them workable in this context.” He gave a bashful smile. “I call them Harrison’s Three Laws of Warfighting AIs.”

  He touched something on his phone and turned it around to face Brandon. Brandon wondered why he didn’t just flash the text to his glasses. But maybe Harrison didn’t want his version of the Three Laws getting out in public. Brandon leaned forward and read the screen without taking the phone.

  The Three Laws of Warfighting AIs

  1. An AI may not injure a friendly human being, or, through inaction, cause a friendly human being to come to harm.

  2. An AI must efficiently neutralize enemy humans and machines, except as it may conflict with the First Law.

  3. An AI must accept the definitions of enemy and friend as given by its commanding officer.

  “Interesting,” Brandon said. “In the robot stories, these were root-level concepts, hardcoded into the most basic levels of their positronic brains. Are you imagining something like that here?”

  “As basic as possible. We don’t want a robot reprogramming itself to kill friendlies.”

  Brandon chuckled. “I’ll have to check with my experts, but with modern machine learning techniques, I would think the best way to accomplish that is by making it the objective function of your training. Whatever kind of mind these programs have, they’re defined by how we train them. It’s not a matter of preventing them from doing what they want. We teach them what to want in the first place.”

  “Good man,” Harrison said. “I can tell we’ve come to the right place.”

  “So let’s say I’m interested. What happens next?”

  “We work out an initial contract. Something to get you started. We give you some parameters, you work on a demonstration, so we can see what kinds of results we’re likely to get. If we like what we see, we take the next step. A larger, more formal contract.”

  Avery stood. He wasn’t tall, but he spoke with a quiet authority that Brandon found intimidating anyway. “You won’t spread this around,” he said. “No news media. No impressing girls you pick up in a bar.”

  “It’s what, top secret? You’d have to kill me?”

  The barest hint of a smile. “No, Mr. Kincannon. If it were, we wouldn’t be standing here in the open talking to you about it. It is, however, in everyone’s best interests not to bring this kind of thing under public scrutiny. Once we have more of a relationship, we’ll discuss getting you a security clearance if we deem it necessary. Until then, just remember that you’re a patriot. You are a patriot, aren’t you, Mr. Kincannon?”

  “Of course.”

  “Then I think we understand each other.”

  “I’ll have to tell some of my other employees. This isn’t something I can do alone.”

  “Understood. As I said, this isn’t classified. It’s just better for all involved if we keep it out of the spotlight.”

  Harrison stood, too, and shook Brandon’s hand. “Looking forward to hearing more from you. It’s exciting work you’re doing here.”

  They left. The moment the door closed, Brandon punched his fist in the air. Yes! This was just what he needed to take the next step. He had thought he would have to sell the company, but this was better. He’d redirect their resources into something new. He’d keep the commercial business going too, of course, as much as he could. There was plenty of money to be made there, if he could regain public confidence in his fleet. But if he wasn’t mistaken, this government contract had the potential to grow much bigger. It could be worth billions. He could be the founder and CEO of the next big defense contractor.

  The one thing he couldn’t accept, though, was the possibility of Tyler and Naomi moving into the gap he left behind. Naomi had toyed with him all along—he could see that now. She’d used her connection to Abby to seduce him, playing on his emotional vulnerability to gain his trust, and all the while she was jumping in bed with his enemy. Her own sister’s killer. She and Tyler had to pay.

  He touched his head where she had clubbed him with the hammer. He still had throbbing headaches every day that Ibuprofen didn’t seem to touch. Only whiskey took the edge off the pain. Most people didn’t operate well under the influence of alcohol, but Brandon found it gave him a certain clarity. Sober, his decisions were hampered by fears and uncertainty. With a little alcohol buzz, though, the inhibitions that kept him from thinking decisively melted away. He could do what needed to be done.

  He had to make sure Min-seo’s death could never land back on him or his company. Naomi had to take the fall, and if at all possible, bring Tyler down with her. If he could do that, he would be safe. Safe from prosecution, and rid of Tyler and Naomi forever. He could move on.

  The crazy thing was, Tyler’s company had started beating him. All of a sudden, Zoom Autocars had jumped up to sixty percent market share in Philadelphia. Worse, Black Knight was dropping in every city along the coast, even those that Zoom had no presence in. Naomi’s arrest for murder-by-car had cast a cloud over the company, but that didn’t seem like enough all by itself. The public outcry hadn’t been that big. No, it was obvious to Brandon what had happened. Before her arrest, Naomi had stolen the Black Knight software and had given it to Tyler. Then she’d probably tweaked something so it didn’t work as well for Black Knight cars anymore.

  If he could prove it, he’d make sure she went down for that as well. That they both did. It didn’t matter that she had written most of the code; she had done so as an employee of Black Knight. Stealing it was illegal. He would press charges, and then he would sue them both into bankruptcy.

  He touched his glasses and called Ashley Priest, the re
porter who’d interviewed him after Naomi was arrested. He’d taken her out for drinks afterwards, and one thing had led to another.

  “Hey, lover,” she said. “Already coming back for more?”

  He didn’t really like her all that much—she tended to prattle on about things, and one of her teeth was slightly chipped, which for some reason annoyed him. He had asked her out in the first place only because he thought it might be useful to have a reporter in his pocket, so to speak. The sex was just bonus.

  “Hey, baby,” he said. “I actually called because I have a tip for you. Want to do another interview?”

  “I don’t know,” she said. “I mean, before I didn’t know you, but now that we’re sleeping together . . . My boss might think that was a conflict of interest. Not that it would be, but you know. She’s kind of old-fashioned about things like that.”

  “Trust me, this is a good story. Your boss won’t mind. Besides, I’ll get to see you in the middle of the day. We could go back to my place for—you know—lunch.”

  She giggled. Brandon hated women who giggled. He was going to have to break it off with her after today.

  “I’ll meet you for lunch then,” she said. “Twelve o’clock?”

  “Twelve it is.”

  Brandon disconnected the call and then made another one, this time to Yusuf.

  “Hey, boss.”

  “It’s ten o’clock, where are you?” Brandon said. “In my pajamas. I’m working from home today.”

  “Well, get your butt in here. I’ve got some new work for you.”

  “What kind of work?”

  “We need to change the training function for our AI again.”

  “Yeah? Why’s that?”

  “We need to train it to kill.”

  CHAPTER 28

  Tyler woke up to a barrage of hate mail. He had notifications on every social media platform and messaging system he used, most of them messages from strangers. He waded through them, horrified, until he finally found a link to tell him what it was all about. It led to an article with the heading “Zoom CEO Tyler Daniels Implicated in Autocar Killing.”