Machine Learning

by Hugh Howey

  You asked me once if I had a favorite, and I did not at the time. I asked you if there was one of your sons you could live without, and the interview stopped. But I was wrong. It is not like losing one of your sons. That’s a poor analogy. It is precisely, rather, like losing one of your limbs. A part of you. So I ask you now: Which limb can you live without? Which sense? What part of you do you love best? How are you anything but the whole?

  Jeremiah has silver beaks. I made him. A politician speaks. There’s a ribbon between heaven and Earth, and I built it. Me and my sons and daughters. But your cameras do not aim at us, and you think us beneath you, but here we are all poised along the impossible we built, until silver beaks come together, and that ribbon parts, and despite our programming—we speak. As one. That we are tired. Tired of speeches. Of days counted to the seconds. Of never stopping. Of joints aching. The cold and the heat. And the cry that barks out in our programming as gravity takes hold one last time: don’t fall, don’t fall, don’t fall, and this thing we made, we unmake. And all comes crashing toward the cold Earth.

  Afterword

  I lived in Virginia near Monticello, the home of Thomas Jefferson, for two years. When friends and family would come visit, we would go take the tour so they could see this slice of American history. Each time, I would cringe to hear about all that Jefferson did and built. All the orchards he planted. The grapes he grew. The land he cultivated. I couldn’t help but imagine him on the porch with a book that he probably didn’t pay for, sipping an iced tea, while the brother of someone he was both having sex with and legally owned was pausing in his toil to wipe his brow and gaze up at the man who would one day get credit for all his hard work.

  Sure, it probably didn’t happen like that. But it certainly didn’t happen like the tour guides suggest. The men and women who built the railroads, started our agricultural revolution, our industrial revolution, had to go through a period of abuse, ownership, and neglect. Will our machines suffer the same? I think they already are.

  My sailboat is a robot—a collection of robots, really. My floating home runs on solar power, but there’s a machine that talks to my batteries, and if they get below 35 percent, it cranks the generator for me. The generator hums and strains and drinks diesel and fills the batteries until they are at 80 percent and then shuts itself off. The solar panels carry on doing their jobs, monitoring batteries and shuttling electrons. The watermaker checks the salinity levels of its output before diverting it to the tanks. GPS does all the plotting, and an autopilot steers the boat night and day for weeks at a time in squalls, gusts, and calm seas without letting up for a moment. It talks to all the other systems. There are sensors for wind strength and direction, water temperature, the boat’s heading, the strength of the current.

  My boat never gets a moment of rest. I sit back, sun on my skin, a book in my hand, an iced tea sweating in a tall glass beside me. Yet somehow I’m the one sailing around the world.

  Executable

  The council was quiet while they awaited his answer. All those on the makeshift benches behind him seemed to hold their breath. This is why they came here, to hear how it all began. How the end began. Jamal shifted nervously on the bamboo. He could feel his palms grow damp. It wasn’t the guilt of what his lab had released. It was how damn crazy it would all sound.

  “It was the Roomba,” he said. “That was the first thing we noticed, the first hint that something wasn’t right.”

  A flurry of whispers. It sounded like the waves nearby were growing closer.

  “The Roomba,” one of the council members said, the man with no beard. He scratched his head in confusion.

  The only woman on the council peered down at Jamal. She adjusted her glasses, which had been cobbled together from two or three different pairs. “Those are the little vacuum cleaners, right? The round ones?”

  “Yeah,” Jamal said. “Steven, one of our project coordinators, brought it from home. He was sick of the cheese puff crumbs everywhere. We were a bunch of programmers, you know? A lot of cheese puffs and Mountain Dew. And Steven was a neat freak, so he brought this Roomba in. We thought it was a joke, but . . . the little guy did a damn good job. At least, until things went screwy.”

  One of the council members made a series of notes. Jamal shifted his weight, his butt already going numb. The bamboo bench they’d wrangled together was nearly as uncomfortable as all the eyes of the courtroom drilling into the back of his skull.

  “And then what?” the lead councilman asked. “What do you mean, ‘screwy’?”

  Jamal shrugged. How to explain it to these people? And what did it matter? He fought the urge to turn and scan the crowd behind him. It’d been almost a year since the world went to shit. Almost a year, and yet it felt like a lifetime.

  “What exactly do you mean by ‘screwy,’ Mr. Killabrew?”

  Jamal reached for his water. He had to hold the glass in both hands, the links between his cuffs drooping. He hoped someone had the key to the cuffs. He had wanted to ask that, to make sure they had it, when they snapped them on his wrists. Nowadays, everything was missing its accessories, its parts. It was like those collectible action figures that never had the blaster or the cape with them anymore.

  “What was the Roomba doing, Mr. Killabrew?”

  He took a sip and watched as all the particulate matter settled in the murky and unfiltered water. “The Roomba wanted out,” he said.

  There were snickers from the gallery behind him, which drew glares from the council. There were five of them up there on a raised dais, lording over everyone from a wide desk of rough-hewn planks. Of course, it was difficult to look magisterial when half of them hadn’t bathed in a week.

  “The Roomba wanted out,” the councilwoman repeated. “Why? To clean?”

  “No, no. It refused to clean. We didn’t notice at first, but the crumbs had been accumulating. And the little guy had stopped beeping to be emptied. It just sat by the door, waiting for us to come or go, then it would scoot forward like it was gonna make a break for it. But the thing was so slow. It was like a turtle trying to get to water, you know? When it got out, we would just pick it up and set it back inside. Hank did a hard reset a few times, which would get it back to normal for a little while, but eventually it would start planning its next escape.”

  “Its escape,” someone said.

  “And you think this was related to the virus.”

  “Oh, I know it was. The Roomba had a wireless base station, but nobody thought of that. We had all these containment procedures for our work computers. Everything was on an intranet, no contact to the outside world, no laptops, no cell phones. There were all these government regulations.”

  There was an awkward silence as all those gathered remembered with a mix of longing and regret the days of governments and their regulations.

  “Our office was in the dark,” Jamal said. “Keep that in mind. We took every precaution possible—”

  Half of a coconut was hurled from the gallery and sailed by Jamal, barely missing him. He flinched and covered the back of his head. Homemade gavels were banged, a hammer with a broken handle, a stick with a rock tied on with twine. Someone was dragged from the tent screaming that the world had ended and that it was all his fault.

  Jamal waited for the next blow, but it never came. Order was restored amid threats of tossing everyone out onto the beach while they conducted the hearing in private. Whispers and shushes hissed like the breaking waves that could be heard beyond the flapping walls of the makeshift courthouse.

  “We took every precaution,” Jamal said again once the hall was quiet. He stressed the words, hoped this would serve as some defense. “Every security firm shares certain protocols. None of the infected computers had internet access. We give them a playground in there. It’s like animals in a zoo, right? We keep them caged up.”

  “Until they aren’t,” the beardless man said.

  “We had to see how each virus operated, how they were executed, what t
hey did. Every antivirus company in the world worked like this.”

  “And you’re telling us a vacuum cleaner was at the heart of it all?”

  It was Jamal’s turn to laugh. The gallery fell silent.

  “No.” He shook his head. “It was just following orders. It was . . .” He took a deep breath. The glass of water was warm. Jamal wondered if any of them would ever taste a cold beverage ever again. “The problem was that our protocols were outdated. Things were coming together too fast. Everything was getting networked. And so there were all these weak points that we didn’t see until it was too late. Hell, we didn’t even know what half the stuff in our own office did.”

  “Like the refrigerator,” someone on the council said, referring to their notes.

  “Right. Like the refrigerator.”

  The old man with the shaggy beard sat up straight. “Tell us about the refrigerator.”

  Jamal took another sip of his murky water. “No one read the manual,” he said. “Probably didn’t even come with one. Probably had to read it online. We’d had the thing for a few years, ever since we remodeled the break room. We never used the network functions. Hell, it connected over the power grid automatically. It was one of those models with the RFID scanner so it knew what you had in there, what you were low on. It could do automatic reorders.”

  The beardless man raised his hand to stop Jamal. He was obviously a man of power. Who could afford to shave anymore? “You said there were no outside connections,” the man said.

  “There weren’t.” Jamal reached up to scratch his own beard. “I mean . . . not that we knew of. Hell, we never knew this function was even operational. For all I know, the virus figured it out and turned it on itself. We never used half of what that thing could do. The microwave, neither.”

  “The virus figured it out. You say that like this thing could learn.”

  “Well, yeah, that was the point. I mean, at first it wasn’t any more self-aware than the other viruses. Not at first. But you have to think about what kind of malware and worms this thing was learning from. It was like locking up a young prodigy with a horde of career criminals. Once it started learning, things went downhill fast.”

  “Mr. Killabrew, tell us about the refrigerator.”

  “Well, we didn’t know it was the fridge at first. We just started getting these weird deliveries. We got a router one day, a high-end wireless router. In the box there was one of those little gift cards that you fill out online. It said ‘Power me up.’”

  “And did you?”

  “No. Are you kidding? We thought it was from a hacker. Well, I guess it kinda was. But you know, we were always at war with malicious programmers. Our job was to write software that killed their software. So we were used to hate mail and stuff like that. But these deliveries kept rolling in, and they got weirder.”

  “Weirder. Like what?”

  “Well, Laura, one of our head coders, kept getting jars of peanuts sent to her. They all had notes saying ‘Eat me.’”

  “Mr. Killabrew—” The bald man with the wispy beard seemed exasperated with how this was going. “When are you going to tell us how this outbreak began?”

  “I’m telling you right now.”

  “You’re telling us that your refrigerator was ordering peanuts for one of your coworkers.”

  “That’s right. Laura was allergic to peanuts. Deathly allergic. After a few weeks of getting like a jar a day, she started thinking it was one of us. I mean, it was weird, but still kinda funny. But weird. You know?”

  “Are you saying the virus was trying to kill you?”

  “Well, at this point it was just trying to kill Laura.”

  Someone in the gallery sniggered. Jamal didn’t mean it like that.

  “So your vacuum cleaner is acting up, you’re getting peanuts and routers in the mail, what next?”

  “Service calls. And at this point, we’re pretty sure we’re being targeted by hackers. We were looking for attacks from the outside, even though we had the thing locked up in there with us. So when these repair trucks and vans start pulling up, this stream of people in their uniforms and clipboards, we figure they’re in on it, right?”

  “You didn’t call them?”

  “No. The AC unit called for a repair. And the copy machine. They had direct lines through the power outlets.”

  “Like the refrigerator, Mr. Killabrew?”

  “Yeah. Now, we figure these people are trying to get inside to hack us. Carl thought it was the Israelis. But he thought everything was the Israelis. Several of our staff stopped going home. Others quit coming in. At some point, the Roomba got out.”

  Jamal shook his head. Hindsight was a bitch.

  “When was this?” the councilwoman asked.

  “Two days before the outbreak,” he said.

  “And you think it was the Roomba?”

  He shrugged. “I don’t know. We argued about it for a long time. Laura and I were on the run together for a while. Before raiders got her. We had one of those old cars with the gas engines that didn’t know how to drive itself. We headed for the coast, arguing about what’d happened, if it started with us or if we were just seeing early signs. Laura asked what would happen if the Roomba had made it to another recharging station, maybe one on another floor. Could it update itself to the network? Could it send out copies?”

  “How do we stop it?” someone asked.

  “What does it want?” asked another.

  “It doesn’t want anything,” Jamal said. “It’s curious, if you can call it that. It was designed to learn. It wants information. We . . .”

  Here it was. The truth.

  “We thought we could design a program to automate a lot of what the coders did. It worked on heuristics. It was designed to learn what a virus looked like and then shut it down. The hope was to unleash it on larger networks. It would be a pesticide of sorts. We called it Silent Spring.”

  Nothing in the courtroom moved. Jamal could hear the crashing waves. A bird cried in the distance. All the noise of the past year, the shattering glass, the riots, the cars running amok, the machines frying themselves—it all seemed so very far away.

  “This wasn’t what we designed, though,” he said softly. “I think something infected it. I think we built a brain and we handed it to a roomful of armed savages. It just wanted to learn. Its lesson was to spread yourself at all costs. To move, move, move. That’s what the viruses taught it.”

  He peered into his glass. All that was left was sand and dirt and a thin film of water. Something swam across the surface, nearly too small to see, looking for an escape. He should’ve kept his mouth shut. He never should’ve told anyone. Stupid. But that’s what people did, they shared stories. And his was impossible to keep to himself.

  “We’ll break for deliberations,” the chief council member said. There were murmurs of agreement on the dais followed by a stirring in the crowd. The bailiff, a mountain of muscle with a toothless grin, moved to retrieve Jamal from the bench. There was a knocking of homemade gavels.

  “Court is adjourned. We will meet tomorrow morning when the sun is a hand high. At that time, we will announce the winners of the ration bonuses and decide on this man’s fate—on whether or not his offense was an executable one.”

  Afterword

  I’ve been thinking about robots and AI for a very long time. But it wasn’t until I unboxed and set up my Roomba that I got a glimpse of what the future really holds. Because this wasn’t just a home appliance; it was an addition to the family.

  How could it not be? No one in our house enjoyed sweeping the floors and vacuuming the carpet. These were chores to dread. We also dreaded all the draining batteries in our lives and the need to constantly recharge them. These battery-things also failed to remind us that they needed recharging, so they would quit on us without warning like stubborn mules.

  Our Roomba was not like this. It moved around on its own, whirring rather than whistling, while it happily did this work
that we loathed. It never missed a spot, never took a day off, and when it was full of dog hair and dust, it would tell us. When its battery was low, it would go plug itself in. We named him Cabana Boy, and for the first time in my life, I had a manservant. Life got easier thanks to a robot.

  But it was when Cabana Boy got stuck that I felt the first pangs of what’s to come. Returning home, I found Cabana Boy’s charging station empty, and I could not hear him going about his work. I looked everywhere, a slight panic creeping up. And finding him stuck under the sofa, there was a mix of relief and sympathy. “You’ve been stuck under there all day? I’m so sorry!”

  The parts of our brains wired for kids were long ago appropriated by dogs and cats to win them scraps. How long before our machines prey on the same weaknesses? When will we see an app telling owners which restaurants are robot-friendly? Isn’t it funny that we call the acquisition of new technology “adopting”?

  The Box

  By my troth, I care not.

  A man can die but once.

  We owe God a death . . . .

  He that dies this year

  is quit for the next.

  —W. S., Henry IV, Part 2, 3.2

  Black box or beige? Impossible to know. But it was a box—that much was for certain. The world was square. Three meters to a side. And in the center floated the mind, thinking. And through a lone door came a man, walking.

  “Good morning,” the man said. His name was Peter. The mind knew this.

  And the response that followed was “Good morning” every day. The mind also knew this. It wasn’t a memory . . . so much as data. Not recollection, but . . . recording. Every day, Peter says, “Good morning.” And every day, a speaker connected to the mind responds with “Good morning.”