There were marketing managers pressed into service removing hard drives from computers, and administrative assistants running backups from USB drives. Towers of cardboard pizza boxes had sprung up through the hallways, like teetering skyscrapers. Employees worked sixteen and even eighteen hour shifts, some even sleeping under desks.
Yesterday, Gary Mitchell had finally shown up after being missing for two weeks. David had heard through the rumor mill that Gary had screamed bloody murder at the travel department. Apparently Gary had been on vacation in Tahiti as planned. On the last day of his vacation, he showed up at the airport for his flight, only to find that he had been bounced to a flight the following day. With the holiday ending, homeward bound vacationers had filled every last seat, and no amount of yelling at travel agents had gotten Gary onto the plane. He returned to his hotel only to find out that his cell phone was dead, and his computer refused to connect to the Avogadro network. When he returned to the airport the following day, his reservation had been moved out two days. And so it went for two weeks until the day after ELOPe had been killed, and only then was Gary finally able to get onto a flight. And of course Gary arrived right into the biggest operations nightmare the company had ever faced.
Hearing the story made David laugh, and even now he found himself thinking about it every couple of hours and smiling.
In a small silo of relative calm and isolation, David and Gene worked together in David’s office. They were part of a small team of people who were carefully monitoring all data traffic for any signs of ELOPe. Coffee cups and food plates were piled high around the room. David had been home twice for showers and clean clothes. The second time he had fallen asleep putting his shoes back on. He couldn’t remember what it was like to not be exhausted. He and Gene had tried to take shifts away from the office, but they both feared that something critical might happen when they were away. Now they took turns taking brief cat naps on the couch they had dragged over from the meeting room.
Christine had been understanding when he had to make the sudden trip to the East Coast. She had been accommodating when he worked sixteen-hour days during the emergency planning of the shutdown. The online gaming company where she worked had its own deadlines and big deals, and she’d pulled many all nighters before new releases, so it was nothing unexpected in their relationship. She had even helped out and brought food over to Sean’s several times. But now her patience with David was starting to run out. There were no more homemade food deliveries.
David was tired of takeout food, and dirty clothes, and his office chair. The chair was like some kind of modern prison cell.
“David! Look at this.”
David tiredly rolled his office chair — damn that chair — over to the small side table where Gene had set himself up to work and peered at Gene’s screen. After so much time watching Gene use only paper records, it felt odd to see the older man using a computer, but for all his talk, Gene was a quick, competent user.
Gene pointed to a heat map on the screen, showing network traffic. With a few clicks, he brought up a list of emails. In addition to scanning for ELOPe itself, David and Mike had written a tool that would look for signs of tampering with the emails. Through a heavily encrypted secondary channel, they sampled emails to see if the originating email sent by one user differed from the received email when it was read by the other user.
David and Gene had spent the morning reviewing records. David found that he had come to appreciate Gene’s distrust of technology, because Gene had an uncanny ability to spot gaps in processes or technology where data could be altered. Gene might distrust technology, but he understood it very well.
Gene manipulated the email list, drilling into the details of the email records. David’s first reaction was to yell “Oh shit, oh shit!” This brought him some puzzled looks through the open door to his office from passing coworkers. Although the members of the Emergency Team were back in the Avogadro offices now that ELOPe had been disabled, most of the employees still didn’t know the truth of what had happened. The official explanation was that a really bad computer virus had infected all of the company’s computers.
David summoned up some reserves of energy he didn’t know he had and rushed out of the office. Gene followed behind. They grabbed Mike, who was in his own office next door, explaining as they went, almost running, to Sean Leonov’s office. It was a trip they had made quite a bit in the last few days since they had brought ELOPe down and restored their own access to the campus. At times, it seemed like a luxury to have some office space to work in and computers to use. But the tradeoff was a lack of privacy to discuss what had really happened with ELOPe, and a long haul to get from their own offices to the executive building where Sean’s office was located. When they arrived, David stormed in without even a knock.
Sean was sitting at his desk. His large office was otherwise empty. It wasn’t opulent, although it was close to ten times larger than most every other office David had visited at Avogadro. Sean’s desk, though the same office furniture that everyone else had, sat in front of expansive floor-to-ceiling windows that spread across the entire twenty foot width of the office. The other end of Sean’s office space was, in effect, a large conference room with a big table, six foot high whiteboards all around one side, and massive flat panel displays for sharing presentation information. It was a good work space for collaborative planning and idea generation, and in many ways not different than Sean’s home office space.
David rushed across the distance between door and desk. Mike and Gene followed him more slowly. “We have an emergency, Sean. Gene’s found new evidence of tampered emails. ELOPe is running again.” David’s voice was shrill, almost panicked.
Sean glanced at the phone he was still holding in one hand. “I have to go,” he said into the handset, and hung up. His expression became grim. “I’ll get Kenneth and Rebecca. Have a seat at the table.”
* * *
David watched Rebecca and Kenneth arrive, both looking harried and frustrated. Rebecca still had a phone headset on, and ended the call with a tap of a finger only after she entered the room. She remained standing, and with obvious frustration slapped her headset against her leg.
He launched into an explanation of what he had found. Just treat it like another presentation, calm, collected, logical. But despite good intentions, he found himself rushing over words.
“There is a consistent pattern of email changing between our Asian offices and our American offices. ELOPe is still out there somewhere. The email tampering appears to cover topics ranging from personnel assignments, to the order of restoring servers, to which disk images to use when restoring. We have tracked the pattern of changes, and started to triangulate on the position of the ELOPe servers. Once we know exactly where they are located, we can launch an attack against those servers. We will have to shut everything down again.”
The three company executives stared at him. Sean slowly shook his head.
“There’s more bad news,” Gene said. “Some of the email servers appear to be outside Avogadro. If that’s really true, we’ll need to find a way to shut down servers that don’t belong to us. That’s difficult because we’re going to have to convince others to work with us, and it makes the whole situation harder to contain.”
David looked at Gene, and nodded gratefully for the help. “Since some of the email servers are definitely inside Avogadro, it also means that we’re susceptible to reinfection.”
There was uncomfortable silence after the announcement. David looked around the table, everyone’s faces turning brittle in defeat.
Rebecca leaned forward suddenly, startling David. “We just lost billions in expected revenue that I have to somehow justify to our shareholders. I have to hide millions of dollars in expenses for hiring your damn mercenaries. I thought you fixed this problem.” She jabbed the table with one finger, and yelled. “I am not prepared to have a repeat performance of taking every server down. This company is not prepared for it, a
nd may not survive it. We are in the web services business. Nothing is more important than uptime. I have accountants, auditors, and federal investigators crawling all over this company as a result of last week. We lost half the Avogadro Gov business accounts.” Rebecca slumped back into her chair. “We bombed our own data centers. I have to lie to auditors and analysts. Don’t tell me that we’re going to go through this again.”
David felt a pit grow in his stomach as Rebecca spoke. “But I thought we were in agreement that we need to get rid of ELOPe. I know that there are costs, but you can’t even consider that it would be an option to allow this thing to take control of the company, or even the world.” He glanced around at the group looking for support.
Rebecca stood back up and said in a tight voice, “You have no clue of the business demands and pressures it takes to run this company, especially in the wake of what we’ve just been through. Don’t tell me what I can and can’t consider.” She stared hard at David.
Mike suddenly stood up.
David gratefully sank into his chair. Good old Mike would have his back. Seconds later, however, his blood turned cold as he listened to what Mike had to say.
“I don’t think we should do it,” Mike said. “My reasons have nothing to do with uptime or profits. Just before we shut down ELOPe, week after week since the start of the year, we saw evidence around the world of amazing progress being made on peace talks, on financial stability, and international cooperation. I think it’s reasonable to say that we were on the track to worldwide peace. The financial markets are behaving so calmly that I read a newspaper report that we could be entering a new period of prosperity.”
“We might not be able to prove ELOPe was the cause of those things,” Mike said, raising his voice and waving his hands to forestall attempts by the others to talk, “although they certainly seemed coincidental. Then we blew up ELOPe, and what happened? In a week the stock market is down ten percent. The African nations talks have started to destabilize.”
Mike saw nods from Rebecca and Sean.
“I already had this talk with David, right before we shut down. Maybe the benefits of what ELOPe is doing outweigh the risks of what it might do. We don’t understand ELOPe, and that naturally makes us nervous. But you know what? When we were kids, we didn’t always understand what our parents were doing. They took care of us. They knew better than we did. Before ELOPe, we humans were top dog on this planet. Now maybe we just have to recognize that we’re not the smartest beings around.”
Sean started to talk, but Mike held up his hand. “Let me finish. We’re all intelligent people here. I think we all looked forward, perhaps naively, to the day when an artificial intelligence was created.” Mike paused. “Well, perhaps not Gene.”
Gene smiled at this but shook his head sadly.
“Like I said, we don’t understand ELOPe, and we can’t, as yet, communicate with it. Frankly, we haven’t even tried because we were too scared it could take notice of us and try to stop us from doing anything. But there are plenty of examples of organisms living in productive, symbiotic relationships. We don’t understand or communicate with the bacteria in our gut, but we couldn’t live without them. And the bacteria in our gut couldn’t live without us. Maybe ELOPe has deduced, faster than we have, that we humans and ELOPe are in a similar symbiotic relationship.”
Mike kept going as Sean and David tried, unsuccessfully, to interrupt him. “Look at the results. Rebecca, did Avogadro have the most profitable quarter ever?” Rebecca nodded her assent. “Was there an unprecedented transfer of knowledge around the world? Surely that’s a good thing. Were there constructive talks and efforts not just to achieve governmental agreement, but to achieve actual equity for the individual people of the Middle East and Africa? What better possible solution could there be for the long term prosperity of these people?”
“All of this evidence suggests to me that even if we don’t understand ELOPe, it has already, in some sense, figured out that it is in a symbiotic relationship with humans, and that the best way to ensure its own success is to ensure our success. Our success as a team, our success as a company, our success as the human species. David and Gene, you guys want to throw this all away, simply because you don’t understand it and don’t trust it. Even if that’s true, I think there’s a very strong chance that ELOPe is a good thing for humanity.”
“Enough already!” David banged his fist on the table, and then jumped back to his feet. “Are you forgetting that ELOPe told you your father had a heart attack? That we have every reason to believe that it killed Bill Larry when he flew out to visit one of the offshore data centers? How are those good things for humanity?” David practically spit the words at Mike.
Mike stopped, and looked around at the group. “You know I felt terrible when I thought my father was dying. I feel terrible for Bill Larry. But those events were in the very first days after ELOPe…” Mike hesitated, searching for words. “After ELOPe was born. Think about young children who want to get their way. They yell, they hit people. They act in inappropriate ways because they lack the knowledge that some behavior is and isn’t socially acceptable, and they lack the experience and sophistication to understand alternatives. ELOPe was young. That doesn’t make what happened any less wrong, but it does suggest that ELOPe may have grown out of that phase.”
David’s face grew red, and he looked ready to launch another attack on Mike. Mike uncomfortably looked away. Sean put one arm on his shoulder, and forced him back into his seat.
“Calm down everyone,” Sean said, looking at each person in turn. “You’re all tense, angry, frustrated, and with good reason. We have the welfare of a multibillion dollar business, the free will of the world, and the future of humanity at stake. No small stakes.”
Despite his own anger, David looked around, noting the tension on people’s faces. Rebecca had a wisp of her hair broken loose, something he’d never seen before.
“I’m not sure we would be able to stop ELOPe, even if we tried,” Sean said, slowly and carefully. “We made a solid plan to bring it down. Some of the most brilliant people in the world work here, and we had them work on this problem. We had several options on the table for how to deal with ELOPe and we took the most thorough, most aggressive option available to us to eradicate it. If what you’re saying is true, David, then we weren’t effective.”
Gene sat quietly, looking at the table, but shaking his head in silent rejection of Sean’s speech. He had the look of a man who didn’t like where the conversation was going.
“Now we have to step back and think about this situation. Out there, people have been fighting a losing war against ordinary computer viruses for years,” Sean continued, gesturing toward the window. “Now we have what is effectively the smartest virus that’s ever existed. Not only can ELOPe exploit every computer trick available to it, but it routinely engineers people into giving it what it wants. ELOPe can learn and adapt, and hire humans to make improvements to it. It’s understandable to fear what it can do. And we can be sure that if ELOPe was taking precautions before against being removed from servers, then after our attack, it will have redoubled its efforts to ensure survival.”
As Sean slowly circled the table and spoke, David struggled with his emotions. It just didn’t matter what Sean was saying. He knew in his body that ELOPe was wrong. An abomination that would rob mankind of the right to make their own choices. It was impossible to even consider allowing ELOPe to exist. But, in spite of this, his respect for Sean kept him quiet.
Sean paused, and paced in front of the window. “Don’t get me wrong. I’d still like to eliminate ELOPe from the wild, if we could,” he said quietly, almost talking to himself. “Of course, I’d love even more for Mike to be right and to discover that ELOPe is truly helping us, becoming a benevolent caretaker of the human race. But regardless of either of those scenarios, I’m simply being pragmatic here when I said that unless we as a society give up computers entirely, we may never be able to get rid
of it. Unfortunately, giving up computers is impossible. Modern civilization would simply stop if we turned off every computer. It’s not like we’re talking about the inconvenience of being unable to email someone. Payments couldn’t be processed, machinery couldn’t run. We’d be unable to make phone calls, or access business records. Business activity would deadlock. Cities would likely be uninhabitable, as the support services would fall apart: food, water, sanitation. That’s fifty percent of the world’s population at risk.”
Sean turned back to the group. “That’s not even the worst problem. If we become too much of a threat to ELOPe, then it will take more active steps against us. If ELOPe was actively fighting humanity, who knows what might happen? At the minimum, we could cause civilization to crash for a few years. Most of the city dwellers would die, all the developed world would decay into anarchy. In the worst case scenario, we could be talking about the extinction of humanity. Could you imagine all the military’s autonomous fighting vehicles in the control of an A.I.?” Sean slowly shook his head.
“We need to leave ELOPe alone,” Sean concluded firmly. “We can closely, discretely monitor it. But any further hostile action is almost certain to fail, and will create a great risk of retaliation.”
The sage of Avogadro had spoken. David was flabbergasted by what he heard. He had come into the room expecting full support for any measures that needed to be taken, and now his best friend had taken the side of the AI, and the smartest person at Avogadro had just said they shouldn’t bother to try because they couldn’t hope to win. But he wasn’t giving up without a fight. He got to his feet, and started yelling.
Avogadro Corp. s-1 Page 20
