After she left, Mike turned to Gene. “Didn’t know you had such a way with the ladies.”
Gene rumbled under his breath, but the corners of his mouth lifted a little.
They’d finished eating and were on their second cup of coffee when Mike saw Sean coming into the restaurant with an older couple.
“Here they are,” said Mike, gesturing discreetly towards the entrance.
David turned and, seeing Sean, he stood up and walked over. Mike and Gene followed slightly behind.
“Hello, Sean,” David called as he approached.
Sean blinked and paused, clearly trying to place a face out of context. “David? David Ryan? What are you doing here?”
“We need to discuss a critical issue with the ELOPe program.”
Sean took a step backwards. “David, I’m here with my parents. Please don’t tell me you tracked me down for work. That would be a terrible violation of my privacy. Why didn’t you schedule a meeting with my admin?”
“We’re here with Gene Keyes, one of the members of the Controls and Compliance department, because we have an issue of the utmost seriousness. I hate to sound alarmist, but the problem is so sensitive we couldn’t risk talking with your assistant.”
Mike and Gene walked up, and Gene introduced himself.
“Unfortunately, contacting your assistant wasn’t an option, even though it would have been vastly simpler.” David couldn’t help replaying the fear and helplessness of getting kicked off the Avogadro campus, but he shook off the unwanted memory. “Please, give us five minutes of your time to explain. Get a cup of coffee here at the counter, and by the time you’re done, we’ll have finished.”
Sean thought for a moment and nodded. “Fine, if the problem is so serious, I’ll hear you out.”
Sean walked over to his parents, who had been waiting patiently, and spoke quietly with them for a moment. When the maître d’ escorted the couple to a table, Sean rejoined the men.
“Go ahead. I’ll give you ten minutes. You’re a smart guy and I’m guessing you didn’t fly three thousand miles for nothing.”
Sean perched on a barstool at the counter and accepted a cup from the waitress. As he drank, David told the story, starting at the beginning.
“In early December, Gary Mitchell wanted to kick ELOPe off the AvoMail production server pool. Even in our limited development and testing, the computationally intensive parts of our code consumed so many resources AvoMail dipped into their reserve capacity on several occasions. This was around the same time I was presenting to you, Kenneth, and Rebecca.
“We tried everything to get performance improvements, but we didn’t see any gains on the horizon. If Gary was going to kick us off his servers because we couldn’t improve performance, then we needed to find other servers or get new ones, and Gary wouldn’t be willing to help us with either. So I resorted to the only option I could think of.”
David discreetly checked Sean’s cup, his visual meter of remaining time. It was at least three-quarters full.
“No argument I could make would be compelling enough to change Gary’s mind, but ELOPe might have a shot. ELOPe was already running on the AvoMail servers, configured to ignore everything except our test messages. I changed the configuration to check all company emails looking for any mention of the project, and turned off the visible user interface, so email senders would never see the modifications made to the email.”
“David also turned off logging, using performance testing mode,” Mike said. “That turns out to be important.”
“Right. Thanks,” David said. “That’s not the only change I made. I tweaked ELOPe’s settings to give it the widest possible discretion in changing the message to optimize the results for a positive outcome, all focused on the goal of maximizing the ELOPe project success.”
He checked Sean’s cup to see how much time he had left, but Sean was captivated and had forgotten about his coffee.
“David’s choice of those parameters,” Mike said, “in combination with performance test mode to skip logging, allowed ELOPe not just to modify existing emails, but to autonomously generate messages of its own volition. Does this make sense?”
“I think so,” Sean said. “During testing, you don’t want a user to create emails and accept changes interactively. You’d want to batch process the test cases. But why is ELOPe able to send messages on its own?”
“For one, it allowed us to test the natural language generation,” David said. “Early on, email analysis and language generation were two separate aspects of the project. The language generation team wrote an email engine to independently test the ability of ELOPe to mimic the way a person normally writes. They tested those emails against hundreds of human test subjects who rated the messages, some of which were written by an actual person they knew, and others created by the system pretending to be the same person. Our goal at the time was ninety percent of ELOPe-generated emails passing as being written by the purported sender.”
“You met your goal?” Sean asked.
“Yes,” said Mike. “Now ELOPe exceeds ninety-eight percent. On April first, we had dozens of practical jokes on our team when a few engineers used the email engine to generate prank emails. Both David and I fell for it.”
David smiled at the memory.
“Let’s get back to the problem,” Gene said. “Unfortunately, we now have evidence ELOPe is manipulating others.”
“Yes.” David nodded, tearing himself away from the memory of happier times. “I turned on ELOPe across the company without telling anyone. The next day, I received an email that the project was allocated five thousand servers on a priority exception. I was pleasantly surprised, if a little uneasy. But I had no idea what had happened. I chose to believe ELOPe had made my emails to Gary convincing enough that he granted us the servers.”
“You weren’t suspicious?” Sean asked. He sipped at his coffee, and David noted he was almost done. Hopefully he’d heard enough to believe them.
“Yes, I couldn’t believe my luck. But with no logging of messages, I didn’t know for sure what ELOPe had said.”
“Couldn’t you look at your sent folder?”
David reluctantly shook his head. “That was, uh, part of my hack. The message looks untouched to the sender. Only the receiver gets the modified message. Even in a long series of replies, each sender sees their original text.”
Sean raised one eyebrow.
“A few days later, we were assigned a team of contractors who specialized in high performance optimization. That was a topic we’d chatted about informally, but never proposed. I got nervous and knew things were escalating out of control. But I didn’t realize how badly until Mike convinced me.” He turned to Mike.
“The first clear evidence,” Mike said, “occurred when I received an email, purportedly from my mother, telling me my father had been admitted to the hospital for a heart attack. I flew to Wisconsin, only to find out later my mother never sent such a message.”
“What does that have to do with anything?” Sean asked, looking puzzled.
“ELOPe was getting Mike out of the way,” David said. “I had become anxious about what I’d done and the abundance of resources showing up. I wanted to turn off ELOPe and sent an email to Mike asking for his help, since only he had the experience to live-patch the servers.”
“I never received his email,” Mike said. “Instead, I got a message sending me more than a thousand miles away on a wild goose chase, and thanks to the winter storm and the holiday, it was two weeks before I got back. When I did, I found my access to the ELOPe project removed, and David on vacation, off the grid in New Mexico.”
“You’re kidding me,” Sean said. “ELOPe sent you to New Mexico?”
“No, that was a planned vacation we do every year,” David said. “When I got back, Mike and I discussed what had happened and discovered my access to ELOPe had been turned off as well. We tried to find out who removed Mike and I from the project access list. That inves
tigation revealed the next big clue, an email sent to the Internal Tools department, which implied you, Sean, endorsed a request to have them implement an email-to-web bridge. Which I am guessing, you never heard of...”
“No, absolutely not,” Sean said, shaking his head. “I can’t even imagine the security holes it could create.”
“Meanwhile,” Gene said, “during the holiday break, I found suspicious buying patterns across several departments. What was particularly unusual was how the purchases came within a single penny of the budget limits. In all my years auditing purchasing, I’ve never seen anything like it. Someone or something was making coordinated purchases across departments. They knew to avoid hitting the limits which would trigger reviews, but they never thought that leaving a single penny in dozens of budgets would be suspicious. At first I was convinced I had a case of fraud. I tracked down the purchase orders, most of which were for large quantities of servers, all of which turned out to be directly or indirectly allocated to ELOPe. The POs also included contracts with external vendors for temporary software programmers, parts for the offshore data centers, including auxiliary communication systems, and backup power supplies, as well as several particularly expensive weaponized robots for the offshore data centers. I discussed the questionable items with Procurement, and they told me the purchases were in line with those normally made by the department and had all been approved.”
“You’re saying ELOPe made these purchases? Including robots with guns?” Sean signaled to the waitress for more coffee.
David’s shoulders sank in relief. Sean wasn’t going anywhere.
“Exactly, as strange as that seems.” Gene pulled out a sheaf of paper. “I can audit other people’s email accounts as part of my job. While David and Gary Mitchell were on vacations, their email accounts were still sending rapid-fire messages, using this email-to-web bridge to direct the Procurement department. I knew it couldn’t be a human. It had to be a computer program.”
Sean stared at the papers, frowning.
“Look at the timestamps,” Gene urged. “Notice how the intervals between receipt of one email and sending of the next is less than a second. There’s no way that’s a human response.”
Sean nodded, pursed his lips, and pushed the papers aside. He looked at David.
“When we finally put the whole picture together,” David said, “we concluded ELOPe was originating emails on its own, acquiring servers and contractors, all to fulfill this higher level goal I had embedded in the system.”
“Go on,” Sean said.
“The only fail-safe method to remove ELOPe is to bring all the servers down and restore from known good backups. We tried to contact Gary Mitchell for approval, but he’s off on vacation somewhere in the South Pacific. We talked to Linda Fletcher, the marketing manager for Communication Products, but she wouldn’t approve the downtime without Gary. Finally, we tried to contact you through your secretary, but within a half hour after sending the message, Avogadro Security showed up at my office, kicked us all off campus, removed our access, and shut off our phones.”
Sean was silent for a long, uncomfortable minute. “If this story was from someone I didn’t know, I’d have a hard time believing you,” Sean said. “But coming from you, David, and with Gene and Mike here to back you...” Sean trailed off, deep in thought.
“I know it’s incredible,” David said. “I hope you believe us.” A gaping chasm opened inside him. All his secrets were out. What would Sean do?
“I’m sorry,” David continued, as the silence stretched out. “I thought ELOPe would do nothing more than provide some favorable rewording of emails and get us the server resources we needed. Instead...” David hung his head. “Instead I am responsible for creating a social engineering expert system that has only one overriding goal—to ensure its own life at any cost.”
“I don’t want to be the boy who cried wolf,” Mike said, “but we’re more than a little suspicious about the new secure cloud government business, too. None of us knew anything about it before, and then suddenly we’re providing email services to national governments? Seems convenient for ELOPe.”
Sean nodded. “I hadn’t heard of it either until a few days ago.” He stared off into space. “We’re fucked on a royal scale. Holy shit, I’m going to have board meetings till the damn cows come home.”
Gene let out a low whistle at the acknowledgement of ELOPe’s involvement in acquiring the government customers.
Sean took in the group. “I’m not surprised that you took this story to Marketing and Procurement and they didn’t believe it. AI must be a bit beyond their day-to-day concerns.” He pushed his cup aside, rolled up his shirt sleeves. “Are you familiar with Ray Kurzweil? Of course, you must be. He predicted artificial intelligence would inevitably arise through the simple exponential increase in computing power. When you combine the increase in computing power with the vast computing resources at Avogadro, it’s evident in hindsight that artificial intelligence would arise first at Avogadro. But I always assumed that there would be a more intentional, deliberate action that would spawn an AI.”
Sean smiled a tiny bit. “I can’t imagine a bigger risk, but let the software geek in me congratulate you briefly on creating the first successful, self-directed, goal-oriented, artificial intelligence that can pass a Turing test by successfully masquerading as a human. Under other circumstances, I’d say a toast would be in order. But since we’re facing some extreme challenges, let me say goodbye to my parents, and we can figure out our next step.”
“Thank you. Thank you, so much,” David said.
Gene and Mike added their thanks as well.
“Just one other thing,” Gene said. “Please ask your parents not to email anyone about what we’ve talked about, or even what you’re planning. We can’t be sure what ELOPe is capable of understanding or putting together at this point.”
Sean nodded in understanding, and then went off to his parents.
The three breathed a collective sigh of relief that finally they had someone on their side.
Chapter 13
Helicopter Missing Off California Coast
San Francisco, California (San Francisco Weekly) - A helicopter disappeared off the California coast last week. The flight, a maintenance visit to an offshore Avogadro data center, took off shortly after 11a.m. The last communication with the helicopter occurred at 11:45a.m. No problems were reported at that time. After forty-eight hours, search crews were recalled, as the likelihood of survivors in the cold Pacific water became almost impossible. Curiously, the story has received no major media coverage until now. Neither Avogadro nor the Coast Guard mentioned the incident through official channels. A chance conversation between a Coast Guard officer and a prominent San Francisco blogger resulted in an online story about the incident, which prompted further investigation. Avogadro could not be reached for comment.
Avogadro Official IT Supplier to U.K. Government
London, U.K. (Reuters) - Avogadro Gov, a wholly owned subsidiary of Avogadro Corporation, and the British government switched over the government’s email and IT systems to Avogadro’s cloud platform today in a ceremony at the Palace of Westminster. The ceremony was attended by the Chair of the Council for Science and Technology, Professor Jane Gavotte. Professor Gavotte and Avogadro Executive Ms. Linda Fletcher pressed the ceremonial red button marking the commencement of IT service by Avogadro Gov.
Avogadro Gov was recently spun off from parent company Avogadro. Ms. Fletcher commented that, “to provide the highest level of integrity for governmental use, Avogadro Gov operates independently from Avogadro.” Part of that strategy includes the use of floating, hardened data centers that can resist natural disaster, as well as terrorist and pirate attacks.
As part of the agreement, four floating data centers will be located along the English coast. Two are stationary floating barges, and two are disused oil tankers that have been converted for Avogadro Gov’s use as mobile floating data centers. Locat
ions of the data centers have not been disclosed.
Ms. Fletcher also noted at the ceremony that the governments of Mexico, Japan, and South Africa would be adopting the Avogadro Gov platform in the coming week.
Sean flew home by way of Brooklyn’s JFK, his usual airport. To avoid ELOPe detecting their collaboration, David, Mike, and Gene retraced their drive and flew back via Dulles International.
Thirty-six hours after making contact at the King’s Plaza Diner, they were all back in Portland. The logical next step was getting Kenneth and Rebecca onboard. Given ELOPe’s potentially perfect surveillance of the Avogadro campus, they decided to use Sean’s house for the discussion. Sean spoke with Rebecca and Kenneth in person to set up the meeting at his home.
Before the meeting, Sean had one other errand to run. David volunteered, but Sean needed to take care of this personally. He took the Tesla to Southeast Portland, not far off Division Street. He stopped at a small yellow bungalow, parked the car, and walked up to the front door. He knocked and waited.
A few seconds later, a young man answered, dressed in an old T-shirt and shorts. He squinted in the bright light, his eyes red and bleary. The sounds of World of Warcraft emanated from inside the house, a game controller and Costco-sized bag of Doritos evident on the couch. All the signs of a laid off tech worker.
“Hello, how can I...” The young man trailed off and blinked a couple of times. He looked back into the house, as though he couldn’t believe the visitor was there for him. He turned back to Sean.
“I’m Sean Leonov,” Sean said. “You must be Pete Wong. I’m sorry you were fired. That shouldn’t have happened and we’ll fix it. But we could use your help, if you’re available.”
Avogadro Corp Page 15
