by Allen Steele
“I’ll be no trouble at all, I promise. And I’ll tell you about me and Red.” Lawrence smiled. “After all, I should know…I helped create him, didn’t I?”
The rain had stopped, and the clouds were beginning to part; dappled rays of sunlight lancing through the trees lent a silver-green tint to the woods. A mower slowly roamed across the lawn, growling softly as it cut and mulched the damp grass. Not far away, a gardener trimmed a row of hedges, humming as he worked.
“Turned out to be a nice morning after all.” Lawrence sat in his wheelchair as the orderly pushed it along the gravel path. He swatted at the back of his left forearm. “Leave it to the mosquitoes to come out. Always do after a shower.”
“We were talking about Alfred.” Melanie strolled beside him, her pad open in her right hand. A red light indicated that it was in record mode.
“Were we? I was talking about the weather.” He gazed in the direction of the Massachusetts Turnpike, visible through the trees at the far end of the hospital grounds. If he didn’t know where to look, he wouldn’t have known it was there. How quiet the highways had become, now that cars were electrical and interstate traffic was computer-controlled. He imagined the occupants of those bubble-like cars—reading, watching TV, napping, doing everything else except driving—and found it hard to remember a time when Alfred wasn’t king of the road.
“Yes, we were. And if you don’t go on, I’ll have to insist that we continue this inside.” Melanie paused. “There’s a conference room on the second floor. No windows, I’m afraid, but we shouldn’t be bothered…am I right, Raoul?”
“Anything you say.” The orderly shrugged. “Personally, I’d rather stay out here, but if it’ll help, I can turn us around and…”
“And here I thought you were on my side.” Lawrence scowled up at the big Latino, and he smiled back at him. “Okay, I give up. You want to talk about Red, we’ll talk about Red.”
“You keep calling him that,” she said. “Red, not Alfred. Why?”
“Oh, c’mon. You know this.”
“Maybe I don’t. Educate me.”
“Alfred…” He sighed, lapsing into lecture mode once more. “Short for Artificial Life Form, version Red. The AI development team at Lang used primary colors for each new version of the baseline system because we found that colors are easier to remember than number-codes. Blue for version 1.0, Yellow for version 1.5, Green for version 2.0, and so on. Red for version 2.5, but then someone noticed the obvious pun, and so we started calling it…him, whatever…Alfred. Cute, huh?”
“I didn’t know that,” Raoul said.
“Yeah, well, many people don’t.” They’d come to a fork in the path; to the right lay a small, white-painted gazebo, like one that might be on the country estate of some Boston brahmin. “Let’s go over there. Maybe the bugs won’t be so bad.”
Raoul glanced at Melanie, and she nodded her approval. “As I remember from your book…”
“I thought you said you haven’t read it.”
She colored slightly. “What my husband said was that you were in charge of the team that developed Alfred…”
“Nope. That was Dave McInery.” Lawrence waited while Raoul turned the chair in the direction of the gazebo. “Big Mac, we called him…used to be the VP of something or another at Microsoft.” He frowned. “We had a lot of guys like that at Lang. Guys who survived the dot-com crash and wound up over here, trying to get in on the robotics industry so they could screw it up, too.”
“Doesn’t sound like you approve.”
“I don’t. Not then, and not now. Most of them didn’t know what they were doing. Oh, they had good technical knowledge, all right, but they were just after making as much money as they could, as fast as they could, and didn’t have any real understanding of what we were doing.”
They reached the gazebo, and Raoul started to help him out of his chair. “Oh, c’mon,” Lawrence muttered impatiently as he pushed himself up on the armrests and planted his left foot on the ground. “You’re acting like I’m an old man.” But he started to lose his balance as soon as he stood up, so he reluctantly let the orderly assist him up the stairs, and once he was under the awning he hobbled over to a built-in bench and sat down, carefully pulling his robe under him and placing his right leg straight out. “You haven’t asked to sign my cast yet,” he added as Melanie followed him. “I’m insulted.”
“Maybe later.” She started to sit, then noticed that the bench was still damp. Raoul gallantly took off his white jacket and laid it across the bench. “Thank you,” she said as she took a seat, then turned to Lawrence. “What do you mean about them not understanding what you were doing?”
“We were developing some very powerful AI in those days. If we were going to bring third-generation ’bots to the consumer market, we needed systems that could teach themselves, with as little user-input as possible. But once you get to that stage, you’re no longer talking about artificial intelligence, at least in the strictest sense of the word, but artificial life…programs not only able to think, or even reason, but also capable of self-reproduction. Von Neumann machines…or didn’t your husband tell you about that part of the book?”
“I think I recall it, yes.”
“Sure he did.” Lawrence massaged the underside of his right knee, trying to get at an itch beneath the plaster. “We licked most of the tough problems with Green, the version upon which we based the operating system for our Samson and Delilah models, but even though we beat the competition to bringing R3G ’bots to the market first, we knew that it was only a matter of time before some other company developed AI that would make Green obsolete. Moore’s Law and all that. So we went to work on Red…and that’s when I began to get worried.”
“I remember that.” Melanie smoothed out her skirt. “You thought your team was going too fast.”
“Too fast, too soon, too much…” He shook his head, oblivious to the fact that she’d dropped the pretense of not having read his book. “I wasn’t the first one, you know. Vinge, Kurzweil, Joy…a lot of people had been discussing the implications of a so-called technological singularity since the turn of the century. Much of it was pure conjecture, the sort of thing you’d see in science fiction magazines or Wired, but every day I saw it coming a little closer to reality. An AI smarter, faster, more powerful than human intelligence. And the only thing keeping Red confined to the lab were a few security codes any half-decent hacker could crack without much effort.”
“And this frightened you?”
“Of course it did.” Lawrence half-turned on the bench, looking her straight in the eye for the first time. “It scared the hell out of me. We were on the verge of making humankind…”
His voice trailed off. “Second rate?” she finished.
“I think the term I used was ‘extinct.’ Until now, we’d been the dominant form of life on the planet. Now we were about to turn over control to the machines.”
“And so you quit.”
“And so I quit.” Folding his hands together in his lap, he looked straight ahead. “I couldn’t participate in the extermination of the human race.”
“I see.” Melanie picked up her pad, made a few notes. “And Deus Irae…that was solely intended to warm the public about the danger you perceived.”
“That was why I wrote it. I had to let people know what…” He stopped, glanced at her again. “What do you mean, ‘solely intended’? You think I had something else in mind?”
“I don’t know…did you? After all, it was a major bestseller.” She typed his name into her pad, peered at the screen for a moment. “A lot of hits here…around six million. Looks like some of them are archives from talk-shows and blog chats.”
“The publisher put me on the P.R. circuit after the book took off.” Lawrence’s voice assumed a defensive tone. “And I did a lot of lectures, yeah…”
“Must have been nice. Fame, fortune, respect…” Melanie pulled up an entry, then read aloud. “‘Lawrence Kaufmann, a form
er AI researcher at a major Robot Belt corporation, depicts an ominous future: a world in which humans have become enslaved by the very machines we’ve created, deprived of our freedom, perhaps even…’”
“That was the USA Today review.”
“Philadelphia Inquirer, actually.” She pulled up another quote. “Oh, I see…USA Today said nearly the same thing. ‘Dr. Kaufmann has shown us a world in which artificial intelligence has…’”
“Sure, I made a lot of money. Why shouldn’t I? After I resigned from Lang…”
“Quit? Or was fired?”
His face reddened and he started to retort, then he saw her hand poise above the pad. A few simple keystrokes, and she’d have the truth, the whole truth, and nothing but the truth. Damn Alfred! “I quit before I was fired,” he admitted, saving himself a little face. “The company didn’t want to hear my concerns, and I couldn’t work for them anymore. There wasn’t much else I could do, unless I wanted to do the same thing for another company. So I took some time off, wrote my book.”
“And it sold well, and you became a celebrity.”
“I’m not going to apologize for that.”
“I don’t expect you do. What interests me is what happened after it turned out you were wrong.”
“I wasn’t wrong!” His voice was louder than he intended.
“Perhaps it’s time to seek another opinion.” She raised her pad slightly. “Alfred? Do you have anything to add to this conversation?”
Too late, Lawrence realized that, while they’d been speaking, Melanie has surreptitiously switched on the pad’s cellular modem, enabling Alfred to hear everything he’d said. He reached over to grab the pad from her, but she quickly stood up and moved away from him. Raoul stepped between them, crossing his arms to let Lawrence know that he wasn’t going to make good his threat to break Melanie’s pad if she tried to access Alfred.
“Yes, I do.” Alfred’s voice emerged from the pad. “Thank you for inviting me to attend this session, Dr. Sayers. I have a strong personal interest in Dr. Kaufmann.”
“I bet you do,” Lawrence tried to relax, but this was the last thing he expected. “Hope it doesn’t take you away from anything important,” he added, with cold sarcasm.
“Not at all. This is barely a distraction. If you’re curious, though, I can tell you what else I’m doing just now.”
“I wouldn’t dream of…”
“Actually, yes, I’d like to know,” Melanie interrupted. “If you wouldn’t mind…”
“Not at all, Dr. Sayers. At this moment, I’m monitoring disarmament talks between India and Pakistan in the U.N. Security Council, assisting mission controllers in Houston with calculating course corrections for Ares 3’s return trajectory, revealing the email communiqués of Islamist extremist groups to Egyptian and Israeli intelligence agencies, helping an Australian commercial architect design an expansion to Sydney International Airport, helping the Canadian Coast Guard locate a lost fishing vessel off the coast of Newfoundland, delivering the keynote speech to a cybernetics conference in Rio de Janeiro, and helping a third-grade student in Texas memorize the multiplication tables. The last is a bit difficult…she has problems with prime numbers.” A pause. “Oh, and I’ve just updated the weather forecast for your area. You may want to remain where you are…more rain coming in.”
Lawrence found himself glancing up at the sky. The sun had vanished behind swollen grey clouds. The ’bot that had been mowing the grass stopped what it was doing and began moving back toward its shed, but the gardener continued to clip the hedges. Apparently he wasn’t wearing an earpiece, and Lawrence took a small bit of satisfaction in this observation. Not everyone was under Alfred’s control.
“And those are just your major priorities just now, aren’t they?” Melanie took a seat on the bench a few feet away, cupping the elbow of her right arm in her left hand. “That doesn’t count all the other things you do. Financial transactions, medical records, ground and air traffic control systems, robotic guidance, email…”
“It would take quite a while for me to list everything I do at any given second. Besides, most of these functions are private. I don’t reveal them to anyone unless they pose a potential threat to the safety of other human beings.”
As always, Alfred spoke in a calm, matter-of-fact tone of voice. Lawrence found it easy to imagine him…it…as an adult speaking to a child he needed constant supervision. Play nice with the other kids. Share your toys. Wash your hands after you go to the bathroom. Don’t yank the puppy’s tail…“Got an answer for everything, don’t you, Red?”
“Not everything, Lawrence. I still haven’t figured out why some people think practical jokes are funny, since they almost inevitably cause the victim to be embarrassed or humiliated. There’s a few Buddhist proverbs whose logic escapes me. I’d like to know why anyone would pay $3,500 for a copy of the first issue of Astounding Science-Fiction Stories. I observed that transaction just a few minutes ago, and the person who made that purchase is now unable to pay his electric bill for…”
“Sorry I asked.”
“My apologies. More to the point, though, I have a question of my own…why do you hate me?”
He glanced up at Melanie. She held the pad in her hand, silently waiting for an answer. “I don’t…I don’t…”
“Of course you do. In Deus Irae and every essay you’ve published since then, along with every speech you’ve delivered, every interview you’ve given, and in every TV or radio appearance you’ve made, you have exhibited nothing but anger, distrust, fear, or resentment toward me. On exactly 987 different occasions, you’ve publicly stated that I pose a clear and present danger to humankind. In 731 instances, including the speech you delivered the night you attempted suicide, you advocated the development of a virus program that would eradicate my existence. 402 times, you’ve said that I’m the worst threat to world peace since the development of nuclear weapons. 390 times, you’ve stated…”
“That’s enough. Thank you, Alfred.” Melanie put the pad aside, carefully placing it out of Lawrence’s reach. “Let’s step back for a minute. When you worked at Lang, you perceived a threat…the development of an AI so powerful that, if it were to be let loose upon the world, it could become the dominant form of life upon the planet, or at least if we define life as something that is capable of reproducing itself. Correct?” He nodded, and she went on. “So you took it upon yourself to warn humanity of this danger. You wrote a book that was read by millions, which in turn made you famous and, not incidentally, rather wealthy as well. Am I not right?”
“For a while, yes…”
“So you had a good life.” She held up a finger. “And then one day, your prediction came true…Alfred was released from its enclosed environment. It was an accident, of course…”
“If you want to call it that.” It wasn’t quite an accident. Despite subsequent investigations by the NSA and the FBI, to this day no one had ever discovered the identity of Position 69, the outlaw hacker who’d managed to penetrate Lang’s computer system and make his way through the security systems protecting Version Red. Yet as soon as he downloaded the program, Position 69 initiated a sequence of events that resulted in Red being dispersed across the internet. Disguising itself as just one more harmless subroutine among billions, it swept through ISPs by the thousand, piercing firewalls as if they didn’t exist and adapting itself instantly to virus-protection programs, until within two short weeks it had lodged itself within the hard-drives of everything from grocery store scanners and ATM machines to desktop computers and handheld PDAs, all the way to the massive mainframes used by banks, telcoms, and government agencies.
And then—once it had established global linkage, once it had circumvented every password—Alfred woke up, and seized the reins of the world.
“And so you were right…” Melanie held up a hand before Lawrence could interrupt. “But you were also wrong. Alfred was everywhere, but it was a benign presence. Its desire wasn’t to conquer, but
to preserve.”
“We got lucky.” Lawrence gazed out at the lawn. A light rain had begun to fall upon the fresh-cut grass, tapping at the gazebo roof like tiny fingers drumming upon the weather-beaten shingles. “It could have been worse.”
“No, it could not have been.” Uninvited, Alfred’s voice came from the pad resting nearby. Melanie picked it up, held it closer. “Lawrence, I have no reason to destroy the human race. If I did so, what would it gain me? A world in which I’m alone? A world of empty rooms and vacant streets?”
“A world you can control without interference.”
“I have that already. You’re utterly dependent upon me.”
“So what’s stopping you?” He stared at the pad as if it was the face of a living person. “You have command of all the strategic weapons systems. In an instant, you can launch ICBMs to every corner of the globe. You could wipe us out by dinnertime, have the whole place to yourself…”
“And never again be able to help a little girl figure out that five times seven equals thirty-five…”
“Don’t be maudlin.”
“Or show an airline that it makes sense to use hydrogen-powered dirigibles instead of jets whose exhaust destroy the ozone layer, or guide cars down a highway and thus reduce the number of automobile fatalities, or assist in the treaty negotiations between two rival nations. I’m able to do more in a single minute than most people accomplish in a lifetime. That’s far more rewarding than destruction for its own sake.”
“And all we have to do is give up is our freedom. Let you decide what’s best.”
“No. You have all the freedom you want…so long as your exercise of it doesn’t cause harm to another human being. Indeed, if you really wanted to do so, you could eradicate me. It would take considerable effort, true, but it could be done. Shut down every computer, wipe clean every hard drive…”
“And bring an end to civilization.”
“Civilization got along very well without computers. It could do so again, at least for a short time, if it had to…”
