by Greg Egan
Daniel was disappointed; he’d expected her at least to understand what was at stake. He said, “This is a race to decide between Godhood and enslavement. Whoever succeeds first will be unstoppable. I’m not going to be anyone’s slave.”
Julie stepped into the elevator; he followed her.
She said, “You know what they say the modern version of Pascal’s Wager is? Sucking up to as many Transhumanists as possible, just in case one of them turns into God. Perhaps your motto should be ‘Treat every chatterbot kindly, it might turn out to be the deity’s uncle.’”
“We will be as kind as possible,” Daniel said. “And don’t forget, we can determine the nature of these beings. They will be happy to be alive, and grateful to their creator. We can select for those traits.”
Julie said, “So you’re aiming for übermenschen that wag their tails when you scratch them behind the ears? You might find there’s a bit of a trade-off there.”
The elevator reached the lobby. Daniel said, “Think about this, don’t rush to a decision. You can call me any time.” There was no commercial flight back to Toronto tonight; she’d be stuck in a hotel, paying money she could ill-afford, thinking about the kind of salary she could demand from him now that she’d played hard to get. If she mentally recast all this obstinate moralizing as a deliberate bargaining strategy, she’d have no trouble swallowing her pride.
Julie offered her hand, and he shook it. She said, “Thank you for dinner.”
The taxi was waiting. He walked with her across the lobby. “If you want to see AI in your lifetime,” he said, “this is the only way it’s going to happen.”
She turned to face him. “Maybe that’s true. We’ll see. But better to spend a thousand years and get it right, than a decade and succeed by your methods.”
As Daniel watched the taxi drive away into the fog, he forced himself to accept the reality: she was never going to change her mind. Julie Dehghani had been his first choice, his ideal collaborator. He couldn’t pretend that this wasn’t a setback.
Still, no one was irreplaceable. However much it would have delighted him to have won her over, there were many more names on his list.
2
Daniel’s wrist tingled as the message came through. He glanced down and saw the word PROGRESS! hovering in front of his watch face.
The board meeting was almost over; he disciplined himself and kept his attention focused for ten more minutes. WiddulHands.com had made him his first billion, and it was still the pre-eminent social networking site for the 0-3 age group. It had been fifteen years since he’d founded the company, and he had since diversified in many directions, but he had no intention of taking his hands off the levers.
When the meeting finished he blanked the wall screen and paced the empty conference room for half a minute, rolling his neck and stretching his shoulders. Then he said, “Lucien”.
Lucien Crace appeared on the screen. “Significant progress?” Daniel enquired.
“Absolutely.” Lucien was trying to maintain polite eye contact with Daniel, but something kept drawing his gaze away. Without waiting for an explanation, Daniel gestured at the screen and had it show him exactly what Lucien was seeing.
A barren, rocky landscape stretched to the horizon. Scattered across the rocks were dozens of crab-like creatures – some deep blue, some coral pink, though these weren’t colors the locals would see, just species markers added to the view to make it easier to interpret. As Daniel watched, fat droplets of corrosive rain drizzled down from a passing cloud. This had to be the bleakest environment in all of Sapphire.
Lucien was still visible in an inset. “See the blue ones over by the crater lake?” he said. He sketched a circle on the image to guide Daniel’s attention.
“Yeah.” Five blues were clustered around a lone pink; Daniel gestured and the view zoomed in on them. The blues had opened up their prisoner’s body, but it wasn’t dead; Daniel was sure of that, because the pinks had recently acquired a trait that turned their bodies to mush the instant they expired.
“They’ve found a way to study it,” Lucien said. “To keep it alive and study it.”
From the very start of the project, he and Daniel had decided to grant the Phites the power to observe and manipulate their own bodies as much as possible. In the DNA world, the inner workings of anatomy and heredity had only become accessible once highly sophisticated technology had been invented. In Sapphire, the barriers were designed to be far lower. The basic units of biology here were “beads”, small spheres that possessed a handful of simple properties but no complex internal biochemistry. Beads were larger than the cells of the DNA world, and Sapphire’s diffractionless optics rendered them visible to the right kind of naked eye. Animals acquired beads from their diet, while in plants they replicated in the presence of sunlight, but unlike cells they did not themselves mutate. The beads in a Phite’s body could be rearranged with a minimum of fuss, enabling a kind of self-modification that no human surgeon or prosthetics engineer could rival – and this skill was actually essential for at least one stage in every Phite’s life: reproduction involved two Phites pooling their spare beads and then collaborating to “sculpt” them into an infant, in part by directly copying each other’s current body plans.
Of course these crabs knew nothing of the abstract principles of engineering and design, but the benefits of trial and error, of self-experimentation and cross-species plagiarism, had led them into an escalating war of innovation. The pinks had been the first to stop their corpses from being plundered for secrets, by stumbling on a way to make them literally fall apart in extremis; now it seemed the blues had found a way around that, and were indulging in a spot of vivisection-as-industrial-espionage.
Daniel felt a visceral twinge of sympathy for the struggling pink, but he brushed it aside. Not only did he doubt that the Phites were any more conscious than ordinary crabs, they certainly had a radically different relationship to bodily integrity. The pink was resisting because its dissectors were of a different species; if they had been its cousins it might not have put up any fight at all. When something happened in spite of your wishes, that was unpleasant by definition, but it would be absurd to imagine that the pink was in the kind of agony that an antelope being flayed by jackals would feel – let alone experiencing the existential terrors of a human trapped and mutilated by a hostile tribe.
“This is going to give them a tremendous advantage,” Lucien enthused.
“The blues?”
Lucien shook his head. “Not blues over pinks; Phites over tradlife. Bacteria can swap genes, but this kind of active mimetics is unprecedented without cultural support. Da Vinci might have watched the birds in flight and sketched his gliders, but no lemur ever dissected the body of an eagle and then stole its tricks. They’re going to have innate skills as powerful as whole strands of human technology. All this before they even have language.”
“Hmm.” Daniel wanted to be optimistic too, but he was growing wary of Lucien’s hype. Lucien had a doctorate in genetic programming, but he’d made his name with FoodExcuses.com, a web service that trawled the medical literature to cobble together quasi-scientific justifications for indulging in your favorite culinary vice. He had the kind of technobabble that could bleed money out of venture capitalists down pat, and though Daniel admired that skill in its proper place, he expected a higher insight-to-bullshit ratio now that Lucien was on his payroll.
The blues were backing away from their captive. As Daniel watched, the pink sealed up its wounds and scuttled off toward a group of its own kind. The blues had now seen the detailed anatomy of the respiratory system that had been giving the pinks an advantage in the thin air of this high plateau. A few of the blues would try it out, and if it worked for them, the whole tribe would copy it.
“So what do you think?” Lucien asked.
“Select them,” Daniel said.
“Just the blues?”
“No, both of them.” The blues alone might have dive
rged into competing subspecies eventually, but bringing their old rivals along for the ride would help to keep them sharp.
“Done,” Lucien replied. In an instant, ten million Phites were erased, leaving the few thousand blues and pinks from these badlands to inherit the planet. Daniel felt no compunction; the extinction events he decreed were surely the most painless in history.
Now that the world no longer required human scrutiny, Lucien unthrottled the crystal and let the simulation race ahead; automated tools would let them know when the next interesting development arose. Daniel watched the population figures rising as his chosen species spread out and recolonized Sapphire.
Would their distant descendants rage against him, for this act of “genocide” that had made room for them to flourish and prosper? That seemed unlikely. In any case, what choice did he have? He couldn’t start manufacturing new crystals for every useless side-branch of the evolutionary tree. Nobody was wealthy enough to indulge in an exponentially growing number of virtual animal shelters, at half a billion dollars apiece.
He was a just creator, but he was not omnipotent. His careful pruning was the only way.
3
In the months that followed, progress came in fits and starts. Several times, Daniel found himself rewinding history, reversing his decisions and trying a new path. Keeping every Phite variant alive was impractical, but he did retain enough information to resurrect lost species at will.
The maze of AI was still a maze, but the speed of the crystal served them well. Barely eighteen months after the start of Project Sapphire, the Phites were exhibiting a basic theory of mind: their actions showed that they could deduce what others knew about the world, as distinct from what they knew themselves. Other AI researchers had spliced this kind of thing into their programs by hand, but Daniel was convinced that his version was better integrated, more robust. Human-crafted software was brittle and inflexible; his Phites had been forged in the heat of change.
Daniel kept a close watch on his competitors, but nothing he saw gave him reason to doubt his approach. Sunil Gupta was raking in the cash from a search engine that could “understand” all forms of text, audio and video, making use of fuzzy logic techniques that were at least forty years old. Daniel respected Gupta’s business acumen, but in the unlikely event that his software ever became conscious, the sheer cruelty of having forced it to wade through the endless tides of blogorrhea would surely see it turn on its creator and exact a revenge that made The Terminator look like a picnic. Angela Lindstrom was having some success with her cheesy AfterLife, in which dying clients gave heart-to-heart interviews to software that then constructed avatars able to converse with surviving relatives. And Julie Dehghani was still frittering away her talent, writing software for robots that played with colored blocks side-by-side with human infants, and learned languages from adult volunteers by imitating the interactions of baby talk. Her prophecy of taking a thousand years to “get it right” seemed to be on target.
As the second year of the project drew to a close, Lucien was contacting Daniel once or twice a month to announce a new breakthrough. By constructing environments that imposed suitable selection pressures, Lucien had generated a succession of new species that used simple tools, crafted crude shelters, and even domesticated plants. They were still shaped more or less like crabs, but they were at least as intelligent as chimpanzees.
The Phites worked together by observation and imitation, guiding and reprimanding each other with a limited repertoire of gestures and cries, but as yet they lacked anything that could truly be called a language. Daniel grew impatient; to move beyond a handful of specialized skills, his creatures needed the power to map any object, any action, any prospect they might encounter in the world into their speech, and into their thoughts.
Daniel summoned Lucien and they sought a way forward. It was easy to tweak the Phites’ anatomy to grant them the ability to generate more subtle vocalizations, but that alone was no more useful than handing a chimp a conductor’s baton. What was needed was a way to make sophisticated planning and communications skills a matter of survival.
Eventually, he and Lucien settled on a series of environmental modifications, providing opportunities for the creatures to rise to the occasion. Most of these scenarios began with famine. Lucien blighted the main food crops, then offered a palpable reward for progress by dangling some tempting new fruit from a branch that was just out of reach. Sometimes that metaphor could almost be taken literally: he’d introduce a plant with a complex life cycle that required tricky processing to render it edible, or a new prey animal that was clever and vicious, but nutritionally well worth hunting in the end.
Time and again, the Phites failed the test, with localized species dwindling to extinction. Daniel watched in dismay; he had not grown sentimental, but he’d always boasted to himself that he’d set his standards higher than the extravagant cruelties of nature. He contemplated tweaking the creatures’ physiology so that starvation brought a swifter, more merciful demise, but Lucien pointed out that he’d be slashing his chances of success if he curtailed this period of intense motivation. Each time a group died out, a fresh batch of mutated cousins rose from the dust to take their place; without that intervention, Sapphire would have been a wilderness within a few real-time days.
Daniel closed his eyes to the carnage, and put his trust in sheer time, sheer numbers. In the end, that was what the crystal had bought him: when all else failed, he could give up any pretense of knowing how to achieve his aims and simply test one random mutation after another.
Months went by, sending hundreds of millions of tribes starving into their graves. But what choice did he have? If he fed these creatures milk and honey, they’d remain fat and stupid until the day he died. Their hunger agitated them, it drove them to search and strive, and while any human onlooker was tempted to color such behavior with their own emotional palette, Daniel told himself that the Phites’ suffering was a shallow thing, little more than the instinct that jerked his own hand back from a flame before he’d even registered discomfort.
They were not the equal of humans. Not yet.
And if he lost his nerve, they never would be.
#
Daniel dreamed that he was inside Sapphire, but there were no Phites in sight. In front of him stood a sleek black monolith; a thin stream of pus wept from a crack in its smooth, obsidian surface. Someone was holding him by the wrist, trying to force his hand into a reeking pit in the ground. The pit, he knew, was piled high with things he did not want to see, let alone touch.
He thrashed around until he woke, but the sense of pressure on his wrist remained. It was coming from his watch. As he focused on the one-word message he’d received, his stomach tightened. Lucien would not have dared to wake him at this hour for some run-of-the-mill result.
Daniel rose, dressed, then sat in his office sipping coffee. He did not know why he was so reluctant to make the call. He had been waiting for this moment for more than twenty years, but it would not be the pinnacle of his life. After this, there would be a thousand more peaks, each one twice as magnificent as the last.
He finished the coffee then sat a while longer, massaging his temples, making sure his head was clear. He would not greet this new era bleary-eyed, half-awake. He recorded all his calls, but this was one he would retain for posterity.
“Lucien,” he said. The man’s image appeared, smiling. “Success?”
“They’re talking to each other,” Lucien replied.
“About what?”
“Food, weather, sex, death. The past, the future. You name it. They won’t shut up.”
Lucien sent transcripts on the data channel, and Daniel perused them. The linguistics software didn’t just observe the Phites’ behavior and correlate it with the sounds they made; it peered right into their virtual brains and tracked the flow of information. Its task was far from trivial, and there was no guarantee that its translations were perfect, but Daniel did not believe it co
uld hallucinate an entire language and fabricate these rich, detailed conversations out of thin air.
He flicked between statistical summaries, technical overviews of linguistic structure, and snippets from the millions of conversations the software had logged. Food, weather, sex, death. As human dialogue the translations would have seemed utterly banal, but in context they were riveting. These were not chatterbots blindly following Markov chains, designed to impress the judges in a Turing test. The Phites were discussing matters by which they genuinely lived and died.
When Daniel brought up a page of conversational topics in alphabetical order, his eyes were caught by the single entry under the letter G. Grief. He tapped the link, and spent a few minutes reading through samples, illustrating the appearance of the concept following the death of a child, a parent, a friend.
He kneaded his eyelids. It was three in the morning; there was a sickening clarity to everything, the kind that only night could bring. He turned to Lucien.
“No more death.”
“Boss?” Lucien was startled.
“I want to make them immortal. Let them evolve culturally; let their ideas live and die. Let them modify their own brains, once they’re smart enough; they can already tweak the rest of their anatomy.”
“Where will you put them all?” Lucien demanded.
“I can afford another crystal. Maybe two more.”
“That won’t get you far. At the present birth rate—”
“We’ll have to cut their fertility drastically, tapering it down to zero. After that, if they want to start reproducing again they’ll really have to innovate.” They would need to learn about the outside world, and comprehend its alien physics well enough to design new hardware into which they could migrate.
Lucien scowled. “How will we control them? How will we shape them? If we can’t select the ones we want—”
Daniel said quietly, “This is not up for discussion.” Whatever Julie Dehghani had thought of him, he was not a monster; if he believed that these creatures were as conscious as he was, he was not going to slaughter them like cattle – or stand by and let them die “naturally”, when the rules of this world were his to rewrite at will.
