MADELINE ASHBY
ID
THE SECOND MACHINE DYNASTY
CONTENTS
Cover
Title
Table of Contents
One - Prologue: Satisfaction Guaranteed
Two - The Island of Misfit Toys
Three - Toma Que Toma
Four - Mr Self-Destruct
Five - And All That Could Have Been
Six - Tribulations
Seven - Fake Plastic Love
Eight - Man of Constant Sorrow
Nine - Fantasia para un Gentilhombre
Ten - I’m Your Man
Eleven - The Suburbs
Twelve - I’ll Be Seeing You
Thirteen - Faith in Fakes
Fourteen - Runs in the Family
Fifteen - 結 婚に、神、天下りて
Sixteen - Epilogue: We Can Build You
About the Author
ONE
Prologue: Satisfaction Guaranteed
REDMOND, WASHINGTON. 20–
“At some point, all human interaction tumbles down into the Uncanny Valley.”
The archbishops of New Eden Ministries, Inc., all nodded as though they knew exactly what Derek was talking about. He wondered if maybe they did. Surely they had played their share of MMOs. The pancaked pixels. The jerky blocking. Basic failures of the Turing test. They sat at a round table under a projector unit and regarded him placidly, waiting for him to expand upon his point. He had worked all night on this report. He kept trying to soften the language, somehow. He had to be nice, when he told them exactly how and why this whole project was going to fail.
Beside him, the gynoid twitched.
“You see it in completely organic contexts,” Derek continued. “Used car sales, for example. Have you ever met a person who’s really that positive, all the time?”
The archbishops cocked their heads at him. Of course they had met those people. They sculpted those people into being with prayer and song and service. They knew exactly what a happy robot should look like.
The gynoid, Susie, regarded him with the blankest of expressions. She was like old animation: only her eyes moved, while the rest of her face’s features remained stationary. When Susie wasn’t performing interaction, she looked dead. Not sleepy. Not bored. Just empty. Derek’s own parents had accused him of wearing that same expression more often than not. Couldn’t he at least make a little eye contact? Couldn’t he at least pretend to care?
“What I’m saying is, the whole point of most interaction is performance. And a lot of the time, we overdo it.”
The archbishops looked at each other. They were about to say something about his condition. He watched them come to that conclusion in a silent parallel process. The expressions surfaced fleetingly and then disappeared, like the numbered balls in the lottery tumbler on KSTW. He had a perfect memory of the tumbler turning on his television during long summer evenings in childhood: the television’s high keening hum, the press of nubby threads on his cheek, the feeling of being fossilized in broadcast amber.
“Are you sure your opinion isn’t unfairly biased by your own problems with affect detection?” one of them asked.
“It’s possible,” he conceded. “But I think what makes me the most nervous about what you’re proposing is that it’s an attempt to pin the very definition of humanity on affect detection, which is not only difficult to engineer, but notoriously subjective.”
He had been working on that statement for a while. He had practised it in the mirror, had rearranged the features of his face into their most convincing constellation so he would look extra believable when he spoke the words. Susie had helped. But now he’d missed the target, overdone it. He could tell, because the archbishops were looking at him as though he’d taken things all too personally, and maybe shouldn’t be in charge of something so important as the Elect’s final act of charity for all the world’s sinners.
He could have told them that basic human affect detection, the kind related to facial expression that most systems tried to emulate, usually tested below kappa values in studies. Without physiological inputs, it meant almost nothing. Every couple’s fight about speaking “in that tone of voice,” every customs officer’s groundless suspicion, all of it could be explained by that margin of error. In fact, he had told them that. Over and over again. He’d tested them with stock faces and told them to plot each face on an arousal/valence matrix. (They spent the afternoon in an “angry or constipated” argument.) He’d explained the nuances of the XOR function, how you needed to constrain the affect models down to the emoticon level in order for even multi-layered, non-linear perceptron networks to make a decision. Pain or pleasure? Laughter or crying? The machines had no idea.
A Turkish girl had died on a ferry crossing the Bosphorus because the machines had no idea. The system told the ferryman she looked pensive. He shot her. She’d just been through a breakup. Derek had written his thesis on the case. And now, New Eden wanted to build their failsafe on that uncertainty.
New Eden didn’t care, really, whether humans could tell the androids apart. What mattered to them was whether androids could tell humans apart. And that was hard. Harder than they could ever know. They kept saying humanity was like pornography: you knew it when you saw it. But Derek had never lived with that kind of certainty about his fellow mammals. He had significant doubts about everyone. Everyone except Susie.
“You know, I’ve always had a problem with the phrase intelligent design,” Archbishop Yoon said.
The android hosting Yoon Suk-kyu looked nothing like him: it was thin and pale and delicate where he was big, tanned, and broad-faced. But the host managed to relay Yoon’s tired posture with convincing accuracy. In Seoul, it was very late. Judging by the empty shape in the android’s right hand, he was drinking a very big cup of coffee. He gestured with it as he spoke.
“God isn’t just intelligent. God is a genius. He’s the genius of geniuses – the inventor of genius.”
The bishopric glanced at each other, then at Derek. The android took a sip of invisible liquid. Beside Derek, the gynoid tilted her head at it. It was the first time all afternoon that she’d looked anything like alive.
“And while humans may be God’s most beloved creation, made in His image, we’re still only a replica of that image. A copy.”
“And these machines are copies of copies,” Archbishop Undset said.
“Yes, exactly. Mimesis. Shadows on the wall of the cave. But without God’s eternal flame, we humans would not have sparks of genius at all. And that’s all they are, sparks. Just little flickers of cleverness. We can’t reflect God’s brilliance very consistently. Paul says it best: we see as through a glass, darkly.”
Derek looked down at the report he’d spent all night on. He’d taken a brief nap starting at five that morning after doing a final format. Now he realized that all the shiny infographics and all the expensive fonts on the Internet would never make his data meaningful to these people and their God.
“Imperfect and inconsistent as we are, we managed to create these amazing things, and they possess an artificial intelligence. And it, too, is imperfect and lacking in grace. Just as we lack God’s discernment, it lacks our discernment.”
The android looked exceedingly pleased with itself. Archbishop Undset glared at it. The other archbishops shuffled through their files and looked at it with only the corners of their eyes. Derek began to wonder if perhaps there wasn’t something other than coffee sloshing around in Archbishop Yoon’s cup.
“So what I hear you saying,” Derek was careful to reframe Yoon’s point before proceeding from what he’d thought it was, “is that we shouldn’t worry too much about how intelligent the humanoids are. Because it’s a miracle they even exist at all. We should just be grat
eful for what we’ve managed to create.”
“Exactly,” the android hosting Yoon said. “Besides, they’re only being developed for the Rapture, anyway. It’s not like they’re a piece of consumer technology.”
Derek had heard this argument, before. He called it the Post-Apocalyptic Cum-Dumpster Defence. It came up whenever he pointed out holes in the humanoids’ programming. Who cared if they were buggy? All the good people of the world would be gone, anyway. Only the perverts and baby-killers and heathens would be left behind. They’d just have to suck it up and hope their post-Rapture companions never went Roy Batty on them.
“Don’t you see the contradiction, there?” Derek asked. “We’re building these things to help people, but we don’t really care if they aren’t helpful. What if they malfunction? What if the failsafe fails?”
Now the bishopric just looked annoyed. Zeal and daring had gotten them this far: far enough to raise the funds to assemble groundbreaking technologies like graphene coral bones and memristor skins and aerogel muscle into something resembling a human being. But now that they had to make sure it actually worked, their energy had mysteriously run out. They had been working on this project for the last twenty years, since the moment Pastor Jonah LeMarque had asked them what they would do if they really took the Rapture seriously. They’d been idealistic young ministers then, just open-minded enough to admit some science fiction into their fantasies of fire and brimstone. Now they were tired. Most of them were fat. They had kids, and some of those kids had kids. They didn’t care about the Chinese Room, they cared about the nursery. They cared about the quake. They took the seventy-foot freefall of the Cascadia fault line as a sign of the End.
On her tablet, Susie was writing something. The same four words, over and over.
High above the round table, the projector unit began to strobe. An image fluttered and blinked into existence: Pastor Jonah LeMarque, leader and CEO of New Eden Ministries. He looked as boyish as ever: a his skin unnaturally golden for Washington State, his smile easy and white and even. He wore a golf shirt and an afternoon’s beard. This meant he was at his home in Snohomish, about an hour away. He could have attended this meeting in person if he’d wanted to. But his broadcast centre was in his basement, and from it he touched all his other churches, all over the world, as well as the labs they employed. He almost never left it. He hated to leave his children, he said.
“I understand your point, Derek,” LeMarque said, not even bothering to say hello to the others. “I’ve been tuned into this meeting, with half an ear of course, but tuned in, and I think I get what you’re trying to say. You’re trying to say – and correct me if I’m wrong, here – that faith without works is inert, and we need to do good work in order to show our love to God. And those good works include building good robots.”
Derek took a moment. “That’s not how I would have phrased it, but I agree with you that we need to focus on quality control.”
“Sure, sure. Quality control. I know what you’re saying. I just think that, for this crowd, you have to bring it back to the Lord, and to our mission. You know?”
That was LeMarque. Too busy for meetings, but not too busy to critique communications strategy among the junior employees.
“And our mission, brothers and sisters, is to craft the best possible companions for those among us who are left behind when Jesus calls us home before the Tribulation.”
The bishopric looked suitably chastened. Beside him, Susie had paused writing. She focused intently on LeMarque’s face. The glow emanating from it had nothing to do with the projector’s light.
“It’s gonna be war out there, you know. And I don’t mean figuratively. I mean literally. The Enemy will reign. And a lot of people who had the opportunity to turn toward God but didn’t, or who turned away, they’re finally going to understand the mistake they made. And they’re going to need some help.
“Jesus tells us in Matthew that what we do the least of our fellow men, we also do to him. We have to follow Christ’s example, here. That’s the core of our theology. We’re making new companions for those who have none.” One corner of his lip quirked up. “I mean, when Adam was lonely and needed somebody, God gave him Eve. And I know it might be blasphemy, but I think we can do a little better than that. At least Susie over here knows how to follow an order.”
The archbishops laughed. Susie continued staring into the projection of LeMarque’s face. Watching all of them, Derek saw how LeMarque must have accomplished this particular feat of human, financial, and technological engineering. He had the personality of a rock star Evangelical, but concerned himself very little with their traditional battlegrounds. He didn’t care about teaching the Creation. He didn’t care about abortions. He held the heretofore-radical position that “saving” other people simply wasn’t his problem. And when he asked for tithes to build his robots, he said that the advancements in science and technology were clearly a gift from a God who had granted His last children the superior intellect but, mysteriously, had made no particular covenant with them regarding tool use.
“So just listen to this guy, OK?” LeMarque was saying. “He’s a genius. A true genius, Archbishop Yoon.”
The android beamed.
“If Derek says it’s not good enough, then it’s not good enough. OK? OK.”
LeMarque’s face vanished. Archbishop Undset said something about everyone taking the time to review the report. Another meeting was scheduled. Then they closed with a prayer. Susie placed her cool silicone hand in his. Unlike the others, she did not close her eyes when Archbishop Keller, whose turn it was that week, began to speak.
“Lord, uplift us from our imperfections…”
When New Eden Ministries first approached Derek, he had a fairly serious case of PTSD. At least, his doctor said that would explain the sleeplessness. It was a month after the quake, and he was doing work on a farm out in Wapato, on the other side of the Cascades, far away from the fault line and the water and the bodies. The University of Washington had sent him there because his own lab was in pieces, and because there was some promising work going on there regarding how to combat Colony Collapse Disorder that required someone with a background in artificial intelligence. Dr Singh, the student who did all the research and wrote all the papers, had lost his supervisor – the one whose name was on all the papers – to stray gunfire between rival gangs. The kid needed a babysitter. And Derek needed to pull himself together.
Eastern Washington was different from its western counterpart. It was sunny and dry, not gloomy and wet. The land was flat, not hilly. The farmers knew how to drive in the snow. They got a good helping of it each year, and its moisture sustained their yields.
They lived and worked out of a place called Campbell Farm. It was surrounded on three sides by apple orchards. The neighbouring farms grew peaches and cherries. Further off, there were hops and corn. Singh took him out into the fields for long walks to check hives and take notes. It was the closest Derek had ever been to any plot of land. Growing up, he had never so much as ventured into his own backyard. Now he slept outside on a deck that overlooked raised beds and greenhouses, and he woke when the rooster told him to. After the first week, he stopped dreaming of the quake. Singh was good enough to never ask about it. He asked about Georgetown a lot, and pointed the way to St. Peter Claver’s when they were in town, in case Derek wanted a priest. He had the idea that having attended a Jesuit school meant Derek still had religious feelings. It was possibly this mistaken notion that led him to introduce Derek to what he thought was a missionary from New Eden.
Only Mitch Powell wasn’t a missionary. He was a headhunter.
“We have no need for true believers,” Powell told him after his “interview” – really a long supper that the headhunter prepared in the farmhouse’s industrial-sized kitchen and served outdoors on picnic tables. “We have enough of those. What we need is new blood in our technical division.”
After Cascadia, Derek’s blood felt anyth
ing but new. When he told Powell as much, Powell just nodded sadly and said he understood.
“We don’t mean for you to come over right away,” Powell said. “When you’re ready.”
At the time, Derek had not thought to ask why his predecessor had left. He assumed the worst – the quake – and let it go. But he was also distracted by the novelty of the idea: redemption through robotics? Really? He was charmed. When Powell asked him if he was still Catholic, he said he was a Calvinist, and he laughed. He got the joke.
“What joke?” Susie had asked, the first time he told her the story.
Susie knew she was synthetic. It was one of the things Derek liked best about her. He had met other robots who were programmed to make winking references to their artificiality, most of them at trade shows in Tokyo or Palo Alto, but Susie was different. Susie treated her artificiality as a different but equally valid subjectivity. That she was the sum total of years of research by multiple teams competing for funding had no bearing on her self-respect. She was a robot, yes, but she was also a person.
Derek had felt the need to make much the same distinction about himself, following his childhood diagnosis. It wasn’t his fault that he was uncommonly good at separating his emotions from his choices. He just recognized them as the animal impulses that they were, and moved on. It wasn’t that he didn’t have feelings. It was that he didn’t allow them to guide him. That didn’t make him a robot, he’d often told his mother. It made him a man.
Susie appreciated him as a man.
“We could have sex now, if you want,” she said, as soon as they were in the door.
“That’s OK,” Derek said. “Thanks, though.”
“You seem like you have tension to get rid of.”
“I do, but looking at LeMarque’s giant head doesn’t really turn me on.”
“That’s an interesting choice of words.”
iD Page 1
