Robots: The Recent A.I.

by Elizabeth Bear

  It’s the first time Mason’s ever suspected Paul might actually care about what they’re doing.

  It changes a lot of things.

  On Friday, Mason brings in a few of his program’s parameters for structuring a sympathy algorithm, and when Paul shows up he says, “I had some ideas.”

  Paul bends to look, his motorcycle jacket squeaking against Mason’s chair, his face tinted blue by the screen.

  Mason watches Paul skim it twice. He’s a quick reader.

  “Fantastic,” Paul says, in a way that makes Mason wonder if Paul knows more about specifics than he’d admit. “See what you can build me from this.”

  “I can build whatever you need,” Mason says.

  Paul looks down at him; his grin fills Mason’s vision.

  Monday morning, Paul brings Nadia.

  She sits in the back of the office, reading a book, glancing up when Mason says something that’s either on the right track or particularly stupid.

  (When he catches her doing it her eyes are deep and dark, and she’s always just shy of pulling a face.)

  Paul never says why he brought her, but Mason is pretty sure Nadia’s not a plant—not even Paul could risk that. More likely she’s his girlfriend. (Maybe she is an actress. He should start watching the news.)

  Most of the time she has her nose in a book, so steady that Mason knows when she’s looking at them if it’s been too long between page-turns.

  Once when they’re arguing about infinite loops Paul turns and asks her, “Would that really be a problem?”

  “I guess we’ll find out,” she says.

  It’s the first time she’s spoken, and Mason twists to look at her.

  She hasn’t glanced up from her book, hasn’t moved at all, but still Mason watches, waiting for something, until Paul catches his eye.

  For someone who brings his girlfriend the unofficial consultant to the office every day, Paul seems unhappy about Mason looking.

  Nadia doesn’t seem to notice; her reflection in Mason’s monitor doesn’t look up, not once.

  (Not that it matters if she does or not. He has no idea what he was waiting for.)

  Mason figures out what they’re doing pretty quickly. Not that Paul told him, but when Mason said, “Are we trying to create emotional capacity?” Paul said, “Don’t worry about it,” grinning like he had at Mason’s first lines of code, and that was Mason’s answer.

  There’s only one reason you create algorithms for this level of critical thinking, and it’s not for use as secretaries.

  Mason is making an A.I. that can understand as well as respond, an A.I. that can grow an organic personality beyond its programming, that has an imagination; one that can really live.

  (Sometimes, when he’s too tired to help it, he gets romantic about work.)

  For a second-gen creative guy, Paul picks up fast.

  “But by basing preference on a pre-programmed moral scale, they’ll always prefer people who make the right decisions on a binary,” Mason says. “Stockholders might not like free will that favors the morally upstanding.”

  Paul nods, thinks it over.

  “See if you can make an algorithm that develops a preference based on the reliability of someone’s responses to problems,” Paul says. “People are easy to predict. Easier than making them moral.”

  There’s no reason for Paul to look at Nadia right then, but he does, and for a second his whole face falters.

  For a second, Nadia’s does, too.

  Mason can’t sleep that night, thinking about it.

  TO: ANDREW MASON

  FROM: HR—HEALTH/WELFARE

  Your caffeine intake from the cafeteria today is 40% above normal. Your health is of great importance to us.

  If you would like to renegotiate a project timeline, please contact Management to arrange a meeting. If you are physically fatigued, please contact a company doctor. If there is a personal issue, a company therapist is standing by for consult.

  If any of these apply, please let us know what actions you have taken, so we may update your records.

  If this is a dietary anomaly, please disregard.

  The company appreciates your work.

  They test some of the components on a simulator.

  (Mason tells Paul they’re marking signs of understanding. Really, he wants to see if the simulation prefers one of them without a logical basis. That’s what humans do.)

  He pulls up a baseline, several traits mixed at random from reoccurring types in the Archives, just to keep you from using someone’s remnant. (The company frowns on that.)

  Under the ID field, Mason types in GALATEA.

  “Acronym?” Paul asks.

  “Allusion,” says Nadia.

  Her reflection is looking at the main monitor, her brows drawn in an expression too stricken to be a frown.

  Galatea runs diagnostics (a long wait—the text-interface version passed four sentience screenings in anonymous testing last month, and something that sophisticated takes a lot of code). She recognizes the camera, nodding at Mason and Paul in turn.

  Then her eyes go flat, refocus to find Nadia.

  It makes sense, Nadia’s further away, but Mason still gets the creeps. Someone needs to work on the naturalism of these simulators. This isn’t some second-rate date booth; they have a reputation to uphold.

  “Be charming,” Mason says.

  Paul cracks up.

  “Okay,” he says, “Galatea, good to meet you, I’m Paul, and I’ll try to be charming tonight.”

  Galatea prefers Paul in under ten minutes.

  Mason would burn the place down if he wasn’t so proud of himself.

  “Galatea,” Mason asks, “what is the content of Paul’s last sentence?”

  “That his work is going well.”

  It wasn’t what Paul really said—it had as little content as most of Paul’s sentences that aren’t about code—which means Galatea was inferring the best meaning, because she favored him.

  “Read this,” Mason says, scrawls a note.

  Paul reads, “During a shift in market paradigms, it’s imperative that we leverage our synergy to re-evaluate paradigm structure.”

  It’s some line of shit Paul gave him the first day they worked together. Paul doesn’t even have the shame to recognize it.

  “Galatea, act on that sentence,” Mason says.

  “I cannot,” Galatea says, but her camera lens is focused square on Paul’s face, which is Mason’s real answer.

  “Installing this software has compromised your baseline personality system and altered your preferences,” he says. “Can you identify the overwrites?”

  There’s a tiny pause.

  “No,” she says, sounds surprised.

  He looks up at Paul, grinning, but Paul’s jaw is set like a guilty man, and his eyes are focused on the wall ahead of him, his hands in fists on the desk.

  (Reflected in his monitor: Nadia, her book abandoned, sitting a little forward in her chair, lips parted, watching it all like she’s seen a ghost.)

  At the holiday party, Paul and Nadia show up together.

  Paul has his arm around her, and after months of seeing them together Mason still can’t decide if they’re dating.

  (He only sees how Paul holds out his hand to her as they leave every day, how she looks at him too long before she takes it, the story he’s already telling her, his smile of someone desperate to please.)

  The way Paul manages a party is supernatural. His tux is artfully rumpled, his hand on Nadia’s waist, and he looks right at everyone he meets.

  It’s too smooth to be instinctive; his father must have trained him up young.

  Maybe that’s it—maybe they’re like brother and sister, if you ignore the way Paul looks at her sometimes when she’s in profile, like he wouldn’t mind a shot but he’s not holding his breath.

  (He envies Paul his shot with her; he envies them both for having someone to be a sibling with.)

  “Why do you ke
ep watching me?”

  She’s not coy, either, he thinks as he turns, and something about her makes him feel like being honest.

  He says, “I find you interesting.”

  “Because of how I look.” Delivered like the conclusion of a scientific paper whose results surprised everyone.

  “Because of how you look at everyone else.”

  It must shake her; she tilts her head, and for an instant her eyes go empty and flat as she pulls her face into a different expression.

  It’s so fast that most people wouldn’t notice, but Mason is suspicious enough by now to be watching for some small tic that marks her as other than human.

  Now he knows why she looks so steadily into her book, if that’s what happens every time someone surprises her.

  Doesn’t stop him from going cold.

  (He can’t process it. It’s one thing to be suspicious, another thing to know.)

  It must show on his face; she looks at him like she doesn’t know what he’s going to do.

  It’s not how she used to look at him.

  He goes colder.

  Her eyes go terrified, as terrified as any human eyes.

  She’s the most beautiful machine he’s ever seen.

  He opens his mouth.

  “Don’t,” she starts.

  Then Paul is there, smiling, asking, “You remember how to dance, right?”, lacing his fingers in her fingers and pulling her with him a fraction too fast to be casual.

  She watches Mason over her shoulder all the way to the dance floor.

  He stands where he is a long time, watching the golden boy of Mori dancing with his handmade Vestige prototype.

  He spends the weekend wondering if he has a friend in Aesthetics who could tell him where Nadia’s face really came from, or one in Archives who would back him up about a personality Paul Whitcover’s been saving for a special occasion.

  It’s tempting. It wouldn’t stop the project, but it would certainly shut Paul up, and with something that big he might be able to renegotiate his contract right up to Freelance. (No one taps your home network when you’re Freelance.)

  He needs to tell someone, soon. If he doesn’t, and someone finds out down the line they were keeping secrets, Mason will end up in Quality Control for the rest of his life, monitored 24/7 and living in the subterranean company apartments.

  If he doesn’t tell, and Paul does, Paul will get Freelance and Mason will just be put down.

  He has to make the call. He has to tell Compliance.

  But whenever he’s on the verge of doing something, he remembers her face after he’d found her out and she feared the worst from him, how she’d let Paul take her hand, but watched him over her shoulder as long as she dared.

  It’s not a very flattering memory, but somehow it keeps him from making a move.

  (Just as well; turns out he doesn’t have a lot of friends.)

  Monday morning Paul comes in alone, shuts the door behind him, and doesn’t say a word.

  It’s such a delightful change that Mason savors the quiet for a while before he turns around.

  Paul has his arms crossed, his face a set of wary lines. (He looks like Nadia.)

  Mason says, “Who is she?”

  He’s hardly slept all weekend, thinking about it. He’d imagined tragic first love, or some unattainable socialite Paul was just praying would get personality-mapped.

  Once or twice he imagined Paul had tried to reincarnate Daddy, but that was too weird even for him.

  Paul shakes his head, tightly. “No one.”

  “Come on,” says Mason, “if I haven’t called HR by now I’m not going to. Who?”

  Paul sits down, rakes his hair back with his hands.

  “I didn’t want to get in trouble if they found out I was making one,” he says. “It’s one thing to fuck around with some company components, but if you take a customer’s remnant—” He shakes his head. “I couldn’t risk it. I had them put in a standard template for her.”

  Mason thinks about Paul’s black-market baseline, wonders how Paul would have known what was there before he installed the chip and woke her.

  “She’s not standard any more,” he settles on.

  Nadia should be here; Mason would really feel better about this whole conversation if she were here.

  (But Paul wouldn’t be talking about it if she were; he knows that much about Paul by now.)

  “No,” says Paul, a sad smile crossing his face. “I tried a couple of our early patches, before we were working on the full. I couldn’t believe how well they took.”

  Of course they did, thinks Mason, they’re mine, but he keeps his mouth shut.

  Paul looks as close to wonderment as guys like him can get. “When we announce Vestige, it’s going to change the world. You know that, right?”

  He knows. It’s one of the reasons he can’t sleep.

  “What happens to Nadia, then?” he asks.

  (That’s the other reason he can’t sleep.)

  “I don’t know,” Paul says, shaking his head. “She knows what she is—I mean, she knows she’s A.I.—she understands what might happen. I told her that from the very beginning. At first I thought we could use her as a tester. I had no idea how much I would—” he falters as his feelings get the better of him.

  “Not human, but the nearest thing?” Mason says, and it comes out vicious.

  Paul has the decency to flinch, but it doesn’t last.

  “She knows I care about her,” he goes on. “I’m planning for better things. Hopefully Mori will be so impressed by the product that they’ll let me—that they’ll be all right with Nadia.”

  He means, That they’ll let me keep her.

  “What if they want her as the prototype?”

  “I haven’t lied to her,” Paul says. “Not ever. She knows she might have to get the upgrade to preserve herself, that she might end up belonging to the company. She accepts it. I thought I had, too, but I didn’t think she’d be so—I mean, I didn’t think I would come to—in the beginning, she really was no one.”

  Mason remembers the first time Nadia ever looked at him; he knows it isn’t true.

  They sit quietly for a long time, Paul looking wracked as to how he fell in love with something he made, like someone who never thought to look up Galatea.

  She’s waiting in the library, and it surprises him before he admits that of course he’d look for her here; he had a map.

  He doesn’t make any noise, and she doesn’t look up from her console, but after a second she says, “Some of these have never even been accessed.” A castigation.

  He says, “These are just reference books.” He doesn’t say, I don’t need them. He needs to try not being an asshole sometimes.

  She glances up, then. (He looks for code behind her eyes, feels worse than Paul.)

  “I love books,” she says. “At first I didn’t, but now I understand them better. Now I love them.”

  (She means, Are you going to give me away?)

  He wonders if this is just her, or if this is his algorithm working, and something new is trying to get out.

  “I have a library at home,” he says. (He means, No.)

  She blinks, relaxes. “What do you read?”

  “Pulp, mostly,” he says, thinks about his collection of detective novels, wonders if she thinks that’s poor taste.

  She says, “They’re all pulp.”

  It’s a sly joke (he doesn’t think it’s anything of his), and she has such a smile he gets distracted, and when he pulls himself together she’s leaving.

  “I’ll walk you somewhere,” he says. “Paul and I won’t be done for a while.”

  Clearly Paul told her not to trust him before he went in to spill his guts, but after a second she says, “Tell me more about your books,” and he falls into step beside her.

  He tells her about the library that used to be the guest bedroom before he realized he didn’t have guests and there was no point in it. He explains why there a
re no windows and special light bulbs and a fancy dehumidifier to make sure mold doesn’t get into the books.

  (It’s also lined in lead, which keeps Mori from getting a look at his computer. Some things are private.)

  Her expression keeps changing, so subtle he’d swear she was human if he didn’t know better.

  She talks about the library at Alexandria, an odd combination of a machine programmed to access information and someone with enough imagination she might as well have been there.

  (Maybe this is immortality, as far as it goes.)

  She mentions the Dewey Decimal system, and he says, “That’s how I shelve mine.”

  “That explains your code,” she says. When he raises his eyebrows, she says, “It’s . . . thorough.”

  (Diplomacy. Also not his.)

  “It has to be,” he says. “I want Vestige to be perfect.”

  He doesn’t say, You.

  “I know,” she says, in a way he doesn’t like, but by then they’re standing in front of Paul’s office, and she’s closing the door.

  This floor has a balcony overlooking the atrium.

  He sticks close to the wall all the way back.

  He goes home and erases her avatar from his program.

  (Not like he cares what she thinks, but there’s no harm in cleaning house.)

  Marketing calls them in for a meeting about the press announcement.

  They talk a lot about advertising and luxury markets and consumer interest and the company’s planned stock reissue and how the Patents team is standing by any time they want to hand over code.

  “Aesthetics has done some really amazing work,” Marketing says, and Mason fakes polite interest as hard as he can so he doesn’t stare at the photo.

  (It’s not quite Nadia; it’s close enough that Mason’s throat goes tight, but it’s a polished, prettier version, the kind of body you’d use if you wanted to immortalize your greyhound in a way society would accept.)

  “Gorgeous,” Paul says, and then with a smile, “is she single?” and the Marketing guys crack up.

  (One of them says, “Now now, Paul, we’re still hoping you can make a studio match—HR would be pleased,” and Paul looks admirably amenable for a guy who’s in love with a woman he thinks he made.)

  It’s only Paul on the schedule to present, of course—Mason’s not a guy you put in front of a camera—and it’s far enough away that they’ll have time to polish the code.