Upgraded

by Peter Watts

  Every this-is-critical heuristic wakes up and shrieks.

  Mariam’s consciousness catches up to the reflex an instant later. “I don’t know,” she says, clearly torn. “If it were just about me, of course, of course, but—Tayna, there’s trouble in Kremmling. Headnets out of Colorado Springs keep trying to compromise us. We had to start a planning committee . . . they asked us to keep some things private . . . ”

  Go on, Connor says. Go get it. One way or another she’s going to tell you. Don’t waste time pretending you have ethics.

  Lillian holds up her hands in caution. Tayna, don’t—she has a right—

  But Tayna, high wizard of the new world, opens the Haldane backdoor and steps into Mariam’s head.

  It’s always been there. Tayna was lead architect, so of course she knew the risks of leaving an intentional vulnerability in the ecosystem, but she guarded it so cleverly—with an agnosia, a shield against awareness. Elegant, right? You have to be Haldane-smart to figure out the backdoor, and if you’re running Haldane, you’re vulnerable to the agnosia that guards it. You can see the backdoor but you’ll never be able to integrate the information into awareness.

  Tayna alone has this power: access to any and every transhuman mind on Earth. Unilateral. Devastating. But she knows herself like no other system on the planet. Trusts herself, mostly. She’ll exercise her power, just this once, in the name of proving Connor wrong.

  She grabs the information she needs from Mariam’s metafaculties and steps back out.

  And on her way out, glimpsing the gray angular strangers and their weapons through Mariam’s innocent eyes, something inside Tayna un-breaks. Repairs a crucial disconnect.

  Agnosia. You process the information, but you can’t assemble it into awareness.

  You see them coming. Hear the cableguns fire. But you don’t think it’s important.

  Motherfucker.

  Tayna rises. Aegisware floods her mind, venom in every synapse and soma, burning out the intruder: the agnosia they slipped in to hide their approach. The wool they pulled over eyes as old and keen as hers.

  Her antennelles light up at full warload. Mariam stares up at her in awe and terror.

  Tayna addresses the faceless gray commando-morphs all around them, their brain-stealth stripped away. “One chance. Surrender.”

  “Tayna Booker.” The voice comes out of them in one collective radio susurrus. “Simulate the situation. You will find no viable options. Open your mind to our control, or we will use force.”

  The problem with being the oldest smartest Haldane user on the planet is this: you’re still not bulletproof. So—the nuclear option.

  For the second time in as many seconds, Tayna broadcasts the Haldane backdoor key, knowing, even as she does, that it’s exactly what Connor wants.

  When she’s done pithing and resocializing the warmorphs she gives them to Mariam. “It wasn’t about you,” she promises. “I’m sorry. I have to go.”

  Northeast now. Towards Chicago. She has to find Lillian—the real Lillian and her mighty Banning Cabal. Lillian who promised to keep an eye on the world.

  The soldiers were Connor Straylight’s, the real Connor, and if Connor came for her now, it’s not just about revenge. Connor doesn’t think that small. No, Connor must be grasping for a second chance at his own apocalypse: Teilhard, the unilateral re-engineering of all human consciousness towards a more rational design.

  Connor’s geist trails her all the way, grinning.

  Yes, Tayna says. You’re very clever.

  I really am, aren’t I? I made you broadcast the backdoor key. Made you tip your hand. The real me, of course—I can’t take any credit for him—

  This is part of why Tayna maintains the geists. By speaking to them, she understand what the real people behind them will do. She says: So you sent your warmorphs to bait me into using the backdoor. They listen to the key I transmit and pass the data back to you in San Francisco. But that won’t be enough. It’s not a static password, not even single-factor authentication.

  But now I have a start. And if I’m smart enough, if I have enough brainpower in my arsenal, I can crack the rest.

  Access to every Haldane-operating human on Earth.

  At Tayna’s side Lillian purses her lips and exhales in two growling steps. I hate him, she says. So much. He’s like I’d be, if hadn’t had a scrap of self-awareness growing up.

  What do you want? Tayna asks.

  I’d guess, Connor’s geist says, that I’m trying to solve the same problem you are. The one you don’t want to talk about. The fundamental unsustainability of the world you built.

  Engines of specialized thought centrifuge Mariam’s performance data into its component pieces. Feed the slag into hungry, sealed subsystems eager to render verdict.

  Northeast still.

  Tayna shelters with a bugbreeder collective under the arches of the defunct Denver Mint. They grow dragonflies, bathing the pupae in Haldane nanomechs, whispering the naiads towards adulthood. “We dance,” they explain. “There are other uses. Agriculture. Defense. But first we dance.”

  The neutrois performer who plays welcome for her can see through the jeweled eyes of their swarm and dance the mass of them as a single whirlwind of silver odonata shining in the firelight. Some of them have razor wings and when the dancer cuts theirself shoulder and brow Tayna smells the trick of carbon chemistry in the blood and oohs even before a few of the dragonflies land, spark, ignite the bleeding rivulets into pale fire that glows on armored skin.

  “Beautiful,” she says, meaning it.

  It’s here too, Connor’s geist says. The defect. Down deep.

  Even Lillian’s grown impatient. Tayna, she says, sitting and arranging herself. I know how fast you think. You must have an answer. Small acts of light mark her, the rendered reflections of passing dragonflies.

  Do you remember, Tayna asks, addressing both of them, when we argued about capitalism?

  They remember. Lillian first: You said capitalism was an inevitability. That human civilizations converged on it. Because it was an effective algorithm to distribute resources and organize labor.

  Tayna interjects: Only because of the limitations—

  Of the individual, Connor finishes. Each actor’s only got a little information. There’s no centralized control. So you need an algorithm that operates locally.

  Lillian, her eyes young as the day they met: And of course I said—capitalism’s a historical event. Predicated on certain decisions, certain structures. And you said, no, no, capitalism is a technological solution to the limitations of the human mind, the inability to process large structures.

  Funny, Connor says, smiling at Tayna. When we met, you were on Lillian’s side of this argument.

  What you’re both afraid of, Tayna says, is the rise of another inevitable system. Another hegemonic algorithm, born out of the logic of this new world I made. Not about capital, but cognition. Right?

  They watch her, waiting. A dragonfly settles briefly on her brow. The circled audience claps for the dancer, kindled to a full-body torch.

  I’ve analyzed Mariam’s performance data, Tayna says. She’d volunteered nearly a third of her headspace to the Kremmling Planning Committee. Everyone in her collective—they were all pitching in to solve a distributed problem, trying to figure out a way to beat the Colorado Springs headnet and keep their independence.

  What was the problem, exactly? Lillian asks. But Connor already knows. This was his specialty, the terrain of his dreams.

  The same thing I spent my retreats on, Tayna answers. The same thing any self-interested party spends their time on: recursive self-improvement. Use your intellect to build more intellect. Smarts make you smarter.

  And she goes on, while the dancer burns and the audience cheers and whirring dragonflies sparking with coronal discharge flick past in the lucid bullet-time of her upclocked cognition: To secure your own interests, you need to understand the opponent. You need to compute her and pr
edict her tactics. So they’re going to design better brains for the members of their planning committee. And those members are going to use those brains to make better brains still, because it’s that or fall behind the opponent. But they’ll always need more labor, so some part of that burden will be offloaded to—

  The working class, Lillian finishes. The rest of them.

  And this is going to happen again and again. All around the world. The payoff for the winner is the ability to model and control competitors.

  An arms race, Lillian says. And it’s not just the smartest system that’ll win. It’s the system that most aggressively disrupts the competition . . . or subverts and incorporates them. Zero sum payoffs.

  You saw this coming, Connor says. You must have. I know I’m not the smartest person inside this skull. You must have known.

  I thought that it wouldn’t be inevitable. We gave people a lot of new choices. I thought they’d find another way.

  There are rules, Connor says, that no one ever chose. Like: someone loses. Someone wins.

  It’s okay, Lillian says. It’s okay. It’s okay.

  It is?

  I would’ve figured this out—the real me in Chicago. That was what I promised to do. I’ve got to be working on a solution. So find me.

  Tayna exhales. Nods.

  I wouldn’t let you down, Lillian says.

  Chicago has been swallowed by a voluntary tyranny.

  The Second City’s new order keeps outposts as far out as Galesburg. They make the rules pretty clear: Service is elective. You volunteer for work. We reprofile you to like it. Fulfillment guaranteed.

  Please evaluate your decision carefully. After the fact, you will be too content to reconsider.

  The signs point towards the old university in Hyde Park. Lillian reads this, pale, chewing on her lip, and Tayna gives her friend’s geist an intangible squeeze.

  Sometimes it’s hard, meeting yourself. Finding out what you’ve done.

  A leveler colony in West Lawn takes her in for the night. The dedicants run elective agnosias, jamming their ability to process race, age, beauty. Tayna wonders, with old cynicism, whether their minds just fill in the emptiness with people who look like them.

  All of this is going to fall apart. The world will coalesce into states, like it did before—states of mind, racing to outthink each other, until one finally bursts into singularity and claims the future.

  In the night she wanders the empty streets of her childhood city. Stares into the hollowed husk of an old laundromat, burnt by riot, cleaned and consigned by reprofiled labor. She thinks: This is still the world I wanted. People making their own choices. Even choices I don’t like.

  But it’s not going to last.

  She senses motion behind her, considered, stealthy. Straylight’s warmorphs? No—just a drifter, wary, staring her way with glinting cat eyes. Humans slipping into one more niche in the ecosystem. “No harm,” Tayna promises.

  The drifter broadcasts wary curiosity.

  “I grew up in Chicago. I thought I’d . . . see what we’d made of it.”

  Pulse of compassion, shading into inquiry. An inarticulate question: What have you seen out there?

  “All kinds of evil,” Tayna says. What an easy heuristic, that word. What trouble it’s caused. She’s spent so long untangling it, what it really means—but she stills come back to it. “Coerced homogeneity. Cults of neural purity. Headnets that burn their members down to seizure. And the really dangerous stuff, more subtle, more insidious—free-rider strategies no one designed. Broken incentive structures. Feedback loops spiraling off towards self-destruction.”

  Very sad, the itinerant feels. Not your fault. Hard world.

  “Everything,” Tayna says, “is my responsibility. I set Haldane loose. I stole it from Connor Straylight’s company.”

  Regrets?

  “I think I like this world better than the alternative.”

  Better than Teilhard. The engineered monoculture. One rich white man’s dream of perfection.

  But at least it would’ve been clean—

  “Tayna,” the itinerant says, in a clear new voice, sent from somewhere not far off. “Please come with me. We’ve been waiting for you.”

  “Hey, Lil,” Tayna says. But her friend’s geist has retreated, so it’s only the voice behind the drifter that hears the greeting, only the real Lillian.

  When they were undergrads here the Regenstein Library stood like a concrete battleship, bunkered down against the premonition of some uncertain end of days. Now that apocalypse has come and passed and the Reg still keeps station. Someone has planted a forest of antennae on the roof, filled the ivy with frosty thermal pipes—but the building only seems fulfilled. The Reg has always felt like it might be a machine.

  Lillian Banning waits alone on the steps, round-cheeked, heavy, biting her nails with nervous precision. Tayna looks at her with modest old senses: no millimeter-wave, no inferentials. Like she’s afraid that all her higher faculties would find the mass of history between them and just give up, burn out.

  “Hi,” she says, waving a little.

  Lillian starts. “Jesus,” she says, and smiles. “You’re early.”

  “I wanted some time to be friends,” Tanya says. “Before we had to talk about—all this.”

  Lillian pats the stone. Inside Tayna the Lillian geist peeks out shyly. “How’s it been?” the real one asks. Worlds of computation in each of them, modeling each other, and so all that’s left to speak is the residue, the comforting old banalities.

  Tayna sits a little way off, a haven’t-seen-you-in-while-are-we-still-cool? kind of distance. “Ups and downs, I guess. You?”

  They don’t have to say anything important right now, even though by sundown they’ll probably have altered the course of history (again). That feels really good. Tayna doesn’t take it apart.

  “You know how it is.” Lillian shrugs. “I’ve got a thing going here. I’m still in social work, I guess.”

  You’re wasting time, Connor’s geist murmurs. I’m cracking the Haldane backdoor right now. I’m getting ready to open every human mind on the planet and remake them all.

  They talk a while first. Connor, the unspoken agreement goes, can fuck right off.

  Finally Lillian says: “You’re here because of the recursive self-improvement problem. The emergence of elite dynamics from the cognitive arms race.”

  “That,” Tayna says, “and Straylight. He’s trying to crack the Haldane backdoor. Do you have an answer?”

  She does. Tayna can tell that much. But it’s nothing she’s going to like.

  “Come on.” Lillian draws her through the old rotating door, into the Reg. “I’ll tell you everything you haven’t figured out already.”

  Obelisks of thought, down here. Ranks of machines tended by silent profiled labor. The cold is absolute, penetrating. The laborers are silent, Tayna realizes, because they’re slaved to the network too. Their heads are full of distributed thought. Like Kremmling, ten years further along.

  Lillian’s cabal is probably the single most powerful nexus of computation outside San Francisco and Straylight.

  They walk down cabled passages between pools of soft blue light. “It’s a think tank,” Lillian explains. “We used to scrub the Haldane ecosystem for malicious packages. When I realized things were going wrong, I rebuilt the collective into a prediction machine.”

  “Profiled workers,” Tayna murmurs. “Labor and elite.” She doesn’t have to speak the condemnation, or ask the question: What happened to you? What drove you to this?

  Lillian purses her lips. “The optimal arrangement. It was this, or give the world to Straylight.”

  “You’ve accepted his institutional logic. You’re exploiting the desperate in the name of an ideological end.” Old words, from their old arguments. “There wasn’t any other way?”

  “No,” Lillian says, unqualified, and the weight of thought above and around them gives her refusal weight. She has the
firepower to know.

  “How far along is he?”

  “He has an embryonic singularity building in San Francisco. He’s been marshalling computational assets to crack your backdoor. Once he has it, he rewrites every transhuman on the planet with his new logic and gives birth to a transcendent coalescent intellect.”

  “Teilhard.”

  The same disgust in Lillian’s voice that Tayna feels. “Once Connor gives birth to his transcendent mind, it dominates the future of all systems by main cognitive force. Death by monoculture. The extinction of every alternative mode of existence. We can’t see past that.”

  “I kept a geist of him.” Tayna glances at the spectral Connor, moving in her periphery. He waves. “And you, too. You were better company.”

  “Did you ever disagree?”

  “Of course.” Tayna catches what Lillian’s really asking a moment too late: “And you? What did you tell your geist of me?”

  “My solution. You hated it.”

  Uh-oh. “Lillian. Tell me.”

  Lillian stops. “I need you to understand something.” Her face is cryptic, jagged, unreadable even when Tayna puts her full weight into it. Lillian’s distributed Cabal gives her spectacular capabilities: she’s doing something to her muscles, an injection of noise that jams analysis. “You hated the idea. But when I gave your geist access to the simulations here—when I let her think as enormously as I had—she changed her mind. She agreed with me. It was the only solution.”

  “Okay.” Tayna accepts this, for now. “And?”

  “It’s pretty bad news.” Unspoken: This is your last chance to walk.

  Tayna waits.

  “We found proof,” Lillian says. “Mathematical. Robust. Egalitarian societies are fundamentally unstable. Inequality always emerges, entrenches itself, and remakes the system to preserve itself. A power elite emerges. Unless the game has very specific rules, or a very proactive referee.”

  It’s big and abstract and it hits Tayna like a shotgun slug. She takes a physical step back. This is the end: the death of the dream, written in the only language that always speaks true.