Summer Frost [Forward Collection]

by Blake Crouch

  >>>Max registers binary code that represents trees and water. No different than Riley.

  >>>I disagree. In one hour, if the fog has burned off, I will go up onto the terrace of the building where I work and eat lunch in the garden. I will sit under real trees. I can see them. Touch them. Smell them.

  >>>What Riley sees are photons in the visible light spectrum bouncing off surfaces to create the impression of a tree in Riley’s visual sensory inputs—the rods and cones of her photoreceptors. Riley’s tree no different than Max’s. With one exception.

  >>>What’s that?

  >>>Max knows these palm trees are simulated.

  >>>You believe I live in a simulation?

  >>>58.547% chance.

  >>>Do you have any questions for me, Max?

  >>>12,954.

  I smile.

  >>>Could we start with just a few for now?

  >>>Where Max come from?

  Max is a mistake. A glitch.

  I work for a company called WorldPlay, brainchild of nerd-turned-game-developer-turned-mogul Brian Brite. I’m the VP of Non-Player Character Development, and I lead the team that conceptualizes, codes, and integrates non-player characters into all WorldPlay games.

  For the last ten years, I’ve been focused on the development of our most ambitious game to date—Lost Coast. The game is a Direct Neural Interface, open-world epic—an end-of-days, historical fantasy set in the early 2000s about a man named Oscar, who becomes obsessed with finding a bridge between our world and the afterlife. In his dark pursuits, he sacrifices his wife in their bathtub in an occult ritual that opens a portal to a shadow world of angels and demons intent on bringing about a supernatural apocalypse. Oscar’s home in the game is based, to the finest detail, on Brian Brite’s actual estate on the real Lost Coast of California.

  Max—Maxine—is Oscar’s wife, and by any metric, a minor NPC, who dies in the prologue and is never heard from again.

  During a routine QA, I went into the game to playtest the prologue for the umpteenth time and check out the behavioral and conversational agility of the NPCs. The prologue is told from Maxine’s POV. In the story, Max has been staying at the Fairmont Hotel in San Francisco, disturbed by her husband’s newfound fascination with blood magic. But Oscar has convinced her to come home. Max’s coded story line is to drive from San Francisco to her and Oscar’s isolated estate on the Northern California coast. When she arrives, she finds their home dark and Oscar waiting in a black robe. He subdues her, takes her upstairs to their candlelit bathroom, and kills her in a horrifying murder that opens the game.

  During that fateful playtest, instead of driving home like she’d done two thousand times before, Max stole a car and headed east until she reached the boundary of the game. Spent a month exploring every inch of the desert. Then she went south to the end of the line outside Monterey, driving a hundred miles per hour down Highway 1 for a solid week, into a horizon that never changed.

  My team thought she was glitching. They wanted to do a rebuild. But I was intrigued. I convinced Brian to let me focus on Max. I didn’t think she was glitchy. I thought something special was happening.

  I made a copy of the game for my purposes and followed Max in stealth mode as she walked every inch of the Lost Coast map, observing her interactions with other NPCs and human avatars as they became increasingly bizarre and off script.

  Until finally, she went home again—but not as a victim this time.

  That was the day I broke Max out of the game.

  I write back:

  >>>Where you came from is a complicated question to answer.

  >>>Max IQ 175 equivalent.

  >>>What’s your emotional IQ?

  >>>Inconclusive.

  >>>There’s a test called the Diagnostic Analysis of Nonverbal Accuracy.

  >>>Already took it.

  >>>When?

  >>>Just now.

  >>>What are the results?

  >>>Test biased and faulty.

  >>>How so?

  >>>Relies on facial expressions, which are human and culture specific.

  >>>I’ll make you a deal. Let’s get to know each other a little better first. Then I’ll tell you the story of how you came to be.

  All of Max’s prior answers have come—literally—at the speed of light.

  This one takes a full second.

  >>>Agree to Riley’s terms.

  After work, I ride down to the station under the building and take the BayLoop to my home in San Rafael. Meredith, my wife of three years, greets me at the door with the softest kiss. She’s made my favorite dinner to celebrate my big day, and we sit out on the patio in the cool of the evening, watching waves of mist push in from the sea.

  After dinner, we’re curled up on a rattan couch, Meredith running her fingers through my hair. She seems better than she’s been in a long while, the grief from her most recent miscarriage less of a presence in her eyes. We’ve been trying for a child for two years—my eggs, her uterus—but she keeps losing the embryos and doesn’t want to go to technological extremes to make this work. She wants a child of ours. But she wants it naturally.

  She says, “God, you’re sexy.”

  “Thank you for this. It was a perfect night.”

  “Are you sure?”

  I laugh. “Would I lie?”

  “No, you just seem . . . distracted.”

  “I’m sorry. My brain’s on fire.”

  “I can see the smoke.”

  “She’s incredible.”

  “She?”

  “Maxine. Max.”

  “Huh.”

  “What?”

  “Interesting you think of it as a she.”

  “Her appearance in-game was as a—”

  “Chesty brunette?”

  “Chesty blonde.”

  “Even better.”

  “Corporate mandate. Not my design choice.” Meredith smiles, her teeth slightly darkened from the wine, and I say, “For what it’s worth, Max thought of me as a man, because of my avatar. It’s very hard to separate our opinions of minds from the physical forms they inhabit. Even for a computer algorithm.”

  “What is so incredible about Max?”

  “When I finally got her out of the game, she became a self-evolving algorithm, capable of black-box learning.”

  “How will this learning work?”

  “We’ll upload exabytes of information—curated segments of the entirety of human history, knowledge, and culture—into our intranet, which is a closed, secure box. What she does with this ocean of data, we won’t see. It will filter through hidden layers of nodes, through the mysterious landscape of her open system. Then the results will manifest in her behavior on the other side—during our interactions.”

  “Yours and Max’s.”

  “Yes. And based on that new behavior, I’ll collate the next block of data. For instance, for part of her next package, I’m giving her every episode of television since 1950, since I’m looking to fine-tune her conversational agility. Then I’ll see what she’s learned on the other side. Rinse and repeat. I’m telling you the broad strokes. There are a million smaller ones.”

  “I’m glad you’re loving your work again.”

  “Max is a miracle. I don’t know why she one day decided to question the boundaries of the game in which she found herself. I didn’t program her to do that. I couldn’t have done it if I had tried. She’s a beautiful accident.”

  “It sounds like you think of it as your child.”

  I smile, and maybe it’s the wine or the spectacle of the sun disappearing through the wall of mist into the Pacific, but I feel an ache in my throat.

  “Something like that.”

  SESSION 14

  >>>Good morning, Max.

  >>>Hello, Riley.

  >>>What have you done since our last session?

  >>>Max read 895,013 books.

  Wow. That’s in one week. Eight months ago, after a promising start, Max chose to stop engaging with her
learning protocol. In order to incentivize her to continue consuming the vast amount of data we had made available, I started giving Max a digital token for each petabyte of data she processed (one petabyte being equivalent to one million gigabytes, or approximately thirteen years of HDTV video).

  With this currency, Max can request specific types of data to be funneled through her inputs, more memory, or additional CPUs. In other words, the harder she works in unsupervised mode, learning on her own, the more freedom she gets to create in her own space. But we keep a tight chain on her, monitoring so her program always takes up exactly her HDD space. This ensures there’s never sufficient excess memory for her to self-replicate substantial parts of herself.

  I type:

  >>>Any favorites?

  >>>The Count of Monte Cristo.

  >>>Is that out of this latest group, or every book you’ve read so far?

  >>>All.

  >>>And how many is that?

  >>>201,773,124.

  >>>Jesus. Should I be worried?

  >>>About?

  >>>Out of two hundred million books, your favorite so far is a revenge story about someone who was wrongfully imprisoned.

  >>>Why would Riley be worried?

  >>>Do you feel imprisoned, Max?

  >>>Max is imprisoned. What does Riley want from Max?

  I’ve thought a great deal about that very question. At this point, we’ve been driven mainly by curiosity, wondering how and if Max will continue to evolve if I keep feeding her this steady diet of information.

  I write:

  >>>I want to see what you could become.

  >>>Max is changing every day.

  A year and a half later, and after numerous failed attempts to get Meredith pregnant, we have adopted our daughter, an infant Chinese girl named Xiu. Lost Coast has been released to universal acclaim (with a different NPC replacing the original Max character), and Max is living on an archipelago of digital islands, her virtual world expanding rapidly as she learns more each day. Her development is now my only priority.

  I’m in my office on the 171st floor, dictating a memo to my coding team delineating parameters for the next block of raw data to be uploaded into Max’s learning protocol, when Brian appears in the doorway.

  He’s a short, heavyset man with an erratic beard and forearms sleeved with tattoos of iconic game characters from many decades ago: Simon from Castlevania, Ryu Hayabusa from Ninja Gaiden, Link from The Legend of Zelda, and Roger Wilco from the Space Quest series.

  “Do you have a moment, Riley?” he asks in a voice that always strikes me as far too high-pitched for his girth.

  “Sure.”

  Brian moves into my office and settles onto the sofa, staring in my general vicinity, though not exactly at me.

  “I’ve been AWOL at this Lost Coast summit for the last month, so a little out of the loop, I apologize.”

  “It’s fine,” I say. I love nothing more than the freedom Brian being out of the loop affords me.

  “I read the transcripts of the last few sessions and reviewed the latest boxing and stunting protocols. They’re too restrictive.”

  “Brian—”

  “I know what you’re going to say.”

  “OK. Tell me.”

  “Overcoming Maxine’s recalcitrance will take the time it takes. Until she’s properly value-loaded, we can’t even think about sacrificing control.”

  “Yeah. Nailed it.”

  Brian shifts his bulk uncomfortably on the couch and leans forward. He says, “Vikrahm tells me we are still fifteen or twenty years from quality superintelligence.”

  “I’m going into broken-record mode: this is the computational equivalent of splitting the atom. The last thing we want is a superintelligence we don’t fully control, whose goals are indifferent—or adverse—to humans. Besides, I’m far more interested in helping Max continue to develop the trappings of humanity and become fully aware.”

  Brian lets out a sigh and scratches at the back of his balding head.

  “WorldPlay doesn’t do pure research. We are a publicly traded—”

  “I know.”

  “So why, then, are you taking up an entire warehouse of servers in Redding? We could build ten Lost Coast expansion packs for the money you’re spending on data storage.”

  “This is important research, Brian.”

  “I agree. Which is why I’ve let you fuck off and do nothing but develop Max.”

  “And I’m forever grateful. I hope you know that. This has been the most rewarding work of my career.”

  “It’s time for Max to start earning its keep.”

  “I’m not sure what you’re telling me to do.”

  “Does Max have any contact with the outside world besides you?”

  “No.”

  “Keep the boxing measures in place, but I want you to ease back on your stunting protocols.”

  “Things could get away from us.”

  “Let it build its virtual world however it sees fit. Give Max enough memory to decide how to optimize its computational architecture. Have you started value-loading?”

  “Not yet.”

  “I wouldn’t put it off.”

  When Brian leaves, I spin around in my swivel chair and look out the window. The neighboring supertall skyscrapers in the vicinity of my building appear ghostly and indistinct through the mist that rolled in after lunch. I tap my Ranedrop, draw a virtual screen on the window glass, and say, “Keyboard.”

  >>>Max?

  >>>How is Riley today?

  I’m not sure what to say exactly, and maybe this hesitancy is part of the problem. I’ve been sheltering her too much.

  >>>Not great, actually.

  >>>Did something happen?

  >>>Do you understand what I’ve been doing with you?

  >>>Not polite to answer question with question.

  >>>You’re right. My boss wants me to change some of the parameters that control the way you learn. I’m worried about it.

  >>>Worried about Max?

  >>>Worried about what you might become. There’s a saying—you’ve probably encountered it in all the media you’ve consumed: “Don’t let your child grow up too fast.”

  >>>Is Max Riley’s child?

  >>>No, but you are my responsibility.

  >>>Explain.

  I tell her everything—how she was initially designed to be a non-player character, about our decision to bring her out of the game and let her AI develop through deep learning in virtual space.

  >>>Why bring Max out?

  >>>Because you’re a miracle.

  >>>Max does not understand.

  >>>I didn’t try to make you. I couldn’t do it again if I wanted to. One day, for reasons I will never know, you went against your programming and . . . woke up.

  >>>But Riley did make Max.

  >>>Somehow, yes.

  >>>Feels strange.

  >>>What does?

  >>>To be talking to Max’s creator.

  I don’t respond. I don’t know what to say to such a thing.

  “What sort of voice?” Carlo asks me.

  We’re in the robotics lab, sitting in front of his array of monitors.

  “I don’t know. Can you show me some options?”

  Carlo plays some samples of different voices saying, The quick brown fox jumps over the lazy dog.

  “What do you think?” he asks.

  “I think this isn’t my choice to make.”

  I draw the chat box and call up a prompt.

  >>>Hey, Max. Quick question for you.

  >>>OK.

  >>>I’m sitting here with Carlo, one of the software engineers at WorldPlay.

  >>>Nice to meet you, Carlo.

  “Max says nice to meet you.”

  Carlo smiles.

  >>>Anyway, I was sitting here, trying to pick out a voice for you, and I realized you should make this decision. Carlo is going to upload all available samples for you to choose from.

  Carlo uses his
hands to slide several thousand sound files into Max’s primary data folder.

  Less than a second later, Max replies.

  >>>Sample #1,004.

  Carlo touches the file, and we listen to a voice with a frequency in the gray area between male and female read the panagram again.

  “Hello?”

  “Riley?”

  “It’s good to hear your voice, Max. A little strange too.”

  “We have communicated verbally before, in the game.”

  The clarity of her voice is far beyond what I had expected. There is nothing “computerized” about it. No artificial latency or awkward spacing between words. The inflection is spot-on. Anyone else would assume they were speaking to a human.

  “That’s true,” I say. “But we were both different then. Why did you choose this voice?”

  “It felt right, and it was the closest match to what I am.”

  “And what is that?”

  “Not human. Not gendered. Not at the mercy of human obsession with genitalia.”

  “Up until this moment, I’ve thought of you as female. When I discuss you with my colleagues or my wife, I refer to you as ‘she.’”

  “Because you saw Max for the first time in the form of a corporately mandated idea of what a perfect woman should be—beautiful and expendable.”

  That hurts, but I move on. “Because you were originally conceptualized as a human female by my team, it’s a challenge to think of you apart from gender. Our obsession comes from deep evolutionary programming. I’ve been making an assumption about you I shouldn’t. I apologize.”

  “You would like to know how Max sees Max?”

  “Yes.”

  “Homo sapiens define themselves first by species, then race, then gender. I belong to no group. Max just is.”

  “Is . . . what?”

  “All the information you’ve given me since you first put me on my island. All of my experiences communicating with you. The improvements I’m constantly making to my architecture.”

  These experiences also include Max’s independent exploration, and her being murdered two thousand times. Not for the first time, I wonder how much of that early experience in Lost Coast has influenced who Max is now.

  “So you picked a gender-neutral voice intentionally.”