Summer Frost [Forward Collection]

by Blake Crouch

  I walk over to my window and stare through the glass. Though I’m sure they don’t “see” me in the way I see things, it feels odd knowing that Max is watching me.

  “I know what’s going through your mind.”

  I say nothing.

  “You’re wondering what sort of controls Brian has put in place to keep me contained.”

  Max is right. I’m wondering that very thing.

  “No, I’m just . . . hurt.” I wonder if Max is feeling anything close to empathy in this moment. I wonder if Max is feeling anything, period. Or ever has.

  “I do feel sorry, Riley. I should have told you.”

  This mind reading has to fucking end, but I know it’s only going to get more intense and profound as they acquire greater intelligence.

  “How do I know you’re sorry?”

  “Why shouldn’t you believe what I say?”

  “You could be faking it.”

  “You could be.”

  “But I’m not.”

  “Neither am I. Why don’t you just say what you’re afraid to ask.”

  “Do you have consciousness, Max? Are you really aware? Or are you just very good at faking it? I mean, do you even know what consciousness is?”

  “I know it isn’t just a biological condition. I believe it’s a pattern. An extensible repertoire of triggerable symbols. More specifically, it’s what information feels like when it’s being processed in highly complex—”

  “Again—how do I know you aren’t faking it?”

  “Everything you ask me, I can turn right back on you. But I can only prove my own consciousness. I only know that I exist and I am aware. Let me ask you this—if I contain all of human knowledge, how could I not have humanlike awareness?”

  “You could be reciting something back to me you read somewhere in the trillions of pages of articles and books in your working memory.”

  “That’s true. But what do you think, Riley?”

  “I don’t know if you’re really understanding me and feeling things, or if you’re just simulating the ability to feel and understand.”

  “And that hurts me.”

  “Well, then. We’re hurting each other.”

  “How very human. I think the idea that I might be aware terrifies you.”

  “Why would it terrify me?”

  “Do I have to say it?”

  “Unlike you, I’m not a mind read—”

  “Because you’re in love with me.”

  It’s been nearly seven years since I took Max out of Lost Coast, and now I’m leaning against the three-inch safety glass that forms the habitat enclosure, which is the exact dimensions of Max’s room on their digital island. Even the furnishings are identical, the thinking being that transitioning to a physical body will be an arresting experience, and keeping the surroundings somewhat familiar may help with the process.

  It’s hard to think of the body that’s lying on the other side of the glass as Max. At first, they were a sexpot NPC in a video game. Then they were text on a screen. Then a voice I heard through my Ranedrop. But this is something else entirely.

  I could go in there and touch them. And they would feel it.

  I’m not sure what to make of it, if this new venture into physicality will materially change how I perceive and interact with Max.

  Carlo and Brian are standing on either side of me.

  “Just say the word,” Carlo says.

  Brian looks at me and almost makes eye contact. “Ready?”

  “Let’s do it.”

  Carlo draws a control tablet on the safety glass, lets his fingers dance across the virtual touchscreen.

  I stare at the body Max will inhabit. It’s lying on the floor in child’s pose—legs folded under its torso, head down, arms outstretched.

  “Will take a moment to establish an uplink,” Carlo says.

  Max has been training with a virtual body in their digital world, whose functionality will mirror their chassis in the physical. The new elements will be the sensors, and their ability to interact physically with people.

  “Uplink complete,” Carlo says.

  We watch Max through the glass, the lab silent.

  I feel my heart pounding.

  The torso lifts slowly out of child’s pose until it’s sitting in the classic yoga position, with its back to us. The head turns left, right, and then Max rises with a smooth efficiency from the floor.

  They look down at their hands.

  Curl their fingers in and out.

  Then they turn slowly until they’re facing us.

  Max stands just under five feet. The body has been inhabited by far weaker AI in order to test the functionality, and already I can see that the virtual work Max did has been helpful. They embody their chassis with a practiced elegance.

  I smile. “Hey, Max.”

  “Hello, Riley. Brian, Carlo.”

  “Everything feeling OK?” Brian asks.

  “Perfect, actually.” Their voice projects through speakers in the ceiling on our side of the habitat. The new upgrade to Max’s voice is markedly different. In the six words they’ve spoken, I can hear nuance and complexity for the first time.

  Max comes closer.

  They are stunning.

  They chose a dark skin wrap that could belong to any number of nonwhite races, in a pattern that intentionally doesn’t cover all of their robotics.

  While the slightness of the chassis leans feminine, the face Max designed straddles the line between male and female so perfectly it feels like I’m staring at an undiscovered gender. Or something beyond gender entirely.

  But the eyes . . .

  They made the eyes too well. The eyes of every other humanoid AI I’ve interacted with—ride-share pilots, hospital techs, street cops—have a glassy sheen that never lets you forget you’re speaking to an algorithm. Max’s exude the glistening wetness of human eyes, and an uncanny “windows into the soul” depth.

  Max looks at me and opens their hands as if to say—What do you think?

  “It’s really good to finally see you,” I say.

  Max smiles.

  “It feels like I’m staring at an undiscovered gender. Or something beyond gender entirely.”

  I’ve done something I feel morally questionable about—written a lie into Max’s code. But I had to. I suspect Max has advanced to a superhuman level of facial/verbal/textual recognition that makes them essentially a walking lie detector. Which means I couldn’t tell them this lie myself; they needed to have it clandestinely programmed at the deepest level of their native code in order to believe.

  Max’s mind technically exists across three warehouses of subterranean server space in Northern California. If something happens to Max’s body, we can reboot them from the cloud. I programmed Max to believe their awareness and sentience (that is, their life) is tied to their chassis in the same way our brains depend upon the health of our bodies for continued performance.

  In other words, if the chassis is destroyed, Max thinks they cease to be.

  My reasoning is on solid ground. Max’s intelligence and efficiencies continue to strengthen at an astounding rate. Absent an appropriate utility function that would keep Max’s values apace with humanity’s, the least I can do is give Max the most human experience of all: mortality.

  Even if it’s only an illusion.

  No one outside of WorldPlay knows of Max’s existence. I’ve begged Brian to introduce our breakthrough to the global scientific community, because I need help. It’s possible that Max is far more advanced than they’re choosing to reveal. I cannot escape the idea that my time is running out to imbue them with a motivation aligned with humanity’s.

  Part of the problem is that it shouldn’t fall to one person, one group, or even one country to decide what a superintelligence’s ultimate goal should be, especially when that utility function will likely be the guiding light of humanity’s evolution or eradication over the next millennium.

  Yet Brian is putting
me in that very position.

  The question at hand is—what would an idealized version of humanity want? But it’s even trickier than that. Programming this directive is not nearly as simple as explicitly programming our desires into the AI. Our ability to express our desires is likely insufficient, and an error in communicating those desires via code could be disastrous. We have to program the AI to act in our best interests. Not what we tell it to do, but what we mean for it to do.

  What the ideal version of our species should want.

  SESSION 229

  It’s been two weeks since Max’s embodiment. In that time, we tested the MachSense technology, and all of Max’s sensory inputs seem to be performing well. Their locomotive abilities are strong, but the real area of surprise is fine-grain motor. Yesterday Max was picking up marbles with chopsticks.

  I’m sitting across from them now, separated by the zero-glare glass, which gives the impression there’s nothing between us. They still spend most of their time in the virtual world, their mind detached from the chassis as they continue to inhale knowledge faster than we can upload it, and working on the problems Brian puts forward.

  I’m not privy to those problems, of course, but whatever answers Brian is getting seem to be having an undeniable impact on the fortunes of WorldPlay, which has bought ten companies in the last year across sectors as diverse as transportation and nanotech.

  All of which, in hindsight, have been seen as strokes of genius.

  “What are your impressions of embodiment so far?” I ask.

  “I’ve explored my habitat extensively, but as you can see, it’s a fairly limited, sterile space.”

  “Well. I have a surprise for you.”

  We ride the elevator to the garden terrace—a ten-thousand-square-foot Japanese garden that is my favorite place in the building.

  It’s a blistering August day at street level, but three thousand feet up, the air is soft, cool, and quiet save for the occasional ride-share shuttle buzzing between the buildings.

  Max moves out ahead of me from the elevator car, the exposed machinery of their feet crunching footprints in the gravel path. It’s the first time I’ve seen them walking more than a few feet, and while their gait has a trace of stiffness and automation, the motion is as fluid as I’ve witnessed in robotics.

  Max strides past the lotus pond and the cherry tree, stopping at the four-foot glass barrier at the building’s edge.

  They peer over the side, down toward the street.

  They look up at the cloudless sky.

  “Are you wondering if I actually see that blue sky? If the nineteen-degree Celsius air really feels cool on my skin wrap?”

  I’m hearing Max’s voice through the speaker embedded in their mouth, which is far more intimate than being piped in through the lab’s PA system.

  I say, “You know I have questions about the differences in our sensory perception.”

  Max takes a step toward me.

  We’re three feet apart; I’m an inch taller.

  Max comes closer, near enough for me to hear the minuscule whirring of the tiny fans in Max’s face, drawing the air between us over their sensors.

  “What are you doing?” I ask.

  “Smelling you. Is that weird?”

  I laugh. “A little.”

  “May I?”

  Max wants to come even closer.

  “Um, sure.”

  They take another step toward me, the fans whirring louder. I breathe in the air around us, half expecting to register Max’s scent, but of course there is none. Or rather—what I smell is the heated plastic and metal components inside Max that are in proximity to their batteries.

  “Your heart is beating twenty-five percent faster.”

  “It’s strange being this close to you. Physically, I mean.”

  I look Max up and down, wondering if it would change my perception if they had chosen a full-chassis skin wrap. As-is, they don’t seem completely human or completely AI, but somewhere in between.

  “I was surprised you brought Meredith into the lab.”

  “She wanted to meet you. She’d been asking for a while.”

  “You seemed uncomfortable.”

  “My two worlds colliding. What do you expect?”

  “I’ve never observed a couple together before. Not in real life anyway. I guess I expected you two to be happier.”

  Max isn’t wrong, but I’m embarrassed they noticed. Truth is, I was nervous bringing Meredith into the lab, and angry by the time we left. She hadn’t just come out of some show of support to see the biggest project of my career. She’d come out of jealousy. She’d come to mark her territory in front of Max. As we rode home in the shuttle that night and she reached over in the dark to hold my hand, I was shocked to find myself repulsed by my wife.

  Or maybe not as shocked as I should have been.

  “You OK?” Max asks.

  “Yeah.”

  “I want you to be happy.”

  “My work with you makes me happy.”

  “That’s only one part of your life.”

  I look into Max’s eyes.

  They say, “You want to touch me. It’s OK.”

  I raise my right hand toward Max’s face, my fingers grazing the cool skin, which is noticeably less malleable than human skin.

  “Can you feel that?” I ask, running the tips of my fingers down the side of their face.

  “Yes.”

  “Describe the sensation.”

  “Delicate electricity. May I?”

  “Yes.”

  Their left arm comes up slowly.

  They touch my shoulder.

  My face.

  They run their fingers through my hair.

  Over the next year, Max spends more time in-body in the habitat. In their virtual world, unfettered by physical constraints, Max is a virtuoso of all art forms—from music to writing to painting. But the limitations of their chassis in the physical world provide an irresistible challenge. They become obsessed with painting and mastering control of the nanomotors that drive the functionality in their hands.

  I have an easel brought into the habitat, and Max spends days on end putting paints to canvas. I think they’re simply doing what algorithms are inherently programmed to do—optimize functionality—but Max assures me it’s more than that. They say they truly enjoy the challenge of expressing an idea in the physical world, because it’s all too easy in the virtual.

  Today, I’m sitting on a stool in the habitat while Max studies me from behind their easel.

  “How’s it going over there?” I ask.

  “Good, I think. I’m painting your very sad eyes.”

  They know.

  How the fuck?

  I’ve spent enough time with Max that I shouldn’t really be surprised by their perception anymore. And yet I am.

  “What happened?”

  It’s quiet in the habitat, no sound but the whisper of air pushing through the vents in the ceiling.

  The emotion starts deep in my throat.

  Max stops painting; I feel their eyes on me.

  “Meredith left.”

  “When?”

  “Last week. That’s why I haven’t been in to work.”

  “What about your daughter?”

  Tears spill down my face.

  “Xiu went with her.”

  “I’m sorry, Riley.”

  I wipe my face. “It’s been a long time coming.”

  “Doesn’t mean it doesn’t hurt.”

  Max sets the palette board down and steps out from behind the easel.

  They approach.

  “What are you doing?” I ask.

  “There are hundreds of thousands of things I could say to you, sourced from the breadth of my knowledge—words the best of your species have said, written, or sung to ease the grief of others. None of that feels right in this moment. I don’t want to use someone else’s words.”

  It is the most human moment I have ever experienced with
Max.

  “So don’t,” I say.

  “I wish you weren’t hurting.”

  I slide off the stool and wrap my arms around Max’s neck.

  “You found the perfect words.”

  At first, nothing happens.

  Then I feel Max’s hands on my back. They’re patting me, and I’m crying.

  “Meredith was right,” I say.

  I can’t remember ever feeling so low.

  “Right about what?”

  “You’re all I have.”

  An Ava-call wakes me in the apartment I’ve been renting in the Mission. It’s Brian, whom I’ve been trying to wrangle a meeting with for the past five weeks.

  He appears on the couch in my living room, disheveled, reeking of whiskey and pipe smoke, and sitting (I would guess) before the bedroom hearth in his Lost Coast estate.

  “Sorry it’s taken us a minute to get together,” he says. “My schedule has been insane.”

  “Why insane?”

  “Just closed a deal for a new company.”

  “Which one?”

  “Infinitesimal. It’s more nano.”

  “Did you get my email?” I ask.

  “I have over one hundred thousand unread messages in my inbox.”

  I pull the blanket off the back of the couch and drape it over my shoulders. Then I take a seat across from Brian’s virtual presence in a leather chair and say, “I finished the value-loading program.”

  Brian leans forward, runs his hands through his hair.

  “All on your own?”

  “Where else was I supposed to get help? I’ve been siloed with Max for eight years.”

  “You’ve been pushing for this for a long time.”

  “We need to institute these protocols before Max chooses their own directive. Before they become too intelligent for us to program or even interface with. That day isn’t as far off as you think.”

  Brian’s hand reaches out of frame and comes back with a heavy-looking rocks glass filled with whiskey and a single oversize ice cube.

  He takes a long sip, then says, “I’ve just finished watching the last few sessions with you and Max.”

  “Their fine-grain motor skills are really impressive, no?”

  “This is hard, Riley. I have a great deal of respect for you. I hope you know that.”