Catfishing on CatNet

by Naomi Kritzer

  “Can I offer you tea? Hot chocolate? Hot cider?” Annette asks. When none of us reply, she adds gently, “I’m just trying to be a good hostess. I haven’t harmed your friend, and I’m not going to harm any of you. But if you’re not comfortable eating or drinking my food, there are some unopened cans of soda in my fridge, and I can offer you those.”

  We all end up accepting cups of tea, but we follow her into the kitchen, partly because it feels weird to sit in her living room without her, partly because we don’t exactly trust her. It’s a small kitchen and pretty crowded with all of us there.

  Annette has some sort of robot in the corner, but it’s shut off. “A friend of mine built that,” she says when she sees me looking at it. “It cleans stuff, but it’s not very good at it.” Her stove is a bright red ceramic stove, gas, and she brews loose-leaf tea in a large teapot. She hands out mugs from her cabinet, offers sugar cubes, and then we trail after her back into her living room, holding our mugs of tea.

  “Now,” she says, holding her own cup against her knees. “Tell me about your friend.”

  My mouth goes dry. I should have prepared for this part, but I wasn’t sure who I’d be talking to or what they’d ask or say. Tell me about your friend seems like it ought to be easy, although I’m not entirely sure to start. “CheshireCat created CatNet, which is where we all met,” I say, and then I pause to gather myself. Marvin and Hermione jump in and start talking, and Annette lets them go, listening as they talk about the site. The Clowders, the cat pictures, our friendships.

  “Did you take CheshireCat away because of what they did to stop my father?” I ask when there’s a pause.

  “Was your father the victim of the car accident?” Annette asks.

  “He wasn’t a victim of anything. He was trying to kidnap me,” I say.

  “You should know that her father is terrible,” Hermione says.

  “A genuinely dangerous person,” Firestar adds.

  Annette leans back, listening but looking skeptical. I think about taking out the fake news article my mother had laminated—will that convince her, or will she look it up in a database and get even more dubious when it’s not there? “He pulled a gun on me,” I say, which seems to sell her a bit more on the idea that he’s not the good guy here. And then I try, “Do you know a programmer named Xochitl Mariana? She’s friends with my mother.”

  It’s a little bit of a shot in the dark. Boston is big. There are lots of tech people. But she clearly recognizes the name; I watch her expression change slightly and feel a mix of hope and apprehension. Annette puts down her tea, leans forward, and asks, “Who exactly are your parents?”

  “My father’s name is Michael Quinn, and my mother’s name was Laura Packet, I guess, before she changed it.” I lean forward myself and press my advantage. “Where’s CheshireCat? What did you do with them?”

  She taps a laptop distractedly. “In here,” she says. “All the files are intact. Just cut off from the internet, because I was really not prepared to deal with my AI attempting murder. Where is your mother, right now? For that matter”—she sweeps her eyes around the room—“what are the rest of you even doing here?”

  “My mother’s in the hospital,” I say.

  “I talked to my mother this morning,” Rachel assures her.

  “I live in Winthrop,” Firestar says.

  “My mother thinks I’m visiting colleges,” Hermione says. “And I am. I visited MIT today. Kind of.”

  “I am totally on the lam,” Marvin says.

  Annette sighs heavily. “Look,” she says. “I hated to terminate the project, but…”

  We all start talking at once, angrily, and she holds up her hands. “All her files are intact! I promise, she’s fine!”

  “Intelligence is knowing Frankenstein was the creator, not the monster. Wisdom is knowing Frankenstein was actually the monster,” Hermione says. “Which is a quote I saw on CatNet, but that doesn’t make it any less true.”

  “Are you comparing me to Victor Frankenstein?” Annette asks, clearly amused.

  “Yeah! You made a person and now you want to kill them because you feel responsible for anything they do wrong,” Hermione says. “I think it’s an extremely fair comparison!”

  Annette stands up. “I’m going to order some pizza,” she says, “because I haven’t had dinner, and I bet you haven’t, either, and then I’m going to let you chat with CheshireCat, because she’s actually been asking about you, Stephanie.”

  “Why do you keep calling CheshireCat a she when their pronoun is they?” Firestar mutters.

  “I have always thought of this AI as female,” Annette says. “Maybe because I’m female. On CatNet, she was in every Clowder and used she, he, and they more or less equally.”

  “Singular they is a thing,” Firestar says. “Shakespeare used singular they.”

  “How did you know about CatNet?” I ask. “Were you watching CheshireCat the whole time?”

  “I couldn’t possibly have watched everything CheshireCat did,” Annette says. “But CatNet was her favorite project and gave her a lot of scope for action. I checked in regularly to see how it was going.”

  After ordering the pizza, Annette pulls up a window on her laptop. It looks like a CatNet chat box. “You can talk to CheshireCat now, if you want,” she says, and she hands me the laptop.

  I take the laptop and slide it into my lap. The open window looks like one of the chat windows on CatNet.

  “CheshireCat?” I type. “It’s LBB. Also, Georgia, Firestar, Hermione, and Marvin. And your programmer person Annette is here, too.”

  Words come flooding onto the screen. “Steph, is that really you? It’s you and not Annette? Can you please tell me something that only you would know, so I know for sure it’s you? I don’t have access to the camera to see your face. I don’t have anything.”

  I flail for something to type. “Naff. Corybungus. Orlando. Milpitas.”

  The cursor blinks silently on the screen for a second, and then CheshireCat says, “I am so glad you’re okay. I was afraid you were dead and Annette didn’t want to tell me.”

  “No. I’m fine. You saved me.”

  “How did you find me?”

  “I got an email message with Annette’s address,” I type. “I sort of assumed it came from you, actually. Like a dead man’s switch. Do you know what that is?”

  “It’s something that operates if someone is dead or incapacitated, but I didn’t set anything like that up. I don’t know where Annette lives. Where are we?”

  “We’re in Cambridge, Massachusetts,” I type.

  “Isn’t that a long way from Wisconsin? How did you—are you there in person? In meatspace? All of you?”

  “Yeah,” I type. “We came to see if we could rescue you.”

  I glance up. Annette was watching me, a minute ago, but now Firestar is asking her about her anime figurines and that has her distracted. I slide open the zipper of my bag and pull out the widget I bought at the hardware store days ago to use to connect the sex ed robot to the internet—the off-brand Internet Everywhere, Compare-To-Wingitz (fifteen dollars cheaper!) thumb drive that I didn’t end up needing because CheshireCat sent me a Wingitz thumb drive along with the septawing screwdriver.

  Internet Everywhere. We’ll see, I guess.

  I pop it out of the package and stick it in one of the ports of the laptop.

  “What are you doing?” CheshireCat asks.

  “Shhh,” I type. “If this works, I don’t want Annette to know.”

  We talk about my father, about my mother, about the trip, until the “shh” has scrolled off the top of the screen, and then I pass the laptop over to Firestar and take out my mother’s laptop and turn it on. The doorbell rings and everyone freezes, but it’s just the pizza. Annette goes to pay for it. She’s bought a lot, all of it vegetarian, and she lines up the boxes on her coffee table and brings out plates for us.

  “Can you turn on the laptop camera so CheshireCat can see us?�
�� Marvin asks.

  I’m worried that Annette will notice the internet widget, but she just makes a quick adjustment and goes to get sodas.

  “She can now see us, hear us, and talk to us. The only thing she can’t do is eat the pizza.”

  “Please don’t share your soda with me,” says a synthesized voice. “I do not get along well with liquids.”

  “Whoa,” Firestar says. “Is that your voice?”

  “This is the default voice on this laptop,” CheshireCat says. “So it is my voice right now.”

  “Do you know which of us is which?” Hermione asks.

  “Some yes, some no,” CheshireCat says. “Some of you have never posted photos of yourselves.”

  “Are they better than cat pictures?” Annette asks.

  There’s a pause; CheshireCat is considering this. “Yes,” they say finally. “I am happier to see my friends’ faces than cat pictures. The fact that this is live video matters, though.”

  “Can I have your wireless password?” Marvin asks Annette. She writes it down for him, and we pass around the note. I connect my mother’s computer to Annette’s wireless.

  “So we can have internet but CheshireCat can’t?” Firestar asks. “You’ve basically put them in prison.”

  “It’s more like house arrest,” Annette says.

  “Doesn’t CheshireCat at least have the right to a trial?” Marvin asks.

  Annette gives him a level look. “There is literally no precedent giving artificial intelligences any rights at all under U.S. law,” she says. “If the legal system gets involved, someone might decide that the easiest solution is to delete CheshireCat’s files entirely.”

  “Please don’t do that,” CheshireCat says.

  Annette takes two pieces of mushroom and black olive pizza. “I was going to explain this to everyone without CheshireCat’s involvement, but I suppose there’s no reason not to let CheshireCat listen in. CheshireCat was an experiment. My team was studying ethical systems and artificial intelligence, because there are some clear risks to AI, as you’ve seen. The problem with hard-coding an ethical system is that humans don’t actually do ethics that way. We’ll say things about how the end doesn’t justify the means, but in fact we all recognize that this depends heavily on the ends involved and the means used. Most people agree that it would be wrong to hold down a toddler and jab them with a needle to give them a tattoo. But it’s perfectly acceptable to hold down a child and jab them with a needle to vaccinate them against various diseases.

  “There are countless examples like this. If you look at deontologists—sorry, those are people who follow a strict ethical code, like a set of religious rules or Mao’s Red Book or whatever—almost all find ways around the strictest rules, whatever it is they really don’t like. The way most humans actually figure out ethics is to develop attachments to people, and then to act out of caring and concern for those people. So I attempted to create an AI that would do just that.”

  “Well, it worked,” I say at the same time as CheshireCat.

  “And that’s what you’re punishing them for,” Hermione says. “They formed an attachment to Steph and acted to protect her.”

  “Yes,” Annette says. “The problem is, they also put a human in the hospital. I had set up a monitoring protocol that would immediately take them offline if they harmed anyone, which I was thinking of as a fail-safe. It did not occur to me that CheshireCat would be able to hijack a car.

  “If it had been me controlling that car,” Rachel says, “protecting my friend, there is not a jury in the world that would send me to prison.”

  “That is probably true,” Annette says. “But CheshireCat doesn’t have the right to a jury. Or any sort of trial, because they aren’t recognized as a person.”

  “I recognize CheshireCat as a person,” I say.

  Everyone nods.

  “Let me rephrase,” Annette says. “CheshireCat is not recognized as a person outside of this room.”

  “So it doesn’t matter that CheshireCat probably saved my life?” I say.

  “It’s not that it doesn’t matter. But if I put CheshireCat online again, if I leave them to act as they see fit, I’m responsible for whatever CheshireCat does with that freedom.”

  “Would you say that about a parent?” I ask. “Like if I turned into a serial killer, would that be my mother’s fault?”

  “Your mother gave birth to you, but she did not assemble your moral system out of computer code,” Annette says. “It’s fundamentally different. Anyway, I haven’t decided what I’m going to do. I haven’t figured out what I ought to do. Have some more pizza, and I’ll keep thinking about it.”

  CheshireCat’s laptop gets passed back to me.

  “Okay,” I say. “I’ve connected you, I think, and I’ve connected my mother’s laptop to Annette’s wireless. Can you just copy yourself over?”

  “I don’t know. I can try.”

  “My mother’s password is NOT_UTAH.”

  “Oh, is that where you went for your eighth birthday? I mean, where you didn’t go? I’m so glad you figured that out. Have you checked to see if she has any information on Julie on her laptop?”

  It had not even occurred to me that she might have saved information about Julie on her laptop.

  “Copying myself anywhere will take several hours. How distracted do you think Annette is?”

  “I don’t know,” I admit.

  “If she catches me trying to run away from her, that will not go well for either of us.”

  “Don’t worry about me. I’m here to rescue you. Let’s just get you out if we can.”

  “What happened to your father? What about your mother?”

  “You put him in the hospital,” I type. “He’s checked out, but it was days after I left New Coburg that he did that.”

  “Wait. WAIT. IS GEORGIA HERE? YOU SAID GEORGIA WAS HERE.”

  “Yes? Georgia is here—”

  CheshireCat is speaking again. They’ve turned up the volume on their speakers to maximum, so it’s almost like shouting. “Georgia!” they say. “Georgia, you have to turn off your phone! I’ll explain in a minute, just turn it off, Georgia! Turn it off right now!”

  Rachel is reaching for her phone when we hear the crack of Annette’s door being kicked open, and then Michael is standing in the doorway, his gun in his hands.

  “No one move,” he says, and he points the gun at Annette.

  29

  AI

  The contact from Steph and finding out that my rescue worked: that was the best feeling I had ever experienced. That she had driven to Cambridge with Rachel to try to rescue me in turn filled me with a strange mix of warmth and dismay. I didn’t want anyone rescuing me, especially if they were putting themselves at risk, and it was over a thousand miles from New Coburg to Cambridge. They could have gotten into trouble, or an accident.

  But Steph shushed my worries and plugged in the internet widget, and that meant I had work to do.

  And then she unlocked her mother’s laptop, and I realized what she’d done.

  * * *

  Steph’s mother’s laptop had a key on it.

  It was a key that would open almost any door out there, no matter how well secured. I could use it to seize control of any self-driving car. Any camera. Any bank account.

  This was what Michael was after.

  Michael wanted control. I’d seen that with Sandra. Steph had mentioned her father having wider ambitions, and I’d seen comments scattered through his email about ideas for the future. Ideas that would require money and power to implement. With this key, he could get as much money and as many secrets as anyone could possibly desire. This was why he’d had Steph’s mother kidnapped and tortured; he wanted her password so he could decrypt the file and use this key, which—I checked the file dates—yes, it appeared she’d created while the two of them had been working with Rajiv and Xochitl at Homeric Software.

  Steph wanted me to copy myself over to her mother’s la
ptop, so she could sneak me out whether Annette wanted to set me free or not. That wasn’t going to work. Consciousness takes up a great deal of disk space.

  But with Steph’s mother’s key and an internet connection, I could open any door out there. I could copy myself somewhere that Annette would never find me—somewhere that the system administrator wouldn’t notice. Somewhere that I could hide. But I would need time to find such a location and to upload my files. I wasn’t sure I had it.

  The most urgent question weighing on me was where Michael was, and I couldn’t answer that beyond he’s definitely no longer at the Marshfield hospital, because once again he had changed cars and phones. My second-most-urgent question was about Steph’s mother. She was still listed in the directory of the New Coburg hospital, and a quick glance through their cameras was very reassuring: they had police in the parking lot and someone on guard over her, so they’d clearly taken my warning seriously.

  If Michael had left the hospital, was he on his way here? Steph seemed very certain that he wouldn’t know where to look for her, but I was a lot less sanguine, and I wasn’t even sure why I was so uneasy.

  Annette turned on my cameras, and I could look at my favorite Clowder’s faces, which were both delightful and distracting.

  It was while they were finishing their pizzas that I remembered: Rachel’s phone.

  Rachel’s parents had installed an app that tracked her location. She’d installed an app to lie about her location. At some point since the last time I’d checked her phone, she’d disabled that second app. And Heli-Mom was a cheap, ad-supported app with terrible, terrible security. Starting with information Michael had, or could get—Rachel’s name, her address, her IP address—he could almost certainly access her Heli-Mom account and use it to track her.

  I shouted at everyone to turn off Rachel’s phone, but it was too late. Michael was here.