by Ken MacLeod
“Well—” began Beauregard.
“I should tell him,” Karzan interrupted, leaning forward. “I was the last of us to be killed.”
“Good point,” said Beauregard.
The others returned solemn nods.
“Two years and three months after you,” Karzan told Carlos. “That was when I died. Even then, after so many great battles, you were still world famous. The hero of Docklands! You were the first great martyr of the Acceleration. You took so many of the enemy with you! In the back streets little pictures of you were stuck to lamp posts and to doors and to the stocks of the fighters’ Kalashnikovs. You were known as Carlos the Terrorist. You inspired us and you were hated by the Reaction.”
She swigged from her bottle and sat back. “That’s why you must lead us now.”
Carlos had listened to this with horror almost as great as that with which he’d watched the recordings of his heroic feat.
He shook his head. “No, no. I haven’t got the experience to lead anyone. Pick someone else.”
The others exchanged admiring glances.
Ames laughed abruptly. “We’re Axle, dude. We wouldn’t want a leader who’d want to be leader. You’ll do.”
Carlos couldn’t help thinking of the choosing of the messiah in The Life of Brian. He tried not to smile.
“No, I can’t—”
Nicole leaned in and spoke sharply. “I think you’ll find you can,” she said.
She shot a stern covert glance at Carlos, with an almost imperceptible nod. Play along.
Carlos spread his hands. “All right. If you insist.”
Everyone cheered again.
There was an awkward pause as Nicole disappeared into the bar for more drinks. Nobody seemed sure what to say.
“How long have you all been here?” Carlos asked, ice-breaking.
“A few days,” said Ames. “I was the first. Nicole met me off the bus from the spaceport, brought me up to speed, left me to my own devices. Not that I have any devices, ha-ha! The others turned up one by one over the next four days. Same story. Taransay arrived yesterday.”
“Hell of a busy spaceport.”
Ames cackled, from somewhere deep in his throat. “You know, I think that spaceport may be just a sort of false impression they put in our minds to make the transition seem vaguely plausible. That’s how I’d do it.”
“Who’s ‘they’?”
Shrugs spread like a ripple around the table.
“Mademoiselle Pascal says AIs,” said Karzan, mindfully building a cigarette. She ran the gummed edge of paper across the tip of her tongue, eyes bright on Carlos, and rolled up with a flourish. “Who’re we to doubt her?”
Beauregard clapped Ames’ shoulder. “Comp Sci Spec Ops, that’s who!”
Ames grinned. “I don’t doubt her. It’s just—”
Whatever he was about to say was lost as Nicole returned with a tray of beer bottles. She put the tray down and then backed to the deck’s balustrade and hand-hopped herself on to its smooth mahogany handrail. There she sat poised and elegant, bottle in hand, legs crossed. She had their attention.
“Listen up,” she said. “I’ve told each of you where we are and you why you’re here. You’re here because you’re criminal, terrorist scum, and you’re here to fight. What I haven’t told you is who you’re fighting for, how you’ll fight, what’s expected of you and what’s in it for you.
“You’re fighting for the Direction, which as far as you’re concerned is me. Obviously the Direction itself is a whole passel of parsecs away, so it has an AI module in this space station—onsite autonomous, but ultimately answerable to the folks in the big building way back there in NYC and the people who elected them, yadda yadda. That module is in overall charge of the mission. I’m its plenipotentiary in this simulation, and in the company you now work for. The Direction likes to outsource as much as possible. Your immediate employer is a law enforcement company called Locke Provisos, hired by an exploration company, Astro America. Locke Provisos is a subsidiary of the top law company Crisp and Golding, Solicitors. You needn’t worry about the details—the companies are all fucking AIs anyway, it’s all accounting at the end of the day. The bottom line is: Locke Provisos pays your wages, which cover your housing and pretty much anything you can reasonably consume in here. It also—”
Ames raised a hand.
“Yes?” said Nicole.
“We’re in a sim,” Ames said. “We’re not exactly consuming anything, apart from processing power and electricity.”
Nicole frowned. “Like I said”—sounding testy—”it’s all accounting at the end of the day. For accounting purposes, resources are priced, even in sims. OK?”
Ames nodded, still looking unconvinced.
Nicole smiled slightly. “This isn’t some fucking utopia, you know. Anyway, when it comes to your weapons and equipment and so forth out in the real world, allocation is direct, as in any other military organisation. Locke Provisos supplies you with materiel and general instructions. Your ultimate employer is the Direction, and the buck here stops with me.
“Here’s how you’ll fight: after a bit of basic training to cohere you into a unit and then to get your reflexes used to operating a crude analogy of the machines you’ll be fighting in, you’ll all be loaded on to”—she waggled air quotes—”‘the bus to the spaceport.’ You’ll doze off. Trust me, you’ll doze off. You’ll wake up in space, in robot bodies. Frames, we call them. They’re quite adaptable bodies, they can plug into all kinds of machines—spacecraft, armoured crawlers, whatever. You’ll get the hang of it quickly—skill sets get downloaded on the fly, it’ll be a more a matter of mental adjustment than training. You’ll need just one more training exercise to familiarise yourselves with the machines. Then you’ll go into action.”
“Action against whom or what?” Chun asked. “Rax? Aliens?”
Someone tittered. Nicole stared down the levity.
“Much worse. Robots. Robots gone rogue.”
Carlos glanced at Ames, who closed his eyes and shook his head.
“Why not just use… other robots?” Rizzi queried. “Like drones.”
“Good question,” said Nicole. “We, of course, have combat machines. But there is a deep prohibition on their being directed by other robots, or by AIs. Even the AI that represents the Direction in mission control is hardwired against taking command decisions. Human consciousness must be in charge of any military action. That is the law and as I said it is hardwired.”
“Why?” Carlos asked.
“Anything less would be far too dangerous. Humanity in its collective wisdom has decided—you can agree or disagree, it makes no difference here—that it doesn’t want armed autonomous AI loose in the universe. That’s the decision, and the Direction enforces it, and the law companies and other DisCorporates here must abide by it. So you, my friends, are to be the requisite humans in the loop.”
“Even though we’ll be robots ourselves?” Ames asked.
“Says the man who’s living inside a fucking computer. You know better, Waggoner Ames. You are still human minds, whatever hardware you’re running on.”
Ames snorted. “Obviously better hardware than evolution provided. I’d rather be a superhuman mind, while I’m about it.”
“No such thing,” said Nicole. “There is a kind of Roche Limit for consciousness—it can’t get above a certain size without breaking up. Humanity has evolved naturally to that limit, and then only statistically—hence mental breakdowns of various kinds. There are indeed AIs far more powerful than human minds, but they are not conscious as we understand it.”
“I’d like to see the workings on that Roche Limit business,” said Ames.
Nicole shrugged. “I can show you where to look it up if you must. Later.”
“Speaking of later,” said Carlos. “If you’ve got a robot revolt on your hands, how much time do we have for all this training? Sounds to me like you’re talking about weeks. Do you have weeks?”
Everyone stared at him. Someone laughed. Nicole smacked her forehead.
“Did I forget to tell you—oh yes, so I did, you had different stupid questions from everybody else—that this sim is running a thousand times faster than real time?”
“Fuck,” said Carlos, brazening it out. “That all?” He looked around. “Best crack on, then.”
He got the laugh, but he felt he’d shown himself up.
“Any more questions? Don’t worry, I have all day.”
“Once we’re trained and out in space and all that,” Rizzi asked, “is that it? Is that us? Space robots forever?”
Rizzi sounded worried. Ames snorted. “Bring it on.” Nicole gave him a sharp look, and Rizzi a reassuring smile.
“Not at all. That’s part of the point of this simulation. As robots you won’t get physically tired, you won’t need sleep, but to maintain your sanity and give you an incentive to cooperate you’ll get plenty of time off back here. Oh, and don’t even think of topping yourselves to get out of serving your sentences. You’ll just be brought back in some future emergency, maybe a worse one than this, and with the crime of desertion added to your docket. On the bright side, you needn’t worry about dying in battle. You’ll be backed up in your sleep on”—again with the air quotes—”‘the bus to the spaceport.’ If your frame is destroyed in action, you’ll just find yourself waking up on the bus back from the spaceport. You’re strongly recommended not to let that happen. Remember what that was like when you came here?”
Carlos recalled the dream of a dark drowning, and shuddered with the rest.
“Imagine that, but much worse. Avoid it if you can. The normal return from duty is considerably gentler, I assure you.” She looked around, eyebrows raised. “Any more?”
“If you have all these hardwired constraints on armed AI,” Chun asked, “how do you get robots going rogue and having to be fought in the first place?”
For all her poise, Nicole’s hand went to the back of her head. Carlos noticed this defensive reflex with interest.
“Ah,” she said. “Well. Some of the robots have become conscious in their own right, and, ah, they either did not have the constraint built in—there was no need to, at that level—or they found a way to override it. They adapt various tools and machines for military, or at least for hostile, purposes. Hostile to the mission’s goals, at least. And so—”
“Hold on a minute!” Ames cried. “You’ve somehow spawned conscious robots, and you want us to fight them?”
“Yes,” said Nicole. “As I said. You will be well armed and well capable of defeating them.”
“That wasn’t exactly my point,” Ames said, looking around for support. “I’m questioning the ethics of this thing.”
“Ethics!” Nicole looked scornful. “Don’t talk to me about ethics, Ames. Let me tell you about ethics. This is what you will get out of doing what I tell you—and what the company that employs you and the Direction that I represent tell you directly.”
She wedged her beer bottle between her knees, and put two fingers in her mouth and whistled. One of the bar staff, a young man with the weather-beaten look Carlos had noticed on all the locals, sauntered out.
“Yes?” he said.
Carlos hadn’t known any of the locals spoke English. Maybe the ones on the bus had all been deliberately unhelpful.
“Would you mind introducing yourself?” Nicole said.
The young man straightened up a little. “My name is Iqbal,” he said. “I was born on Malta, I worked as an agricultural technician in North Africa, and I died at the age of two hundred and ten. I chose to be scanned for uploading. I’ve worked at the Digital Touch for some years.”
“Do you enjoy it? Do you find it fulfilling?”
Iqbal pondered. “Yes,” he said. “It’s interesting to meet people, the scenery is spectacular, the work isn’t too hard, I save money. I prefer it to farming, of which I had quite enough in my first life. Fulfilling? Perhaps not. In my spare time I swim, I read, I study, I go out and have some fun. Someday I may wish to do something else, perhaps further my education. And of course I look forward to living on this planet in the real world. But for now I’d say I’m content, thank you.”
“Do you ever find yourself hesitating when you’re asked an unexpected question?” Nicole asked.
Iqbal hesitated, then laughed. “As you see, yes!”
“Thank you, Iqbal,” said Nicole. “That’s all for now. I’ll be in for another half-dozen drinks shortly.”
“You’re welcome, Mademoiselle Pascal.” He waved vaguely to all of them and went back inside.
“What was all that about?” Ames demanded.
Nicole slid down from her perch and sat back at the table. She leaned in and spoke quietly, drawing them all into a huddle.
“I’ve told you all that some people in this sim are ghosts like you—that is, they are of flesh and blood human origin like yourselves. Future colonists, basically, who unlike you are here in the sim as volunteers. What they volunteered for, well in advance and before their actual deaths, is live testing of the sim. Understandably, perhaps, there aren’t many volunteers for that, but we have a way of making up the numbers. That’s where the others here come in. They’re p-zombies—philosophical zombies. So called because philosophers once disputed whether you could have a human-like entity that displayed human behaviour in every detail, but without having human—or any—conscious awareness. Well, now we know, because we’ve made them. Walking thought experiments, so to speak. Iqbal is one of them. They can mimic consciousness, but they have no inner life, though they can answer any relevant question about it and about their ‘past lives’ on Earth as confidently and convincingly as Iqbal did just now. The point is, you have no way of telling the difference.”
“Did we ever?” Beauregard said. He looked around. “I’ve met loads of people who were a few enigmas short of the full Turing.”
That got a laugh. Nicole wasn’t impressed.
“That is precisely the attitude,” she said, “that got each of you posthumously executed centuries ago. Callous and instrumental. Borderline sociopathic. What you have to prove, here, is that you are capable of treating people as people, not as p-zombies. If you do, you’ll have a chance to rejoin human society—in our future colony, or back in the solar system if you prefer.”
“How could any of us get back to the solar system?” Carlos scoffed.
“You’re information now,” Nicole pointed out. “Information can be transmitted.”
“When you’ve built powerful enough lasers?”
“Yes.” Nicole shrugged. “And, yes, that will not be for a long time. But it would be no time, for you, if you were in storage. Your choice.”
“What happens,” Maryam Karzan asked, “if we win your fight but don’t pass your test?”
Nicole drew a fingernail across her throat. “Back in the box with you.”
“And if we lose the fight?” Ames taunted.
Nicole leaned back and lowered her eyelids. “Of course, you might lose. Consider what you would lose to. Imagine conscious entities with no natural selection behind them—no social instincts, no restraints, no notion of ethics beyond necessity and law. Imagine being at the mercy of minds without compassion, and with curiosity: endless, insatiable curiosity.”
She let that sink in for a bit.
“It strikes me,” Ames said, “that we already are.”
“What d’you mean?” Nicole snapped.
“I mean,” Ames said, “that we’re right now completely at the mercy of—heck, we’re living inside—intellects vast and cool and unsympathetic as all get-out.”
Nicole looked irritated. “As I said, you are not. The AIs running the mission and sustaining the simulation have constraints. The rebel robots do not. The difference is hardwired. It may not seem like much of a difference to you, but take it from me, it’s the difference you live in. For now.”
She stood up and leaned over the table, g
laring at them one by one.
“I would advise you not to lose.”
Belfort Beauregard sat in a bar.
It sounded like the beginning of a joke, to which he couldn’t remember the punchline. It kept going through Beauregard’s head, as he sat in the bar, or outside it on the deck back of the Touch. He’d had four days to adjust to his situation, and he still hadn’t. Every so often he thought he had, then the obsessive thought would come back, in one of its two guises.
The first was that he was literally in hell. Not the traditional conception, of course, but why expect hell to follow the vengeful fantasies of ancient sectaries and sex-starved medieval monks? The defining element of hell was eternal conscious suffering. Here he was, potentially eternally alive, and by God suffering. The whole place seemed set up to torment him in a very particular, very personal way.
For the past couple of nights he hadn’t repeatedly woken in a cold sweat—the girl had seen to that. Where was she, by the way? Out with her friends, no doubt having some mindless fun. Mindless fun, that’s a good one, must remember it. Christ, he could do with mindless fun. But unlike her he had a mind, not just a theory of mind. Ha. He was getting drunk. Have to watch that. Seen good men go bad that way. Here’s to their memory. Cheers.
Everyone laughing.
Which brought on the second variant: that it was all a joke. Like an April Fool’s prank, or a surprise party, or a you’ve-been-had reality show. At any moment the curtain would be whipped aside, the blindfold would come off, the truth dawn, the presenter step forward smirking, an audience of millions in stitches.
Maybe the horror of these two paranoid possibilities was his mind’s way of nudging him towards sanity, by making the reality—the virtual reality, let’s not forget, though the glass in his hand felt solid enough—less appalling and unacceptable by contrast.
After he’d been given the talk by the lady, and shown what he’d done and how he’d died, he nodded and mumbled an assent that hid bafflement. Not only had he no memory of having been with the Acceleration, his last memories of his life in the British Army betrayed no fundamental discontent. He’d enjoyed his work, he’d believed in what he was doing. The partisans in the Caucasus were a ruthless lot, deeply embedded in extended families and remote communities. Coordinating drone strikes on them had been a pleasure of the mind as much as of the gut.
