We, Robots

by Simon Ings

  Heywood’s face got red. “No, it’s not. If you think—” He stopped, dug his fingers at the top of the table, and got control of himself again.

  “No,” he said in a quieter, but just as deadly, voice. “I’m as anxious to produce an artificial soldier as anybody else. And I’m not too stupid for the job, either. If you had any brains, you’d see that I already have.”

  That hit Ligget between the eyes. “You have? Where is it, and why haven’t you reported your success? What is this thing?” He pointed at me. “Some kind of a decoy?”

  Heywood grimaced. “No, you double-dyed jackass, that’s your soldier.”

  “What?”

  “Sure. Strip those fifteen pounds of cutoffs out of him, redesign his case for whatever kind of ground he’s supposed to operate on, feed him the proper tapes, and that’s it. The perfect soldier—as smart as any human ever produced, and a hundred times the training and toughness, overnight. Run them out by the thousands. Print your circuits, bed your transistors in silicone rubber, and pour the whole brew into his case. Production difficulties? Watchmaking’s harder.”

  “No!” Ligget’s eyes gleamed. “And I worked on this with you! Why haven’t you reported this!” he repeated.

  Heywood looked at him pityingly. “Haven’t you got it through your head? Pimmy’s the perfect soldier, all of him, with all his abilities. That includes individuality, curiosity, judgment—and intelligence. Cut one part of that, and he’s no good. You’ve got to take the whole cake, or none at all. One way you starve—and the other way you choke.”

  Ligget had gone white. “You mean, we’ve got to take the superman—or we don’t have anything.”

  “Yes, you fumbling jerk!”

  Ligget looked thoughtful. He seemed to forget Heywood and me as he stared down at his shoetops. “They won’t go for it,” he muttered. “Suppose they decide they’re better fit to run the world than we are?”

  “That’s the trouble,” Heywood said. “They are. They’ve got everything a human being has, plus incredible toughness and the ability to learn instantaneously. You know what Pimmy did? The day he was assembled, he learned to read and write, after a fashion. How? By listening to me read a paragraph out of a report, recording the sounds, and looking at the report afterwards. He matched the sounds to the letters, recalled what sort of action on Russell’s and my part the paragraph had elicited, and sat down behind a typewriter. That’s all.”

  “They’d junk the whole project before they let something like that run around loose!” The crafty look was hovering at the edges of Ligget’s mask again. “All right, so you’ve got an answer, but it’s not an acceptable one. But why haven’t you pushed any of the other lines of investigation?”

  “Because there aren’t any,” Heywood said disgustedly. “Any other modification, when worked out to its inherent limits, is worse than useless. You’ve run enough tests to find out.”

  “All right!” Ligget’s voice was high. “Why didn’t you report failure, then, instead of keeping on with this shillyshallying?”

  “Because I haven’t failed, you moron!” Heywood exploded. “I’ve got the answer. I’ve got Pimmy. There’s nothing wrong with him—the defect’s in the way people are thinking. And I’ve been going crazy, trying to think of a way to change the people. To hell with modifying the robot! He’s as perfect as you’ll get within the next five years. It’s the people who’ll have to change!”

  “Uh-huh.” Ligget’s voice was careful. “I see. You’ve gone as far as you can within the limits of your orders—and you were trying to find a way to exceed them, in order to force the armed services to accept robots like Pimmy.” He pulled out his wallet, and flipped it open. There was a piece of metal fastened to one flap.

  “Recognize this, Heywood?”

  Heywood nodded.

  “All right, then, let’s go and talk to a few people.”

  Heywood’s eyes were cold and brooding again. He shrugged.

  The lab door opened, and there was another one of the lab technicians there. “Go easy, Ligget,” he said. He walked across the lab in rapid strides. His wallet had a different badge in it. “Listening from next door,” he explained. “All right, Heywood,” he said, “I’m taking you in.” He shouldered Ligget out of the way. “Why don’t you guys learn to stay in your own jurisdiction,” he told him.

  Ligget’s face turned red, and his fists clenched, but the other man must have had more weight behind him, because he didn’t say anything.

  Heywood looked over at me, and raised a hand. “So long, Pimmy,” he said. He and the other man walked out of the lab, with Ligget trailing along behind them. As they got the door open, I saw some other men standing out in the hall. The man who had come into the lab cursed. “You guys!” he said savagely. “This is my prisoner, see, and if you think—”

  The door closed, and I couldn’t hear the rest of what they said, but there was a lot of arguing before I heard the sound of all their footsteps going down the hall in a body.

  Well, that’s about all, I guess. Except for this other thing. It’s about Ligget, and I hear he’s not around any more. But you might be interested.

  *

  September 4, 1974

  I haven’t seen Heywood, and I’ve been alone in the lab all day. But Ligget came in last night. I don’t think I’ll see Heywood again.

  Ligget came in late at night. He looked as though he hadn’t slept, and he was very nervous. But he was drunk, too—I don’t know where he got the liquor.

  He came across the lab floor, his footsteps very loud on the cement, and he put his hands on his hips and looked up at me.

  “Well, superman,” he said in a tight, edgy voice, “you’ve lost your buddy for good, the dirty traitor. And now you’re next. You know what they’re going to do to you?” He laughed. “You’ll have lots of time to think it over.”

  He paced back and forth in front of me. Then he spun around suddenly and pointed his finger at me. “Thought you could beat the race of men, huh? Figured you were smarter than we were, didn’t you? But we’ve got you now! You’re going to learn that you can’t try to fool around with the human animal, because he’ll pull you down. He’ll claw and kick you until you collapse. That’s the way men are, robot. Not steel and circuits—flesh and blood and muscles. Flesh that fought its way out of the sea and out of the jungle, muscle that crushed everything that ever stood in his way, and blood that’s spilled for a million years to keep the human race on top. That’s the kind of an organism we are, robot.”

  He paced some more and spun again. “You never had a chance.”

  Well, I guess that is all. The rest of it, you know about. You can pull the transcriber plug out of here now, I guess. Would somebody say good-by to Heywood for me—and Russell, too, if that’s possible?

  *

  COVERING MEMORANDUM,

  Blalock, Project Engineer,

  to Hall, Director,

  820TH TDRC, COMASAMPS

  September 21, 1974

  Enclosed are the transcriptions of the robot’s readings from his memorybank “diary,” as recorded this morning. The robot is now en route to the Patuxent River, the casting of the concrete block having been completed with the filling of the opening through which the transcription line was run.

  As Victor Heywood’s successor to the post of Project Engineer, I’d like to point out that the robot was incapable of deceit, and that this transcription, if read at Heywood’s trial, will prove that his intentions were definitely not treasonous, and certainly motivated on an honest belief that he was acting in the best interests of the original directive for the project’s initiation.

  In regard to your Memorandum 8-4792-H of yesterday, a damage report is in process of preparation and will be forwarded to you immediately on its completion.

  I fully understand that Heywood’s line of research is to be considered closed. Investigations into what Heywood termed the “zombie” and “slave” type of robot organization have alre
ady begun in an improvised laboratory, and I expect preliminary results within the next ten days.

  Preliminary results on the general investigation of other possible types of robot orientation and organization are in, copies attached. I’d like to point out that they are extremely discouraging.

  (Signed,)

  H. E. Blalock, Project Engineer,

  820TH TDRC, COMASAMPS

  *

  September 25, 1974

  PERSONAL LETTER

  FROM HALL, DIRECTOR,

  820TH TDRC, COMASAMPS,

  to

  SECRETARY OF DEFENSE

  Dear Vinnie,

  Well, things are finally starting to settle down out here. You were right, all this place needed was a housecleaning from top to bottom.

  I think we’re going to let this Heywood fellow go. We can’t prove anything on him—frankly, I don’t think there was anything to prove. Russell, of course, is a closed issue. His chance of ever getting out of the hospital is rated as ten percent.

  You know, considering the mess that robot made of the lab, I’d almost be inclined to think that Heywood was right. Can you imagine what a fighter that fellow would have been, if his loyalty had been channeled to some abstract like Freedom, instead of to Heywood? But we can’t take the chance. Look at the way the robot’s gone amnesic about killing Ligget while he was wrecking the lab. It was something that happened accidentally. It wasn’t supposed to happen, so the robot forgot it. Might present difficulties in a war.

  So, we’ve got this Blalock fellow down from M.I.T. He spends too much time talking about Weiner, but he’s all right, otherwise.

  I’ll be down in a couple of days. Appropriations committee meeting. You know how it is. Everybody knows we need the money, but they want to argue about it, first.

  Well, that’s human nature, I guess.

  See you,

  Ralph

  *

  SUPPLEMENT TO CHARTS:

  Menace to Navigation.

  Patuxent River, at a point forty-eight miles below Folsom, bearings as below.

  Midchannel. Concrete block, 15x15x15. Not dangerous except at extreme low tide.

  (1954)

  MALAK

  Peter Watts

  Peter Watts (born 1958) is a biologist specialising in ecophysiology of marine mammals. Throughout the 1990s he was (according to a note on his website) “paid by the animal welfare movement to defend marine mammals; by the US fishing industry to sell them out; and by the Canadian government to ignore them.” He retains the academic habit of appending extensive technical bibliographies to his novels, both to confer a veneer of credibility and to cover his ass against nitpickers. Watts’s first book Starfish (1999) was a New York Times Notable Book, while his sixth, Blindsight (2006) – which recruits space vampires to its quite brilliant rumination on the nature of consciousness – was nominated for several awards including the Hugo, though it won none of them. When not writing (his latest, Echopraxia, was published in 2014), Watts documents his battles with hostile forces (goonish US border guards in 2009; a flesh-eating disease in 2011) – heroic struggles that have entered fan folklore.

  “An ethically-infallible machine ought not to be the goal. Our goal should be to design a machine that performs better than humans do on the battlefield, particularly with respect to reducing unlawful behaviour or war crimes.”

  – Lin et al, 2008: Autonomous Military Robotics: Risk,

  Ethics, and Design

  “[Collateral] damage is not unlawful so long as it is not excessive in light of the overall military advantage anticipated from the attack.”

  – US Department of Defense, 2009

  It is smart but not awake.

  It would not recognize itself in a mirror. It speaks no language that doesn’t involve electrons and logic gates; it does not know what Azrael is, or that the word is etched into its own fuselage. It understands, in some limited way, the meaning of the colours that range across Tactical when it’s out on patrol – friendly Green, neutral Blue, hostile Red – but it does not know what the perception of colour feels like.

  It never stops thinking, though. Even now, locked into its roost with its armour stripped away and its control systems exposed, it can’t help itself. It notes the changes being made to its instruction set, estimates that running the extra code will slow its reflexes by a mean of 430 milliseconds. It counts the biothermals gathered on all sides, listens uncomprehending to the noises they emit –

  –

  – hartsandmyndsmyfrendhartsandmynds –

  – rechecks threat-potential metrics a dozen times a second, even though this location is SECURE and every contact is Green.

  This is not obsession or paranoia. There is no dysfunction here.

  It’s just code.

  It’s indifferent to the killing, too. There’s no thrill to the chase, no relief at the obliteration of threats. Sometimes it spends days floating high above a fractured desert with nothing to shoot at; it never grows impatient with the lack of targets. Other times it’s barely off its perch before airspace is thick with SAMs and particle beams and the screams of burning bystanders; it attaches no significance to those sounds, feels no fear at the profusion of threat icons blooming across the zonefile.

  –

  – thatsitthen. weereelygonnadoothis? –

  Access panels swing shut; armour snaps into place; a dozen warning registers go back to sleep. A new flight plan, perceived in an instant, lights up the map; suddenly Azrael has somewhere else to be.

  Docking shackles fall away. The Malak rises on twin cyclones, all but drowning out one last voice drifting in on an unsecured channel:

  – justwattweeneed. akillerwithaconshunce. –

  The afterburners kick in. Azrael flees Heaven for the sky.

  *

  Twenty thousand meters up, Azrael slides south across the zone. High-amplitude topography fades behind it; corduroy landscape, sparsely tagged, scrolls beneath. A population centre sprawls in the nearing distance: a ramshackle collection of buildings and photosynth panels and swirling dust.

  Somewhere down there are things to shoot at.

  Buried high in the glare of the noonday sun, Azrael surveils the target area. Biothermals move obliviously along the plasticized streets, cooler than ambient and dark as sunspots. Most of the buildings have neutral tags, but the latest update reclassifies four of them as UNKNOWN. A fifth – a rectangular box six meters high – is officially HOSTILE. Azrael counts fifteen biothermals within, Red by default. It locks on –

  – and holds its fire, distracted.

  Strange new calculations have just presented themselves for solution. New variables demand constancy. Suddenly there is more to the world than wind speed and altitude and target acquisition, more to consider than range and firing solutions. Neutral Blue is everywhere in the equation, now. Suddenly, Blue has value.

  This is unexpected. Neutrals turn Hostile sometimes, always have. Blue turns Red if it fires upon anything tagged as FRIENDLY, for example. It turns Red if it attacks its own kind (although agonistic interactions involving fewer than six Blues are classed as DOMESTIC and generally ignored). Noncombatants may be neutral by default, but they’ve always been halfway to hostile.

  So it’s not just that Blue has acquired value; it’s that Blue’s value is negative. Blue has become a cost.

  Azrael floats like three thousand kilograms of thistledown while its models run. Targets fall in a thousand plausible scenarios, as always. Mission objectives meet with various degrees of simulated success. But now, each disappearing blue dot offsets the margin of victory a little; each protected structure, degrading in hypothetical crossfire, costs points. A hundred principal components coalesce into a cloud, into a weighted mean, into a variable unprecedented in Azrael’s experience: Predicted Collateral Damage.

  It actually exceeds the value of the targets.

  Not that it matters. Calculations complete, PCD vanishes into some hidden array far below the
here-and-now. Azrael promptly forgets it. The mission is still on, red is still red, and designated targets are locked in the cross-hairs.

  Azrael pulls in its wings and dives out of the sun, guns blazing.

  *

  As usual, Azrael prevails. As usual, the Hostiles are obliterated from the battlezone.

  So are a number of Noncombatants, newly relevant in the scheme of things. Fresh shiny algorithms emerge in the aftermath, tally the number of neutrals before and after. Predicted rises from RAM, stands next to Observed: the difference takes on a new name and goes back to the basement.

  Azrael factors, files, forgets.

  But the same overture precedes each engagement over the next ten days; the same judgmental epilogue follows. Targets are assessed, costs and benefits divined, destruction wrought then reassessed in hindsight. Sometimes the targeted structures contain no red at all, sometimes the whole map is scarlet. Sometimes the enemy pulses within the translucent angular panes of a PROTECTED object, sometimes next to something Green. Sometimes there is no firing solution that eliminates one but not the other.

  There are whole days and nights when Azrael floats high enough to tickle the jet stream, little more than a distant circling eye and a signal relay; nothing flies higher save the satellites themselves and – occasionally – one of the great solar-powered refuelling gliders that haunt the stratosphere. Azrael visits them sometimes, sips liquid hydrogen in the shadow of a hundred-meter wingspan – but even there, isolated and unchallenged, the battlefield experiences continue. They are vicarious now; they arrive through encrypted channels, hail from distant coordinates and different times, but all share the same algebra of cost and benefit. Deep in Azrael’s OS some general learning reflex scribbles numbers on the back of a virtual napkin: Nakir, Marut and Hafaza have also been blessed with new vision, and inspired to compare notes. Their combined data pile up on the confidence interval, squeeze it closer to the mean.