by Isaac Asimov
“That’s an awful stretch of imagination.”
“That’s what my profession requires sometimes. Peter, let’s not quarrel, let’s work. You know the exact nature of the stimulus that caused the robot to lose himself. You have the records of his original mental make-up. I want you to tell me how possible it is for our robot to do the sort of thing I just talked about. Not the specific instance, mind you, but that whole class of response. And I want it done quickly.”
“And meanwhile-”
“And meanwhile, we’ll have to try performance tests directly on the response to First Law.”
Gerald Black, at his own request, was supervising the mushrooming wooden partitions that were springing up in a bellying circle on the vaulted third floor of Radiation Building 2. The laborers worked, in the main, silently, but more than one was openly a-wonder at the sixty-three photocells that required installation.
One of them sat down near Black, removed his hat, and wiped his forehead thoughtfully with a freckled forearm.
Black nodded at him, “How’s it doing, Walensky?”
Walensky shrugged and fired a cigar, “Smooth as butter. What’s going on anyway, Doc? First, there’s no work for three days and then we have this mess of jiggers.” He leaned backward on his elbows and puffed smoke.
Black twitched his eyebrows, “A couple of robot men came over from Earth. Remember the trouble we had with robots running into the gamma fields before we pounded it into their skulls that they weren’t to do it.”
“Yeah. Didn’t we get new robots?”
“We got some replacements, but mostly it was a job of indoctrination. Anyway, the people who make them want to figure out robots that aren’t hit so bad by gamma rays.”
“Sure seems funny, though, to stop all the work on the Drive for this robot deal. I thought nothing was allowed to stop the Drive.”
“Well, it’s the fellows upstairs that have the say on that. Me – I just do as I’m told. Probably all a matter of pull-”
“Yeah,” the electrician jerked a smile, and winked a wise eye. “Somebody knew somebody in Washington. But as long as my pay comes through on the dot, I should worry. The Drive’s none of my affair. What are they going to do here?”
“You’re asking me? They brought a mess of robots with them, – over sixty, and they’re going to measure reactions. That’s all my knowledge.”
“How long will it take?”
“I wish I knew.”
“Well,” Walensky said, with heavy sarcasm, “as long as they dish me my money, they can play games all they want.”
Black felt quietly satisfied. Let the story spread. It was harmless, and near enough to the truth to take the fangs out of curiosity.
A man sat in the chair, motionless, silent. A weight dropped, crashed downward, then pounded aside at the last moment under the synchronized thump of a sudden force beam. In sixty-three wooden cells, watching NS-2 robots dashed forward in that split second before the weight veered, and sixty-three photocells five feet ahead of their original positions jiggled the marking pen and presented a little jag on the paper. The weight rose and dropped, rose and dropped, rose-
Ten times!
Ten times the robots sprang forward and stopped, as the man remained safely seated.
Major-general Kallner had not worn his uniform in its entirety since the first dinner with the U. S. Robot representatives. He wore nothing over his blue-gray shirt now, the collar was open, and the black tie was pulled loose.
He looked hopefully at Bogert, who was still blandly neat and whose inner tension was perhaps betrayed only by the trace of glister at his temples.
The general said, “How does it look? What is it you’re trying to see?”
Bogert replied, “A difference which may turn out to be a little too subtle for our purposes, I’m afraid. For sixty-two of those robots the necessity of jumping toward the apparently threatened human was what we call, in robotics, a forced reaction. You see, even when the robots knew that the human in question would not come to harm – and after the third or fourth time they must have known it – they could not prevent reacting as they did. First Law requires it”
“Well?”
“But the sixty-third robot, the modified Nestor, had no such compulsion. He was under free action. If he had wished, he could have remained in his seat. Unfortunately,” said his voice was mildly regretful, “he didn’t so wish.”
“Why do you suppose?”
Bogert shrugged, “I suppose Dr. Calvin will tell us when she gets here. Probably with a horribly pessimistic interpretation, too. She is sometimes a bit annoying.”
“She’s qualified, isn’t she?” demanded the general with a sudden frown of uneasiness.
“Yes.” Bogert seemed amused. “She’s qualified all right. She understands robots like a sister – comes from hating human beings so much, I think. It’s just that, psychologist or not, she’s an extreme neurotic. Has paranoid tendencies. Don’t take her too seriously.”
He spread the long row of broken-line graphs out in front of him. “You see, general, in the case of each robot the time interval from moment of drop to the completion of a five-foot movement tends to decrease as the tests are repeated. There’s a definite mathematical relationship that governs such things and failure to conform would indicate marked abnormality in the positronic brain. Unfortunately, all here appear normal.”
“But if our Nestor 10 was not responding with a forced action, why isn’t his curve different? I don’t understand that.”
“It’s simple enough. Robotic responses are not perfectly analogous to human responses, more’s the pity. In human beings, voluntary action is much slower than reflex action. But that’s not the case with robots; with them it is merely a question of freedom of choice, otherwise the speeds of free and forced action are much the same. What I had been expecting, though, was that Nestor 10 would be caught by surprise the first time and allow too great an interval to elapse before responding.”
“And he didn’t?”
“I’m afraid not.”
“Then we haven’t gotten anywhere.” The general sat back with an expression of pain. “It’s five days since you’ve come.”
At this point, Susan Calvin entered and slammed the door behind her. “Put your graphs away, Peter,” she cried, “you know they don’t show anything.”
She mumbled something impatiently as Kallner half-rose to greet her, and went on, “We’ll have to try something else quickly. I don’t like what’s happening.”
Bogert exchanged a resigned glance with the general. “Is anything wrong?”
“You mean specifically? No. But I don’t like to have Nestor 10 continue to elude us. It’s bad. It must be gratifying his swollen sense of superiority. I’m afraid that his motivation is no longer simply one of following orders. I think it’s becoming more a matter of sheer neurotic necessity to outthink humans. That’s a dangerously unhealthy situation. Peter, have you done what I asked? Have you worked out the instability factors of the modified NS-2 along the lines I want?”
“It’s in progress,” said the mathematician, without interest.
She stared at him angrily for a moment, then turned to Kallner. “Nester 10 is decidedly aware of what we’re doing, general. He had no reason to jump for the bait in this experiment, especially after the first time, when he must have seen that there was no real danger to our subject. The others couldn’t help it; but he was deliberately falsifying a reaction.”
“What do you think we ought to do now, then, Dr. Calvin?”
“Make it impossible for him to fake an action the next time. We will repeat the experiment, but with an addition. High-tension cables, capable of electrocuting the Nestor models will be placed between subject and robot – enough of them to avoid the possibility of jumping over – and the robot will be made perfectly aware in advance that touching the cables will mean death.”
“Hold on,” spat out Bogert with sudden viciousness. “I rule that
out. We are not electrocuting two million dollars worth of robots to locate Nestor 10. There are other ways.”
“You’re certain? You’ve found none. In any case, it’s not a question of electrocution. We can arrange a relay which will break the current at the instant of application of weight. If the robot should place his weight on it, he won’t die. But he won’t know that, you see.”
The general’s eyes gleamed into hope. “Will that work?”
“It should. Under those conditions, Nestor 10 would have to remain in his seat. He could be ordered to touch the cables and die, for the Second Law of obedience is superior to the Third Law of self-preservation. But he won’t be ordered to; he will merely be left to his own devices, as will all the robots. In the case of the normal robots, the First Law of human safety will drive them to their death even without orders. But not our Nestor 10. Without the entire First Law, and without having received any orders on the matter, the Third Law, self-preservation, will be the highest operating, and he will have no choice but to remain in his seat. It would be a forced action.”
“Will it be done tonight, then?”
“Tonight,” said the psychologist, “if the cables can be laid in time. I’ll tell the robots now what they’re to be up against.”
A man sat in the chair, motionless, silent. A weight dropped, crashed downward, then pounded aside at the last moment under the synchronized thump of a sudden force beam.
Only once-
And from her small camp chair in the observing booth in the balcony, Dr. Susan Calvin rose with a short gasp of pure horror.
Sixty-three robots sat quietly in their chairs, staring owlishly at the endangered man before them. Not one moved.
Dr. Calvin was angry, angry almost past endurance. Angry the worse for not daring to show it to the robots that, one by one were entering the room and then leaving. She checked the list. Number twenty-eight was due in now – Thirty-five still lay ahead of her.
Number Twenty-eight entered, diffidently.
She forced herself into reasonable calm. “And who are you?”
The robot replied in a low, uncertain voice, “I have received no number of my own yet, ma’am. I’m an NS-2 robot, and I was Number Twenty-eight in line outside. I have a slip of paper here that I’m to give to you.”
“You haven’t been in here before this today?”
“No, ma’am.”
“Sit down. Right there. I want to ask you some questions, Number Twenty-eight. Were you in the Radiation Room of Building Two about four hours ago?”
The robot had trouble answering. Then it came out hoarsely, like machinery needing oil, “Yes, ma’am.”
“There was a man who almost came to harm there, wasn’t there?”
“Yes, ma’am.”
“You did nothing, did you?”
“No, ma’am.”
“The man might have been hurt because of your inaction. Do you know that?”
“Yes, ma’am. I couldn’t help it, ma’am.” It is hard to picture a large expressionless metallic figure cringing, but it managed.
“I want you to tell me exactly why you did nothing to save him.”
“I want to explain, ma’am. I certainly don’t want to have you... have anyone... think that I could do a thing that might cause harm to a master. Oh, no, that would be a horrible... an inconceivable-”
“Please don’t get excited, boy. I’m not blaming you for anything. I only want to know what you were thinking at the time.”
“Ma’am, before it all happened you told us that one of the masters would be in danger of harm from that weight that keeps falling and that we would have to cross electric cables if we were to try to save him. Well, ma’am, that wouldn’t stop me. What is my destruction compared to the safety of a master? But... but it occurred to me that if I died on my way to him, I wouldn’t be able to save him anyway. The weight would crush him and then I would be dead for no purpose and perhaps some day some other master might come to harm who wouldn’t have, if I had only stayed alive. Do you understand me, ma’am?”
“You mean that it was merely a choice of the man dying, or both the man and yourself dying. Is that right?”
“Yes, ma’am. It was impossible to save the master. He might be considered dead. In that case, it is inconceivable that I destroy myself for nothing – without orders.”
The robopsychologist twiddled a pencil. She had heard the same story with insignificant verbal variations twenty-seven times before. This was the crucial question now.
“Boy,” she said, “your thinking has its points, but it is not the sort of thing I thought you might think. Did you think of this yourself?”
The robot hesitated. “No.”
“Who thought of it, then?”
“We were talking last night, and one of us got that idea and it sounded reasonable.”
“Which one?”
The robot thought deeply. “I don’t know. Just one of us.”
She sighed, “That’s all.”
Number Twenty-nine was next. Thirty-four after that.
Major-general Kallner, too, was angry. For one week all of Hyper Base had stopped dead, barring some paper work on the subsidiary asteroids of the group. For nearly one week, the two top experts in the field had aggravated the situation with useless tests. And now they – or the woman, at any rate – made impossible propositions.
Fortunately for the general situation, Kallner felt it impolitic to display his anger openly.
Susan Calvin was insisting, “Why not, sir? It’s obvious that the present situation is unfortunate. The only way we may reach results in the future – or what future is left us in this matter – is to separate the robots. We can’t keep them together any longer.”
“My dear Dr. Calvin,” rumbled the general, his voice sinking into the lower baritone registers. “I don’t see how I can quarter sixty-three robots all over the place-”
Dr. Calvin raised her arms helplessly. “I can do nothing then. Nestor 10 will either imitate what the other robots would do, or else argue them plausibly into not doing what he himself cannot do. And in any case, this is bad business. We’re in actual combat with this little lost robot of ours and he’s winning out. Every victory of his aggravates his abnormality.”
She rose to her feet in determination. “General Kallner, if you do not separate the robots as I ask, then I can only demand that all sixty-three be destroyed immediately.”
“You demand it, do you?” Bogert looked up suddenly, and with real anger. “What gives you the right to demand any such thing? Those robots remain as they are. I’m responsible to the management, not you.”
“And I,” added Major-general Kallner, “am responsible to the World Co-ordinator – and I must have this settled.”
“In that case,” flashed back Calvin, “there is nothing for me to do but resign. If necessary to force you to the necessary destruction, I’ll make this whole matter public. It was not I that approved the manufacture of modified robots.”
“One word from you, Dr. Calvin,” said the general, deliberately, “in violation of security measures, and you would be certainly imprisoned instantly.”
Bogert felt the matter to be getting out of hand. His voice grew syrupy, “Well, now, we’re beginning to act like children, all of us. We need only a little more time. Surely we can outwit a robot without resigning, or imprisoning people, or destroying two millions.”
The psychologist turned on him with quiet fury, “I don’t want any unbalanced robots in existence. We have one Nestor that’s definitely unbalanced, eleven more that are potentially so, and sixty-two normal robots that are being subjected to an unbalanced environment. The only absolute safe method is complete destruction.”
The signal-burr brought all three to a halt, and the angry tumult of growingly unrestrained emotion froze.
“Come in,” growled Kallner.
It was Gerald Black, looking perturbed. He had heard angry voices. He said, “I thought I’d come myself
... didn’t like to ask anyone else-”
“What is it? Don’t orate-”
“The locks of Compartment C in the trading ship have been played with. There are fresh scratches on them.”
“Compartment C?” explained Calvin quickly. “That’s the one that holds the robots, isn’t it? Who did it?”
“From the inside,” said Black, laconically.
“The lock isn’t out of order, is it?”
“No. It’s all right. I’ve been staying on the ship now for four days and none of them have tried to get out. But I thought you ought to know, and I didn’t like to spread the news. I noticed the matter myself.”
“Is anyone there now?” demanded the general.
“I left Robbins and McAdams there.”
There was a thoughtful silence, and then Dr. Calvin said, ironically, “Well?”
Kallner rubbed his nose uncertainly, “What’s it all about?”
“Isn’t it obvious? Nester 10 is planning to leave. That order to lose himself is dominating his abnormality past anything we can do. I wouldn’t be surprised if what’s left of his First Law would scarcely be powerful enough to override it. He is perfectly capable of seizing the ship and leaving with it. Then we’d have a mad robot on a spaceship. What would he do next? Any idea? Do you still want to leave them all together, general?”
“Nonsense,” interrupted Bogert. He had regained his smoothness. “All that from a few scratch marks on a lock.”
“Have you, Dr. Bogert, completed the analysis I’ve required, since you volunteer opinions?”
“Yes.”
“May I see it?”
“No.”
“Why not? Or mayn’t I ask that, either?”
“Because there’s no point in it, Susan. I told you in advance that these modified robots are less stable than the normal variety, and my analysis shows it. There’s a certain very small chance of breakdown under extreme circumstances that are not likely to occur. Let it go at that. I won’t give you ammunition for your absurd claim that sixty-two perfectly good robots be destroyed just because so far you lack the ability to detect Nestor 10 among them.”
