The Stories of Ibis
Page 18
“Um,” Kajita said, nervously raising her hand. “H-how much does one of these cost?”
I had been wondering the same thing.
“Er… how much was it?” Takami said. He looked toward his boss. He was clearly not someone who paid much attention to the financial side of things.
“Well, Shion is a prototype,” the boss said. He reluctantly named a figure. We all gasped. It was more than ten times our annual salary.
“Naturally, if we go into full production, the unit cost is a fraction of that, and they run on a new type of fuel cell—as long as you provide Shion with methanol every four hours, she can work twenty-four hours a day. Obviously, she will need to take the occasional maintenance break, so practically speaking she can only work about 120 hours a week, but that’s more than twice what we would expect of a human. With the expansion of methane hydrate resources, the price of methanol has dropped. Even with fuel and maintenance costs, we believe the final production model will be able to pay for itself after five years of operation.”
If they had that kind of money, they ought to treat us better, I thought. There really was a severe shortage of caregivers, but welfare budgets were in sharp decline. They had reduced our staff to the bare minimum and almost never increased our wages. If the government provided more financial aid, this job would be so much easier, and there would be more people who wanted to be nurses or caregivers. I knew welfare was funded from a different source than the money they were pouring into robotics. And the politicians that decided how to fund these programs all had scads of money and were guaranteed to spend their final years in comfort. Why should they care what happened to ordinary retired citizens?
Twenty years ago, a vast majority of them had voted against implementing new technology that would allow people to vote from home. Officially, they claimed this was because the protections against voter fraud were inadequate, but I’d heard there was another reason: if you could vote without visiting a voting booth, the number of voters in nursing care would skyrocket, and politicians who did not support the welfare state would be at a disadvantage. I didn’t know the truth, but it seemed plausible enough.
We grilled Takami for a while.
“How much can it do?”
“It should be able to perform all standard caregiver duties.”
“Specifically?”
“It can pass the caregiver certification exam.”
That got reactions from everyone—half skeptical, half impressed. Could a robot really do that? If it could, that was worth writing home about. We all knew just how difficult caring for the elderly was, knew just how delicate you had to be.
“Let me show you,” Takami said. He produced a demonstration disc and inserted it into the room’s video display.
It showed what was presumably part of the Geodyne labs. The camera looked down diagonally at Shion, who was standing by a bed. I had seen her face on the news. It was specially designed to look as human as possible so that residents would feel closer to their caregivers. She wore white work clothes and a nurse cap; if you didn’t know she was a robot, you would not have guessed.
A middle-aged man in pajamas was lying on the bed. A voice from offscreen instructed her to change the sheets without disturbing the occupant. Shion bent down and said, “May I change your sheets?” When the man nodded, she began working.
First she put her hands on the man’s shoulder and hip and gently rolled him onto the side of the bed closest to her. Then she circled around the bed and lowered the railing that kept him from rolling off. She rolled the dirty sheets up against the man’s back and quickly brushed the mattress itself before laying out a new sheet on that half of the bed. Making sure the sheet lay flat, she folded the ends into triangles and tucked them under the mattress. Then she raised the railing again and gently rolled the man across the rolled-up sheets onto the fresh one. She moved back to the original side, lowered the railing, removed the dirty sheet entirely, and placed it in a laundry bag. Then she spread out the other half of the new sheet and again folded and tucked the corners. Finally, she rolled the man back into the center of the bed.
Flawless.
Other videos showed Shion helping residents out of bed and into wheelchairs or over to portable toilets, teaming up with a human caregiver to lift a patient onto a stretcher, pushing a man in a wheelchair, and helping a woman eat. Our skepticism began to fade. It definitely looked like she could pass the certification exam.
“It took five years for us to teach her how to do this,” Takami said proudly. “None of this behavior is programmed. She learned it just like humans do, through constant repetition and refining of techniques. Her early movements were pretty rough—it could take her twenty minutes to spread a sheet. And she had trouble controlling her strength. We had her using a dummy rather than a human to practice on, but she kept breaking them. All it took was a little too much pressure and the dummy’s joints would snap, or she’d drop the dummy on the floor while trying to seat it in a wheelchair.”
Realizing that we were starting to look worried, he hastily added, “O-of course, nothing like that happens now. We’re absolutely confident she won’t make basic mistakes like that anymore. But we do think she won’t gain any real experience without some on-the-job training. She needs real-world experience if she’s going to level up.”
But the elderly are not RPG monsters, I thought.
“Our ultimate goal is to have Shion perform all her duties under her own discretion, but that may not be possible at first. We’ll need human staff with her, giving her instructions. Ideally, we would have one person devoted to that role. If several people provide conflicting instructions, it might confuse her.”
“You said as much,” the facility manager said. He turned toward me. “We’ve already selected someone for that role. Kanbara, would you be up for that?”
“Yes, certainly.” I agreed readily, but I had only been a nurse for five years and had no idea why I would be selected for such an important task. There were any number of people with more experience than me.
“I recommended you,” Kajita said. “You seem to like robots.”
“Hunh?”
“You said you often watch TV shows with robots in them, right? X-something.”
Erp. Was that all? Takami beamed like he’d found a buddy. This was awkward. I hadn’t been watching Advent of the Metal God X-Caesar because of the robots.
But Kajita could hardly be blamed.
“So the android will work with Kanbara on the second floor,” the facility manager said. “The news may call her an android caregiver, but she is not licensed, so as far as we’re concerned she is just another tool that helps us with our work. At first, she’ll only be working during the day. For the time being, Kanbara will be excluded from the night-shift rotation.”
Awesome. There was no bump in pay for working with the android, and with no night shift, my wages would drop. I couldn’t say I’d miss working on the night shift, but I wasn’t exactly thrilled by it either.
“How do I give instructions?” I asked. “Just tell her what to do?”
“Yes. Even if your grammar strays a little from textbook Japanese she’ll be able to understand. But if you get too abstract or vague she might have trouble grasping your intent—in that case, she’ll ask questions.”
“Does she obey anyone? What if one of the residents asks her to do something strange? Like… ‘Let me touch your breasts.’”
They laughed, but I was serious. There was no telling what seniors would say, particularly those suffering from dementia. If the android followed every command to the letter, it could cause all kinds of problems.
“She will prioritize orders from facility staff. If staff orders and resident orders conflict, she will follow staff orders. In the case of the breast issue… well, if you tell Shion not to obey that command, she’ll ignore it.”
“A lot of the more senile residents often ask to be taken home. Will she refuse those requests?”
The ten years of development had clearly not gone to waste; they had put a lot of thought into this.
“What about emergencies? If a senior suddenly collapses?”
“She will not wait for orders but act according to her best judgment.”
“How accurate will that judgment be?”
“That is hard to know for sure. During testing, Shion reacted to accidents at a fairly high rate of appropriateness. But not all accidents can be predicted. I couldn’t begin to guess whether she will react appropriately if something she has never experienced happens. But no matter what the event, Shion will not freeze. We’ve cleared the frame problem.”
Before I could ask, he explained, “The frame problem—say you tell a robot you’re going out, so it should keep you safe. The robot will walk with you, observing its surroundings, on the lookout for danger. But what qualifies as ‘danger’? There’s a car coming. There’s a chance the driver won’t turn in time, and the car will run into you. There’s a pebble on the ground in front of you. If you step on it the wrong way, you could trip and fall. The man walking toward you might be a terrorist carrying a bomb, and he might blow himself up! That house you’re walking next to might erupt in a gas explosion, there might be an earthquake, an airplane might fall on you—the chance of all of this is always greater than zero.
“If it considered all these possibilities, the robot would be overwhelmed. Every single thing it perceives will tug on its attention, and it will attempt to retrieve and process information related to all sensory input. The processors will lock up, and it will be unable to protect you. This is what we call the frame problem.”
“It can’t just ignore anything that is unlikely to happen?”
“Yes… but it wasn’t easy teaching a robot that trick. It would have to start by calculating that the chance of something happening was very low. But how do you calculate the odds of tripping on a pebble? Frankly, humans don’t even worry about odds when they decide to ignore risks. As an example from everyday life, every time there’s a news story about a child murderer, everyone takes steps to protect their children. But children are far more likely to die in a traffic accident than they are likely to be murdered. In which case, we should, logically, take steps to improve traffic safety. But few people think cars are more dangerous than child murderers. And the number of people who die in traffic accidents is dwarfed by the number of accidental deaths that occur in one’s own home, but nobody thinks their home is more dangerous than the street. People who worry about cell phone radiation or insignificant additives in their food will happily drink alcohol. Even though booze is far more dangerous. Not many people get married on the day of the Buddha’s death, do they? There’s no reason at all to assume something bad will happen if you do, but people choose to avoid a risk that doesn’t even exist.
“The point being: the way humans evaluate risk is nonsense. It is based on feelings rather than logic, subjective reactions rather than percentages and hard data. They draw a line between what they will take seriously and what they will ignore. To avoid the frame problem, we have to do the same. The robots do not calculate the percentage of anything—they simply ignore anything not immediately relevant to them. It took us an awfully long time to teach them how to do that.”
I was somewhat taken aback by all this. “S-so your robot…”
“Shion.”
“So Shion just… ignores danger?”
“Yes, she does.”
A collective grumble rose up in the room.
“What I need you all to understand,” Takami said, not at all daunted—in fact, he sounded quite proud—“is that nothing in this world is ever 100 percent safe. Of course, the job of any good engineer is to make things as safe as humanly possible. But there are no planes guaranteed never to crash. There are no drugs guaranteed to have no unexpected side effects. The idea of perfect safety is an illusion. There is always a compromise somewhere. If we removed everything with even a fraction of a percentage of a chance of causing us harm, there would be almost nothing left. We would be in the Stone Age again… and of course, the way the cavemen lived was far more dangerous than the way we do now.
“We make no claim that Shion is 100 percent safe. We know she is 99.99 percent safe, but we cannot rule out the possibility of some unforeseen accident occurring. But if you’ll excuse the suggestion… the same is true for all of you. Even with human caregivers, unimaginable accidents sometimes occur. The same with Shion.
“This is a problem we’ve been aware of for quite some time. Alan Turing, the father of artificial intelligence, said this in 1946: ‘If a machine is expected to be infallible, it cannot also be intelligent.’ Intelligence allows it to do things an ordinary machine could not do, and it is therefore also capable of making mistakes.
“Shion’s functions—like understanding poorly worded instructions and reacting to emergency situations—are made possible by circumventing the frame problem, which means ignoring certain risks. An android that never takes any risks is an android that never moves. That android is certainly safe but not particularly useful. Shion is very useful. And that brings with it some degree of risk.”
His logic was impeccable, I was sure. He was being completely honest with us, which certainly proved his sincerity, but this was not something we could emotionally accept that easily.
“So,” Okeya said, as if issuing a challenge. “What if something causes this robot to go berserk? It is much stronger than a human, right?”
“Well, yes. That kind of thing happened a lot in old manga but is unheard of in the real world. But in the unlikely event of something like that, keep your distance and call out the shut-down code.”
“The shut-down code?”
“A password that causes an emergency shutdown. It would take too long to fetch a remote control, so we made it into a verbal command. If you say the password, Shion will turn herself off.”
“What’s the password?”
“Klaatu barada nikto.”
“Huh?”
Takami grinned. “Since days of yore, these words have been used to tame rampaging robots. They are words you would never use in normal life, so there’s no chance of accidentally shutting her down in mid-conversation. Don’t mention them in her presence.”
“Right…”
“Well, why don’t we all practice the password?” Takami said. He waved his hand like an orchestra conductor. “And a one, and a two… Klaatu barada nikto!”
“Klaatu barada nikto!” we all chorused.
2
Shion arrived a month and a half later, on the last Monday in June. It was raining.
They had called ahead to let us know she was arriving, and a number of staff with nothing better to do had been hanging around the entrance waiting. Several of the more active seniors were also there to greet the new caregiver.
“How will she arrive?” Kajita said amiably. We were all wondering the same thing. “Will she be in a big box? Wrapped in plastic?”
“I doubt it,” I said. “She can walk, so they’ll probably drive her here. In the usual fashion.”
“But she’s so expensive! What if she gets wet?”
“If she broke that easily, she’d be no good at all,” I said with a laugh. In our line of work, we had things spilled on us all the time. And we had to help some of the seniors take baths. Any machine designed to perform those tasks would not rust over or short out when exposed to a little water.
“You think so?”
“H
ey, maybe we should make a card to welcome her with!” Kasukabe said excitedly. She had just started working at the facility a little over a year ago, leaving her old office job behind. I wouldn’t quite call her a friend, but we worked on the same floor, were roughly the same age, and we had the same taste in manga, so we often chatted when we both got stuck with the night shift. “And a bouquet! This is a big deal—we should make a thing of it!”
Okeya scowled. “Too late now.” We’d decided not to make a big deal out of it a few weeks ago. We were just welcoming a new caregiver and were not to treat her any differently than we would any new hire.
And although none of us said so aloud, none of us was really sure how much help this “new hire” would be. If we threw a big party to welcome her and she turned out to be useless or caused some awful disaster, it would be awkward.
“I dunno. It just seems too relaxed.”
“We’re not all as relaxed as you,” Okeya said, her age and dedication like oil next to the water of Kasukabe’s youth and cheer. Okeya always winced when Kasukabe said something that struck her as childish, but Kasukabe never noticed. Or if she did, she wasn’t the sort to let it bother her.
While we talked, a blue car pulled up in front of the hospital. It stopped, and the door opened. Takami stepped out, and Shion followed him.
What had I expected? Fanfare? A spotlight? Or a shower of roses magically appearing behind her? There was nothing like that. She just got out of the car and walked over to us. No editing, no fancy camera moves, no background music. She was, from the start, a part of the world in which we lived.
I’d seen pictures of robots like her any number of times, but this was my first time seeing one face-to-face. She looked like an ordinary young woman. She wore a simple white sleeveless dress and cute pumps. She was a little taller than me, 165 centimeters. Her bare arms were pale and slender—according to the data files, they were 2.5 times as strong as human arms. She had short hair. Her eyes gleamed, and there was a faint smile on her face. She was not especially beautiful—perhaps they had been worried about other women resenting her—but she had enough of a baby face that you wanted to like her. The luster of her skin and the sparkle in her eyes looked totally natural. It was hard to believe she was man-made.