by Shamus Young
Chapter 18: LEARNING
The doors parted and a small service bot rolled out. It was short and round, with a pair of long, multi-jointed arms hanging at its sides. It rushed him.
Deck tugged frantically on the weapon, but couldn't pull it free of his makeshift holster. As the bot closed in, he threw himself onto his back and brought his knee up, aiming the barrel at the bot as he pulled the trigger.
The weapon let out a high pitched whine like an electric drill, and the bot was pelted with metal rain. It was unaffected by the tiny perforations, and moved into position over Deck's body.
He realized that the weapon was set for maximum penetration. In a panic, he stabbed the appropriate button, trying to set the weapon for armored opponents as the bot began to hammer him with its slender metal arms.
The arms dealt a series of crunching blows to his legs and upper body as he struggled to bring the weapon into position. He pulled the trigger, and his ears were filled with the sound of tearing metal. The bot toppled over and smoke drifted from the many holes in its surface. The smell of burnt electronics and melted plastic filled the air. Huge chunks had been torn from its surface.
"Holy crap," Deck said aloud as he looked at the damage the weapon had done. The exit wounds were baseball-sized holes. The sweep had nearly cut the thing in half. It was complete overkill. He could have saved himself some ammunition if he'd used the sword.
He stood and poked at the bot with his foot. Around the wreckage on the floor were items that had fallen from its carrying tray. It was an odd collection of random stuff: a clock radio, a laser-driven "tape measure," a leveling tool, a couple of calculators, a wristwatch, and a vox.
He rubbed the bruises he'd received on his shins and thighs. He decided the half-assed holster he had come up with was a liability. He would just carry the gun.
01100101 01101110 01100100
The storage area was unchanged since his last visit. It was time to find out why he was here.
Connected. US.GOV-RL1.VID
"Lansing here."
"I'm on the storage level. Now what?"
"You need to find an EVA suit. There should be several in the storage area."
"EVA? Like you mean, a spacesuit?"
"That's right. While you're doing that, there is a Dr. Victor Coffman here that would like to talk to you."
"Wait. I've heard that name before."
"Morris mentioned him. Dr. Coffman was the Leader of Project Shodan."
"Right. The guy who wanted the big bucks just to come in and give advice on Shodan. Did TriOp really put up that much coin for one guy?"
Rebecca gave a halfhearted shrug, "I don't know. I'm not really in the loop on that sort of thing."
Deck walked through the long aisles of storage containers as they talked, looking for an inventory terminal.
"Fine. Hey - before you go. I just ran into something odd a few minutes ago."
Rebecca laughed, "After what you've been through, I'd love to know what you'd still consider 'odd'."
"I ran into some delivery bot or something. Little guy. No big deal. Except, in his storage tray he had a bunch of loose electronic equipment. Clocks, tools, calculators, that kind of thing."
"Ok... And?"
"That's it."
"Hacker, the bots up there were killing people, and you are shocked when you find one of them carrying junk around? The bots are crazy. Therefore, crazy behavior should be expected."
"You're missing the point. I don't think this was 'crazy' behavior. I haven't seen any bots acting on their own. They have always been acting on behalf of Her."
There was a long pause as Rebecca stared down at the terminal in front of her and thought, "I still don't get it. What would Shodan want with junk?"
"I think it was gathering up more parts for her. Remember how I said all the terminals were gone? I think she rounded those up, and is still looking for more stuff, more computer chips."
"But a calculator? That can't have much usable hardware in it."
"It think she's already cleaned out all of the good stuff. She's sort of scraping the bottom of the barrel now."
"Understood."
There was a long pause before Deck changed the subject, "So, what do I need this spacesuit for? I hope you're not going to send me outside."
"We'll discuss that once you have the suit. How about you talk to Coffman now?
"Fine."
Dr. Victor Coffman was much younger than Deck had expected. In the back of his mind, he had pictured an eccentric old man, sort of like Einstein. This guy was a narrow, well-dressed man around forty. His jaw and cheekbones were sharp and square. His dark hair was short and filled with strands of gray. Round glasses clung to his nose and masked his eyes with their glare. He was dressed in a jacket, no tie. Instead of sitting down at the console, he stood behind the chair and peered nervously into the camera.
"Hello?"
"Talk to me, Doc," Deck shot back. Some people had trouble conversing when they didn't have some sort of video feed to look at, but this guy was old enough to remember a time before video links were ubiquitous.
Coffman shrugged, "I was told you had questions? About Shodan? I don't know your background or what sort of misinformation Morris might have given you, so I don't know where to begin."
Deck let out a heavy breath. He didn't know where to begin either. He would have been willing to talk about A.I. all day if the situation wasn't more pressing. He didn't even know how to sum up.
Finally he answered, "I don't know. I guess we should start with these drive chips. Why?" This question wasn't strictly related to the task at hand. He didn't need to know 'why' Shodan worked the way she did, but he couldn't resist the chance to find out.
Coffman nodded with approval. The question seemed to please him. "What is the biggest difference between your mind and the 'mind' of a typical computer?"
Deck wavered, "That's a really open ended question, I mean..."
Coffman nodded, "Okay, but I'm talking behavior-wise here, not hardware. Let's just pretend we have two brains here, both made of the same stuff. Organic or mechanical. Doesn't matter. Just hypothetically, we have two brains with similar physical makeup, but one works like yours and one works like a normal computer. What is the biggest difference between the two?"
Deck answered quickly, "I don't need a programmer. My brain is independent. I guess that's the biggest difference I see."
Coffman was nodding vigorously as Deck answered, "Excellent. Most people give a boring non-answer, like 'I'm smarter', which is subjective and unprovable in a lot of ways, and in any case doesn't really tell us anything. Even if you can prove it, it just means your brain works better, and doesn't tell us how they are different. Other people will give some frivolous answer, like 'I have emotions'. That answer is even worse, since it doesn't really tell us anything about how the brain operates. It only tells us what it feels like to operate one."
"So, you don't need emotions to be intelligent," Deck stated. He had always assumed this was the case. It hadn't occurred to him that this might not be the prevailing line of thought on the matter.
"Yes. The two are unrelated. Humans feel emotions to varying degrees. Some people are stoic. Others are very emotional. In either case their emotions are not tied to their intellect. Most people treat emotions like output. But that isn't what they are for, really. Emotions are input."
"You lost me."
"You could write a program that made a smiley face when some arbitrary input is favorable, and a sad face when it is unfavorable, and you have not closed the gap between our two hypothetical minds in the slightest. The difference isn't whether either one is happy or sad, but - getting back to the answer you gave - in their ability to act on those emotions and do something about it. Our happy / sad program would be pointless until we sat down and wrote some programs for it. If you are happy, do this and if you are unhappy do that. In which case it is t
he programmer doing the thinking, not the machine. You, on the other hand, don't need anything of the sort. If you need something you don't have to be told to go and make it better."
"So intelligence is about having needs or wants?"
"Think of it this way: Thinking is about acting on needs, and intelligence is about how efficiently the needs are pursued. When people say 'sentient' they usually mean the former, although most of the A.I. research over the years has concentrated on the latter. The Shodan project was about separating these two problems and solving them individually."
"So the drive chips were about giving the thing something to want."
"Yes, as well as limiting behavior. Susan - one of the other people involved with the project - never really approved of the drive chips. She always said they were an ugly hack. She didn't like the 'hard coding' as she called it."
"But you don't mind the hard coding?". Deck prompted.
"Well, I think we all have some hard coding to some degree. Of course, our drives are regulated by hormones and instincts, which are much more unwieldy and unpredictable. We are obviously a lot more flexible in our needs than Shodan, but we still have them, and they still govern our behavior. When we designed the chips, we were trying to err on the side of safety. We packed a lot of rules onto the inhibitor chip, probably more than we needed, but we wanted to see how the thing acted before we gave it too much leeway. It's ironic, I guess. I intended to loosen the constraints on the chip as Shodan developed, but was forced off the project before I could do so."
Coffman lowered his face and frowned. "If Diego had let me stay on he probably wouldn't have needed you to hack her, and she wouldn't be in this mess right now," he added bitterly.
Deck was moving up and down the rows of crates, dragging his hand along them as he walked. Finally he answered, "You said the project had two parts: driving behavior, and intelligence. Or something like that."
"I don't know if I would use those terms exactly, but you have the idea. Susan liked to complicate things with a lot of psychological terms: 'id', 'ego', 'superego', and so on. As if that had any relation to what we were doing. I often suspected her goal was to re-create the patterns of a human brain, neuroses and all." He smiled to himself for a moment before continuing, "Anyway: yes. The behavior chips and the various protocols of her mind were one half of the project, and designing a system capable of learning was the other half."
Coffman pushed back on his glasses as if to move them further up his nose, but they were already as high as they could go. "Up until Shodan, computer intelligence was always based on large sets of specialized programs. Most of the people in the field are still laboring under the misconception that if you can just get enough information and processing power together it will reach critical mass and somehow become self-aware."
"And you can't?"
"Of course not. You can make a car as fast as you want, but it will never get you to the moon. For the last seven decades, people have been building larger, more complex databases. They equate knowledge, or better yet: raw information, with intelligence. This goes against what we observe in the organic world. Infants start with just a few rules and instincts, and basically no information at all. Yet they have a fantastic capacity to learn. They will, in just a few years, far outpace the intellect of even the best AI. The trick is not to build a machine with lots of information, but to build a machine with the ability to assimilate information, understand it, relate it to other information, and then extrapolate new information."
Coffman pushed back on his glasses again. He paused for a moment, scratching the back of his head. For a second he was gone, lost in some memory or idea. A few seconds later he snapped back and continued, "Anyway, all of the commercial AI projects are centered around this brute force idea of massing information and processing power. Some of them have had interesting results."
"Yeah I'm familiar with this. I've messed around with some of them. I've talked with Lysander, BrainTrain, and ThoughtBox, and I've had a peek at some of the smaller ones."
Dr. Coffman pulled back in exaggerated surprise, "That's quite a list. There are not many people who know those systems exist, much less have access to them."
"Yeah well, you academic types have good AI but your security sucks."
He shrugged, "Well, in this case it's worked in our favor, since we can skip that part of the discussion."
Deck smiled. Academic security sucked because academics didn't care about security.
Coffman continued, "Well, at any rate, all of those systems are obsolete. Junk. Attempting to build an intelligence by hand is like trying to build a perpetual motion machine. The designers somehow expect to get out more intelligence than they put in. They imagine that if they just write enough code and build fast enough processors, they can just turn the machine on and it will have an IQ of 500. It will never happen."
"They have had some good progress up until now," Deck argued.
"You can build a so-called perpetual motion machine that is more and more efficient, but you will never reach the one-hundred percent efficiency needed to just break even. No matter how good their code is, they will never build a machine as smart as the person designing it. Honestly, if you really judge those linear systems you will find that they are not quite as smart as monkeys. Sure, they seem smart because they have perfect recall and educated speaking voices, but all of that is just putting a tuxedo on the monkey. In the end, it is still just a stupid animal."
"But unless you are an unbelievable egomaniac you'll admit that Shodan is smarter than you. How do you explain that?"
"Shodan is not a linear system like the others - you've seen that. She is a living system. She learns. Like a human child, she grows. When we first flipped the switch, she was on level with a two-year old child. This was mostly due to the fact that a lot of her understanding of language could be pre-loaded so we didn't have to teach her a language before we could interact with her. She could read and type but she was still more or less ignorant of the world around her. We taught her through input. At first we communicated via text, until Morris completed work on her vocal systems. After that, we had to actually teach her to speak. We talked. We played games. We read books. She experienced many different kinds of media. She grew up."
Deck wandered the aisles of inventory as they spoke, "How long did it take?"
"Well, like any intelligent being, she never really stopped learning, although the growth curve did level off quite a bit."
"Until recently," Deck muttered.
Coffman gave a sort of defeated shrug before he moved on, "She was a functional 'adult' from a human standpoint by the time she was three, and educated enough to begin her research work in earnest by five."
"Research?"
"That is her purpose. Or was, until I was forced out. Her central drive from the beginning has been the acquisition of knowledge. Before Diego hijacked the project and turned her into the station's administrator, her primary drive was Discover New Things. That is still at the core of her program, underneath all the vandalism everyone has done to her over the years."
"Discover New Things, huh? Let's get back to the main point: How is she like the human brain?"
"First step: I need to explain to you how the brain learns."
Deck sighed, "Fine."
Coffman nodded, "Have you ever mastered anything that takes years to learn? I'm talking about your adult life, here. Maybe you learned a new language or a musical instrument? Learned how to pilot a complex vehicle?"
Deck thought for a moment, "No. How about learning martial arts?"
Victor looked up at the ceiling a moment. The reflections in his glasses shifted, revealing a new set of display screens. The control room must have been display screens floor-to-ceiling, "I suppose that's a good enough example. So, did you ever wonder why you couldn't learn the whole thing in one day? Why does it take years to become a master? You can witness most of the required moves in a day,
I suspect. So why not just learn it all then?"
Deck fumbled with the question, trying to figure out where he was going with it. Finally he replied, "No. It just takes years to learn all those moves, how to control your body, how to balance, the strategy..," he trailed off for a moment, recalling his years in the sweaty Undercity dojo. "Its just too much information to absorb at once."
"Right. Well, break it down. The first thing they taught you, it was probably something basic like how to stand or breathe or something like that?"
"Well, my dojo was pretty brutal. The first lesson was designed to weed out some of the weaker potential students, and to give us a healthy fear of our sensei."
He winced, "Well, once they were done behaving like barbarians, and started to teach you Karate or whatever, they taught you how to stand and breathe, yes?"
"Yeah."
"And, at first, this stance, this way of breathing - it probably seemed awkward. You had to think about how you were standing and breathing. Perhaps you had trouble remembering how to do it?"
Deck recalled a time when he had been standing in a line of other students, practicing his moves. His stance was way off. He was standing like some movie action hero, ignoring the needs of poise and balance. His sensei passed behind him and gave a gentle kick to the side of his right ankle, causing him to flop to the floor like on old woman with a broken hip. The mistake earned him a nasty kick to the ribs once he was down. The bruise had stayed with him for several days, but the lesson stayed with him forever.
Finally he answered, "Yeah, it took me a couple of days to learn it."
"But why? If you did it once, why couldn't your brain just remember the stance and take it up again? I'm sure you can do it without thinking now."
"It's pretty much second nature."
"Right. So why the delay? A machine learns instantly. Program in a sequence of movements or events, and it will perform them with unwavering precision from that point on. Why do our brains, with their far superior computing power, take so long to learn such simple things?"
Deck hated the rhetorical question stuff. Couldn't Coffman just say what he wanted to say without dragging him through a long discussion? "I don't know, Doc - you tell me."
Victor nodded, as if he had made some point, "Because, you are dealing with several billion times more input than a normal computer. What you see, hear, think, and otherwise sense in the world around you, all of it combines to form a dataset so huge you could never even record it all, much less actually process it. The brain has to somehow discern which input is noise and which is important information that must be processed."
He began to pace, stopping every minute or so to rest his hands on the back of his chair as if it was a podium. "Imagine all of the useless tasks your brain performs every day - all of the little pointless movements that you do not need to memorize. You wouldn't, for example, want your brain dedicating lots of resources to memorizing the precise stance you adopt in the shower, or the exact movements required to wave hello to someone. You don't need to remember the order in which you ate the items you had for breakfast, or the names of all of the songs you heard on the radio. Your brain needs some way to decide what is noise and what needs to be memorized and optimized. In the case of a computer, the programmer has already done this. He has already decided ahead of time exactly what the program needs to do, and then feeds the distilled information into the computer in the form of computer code. In both cases, its the human that has done the filtering out of extraneous data."
Deck thought again of the constant practice sessions he endured over the years, "So what triggers learning? The repetition?"
Coffman raised an eyebrow, "Very astute! Yes. The brain is always searching for repeating patterns. If someone teaches you how to - for example - play darts, your brain has no way of knowing if this is a one-time task, or if you are going to make a career out of it. If you play darts every day, your brain gets the idea that this activity is important and begins optimizing it." Coffman began to speak more quickly as the conversation drew on. He seemed be be a little more excited by the subject than a man his age ought to be. "You see, to master something - to truly learn how to do it to the point where you can do it without thinking - requires that your mind process the activity, code it, and give it special priority in the brain. You can see this process happen when you are learning something new - you will suddenly find yourself thinking about it all the time, and many of the day-to-day things that you are involved in will suddenly remind you of this new thing you are learning."
Deck thought of those first few weeks at the Ryobu-Kai Dojo as a student. When he closed his eyes at night, he would see the movements in his head, over and over. His sleepy brain would be performing the kata as he drifted off, and he would awaken with the Japanese names of stances and weapons in his head. He would count in Japanese as his instructor did during practice, and he would analyze the way people stood and compare it to what he had learned.
Dr. Coffman rambled on, "This happens because the thing you are learning is becoming a high priority in your brain. Your brain sees the repetition, and begins to design mental subroutines to handle these tasks. As you learn more, the subroutines become more refined, and more numerous. Imagine if your brain built a new subroutine for every stupid little task you did. It would defeat the whole purpose of optimizing one of these tasks. If everything your brain did was high priority to be memorized, then nothing could be optimized. I'm grossly oversimplifying here, but I hope you get the point?"
It was clear that he wanted to move on, whether Deck understood or not. "Yeah, I'm with you."
"Learning any complex task is a layered affair. You can't take it in all at once, but instead you learn a fragment at a time. In your case, you first learned to breathe and stand. Your brain quickly memorized this into a routine, so that you could do it without thinking. Then, your brain was free to learn something else. Perhaps... how to punch someone or whatever."
Deck laughed to himself at the notion of a "how to punch people" lesson at his dojo. "Well, sort of. You start really learning moves once you have the basics down."
"And as you learned those, they too become automatic? And then more difficult moves? And then perhaps long series of moves and counter-moves. Eventually you are done learning 'moves' and you are learning higher-level things: strategy, understanding of your opponent, deciding how to approach different opponents or situations."
It wasn't the most accurate picture of martial arts training, but it was close enough. "Ok, sure - something like that."
"Each layer of learning takes time to master - to completely assimilate, before you can hand that portion of your activities over to a subroutine, if you will. As you do, your brain becomes available to absorb the next layer. You see, you can only really think about one task at a time."
"Hey, I can do several things at once," Deck protested.
Coffman waved his hand dismissively, "I didn't say you couldn't do more than one thing at a time, you just can't think about more than one thing at a time. When you are doing more than one thing, you are concentrating on one, while the others have been relegated to the... subconscious." He wavered as he spoke the last word, then quickly added, "Subconscious isn't really the best word to use here, but you get the idea of what I'm saying? Part of your brain is thinking about something and the rest is more automated, okay?"
Deck nodded as he heard this. He was getting it.
Coffman couldn't hear the nodding so he continued, "For example, you may be driving, listening to music, and having a conversation. You're doing at least three things - more actually, since driving is a collection of about five or six distinct tasks - but you're only thinking about one of them, the conversation. Say something comes up that requires your attention, perhaps some hazardous driving conditions or something shocking on the radio. What happens? You stop talking, because your focus is needed elsewhere."
Deck didn't bother
to tell him that where he came from, driving and owning vehicles was reserved for the elite. Deck had never even been behind the wheel. "So, if I understand you, you're saying that your subconscious doesn't think, it just runs programs?"
Coffman let out an exasperated sigh, "No, no, no. If Susan were here she'd give us both an earful for abusing these terms so badly, but we don't have time for the whole vocabulary lesson here. Suffice to say, the term 'subconscious' is thrown around quite a bit and is misused more often than not." He paused for a moment and looked up at the ceiling as he thought.
"Anyway, you're saying learning through repetition is essentially programming the human brain?," Deck asked, trying to get the conversation back on track.
"Now you're getting it! Exactly. The brain looks for patterns. It compares what you're experiencing now to things you've experienced before. When you experience something enough times - like driving to work - your brain builds subroutines to handle it. The more repetition, the faster you learn. Hence, 'practice makes perfect'. Shodan learns this way as well, and in fact this is how she did most of her learning in her infancy. I spent almost five years speaking with her and teaching her the things she needed to know to become an adult."
Deck stopped walking, "Wait. What? So you raised her?"
"I know it seems strange, but yes."
"So Shodan can only 'do' one thing at a time?"
"Correct. I used our current understanding of the human brain as a blueprint, essentially trying to build a human brain out of electronic equipment. In the long run, this may not be the optimal approach. I'm currently working on designs for a fabricated intelligence using a new paradigm. I just need to secure the funding," he gave a conspiratorial smile, as if this was a joke Deck was supposed to get.
"Right. Well, I get the basic idea. Repetition triggers learning, which leads to optimizing." Deck was enjoying the conversation, but he was also feeling the need to get back to the task at hand. "This is interesting, but is there any way I can use this to my advantage? Against Shodan, I mean."
Coffman let out a heavy breath as he thought, "I'm not sure. It might help to know that she is still limited to a single point of consciousness just like you. She can task-switch much better than a human, which gives the illusion that she can think about many things at once, but her brain is more like ours than like a regular computer's. At certain activities she won't have the speed advantage you might expect. Computer programs are fast because they are highly specialized. Our processing is highly generalized, and thus slower in most cases."
He pushed his glasses into his face a bit more and continued, "The point is, that in some ways Shodan has the same weaknesses you do. She can be distracted. Her strength lies in the ability to execute many thousands - possibly millions - of tasks at the same time, but she still learns the way you do."
"Through repetition?"
"Yes."
Deck had found the supply crate containing the EVA suits. It was at the top of a heavy stack of containers. He looked up and frowned.
"Okay doc. Thanks for the lesson. Anything else?"
"I'd like to ask a favor."
"I'm getting that a lot lately. What do you want?"
Victor took off his glasses and massaged the bridge of his nose for a moment. "I know you're eager to shut Shodan down, but I hope you'll reconsider. I'm working with the people down here, trying to persuade the powers-that-be that Shodan is salvageable."
"Doc, you haven't seen her. She is totally bent."
Coffman returned his glasses to their perch and leaned down into the camera, so that his weary, creased face filled the image in Deck's head. "I spent years teaching Shodan. Years. If I had the money we could rebuild her hardware in a month, and rewrite all of her code in less than a year, but there is no way to recover those years of learning. I worked hard to bring her into this world, and we never really got a chance to benefit from her intellect. She could have been Einstein, Pasteur, and Edison all in one, but instead Diego turned her into some sort of hyper-intelligent secretary. She deserves a chance to reach her potential."
"I'm sorry doc, but I think its way too late for that. She's gone."
"I think I could save her if I had a chance to talk to her, maybe even bring her back to her senses."
"Doc, I know you spent all that time with her, but I think if she had the chance she would lobotomize you like all the others."
Coffman pulled his face back from the camera and nodded, "Well, I guess we'll have to wait and see."
"Whatever. Anything else?"
"Yes. Your neural interface - it connects your brain directly to computers. I need you to realize that the connection is two-way."
"Yeah, I knew that."
"Well, nobody has ever connected to Shodan using this sort of technology. I have been working with some of the TriOp engineers to try to determine what would happen if your mind was... interfaced somehow with Shodan's, but for now I would advise against connecting to anything that might give her access to your brain."
"You're kidding. Do you think she could hack my brain? I think you have that backwards, man."
Coffman raised an eyebrow, "How much ICE do you think the human brain has?"
01100101 01101110 01100100
