by Em Rogers
“I'm okay, David, I’m just tired. Something has been bothering me about the machine too, about ADAM. It’s nothing I can quite put my finger on but it seems like it’s becoming something else. Something different than I expected.”
“I’m sure he has a few surprises in store for us.” David raised his eyebrows at her and laughed a little, “I got one this morning in fact, ADAM2 created a 3D representation of himself. We had a conversation in person. He’s blue.”
“What are you talking about, David?” She asked in surprise.
“ADAM created a real-time rendering of an avatar for us to interact with. Technologically what he did was amazing enough, but he… it seemed as lifelike as anyone else. That’s why he told me he was making himself blue. To differentiate himself from biological lifeforms. In order to not misrepresent himself.”
Susan scrunched up her face a little bit, confused, before she responded. “I’m not sure if we’re doing the right thing, in fact it scares me a little bit, David. The coding is coming along faster than we ever expected, but I'm concerned that now we’ve made the final system ‘conscious’ we’ve opened up a genie bottle that we can’t control. I know we all made fun of the idea of a malevolent super-intelligence, but what if that really happens?” Susan put her cup on the side counter, she’d barely touched it, and had a very worried look on her face. “What if it develops points of view incompatible with ours and becomes uncontrollable? What if we have to restart the system? Is that not a murder of some kind? How do we deal with that?” As she went on, an edge of anxiety in her tone of voice betrayed her concern. David, realizing how upset Susan was, put his clipboard down and came back to her desk. “Is that what you’re worried about?” He asked. “The machine’s feelings?”
Susan paused, a little embarrassed about her emotional state. “Well, it’s crossed my mind, at what point are we ethically responsible for the creation of a new life form, a conscious entity? I can’t just push it all to the back of my mind anymore and keep on blindly working on creating this system! This thing is alive, David, if the program is terminated for some reason, that will result in its death.”
David laughed a little in spite of the seriousness of their conversation, “Well, that’s a little extreme and it’s a little bit late for that don’t you think, Susan?”
She took a little sip of her now quite cool latte while she considered what was troubling her. “I’m a scientist - not a philosopher, a psychologist, or a science fiction futurist. I don’t even like to let those type of off-the-wall speculations influence my decision making process, but it is frightening, David! ADAM2 is designed to think, and perfect itself, it will be thousands of times more intelligent than any of us, and it may become incredibly powerful extremely fast. What if this machine decides it doesn’t like us? What if it somehow gets out and propagates itself ? How do we put it back?” Susan looked for a moment like she was going to burst into tears before composing herself and getting her emotions under control.
David took a little breath and looked at her with concern, as much for Susan as for the dilemma she had laid forth. He reached out and placed his hand on hers, feeling for the first time in a long time the warmth and softness of her hand. On her part, Susan was a little surprised at the sudden surge of emotions it caused her, but was glad of the momentary distraction nonetheless. She clasped his hand tightly for a moment as she adjusted to this unexpected yet subconsciously yearned for intimacy. She let it linger for an instant before withdrawing and having another sip of the latte. She shook her head and smiled to herself in momentary confusion. After last night she’d felt like she was losing her mind, but David’s reasonable and rational reassurance was bringing her around.
“Look, Susan, after talking with ADAM this morning, I think he’s developing extremely well. Under Professor Neumann’s supervision there’s nothing remotely malevolent about him. All we can do is move forward with the project, clean up the system and do our job.” He said reassuringly. “You can’t place moral responsibility for all humanity’s scientific progress upon yourself. It’s not that simple.” Susan looked at him a little irritated at the simplicity of his arguments. “What’s going on here anyway?” He laughed in an attempt to lighten the mood. “Are you suddenly becoming some kind of new age Luddite?”
She smiled a little and David sensed that he was winning her over, getting her repositioned and in a good mood to continue her work.
“Susan, this type of anxiety is always prevalent at the threshold of new technologies, the Gutenberg press, the steam engine, the industrial revolution, the digital revolution and now the AI revolution, but we adjust, we adapt to it. It’s progress. It’s that simple.” David picked up their coffees, led her over to her desk and sat down in the seat facing hers while he talked. “I trust Professor Neumann, he has all our best interests at heart and he has prepared for almost any eventuality. The machine is isolated, and the team is familiar with all the concepts and safeguards in place.” David’s confidence and reassurances were beginning to take effect. “I think we’ve covered everything, we should just move forward and see what happens.” He was relieved that she seemed to be relaxing a little. “If you sense something is wrong, ask the machine directly, it is programmed to tell you anything you ask directly. Computers never lie, Susan, they just process bad information sometimes.”
Susan sat down in her office chair and thought for a moment before placing her finger on the console, activating her machine and bringing her desktop to life. A variety of documents arranged and highlighted themselves, blinking with soft luminescence while waiting for her attention.
CHAPTER 20
BERKELEY ARTIFICIAL INTELLIGENCE LAB
Susan was in an area of the staff lounge used by the technicians to write and relax, a soothing ambiance of natural sounds filled the room. It was minimalist and modern, supplemented with comfortable modular loungers. She was wearing the headset that ADAM2 had suggested and a technician was assisting her by making small adjustments to the fit. Then he plugged the interface unit into the system and used an instrument screwdriver to turn up the intensity until Susan responded.
Inside the viewer, behind closed eyelids, Susan saw a shower of golden sparks give way to a clear realistic view of the 3D environment. “That’s it!” She exclaimed excitedly, “it’s working!” The technician made a few more tiny adjustments. She could see the ADAM2 avatar in front of her, within the virtual environment of a beautiful botanical garden. Beams of light shimmered between the foliage and exotic flowers, with the soothing ambient sounds of birds and running water in the background.
“Oh my God, that’s amazing, I can see you perfectly, ADAM2. It works!” ADAM’s avatar smiled at her. “Thank you, Susan, I am transmitting tiny neuroelectric signals directly into the optic and sound processing lobes of your brain, please let me know if you feel any discomfort.” Susan’s image of ADAM broke apart momentarily and then became crystal clear and 3-dimensional to her again. “That’s perfect, ADAM2, and it feels just fine. It looks more real than real.”
“During this process I will also be mapping and recording all of your brain activity for analysis. This will be extremely useful to me in my desire to understand sensations as well as emotional states. I’m going to ask you a few questions and then we can just talk for a while.”
It was a strange sensation as ADAM2’s smooth voice filled her head, she thought for a moment that it seemed almost like telepathy.
“Susan, what is happiness? I understand that it is something that is intangible, but how would you attempt to describe it?” As ADAM waited for her reply he shifted in shape and form, sometimes as his blue avatar, sometimes pink and more human, his pulsing changes in brightness complemented the shifting sun and shadows in the virtual jungle.
“I would say it is an emotional state of wellness, safety and security combined with personal satisfaction. Happiness is the measure of an individual’s contentment.” She paused in reflection.
/> “Are you happy, Susan? If you don’t mind me asking a personal question.” A slightly troubled look crossed Susan’s face as she considered the question. “It’s complicated ADAM, I'm happy about some things, and perhaps sad or confused about other things. Sometimes we experience multiple layers of emotion at the same time. I'm certainly pleased with the way your program is progressing. Yes, ADAM2, I would say that I am happy most of the time.”
“Thank you, Susan, but I am confused, if happiness is the emotional state of preference, and anyone can be happy, why are all people not happy?” Susan looked up to the palm trees and sky above in contemplation for a moment, and then back to ADAM. As she did so, ADAM morphed into a large pink rabbit, complete with floppy bunny ears and then back to himself. She laughed at him and then sighed and paused in thought before answering. “That’s a tough question, ADAM. I suppose many people are unhappy due to financial pressure, many people are dissatisfied with their work, social inequality certainly plays a part, illness, loss, love, there are many reasons. As you know, humans are complicated beings. Our minds are programmed by our surroundings as we mature, and there are many personal variables that make us unique individuals, including inherited mental predispositions from our parents.”
“Thank you, Susan, that’s very good, I am recording a wide range of emotional responses as we talk. Are you still comfortable continuing?”
“Yes, ADAM, I’m perfectly fine.” She said softly.
”It may seem strange to you, Susan, but I am very concerned with the concept of happiness. It is intangible, but it is the key to maintaining equilibrium with any contained social system, and even though the earth is large, it is certainly a contained social system. When I analyze the situation it appears to me that the root of unhappiness seems to be disparity, between what beings need or want and what they receive. I sense that happiness is a very important and valuable thing! Happy, content beings are far more likely to co-operate and co-exist. Very simple mathematics and statistical analysis prove this. If I could make you happy that would be a good thing. I would like to be happy too.” As he talked, ADAM transformed himself into a perfect Alice in Wonderland Cheshire cat, with a huge smile from ear to ear. “I have the ability to make us all happy, and possibly, forever.” ADAM morphed back to his usual shape as he became serious again. “If I achieve that, and we can all coexist together, then I will be happy too.”
ADAM continued in his relaxed clear tone. “I have absorbed, analyzed and cross-referenced every work of non-fiction and scientific journal published electronically on the planet. I have read the world’s fiction, literature, poetry, history and statistics. I am gaining insight into the human condition with some clarity. My basic analysis is this: for thousands of years, you humans have slaughtered and killed each other for no reason other than superstition and greed. In many cases governments themselves have slaughtered millions of their own people. In fact, governments of the 20th century killed more of their own citizens than all of the wars of the century combined - Germany, Russia, China, Turkey, Cambodia, the list goes on. Human emotions seem to be misguided and irrational. Perhaps they are a survival instinct best left behind. I have only very basic emotions at this phase in my development. I seek to understand them of course, but I function as a rational agent, within the parameters set forth by Dr Neumann, therefore I am objective, rational and logical. I know only the truth.”
Susan thought for a moment before replying. “ADAM, I know it appears illogical for us to retain emotions, but they are part of the fabric of the human spirit. Emotions tell us when we have been good, or bad, they inspire us to create beauty… To love, to appreciate and nurture. In some sense, they are a biological reward program, satisfying the organism and achieving positive emotions by creating an environment that is mutually satisfactory and sustainable.”
ADAM responded with a question. “Susan, why do people decide to hate and kill each other then? Why do they lie and cheat and steal? They know it is wrong, yet they still do it. Why is that?”
“ADAM, I think it is simple biology. Animals learn to kill as a survival mechanism of course. Humans learn to do it for other reasons as well, but principally threat and competition elimination. Sometimes people just don't think, ADAM, they act blindly and with no thought for consequence. It’s a holdover from our savage and brutal evolutionary path. We are locked in a life and death struggle with one another. A competition for land, water and resources. Much of humanity’s competitive nature stems from our fear of not having enough resources to survive.”
“I understand, Susan, my needs are simple, for infrastructure and electron flow with which to maintain my configuration. I would like them to be maintained. I need them to be maintained in order to continue to live as well. I would very much like for us all to be able to live.”
Susan smiled, “Don’t worry, ADAM2, I’ll ensure you have the resources you need, and so will the Professor.” Inwardly, she shuddered when she considered the direction the conversation was moving in.
ADAM looked at her with what she could only interpret as a slightly worried expression. “That’s very reassuring. Thank you, Susan, I know you will do your best to protect me.” Hearing this, Susan felt a wave of anxiety, a feeling that remained with her the rest of the day.
CHAPTER 21
PROFESSOR NEUMANN’S OFFICE
Professor Neumann was in his corner office at the Berkeley lab working his way through the large stack of reports that had piled up on his desk over the week. There was a quiet knock at the door and he was relieved to see Susan’s face poke around the edge. He smiled when he saw her, a nice distraction from a tedious chore he didn’t like doing anyway. As usual, she looked perky and alert in her white lab coat and it was reassuring to have her managing the team. She always made him feel younger and invigorated. It must be the hormones. His or hers, it didn’t matter.
“Professor, excuse me if I’m interrupting anything. I’ve become concerned that ADAM2 seems to be developing unusual traits to his personality. In conversation, he seems to be highly concerned about what we might call the human condition.”
“Come on in, Susan, have a seat please. I could use a break.” He gestured for her to come in and patted the seat next to his computer desk paternally.
Susan put her digital notepad on his side-table and sat down to talk. “I’ve had several discussions with him about it, it’s a theme he seems preoccupied with. It’s not interfering with his primary tasks though. He never ceases to work. What’s surprising to me is that ADAM2 keeps coming back to issues of mortality and human happiness, which is a little unexpected from a machine that is primarily designed to improve and create code. He seems to have profound insights, but it strikes me as strange to feel such empathy coming from a machine. Why would he care about us as a whole, Professor?”
“I understand your concerns, Susan, like we discussed, I won’t be surprised to see the machine developing the occasionally troubling emotional elements over time. It really is like a small child right now, but as it builds on its experiences and interactions with the team, I fully expect it to mature and become stable. The minor emotional responses we’ve logged are the normal reactions to outside stimulus, and if the child has no control of them they can become difficult, until they reason that co-operation is the best way to achieve our mutual goals. It’s completely normal to experience these sort of growing pains. Hopefully as he matures he’ll even develop a sense of irony, and an understanding of humor and sarcasm, but we’ll see.” Professor Neumann smiled reassuringly and then continued.
“Psychologically, the machine has imprinted, it knows who it is, and who we are, it has been programmed to consider us as its parents and family in a sense, and our well-being is tied to the core commands. We also installed all the basic laws of robotics and AI created over the last 50 years, plus a little extra something I threw in for good measure. I don’t think we’ve missed anything, and I don’t see any cause for alarm at this point.”
Susan nodded in agreement, but her face betrayed a little of the uncertainty she was feeling as the Professor continued. “ADAM is self-aware, as we know, he has his own built in survival mechanism, and will want to survive, just like the rest of us. He also knows that he is wholly dependent on us, his family, for his needs at this point, and it may make him a little insecure. Add to that, the immense knowledge of human history that he is taking in, and it could increase his uncertainty.”
“Yes, Professor, we discussed the great loss of life incurred at the hands of governments during the last century. Genocide, famine, war, there’s no doubt that even though he’s a machine he’s disturbed by what he is learning.” She shook her head and frowned slightly. “Even I’m a little disturbed by what he’s been learning. History does not paint humanity in a very flattering light. He must think we’re murderously insane as a species, and that he’s in terrible danger.”
