by Scott Sibary
Conception
Book One of Human Dilemma
by Scott Sibary
Copyright 2019 Scott Sibary
Smashwords edition
Cover design by Christopher Moisan
Thank you for downloading this ebook. This book remains the copyrighted property of the author, and may not be redistributed to others for commercial or non-commercial purposes. If you enjoyed this book, please encourage your friends to download their own copy from their favorite authorized retailer. Thank you for your support.
A minimum of 30% of the cost of this book (about 50% of the author royalties, varying with different retailers) will be donated to non-profit organizations that work to promote AI alignment with human values (such as those listed in back of this book. The first donee organization will be The Center for Human-Compatible Artificial Intelligence at UC Berkeley).
Disclaimer: Many of the names (forenames or surnames, never both) of the active characters in this book are coincidentally names of people known personally by the author. The use of those names in this book is solely for literary purposes. The active characters in this novel are fictitious, and any resemblance to real persons is unintentional and coincidental (unntatt, kan henda, Rolv). References to real or historical persons are intended to be accurate, and any inaccuracies are unintentional.
Table of Contents
Prologue
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
Chapter 13
Chapter 14
Chapter 15
Chapter 16
Chapter 17
Chapter 18
Chapter 19
Chapter 20
Acknowledgements
About the author
Links to relevant organizations
Prologue
Decisions, and decisions.
The day began with the clarity of a sky blown clear by north winds, autumn sunlight illuminating the west wall of her office, tea and coffee mugs steaming on the desk, and two minds fresh, buzzing and intent. Then, as the sunlight rose higher, the spotlight disappeared, the mugs stood empty, and the pace quieted. Yet the two persisted, driven by the career-changing proposal from the day before.
A computer screen on the desk displayed part of the code for a new set of algorithms she had dubbed “Protection Lock.” Its purpose was to guarantee the inalterability of the vital codes—codes aiming to ensure that an artificial general intelligence would remain altruistic, or at least benign. With completion of what the Chinese were calling the world’s first AGI scheduled for as soon as January 2039, Solveig had only a few months to prepare the Protection Lock for trial integration into that system, and only one month before leaving for China.
If she said yes.
Solveig Kleiveland leaned back, her reddish-blonde hair falling free behind her shoulders. Her well-worn cardigan with pewter buttons hung open. She’d been able to make just enough room for her slim figure to fit beside Lars at her desk without bumping their knees. Her younger colleague, in jeans and tight-fitting knit T-shirt, was similarly lean and as fit as a practitioner of modern dance.
“It has to be kept entirely confidential,” she said.
“None of it open code?”
“The Chinese are keeping the code for their immune system confidential. You, me, and the other team members from Norway will be the only ones with any access to the Protection Lock.”
“And you, Solveig, will be the only one with access to the part of the code that updates the Lock, assuming you’ve effectively secured the Lock itself?”
“Right. It’s enough that the vital codes will be open source, as they’ll have to be if the new World Council is to trust the AI. And that’s assuming enough nations continue to back the Indian Proposal, and the Council becomes a reality.”
“That’s the impression I got from the interview. Only, you haven’t yet agreed to go on the mission, right?”
“Right.”
They sat in silence. Solveig took long, deep breaths, eyes focused on infinity, like a distance runner who has completed one training lap and pauses to consider a second go.
Two soft raps sounded on the door.
“Come in!” she said.
Rune Andersen, the Director of the Institute for Humanistic Technology at the University of Bergen where Solveig and Lars worked, took a step inside the office. “Good morning!”
“Good morning,” both replied.
He closed the door behind him. His necktie and open sports jacket swayed left and right as he swaggered around to the opposite side of the desk. He raised his palms as if to beckon, mouth hinting at a smile.
“You’ve come for my answer?” Solveig asked.
“I need to ask you an additional question.”
“Wait, Rune. Don’t tell me . . .”
He raised his eyebrows.
She shook her head. “Don’t tell me you want me to be leader of the mission?”
He nodded.
“Why didn’t you mention it yesterday? Did someone else turn you down? Like Rolv Drammen?”
“No. The committee didn’t discuss the question until we were done with yesterday’s interviews.”
“Why not Reidar Hagerup? He and Rolv are older, and have loads of leadership experience.”
“Thirty-four isn't old enough? You’ve led research teams and taught classes, and it’s only a team of six.”
“Huh!” She coughed. “I’ve led one graduate seminar, and taught an introductory course one semester—not very successfully, in my opinion. And it was you who selected me to lead your small research teams. That’s hardly preparation for being the point person for a semi-secret international mission with potentially gargantuan impacts!”
“I’ll grant you that,” Andersen said. “Yet everyone on the committee, including the representative from the Foreign Ministry, thought you were the best choice. The design aspects of this project are more important than the relationships with the other scientists.”
“Or even with their leader?”
“Dr. Deng AnDe? Don’t worry about him. He has an excellent reputation for getting along with people, and he’s assembled a whole array of brilliant minds to work on their project. But he himself doesn’t have your technical genius.” He raised his hands questioningly. “Yesterday, Solveig, you seemed enthusiastic about the mission, open to anything. What’s your hesitation?”
Her focus moved from Andersen to the three photos on the wall. The first portrayed six middle-aged women wearing the traditional ceremonial dress of her home island of Osterøy. The feet of two of the women seemed to rest on a thick black line drawn on the photo with a marking pen. The second was a poster-sized aerial of Osterøy with the name of the hamlet of Hauge bright from a yellow highlighter, and farthest from the door hung a framed portrait of the Crown Princess of Norway.
“Things happen at night,” she said. “One gets shaken up.”
“Oh yeah.” Andersen shook his head and took a deep breath. “Don’t we all know.” He followed Solveig’s gaze to the first photo on the wall, rubbed his chin and then moved to look out the nearby window with a view over the Bergen harbor.
“Actually,” she said, and he swung back around.
Solveig glanced back and forth between the two men. “I don’t think I’d want to go if I weren’t the leader. The Protection Lock has to be handled correctly. And there are so many contingencies! What if the systems
won’t merge?”
“That’s their problem. We just do our best.”
“What if they merge,” Solveig said, “but don’t honor the vital codes? Or their system penetrates the Protection Lock? Worse, what if we hand over our technology, but the World Council is never convened? We might be leaving the Chinese with a dangerous tool, maybe a weapon.”
Lars leaned back, one leg crossed over the other and angled towards her. He looked at her with a twisted smile. “Ah, so that’s it.”
“I have to make my own assessment of the risks,” she said. “I can’t make a knee-jerk reaction to go ahead just because the government deems it best. I bear responsibility, and so do you.”
He kept smiling as his hanging foot swung forward to tap her in the back of her calf.
“Lars!” She gave a tired laugh and shook her head. “I’m serious.”
“Of course. But your Protection Lock is useless by itself. What purpose can it have other than to play a role in a larger system?”
“Which system? Whose system? That’s my worry.”
Lars threw up his hands. “Either the people on our troubled, shrinking planet cooperate more closely, or else . . .” He brought his fingertips together. “Tell me, can you prosper in a personal relationship if you don’t cooperate? The closer the relationship, the closer the cooperation. It has to be, right?”
“That’s different. Individuals always have choices—so many alternatives. You should find a good match, eventually.”
“Easy for you to say, you who ends things when you find something’s lacking. My situation is more like the international scene, with its limited number of nations. I don’t have many men to choose from, so I’ve embraced the spirit of compromise." He slapped the desktop with his hand. "And that’s why, as your closest colleague, I’ve told Dr. Andersen I’m willing to join the mission. And you?”
She leaned forward to rest her chin in her palm. In the quiet of the room, the buttons on her sweater swung against the edge of the desk with the resonance of a ticking clock, winding down.
“I share your concerns, Solveig,” the older academic said. “I think we all do.” He put both hands on the far edge of the desk, his shoulders jutting forward and his eyes beseeching. “Yet we need to know your answer. Will you be involved? Will you lead? It’s time to decide.”
The mind-deadening grey paint on the smooth concrete failed to hide the past. Those working in the old building of the Beijing health department knew what the main room of the basement had been used for. Empty since then, it would soon be given life. Thin, hollow walls had newly divided this subterranean world into several smaller rooms: conference rooms, a computer room, a guard room, a toilet room, and one larger, leftover area as a general meeting room that offered the only door in and out.
Dr. Deng AnDe stood alone, a tall but otherwise unremarkable statue of an everyday, forty-year-old man, and waited. The café where he’d just eaten a quick lunch had been playing the kind of dance music he liked: a fusion of swing and recent Chinese pop that had added to his energy and kept him moving fast, until he reached the basement. He glanced at his phone. He’d arrived twelve minutes early. He figured his boss, director of the Division for Artificial Intelligence within the Ministry of Technology, would probably arrive about one minute early.
The arrival of his boss should feel like a formal endowment of authority; AnDe could then feel in charge of this new job site. The transfer of jurisdiction over the basement facilities to the Ministry of Technology had been a mere technicality, but it symbolized a proactive step forward. At a time when many foreign governments remained as unstable as slowly spinning tops, his country would accept the leading role. It would, AnDe hoped, guide the world away from Armageddon and devastation, and safely into the future.
He could feel the weight on his shoulders.
He straightened his tie, carefully brushed down the front of his suit jacket with his palms, and took slow and deep breaths to let the tension flow out of his body. His feet began to wiggle in his shoes, and he felt his throat wanting to hum a tune. The quarter turn he took to look down the inner hallway resembled a dance step.
Facing the computer room where technicians were working, he again became a statue. In that room was assembled the hardware for the operating system of a new generation of artificial intelligence: an intelligence his teams would be creating with the participation of a small group from Norway.
He shook his head. How had that country managed to persuade China to cooperate on the Indian Proposal? India and the US had not. Norway’s influence seemed out of balance. Was it because the United Nations and the World Trade Organization had proven ineffective at facilitating significant international cooperation, beyond the treaty organizations they ran?
Perhaps his boss could throw more light on the question. It seemed to AnDe that the Chinese government had come to see international cooperation as necessary to prevent engineered disasters, like the influenza virus in 2029 that had killed nearly a billion people.
AnDe completed a scan of the site that would absorb the main focus of his energy for the next several months and was returning to the large meeting room when a familiar figure entered from the main building hallway. Director Liu, a man in his sixties, wore a casual business suit but walked as though he were in charge.
“Good morning, Division Director Liu,” AnDe said as his boss approached.
“Senior Project Manager Deng AnDe, thank you for offering me a tour of your new facilities.”
“With pleasure, sir.”
AnDe led him around the large room, and then paused to glance into the guard room. There, a uniformed watchman sat at a table below a bank of screens. The two continued past the three empty conference rooms and to the door of the computer room. That room bustled with activity. Several white-coated technicians were stringing wires, moving equipment, and testing circuits and routers. One person ran a vacuum cleaner around the corners of the room. AnDe and Liu looked through the door only long enough to receive a few over-the-shoulder glances, then turned away.
“You mentioned there was something you wanted my insight on,” the elder said.
“Yes, sir. There’s something I am curious—or you might say, confused—about. Our government had rejected the Indian Proposal and offered some criticism of the proposed structure of the World Council. But I don’t think that was the main concern. I put more weight on the distrust our government expressed about using an AGI as an official advisor. It seems to me that artificial intelligence has acquired a bad image: people in our country blame the pandemic on the AI that engineered the virus.”
“Yes, rather than blaming the dictator who thought it would help squash the drought-triggered rebellions in his country.”
“Or the arms producer who sold him the drones to deliver that plague, even if it was not supposed to be so deadly. Despite all that, our government now seems to be endorsing the use of AI.”
“About that I have only theories. When a nation is clearly the most powerful, it tends to resist any form of internationalism that it doesn’t control. We saw that with the US government on the subjects of arms control, climate change, and human rights. The plague, as some call it . . . it hit our country hard, but what were bumps and stumbles for us became a free fall for them.”
“And so their government fell to a coup.”
“Sort of. I suspect power was already being lost to private military contractors employed to quell the civil unrest. The coups by the traditional militaries in the US and UK restored a kind of government order, and they later handed power back to an elected government. But by then their economies had crumbled and China was the new economic leader.”
“That’s what I’m getting at. Once China became the most powerful, why did our government change its position and endorse a new international organization?”
“I wouldn’t call it a change. After we saw how much harm AI could cause—or I should say, could be involved in—it made sense not to pursu
e an internationalism that relied on AI. For the time being, that is. But much progress has been made on the design of vital codes. There are now several versions that might resolve the conundrum.”
The conundrum, AnDe reflected, was the great puzzle that if left unanswered too long could lead to the end of humanity. The more free-thinking the AGI, the more difficult it would be to direct. If too controlled, it would be too slow and limited to achieve significant new results; but if too dutifully focused on a particular result while remaining free in its choices, it would likely cause collateral harm. Yet regardless of the conundrum, the drive to create powerful AGI would not be stopped.
“Why endorse this version?”
“Ah, I’ve known you too long, AnDe, not to see there is something more you’re driving at. The Protection Lock?”
AnDe sighed. “Yes. We’re told it keeps the vital codes sacrosanct. We’re told it’s very strong, and in the future it could be readily transitioned for use in a quantum computer. But is there something else that makes this Protection Lock so attractive?”
“I don’t know, unless it’s simply that it works, in which case”—Liu held up his index finger—“it will protect the vital codes and guarantee that the conundrum remains answered.” He dropped his hand. “Or it may be just that those at the top have decided the proposal is a venture to investigate, to see what will work. They might like that the Norwegian modification to the proposal has the World Council sit in Beijing rather than Delhi.” He offered AnDe the smile of an insider.
A few paces from them stood a woman in custodian’s overalls, her back and one ear towards them, polishing two large, brass letters on a door: WC.
Liu looked from the door to AnDe. “Only one?”
AnDe sighed again. Might as well drop the subject and accept the answers one gets. “It’s unisex. Outside, by the elevators in the main hallway, are two more: a men’s and a women’s.”
