by Scott Sibary
“You think it was an ambush?”
“Oh, Solveig, you know he’s been trying to chat you up, get your confidence, get you to make concessions, or at least slip up. You’ve shown you’re more than his match. Let’s see how he continues his ploys.”
“He’s supposed to take me on a tour of the Great Wall one of these Saturdays. I haven’t been yet.”
“Then it will be a good occasion to turn him down. A nice response to the way he behaved today.” He met her questioning look. “Until one shows willingness to walk away from a deal, one will always pay too much. And you can see the Wall anytime, right?”
“Or be up against it. It might send him the wrong message, that we’re not willing to work with them on anything but terms we unilaterally dictate. Although this Sunday, Miriam and I are planning to have an excursion to a spa”—she put a hand to cover her cheek—“to relax, and maybe have a meal.” She lowered her hand.
“Now, Miriam is sweet and innocent enough, I think. You’ll like her husband if you meet him. He is an environmental engineer, like my mother. He has a lively sense of humor.”
“I thought her husband was a real estate manager.”
“That’s Victor, the husband of Elfin, in my subgroup. He’s nice too, but much more quiet.”
“You get around!”
“I like to. It’s wise to learn about those you’re dealing with.”
Dealing with, thought Solveig. Working with or working against? And I haven’t discovered where to draw the lines on when I do one or the other. As she looked into the wine glass moving in her hand, the swirl of self-harassing thoughts trapped her in an isolating eddy. Be cautious in life, don’t take unnecessary chances, don’t fail, . . . and don’t succeed? Her thoughts swirled on.
Per spoke with caution in his voice.
“Where do you get this belief that people or countries always need to cooperate?” He was looking down, moving the almonds on his small plate into one arrangement and then another. “Why shouldn’t they simply balance their positions while pursuing their own interests and trying to stay away from open conflict?” He looked up at her as if to see whether she wanted to answer or simply listen.
She didn’t answer, and he continued.
“This idea of yours, that we must choose between cooperating or not, may be misconstruing our role. I think we need to guarantee that the vital codes protect the values of our culture. We need to live our way. Of course we must pursue our values; that’s a given.” He nodded to his own words, and Solveig rocked her head in general agreement.
“We must keep up our alliances and maintain a balance of power,” he continued, “or our tiny nation and culture will be overrun. Maybe this temporary, limited alliance with the Chinese is the right one at this moment, but we must keep in mind that our values are different. China was the last major nation to ban technology that directly controls the brain, with exceptions for medical use. Why was that? And how do they construe medical and psychiatric? Makes you realize that you need to stay on your toes.”
She’d watched him form different symbols with his almonds: a Zen ensō, a Taoist yin-yang, a peace symbol, a cross. She thought of picking apart his reasoning. She wanted to mention that a standard strategy of powerful nations was to disable possible competitors, as with colonial powers when drawing national boundaries in Africa. Or she could point out that one can always find differences, even within what appears to be a homogenous group. But that would invite an unhelpful discussion of the Chinese and whether they were “essentially” or “critically” different.
Instead, she returned to his conclusion. “Balance of power can mean a buildup of weaponry, eventually leading to war. Aren’t we here to prevent that, to make all buildup seem unnecessary, even wasteful?”
“Yes, when the policy has been mishandled, it has failed, just like any other policy. And yes, we’re here to avoid conflict. But that need not be the same as cooperating to the point of relenting. As you work on the definitions and parameters of altruism for your first imperative, there’s no need to compromise our values.”
“But, Per, an unbridled pursuit of one’s own interests might mean harming someone else’s welfare. For example, don’t we cooperate in society to make rules and laws? It’s how we invent property rights and contract rights. Don’t we need to cooperate to protect a competitive market by making rules against monopolies—even the ones that result from simply being the most efficient and out-producing one’s competition? You see what I mean? Otherwise, how can we make sure that when people pursue their own interests, they don’t interfere with everyone else’s?”
“Yeah, yeah, yeah. Of course. And we reach agreement in our society while guided by our cultural values. In practice, the specifics will vary from country to country. So figuring out how the vital codes can do this on a worldwide scale is the most meaningful thing we can accomplish.”
“Assuming the values are consistent, that they’re part of a coherent system for living, right?”
Per’s eyes widened as he leaned away. “No. Our values are a given.”
“Not questioned? What if they lead to conflict with other values? Or other people?”
“We stand by what we believe.”
“Listen.” She raised her voice. “Where do you get those values? Just by feeling them?”
Per waved his hand once in front of his twisted face and shook his head, as if fending off a persistent mosquito.
“What I’m saying,” she continued, “is that we can’t give in to the appeal of impulse, of letting only our feelings dictate what we do and what we believe. It ends up as just a collage of animal reactions. No coherency to it, no meaning. What we’re doing—these vital codes—can’t be just lists of things we believe on impulse. Competing impulses, at that. No, the system has to have coherency, and the parts must work together.”
“Within the software, of course.”
“And in the results. And it must start with the human values we translate and input to give us those results. That’s why there’ve been interdisciplinary teams developing the philosophy and the descriptive content of the code to make AI’s behavior consistent.”
Per let out a puff through vibrating lips. “That’s a lot to figure out. I suppose it’s another reason we have this painful trial-and-error process.” He chuckled as if to himself, then raised a smile on one side of his face. She’d never debated with Per before now, and he seemed energized by her arguments.
“I hope you don’t approach your personal decisions that way,” he said.
“Machines are easier.”
“OK,” he said, “what about our human issue: relations between different societies? Must we compromise to create coherency? Well, we can look at our own country and ask ourselves whether our values work within our country. I would insist they do. And I have no reluctance in condemning what we see in some other countries, where women are forced into subservient positions or are expected to devote their lives to raising babies and populating their country beyond its resources. I have no guilt if our value of equality and freedom for women infiltrates those cultures.”
“I agree with your example; you chose it well. But what about economic exploitation? How do we share finite resources?”
“We work together, as we’re doing here. But not to the point that we’re assimilated and overrun. It could happen too easily. It’s easy for you to say, ‘Let’s cooperate,’ because as a tiny nation we cannot stand alone. When we cooperate, we have to choose the right partners.”
While her mind spun in contemplation of his word partners, Per explained further. “Do you think it’s just coincidence that Asilomar is in California? That that’s where the people who know and care the most about this issue chose to meet?”
“Then choosing partners depends on what you’re interested in?”
“I’m not sure you understand me.” Per looked straight at her, appealing to her as though she were not already trying to comprehend. “How did you first
come to believe you should cooperate with people you don’t really know?”
Solveig sat up straight and stared into space and back into time. “From competitive sports. If you don’t cooperate on following the rules of the game, the game isn’t possible.” Her voice became assertive. “It’s not a matter of cultural values. Once you’ve agreed you want to play the game, it’s simple logic. I realized when I learned tennis that you must cooperate in honestly calling the shots in or out. If you cheat and get away with it, then you do not in fact win the game. You might win a different game—one where you get to cheat on your opponent—and maybe also a private psychological game you’re playing; but, logically speaking, you don’t win the game you’re supposedly playing. To do that, you must cooperate and play by the rules that define that game.”
“Or you may lose someone to play with in the future,” Per added.
“Also true.”
“And you say this is just a matter of logic, yet it seems in some cultures, cheating in games is understood as part of the competition. The game is about something more than what it pretends to be. Meanwhile, you learned our value of fair play in our culture.”
“Actually, I was taught tennis by my first boyfriend, if I could call him that, and he was a foreign exchange student at my high school.”
“Ah . . .” Per nodded slowly. “From what country?”
“New Zealand.”
“Exactly! A similar culture, a similar setting, etcetera; and, therefore, a natural ally because the values are similar.” He picked up a few almonds at once, popped them into his mouth, and crunched.
Seeing the self-satisfied expression on his face, she suspected he couldn’t help feeling special and more valuable than those who did not come from an exceptional land. She wondered what kind of world that would lead to. Unending competition for status: competition until only one group, or even one person within it, becomes the most powerful of all special people and is able to take control, ignore all the rules of fair play, and manipulate the game in the future. Hadn’t he learned that from history? Or was his psyche so trapped in needless insecurity that he was compelled to feel special?
On their recent Sunday together in a park outside the city, he’d surprised her with his preference for sticking to trails that were paved. Glancing around his living room, she noticed how tidy the papers and devices were at his work station. Maybe it had been cleaned up just for her, but she would not have bothered.
Nearby was a bookcase containing neatly arranged rows of books and expensive-looking art glass. On top were photographs, probably from home, and a bronze sculpture of a medieval warrior holding an exposed longsword in front of himself. On the wall above was a painting that looked close to an exact copy of View from Stalheim by Johan Christian Dahl. A powerfully romantic view of nature, it depicted dark, rounded granite mountains suggestive of enormous living beings, or the living Earth itself. On the shoulders of these, other life—scattered forest, animals in meadows, and a few people—could co-exist. The ominous majesty of the mountains was balanced by a rainbow. The frame of the painting looked perfectly plumb with the wall.
So much to attract her and so much to be leery of in Per. As she picked up another neatly assembled cheese smørbrød, she wondered whether he could step outside his context and see himself as just another actor on life’s greater stage. Or he might be like the thespian who does a better job at his role by subsuming his full potential into it, suppressing other ways of being. There’s the dilemma, she told herself. What if the established patterns of behavior are failing? He must see that happening. He must be able to.
She blinked, then bit into the cheese.
“Delicious,” she said with a nod of approval. “Thank you for preparing these reminiscences of home.” Why argue with someone who knows what he loves?
“I’m glad. That’s just what I was hoping to hear you say.”
She saw sentimentality on his face for the first time.
“My heart is with the people I love,” he said, “where I’m most relevant. Besides,”—he looked directly into her eyes—“if we don’t support our own local communities, we don’t form the building blocks for creating a stable world.”
“Building blocks, yes,” she said, and clicked her tongue to the rhythm of her thoughts. A vision of a moving wall, creeping forward, flashed in her mind. It was an ominously large wall—almost a mountain—able to incorporate or devour anything in its path. “And here we are building something with world application, something that might eventually tie many operating systems together, the way the internet began to with information.”
“That’s my worry. If I can get any message across to you, it’s this: Homogenization is the last thing we need. We must design this system to protect cultures, which often means language groups and the nation-states they form. And in multi-cultural countries, we see mixed success. I’ll be blunt; the US has not managed this issue perfectly, but far better than China. Yet here we are in Beijing. So it must be integral to our concept of harm that cultural diversity is not oppressed. Can you feel what I’m saying?”
“I have felt it. It’s something I’m trying to get AnDe to understand: that the system we design must protect individuals—which I think would also help to protect cultures.”
“Now please, Solveig, consider this. I deal with it every day. I’m fending off attempts to get information, stuff they know they aren’t supposed to get. You know they have this Great Wall acting as a complex immune system? You’re familiar with its general structure, right?”
She nodded.
“It has powerful analytical tools to defend the OS as an immune system should. After many frustrating hours of working with these Chinese, I think I’ve figured out that it’s more than just a firewall and virus filter.” He slowed, raised and lowered his carrot stick as though he was instructing a class. “It’s a sophisticated learning system that probes, tests, and analyzes thoroughly anything it doesn’t recognize.”
“Anything that’s downloaded to it?”
“Yes. That means it’s constantly trying to crack the Protection Lock. Since it’s not able to, the imperatives from the vital codes never get fundamentally incorporated into the AI. After all I’ve learned, I must admit, it’s masterful.” He exhaled a gusty sigh. “You know that panel with the small, removable drives? The people on my team say that it forms the Gateway of the Great Wall, something like the way immigrants might be examined on an island before being allowed in.”
She nodded again.
“Well, just as the real Great Wall is made up of watch towers, those drives act as an analogous part of the OS’s Great Wall, in both the hardware and software. You see, it works on contingencies. If you download an application that asks to do certain things, it creates an artificial, temporary OS to accept and work with the new application or new input on a separate drive and without any resistance—because it’s all hypothetical to the real OS. Then the application is used in a variety of ways, and the proxy OS is examined for undesirable changes—infections, bots, etcetera—and if anything questionable is found, the material is isolated or dumped with only a brief descriptive memory retained. The mainframe OS operates at a faster speed than the separate drives. It’s even able to send a surge and destroy the drive.”
“That’s too funny!” Solveig laughed. “So that’s why one of those little guys started smoking last week?”
“You got it,” said Per, grinning broadly. “You might have come very close to breaching the wall.”
“Then I gather their drive-fry works in some situations. How does their mock laboratory work?”
“Unknown or unreliable inputs are assigned to a replicated group of applications transferred from the OS to the isolated drive. Those applications become the proxy OS that deals with the new inputs within the confines of that drive. It copies or summarizes the input—to create a mirror or abstract representation of it, so to speak. Then the unit generates a commentary about the new inputs, and atta
ches the commentary to the new representation. The commentary may be simply a label or identifier, or it may include a warning or even an analysis. What’s more, the analysis can be an elaboration, or a building upon of the idea contained or suggested by the new inputs. I suspect there’s even more it can do, building upon those elaborations. Who knows?”
More? thought Solveig. There is theoretically no end to what such a system could do, limited only by materials. And how far have they taken it? She nodded for Per to continue.
“The most relevant part is this. The Great Wall sets up laboratory-like situations for incoming software to test it for compatibility with the OS. Well, that’s exactly parallel to what we want: something that tests actions, decisions, recommendations, etcetera, for compatibility with the vital codes. It even determines probabilities of various outcomes, just as we’d want. But then it rejects our system, because we would modify existing priorities—as we’re supposed to when we introduce the vital codes containing the imperatives—and in a manner that remains secretive to it because of the Protection Lock. It tries to learn the contents of the Lock, and it’s made some rather interesting attempts.” He drew out the last three words. “I don’t know if there’s a way around it other than to rebuild their entire OS from the start.” He waited.
Solveig squinted at the wall across the room, as though the intensity of her focus would penetrate the paneling and reveal an incalculable treasure. A key piece remained missing from the project she’d begun in India. An idea began to take shape, in the manner of a mirage over a sweltering landscape: an idea that might help her towards her ultimate goal. Yet until she discovered a solution to the problem Per was describing, the Great Wall would foil her. And as this idea formed, she felt the pull of his eloquent speech, an intellectual magnetism, drawing her towards him.
