It may not feel new. All day every day, we connect with witty apps, we type our information into dialogue programs, and we get information from personal digital assistants. We are comfortable talking at machines and through machines. Now we are asked to join a new kind of conversation, one that promises “empathic” connections.
Machines have none to offer, and yet we persist in the desire for companionship and even communion with the inanimate. Has the simulation of empathy become empathy enough? The simulation of communion, communion enough?
The fourth chair defines a space that Thoreau could not have seen. It is our nick of time.
What do we forget when we talk to machines—and what can we remember?
“A Computer Beautiful Enough That a Soul Would Want to Live in It”
In the early 1980s, I interviewed one of Marvin Minsky’s young students who told me that, as he saw it, his hero, Minsky, one of the founders of artificial intelligence (AI), was “trying to create a computer beautiful enough that a soul would want to live in it.”
That image has stayed with me for more than thirty years.
In the AI world, things have gone from mythic to prosaic. Today, children grow up with robotic pets and digital dolls. They think it natural to chat with their phones. We are at what I have called a “robotic moment,” not because of the merits of the machines we’ve built but because of our eagerness for their company. Even before we make the robots, we remake ourselves as people ready to be their companions.
For a long time, putting hope in robots has expressed an enduring technological optimism, a belief that as things go wrong, science will go right. In a complicated world, what robots promise has always seemed like calling in the cavalry. Robots save lives in war zones; they can function in space and in the sea—indeed, anywhere that humans would be in danger. They perform medical procedures that humans cannot do; they have revolutionized design and manufacturing.
But robots get us to hope for more. Not only for the feats of the cavalry, but for simple salvations. What are the simple salvations? These are the hopes that robots will be our companions. That taking care of us will be their jobs. That we will take comfort in their company and conversation. This is a station on our voyage of forgetting.
What do we forget when we talk to machines? We forget what is special about being human. We forget what it means to have authentic conversation. Machines are programmed to have conversations “as if” they understood what the conversation is about. So when we talk to them, we, too, are reduced and confined to the “as if.”
Simple Salvations
Over the decades, I have heard the hopes for robot companionship grow stronger, even though most people don’t have experience with an embodied robot companion at all but rather with something like Siri, Apple’s digital assistant, where the conversation is most likely to be “locate a restaurant” or “locate a friend.”
But even telling Siri to “locate a friend” moves quickly to the fantasy of finding a friend in Siri. People tell me that they look forward to the time, not too far down the road, when Siri or one of her near cousins will be something like a best friend, but in some ways better: one you can always talk to, one that will never be angry, one you can never disappoint.
And, indeed, Apple’s first television advertising campaign for Siri introduced “her” not as a feature, a convenient way of getting information, but as a companion. It featured a group of movie stars—Zooey Deschanel, Samuel L. Jackson, John Malkovich—who put Siri in the role of confidante. Deschanel, playing the ditzy ingénue, discusses the weather, and how she doesn’t want to wear shoes or clean house on a rainy day. She just wants to dance and have tomato soup. Siri plays the role of the best friend who “gets her.” Jackson has a conversation with Siri that is laced with double meanings about a hot date: A lady friend is coming over and Jackson is cooking gazpacho and risotto. It’s fun to joke with his sidekick Siri about his plans for seduction. Malkovich, sitting in a deep leather chair in a room with heavy wall moldings and drapes—it might be an apartment in Paris or Barcelona—talks seriously with Siri about the meaning of life. He likes it that Siri has a sense of humor.
In all of this, we are being schooled in how to have conversations with a machine that may approximate banter but doesn’t understand our meaning at all; in these conversations, we’re doing all the work but we don’t mind.
I was on a radio show about Siri with a panel of engineers and social scientists. The topic turned to how much people like to talk to Siri, part of the general phenomenon that people feel uninhibited when they talk to a machine. They like the feeling of no judgment. One of the social scientists on the program suggested that soon a souped-up and somewhat smoothed-out Siri could serve as a psychiatrist.
It didn’t seem to bother him that Siri, in the role of psychiatrist, would be counseling people about their lives without having lived one. If Siri could behave like a psychiatrist, he said, it could be a psychiatrist. If no one minded the difference between the as-if and the real thing, let the machine take the place of the person. This is the pragmatism of the robotic moment.
But the suggestions of a robotic friend or therapist—the simple salvations of the robotic moment—are not so simple at all.
Because for all that they are programmed to pretend, machines that talk to us as though they care about us don’t know the arc of a human life. When we speak to them of our human problems of love and loss, or the pleasures of tomato soup and dancing barefoot on a rainy day, they can deliver only performances of empathy and connection.
What an artificial intelligence can know is your schedule, the literal content of your email, your preferences in film, TV, and food. If you wear body-sensing technologies, an AI can know what emotionally activates you because it may infer this from physiological markers. But it won’t understand what any of these things mean to you.
But the meaning of things is just what we want our machines to understand. And we are willing to fuel the fantasy that they do.
Vulnerability Games
We have been playing vulnerability games with artificial intelligence for a very long time, since before programs were anywhere near as sophisticated as they are now. In the 1960s, a computer program called ELIZA, written by MIT’s Joseph Weizenbaum, adopted the “mirroring” style of a Rogerian psychotherapist. So, if you typed, “Why do I hate my mother?” ELIZA might respond, “I hear you saying that you hate your mother.” This program was effective—at least for a short while—in creating the illusion of intelligent listening. And there is this: We want to talk to machines even when we know they do not deserve our confidences. I call this the “ELIZA effect.”
Weizenbaum was shocked that people (for example, his secretary and graduate students) who knew the limits of ELIZA’s ability to know and understand nevertheless wanted to be alone with the program in order to confide in it. ELIZA demonstrated that almost universally, people project human attributes onto programs that present as humanlike, an effect that is magnified when they are with robots called “sociable” machines—machines that do such things as track your motion, make eye contact, and remember your name. Then people feel in the presence of a knowing other that cares about them. A young man, twenty-six, talks with a robot named Kismet that makes eye contact, reads facial expressions, and vocalizes with the cadences of human speech. The man finds Kismet so supportive that he speaks with it about the ups and downs of his day.
Machines with voices have particular power to make us feel understood. Children first learn to know their mothers by recognizing their voices, even while still in the womb. During our evolution, the only speech we heard was the speech of other humans. Now, with the development of sophisticated artificial speech, we are the first humans asked to distinguish human from non-human speech. Neurologically, we are not set up to do this job. Since human beings have for so long—say, 200,000 years—heard only human voices, it takes serio
us mental effort to distinguish human speech from the machine-generated kind. To our brains, speaking is something that people do.
And machines with humanlike faces have particular power as well.
In humans, the shape of a smile or a frown releases chemicals that affect our mental state. Our mirror neurons fire both when we act and when we observe others acting. We feel what we see on the face of another. An expressive robot face can have this impact on us. The philosopher Emmanuel Lévinas writes that the presence of a face initiates the human ethical compact. The face communicates, “Thou shalt not kill me.” We are bound by the face even before we know what stands behind it, even before we might learn it is the face of a machine that cannot be killed. And the robot’s face certainly announces, for Lévinas, “Thou shalt not abandon me”—again, an ethical and emotional compact that captures us but has no meaning when we feel it for a machine.
An expressive machine face—on a robot or on a screen-based computer program—puts us on a landscape where we seek recognition and feel we can get it. We are in fact triggered to seek empathy from an object that has none to give.
I worked at the MIT Artificial Intelligence Laboratory as people met the sociable, emotive robot Kismet for the first time. What Kismet actually said had no meaning, but the sound came out warm or inquiring or concerned.
Sometimes Kismet’s visitors felt the robot had recognized them and had “heard” their story. When things worked perfectly from a technical standpoint, they experienced what felt like an empathic connection. This convincing imitation of understanding is impressive and can be a lot of fun if you think of these encounters as theater. But I saw children look to Kismet for a friend in the real. I saw children hope for the robot’s recognition, and sometimes become bereft when there was nothing nourishing on offer.
Estelle, twelve, comes to Kismet wanting a conversation. She is lonely, her parents are divorced; her time with Kismet makes her feel special. Here is a robot who will listen just to her. On the day of Estelle’s visit, she is engaged by Kismet’s changing facial expressions, but Kismet is not at its vocal best. At the end of a disappointing session, Estelle and the small team of researchers who have been working with her go back to the room where we interview children before and after they meet the robots. Estelle begins to eat the juice, crackers, and cookies we have left out as snacks. And she does not stop, not until we ask her to please leave some food for the other children. Then she stops, but only briefly. She begins to eat again, hurriedly, as we wait for the car service that will take her back to her after-school program.
Estelle tells us why she is upset: Kismet does not like her. The robot began to talk with her and then turned away. We explain that this is not the case. The problem had been technical. Estelle is not convinced. From her point of view, she has failed on her most important day. As Estelle leaves, she takes four boxes of cookies from the supply closet and stuffs them into her backpack. We do not stop her. Exhausted, my team reconvenes at a nearby coffee shop to ask ourselves a hard question: Can a broken robot break a child?
We would not be concerned with the ethics of having a child play with a buggy copy of Microsoft Word or a torn Raggedy Ann doll. A word-processing program is there to do an instrumental thing. If it does worse than usual on a particular day, well, that leads to frustration but no more. But a program that encourages you to connect with it—this is a different matter.
How is a broken Kismet different from a broken doll? A doll encourages children to project their own stories and their own agendas onto a passive object. But children see sociable robots as “alive enough” to have their own agendas. Children attach to them not with the psychology of projection but with the psychology of relational engagement, more in the way they attach to people.
If a little girl is feeling guilty for breaking her mother’s crystal, she may punish a row of Barbie dolls, putting the dolls into detention as a way of working through her own feelings. The dolls are material for what the child needs to accomplish emotionally. That is how the psychology of projection works: It enables the working through of the child’s feelings. But the sociable robot presents itself as having a mind of its own. As the child sees it, if this robot turns away, it wanted to. That’s why children consider winning the heart of a sociable robot to be a personal achievement. You’ve gotten something lovable to love you. Again, children interact with sociable robots not with the psychology of projection but with engagement. They react as though they face another person. There is room for new hurt.
Estelle responded to this emotionally charged situation with depression and a search for comfort food. Other children who faced a disappointing conversation with Kismet responded with aggression. When Kismet began an animated conversation that Edward, six, could not understand, he shoved objects into Kismet’s mouth—a metal pin, a pencil, a toy caterpillar—things Edward found in the robotics laboratory. But at no point did Edward disengage from Kismet. He would not give up his chance for Kismet’s recognition.
The important question here is not about the risks of broken robots. Rather, we should ask, “Emotionally, what positive thing would we have given to these children if the robots had been in top form?” Why do we propose machine companionship to children in the first place? For a lonely child, a conversational robot is a guarantee against rejection, a place to entrust confidences. But what children really need is not the guarantee that an inanimate object will simulate acceptance. They need relationships that will teach them real mutuality, caring, and empathy.
So, the problem doesn’t start when the machine breaks down. Children are not well served even when the robots are working perfectly. In the case of a robot babysitter, you already have a problem when you have to explain to a child why there isn’t a person available for the job.
Treating Machines as People; Treating People as Machines
In all of this, an irony emerges: Even as we treat machines as if they were almost human, we develop habits that have us treating human beings as almost-machines. To take a simple example, we regularly put people “on pause” in the middle of a conversation in order to check our phones. And when we talk to people who are not paying attention to us, it is a kind of preparation for talking to uncomprehending machines. When people give us less, talking to machines doesn’t seem as much of a downgrade.
At a panel on “cyberetiquette,” I was onstage with a technology reporter and two “advice and manners” columnists. There was general agreement among the panelists on most matters: No texting at family dinners. No texting at restaurants. Don’t bring your laptop to your children’s sporting events, no matter how tempting.
And then came this question from the audience: A woman said that as a working mother she had very little time to talk to her friends, to email, to text, to keep up. “Actually,” she confessed, “the only time I have is at night, after I’m off work and before I go home, when I go family shopping at Trader Joe’s. But the cashier, the guy at the checkout counter, he wants to talk. I just want to be on my phone, into my texts and Facebook. Do I have the right to just ignore him?” The two manners experts went first. Each said a version of the same thing: The man who does the checkout has a job to do. The woman who asked the question has a right to privacy and to her texting as he provides his service.
I listened uncomfortably. I thought of all the years I went shopping with my grandmother as I grew up and all the relationships she had with tradespeople at every store: the baker, the fishmonger, the fruit man, the grocery man (for this is what we called them). These days, we all know that the job the man at the checkout counter does could be done by a machine. In fact, down the street at another supermarket, it is done by a machine that automatically scans your groceries. And so I shared this thought: Until a machine replaces the man, surely he summons in us the recognition and respect you show a person. Sharing a few words at the checkout may make this man feel that in his job, this job that could be done by a
machine, he is still seen as a human being.
This was not what the audience and my fellow panelists wanted to hear. As I took stock of their cool reaction to what I said, I saw a new symmetry: We want more from technology and less from each other. What once would have seemed like “friendly service” at a community market had become an inconvenience that keeps us from our phones.
It used to be that we imagined our mobile phones were there so that we could talk to each other. Now we want our mobile phones to talk to us. That’s what the new commercials for Siri are really about: fantasies of these new conversations and a kind of tutelage in what they might sound like. We are at a moment of temptation, ready to turn to machines for companionship even as we seem pained or inconvenienced to engage with each other in settings as simple as a grocery store. We want technology to step up as we ask people to step back.
People are lonely and fear intimacy, and robots seem ready to hand. And we are ready for their company if we forget what intimacy is. And having nothing to forget, our children learn new rules for when it is appropriate to talk to a machine.
Stephanie is forty, a real estate agent in Rhode Island. Her ten-year-old daughter, Tara, is a perfectionist, always the “good girl,” sensitive to any suggestion of criticism. Recently, she has begun to talk to Siri. It is not surprising that children like to talk to Siri. There is just enough inventiveness in Siri’s responses to make children feel that someone might be listening. And if children are afraid of judgment, Siri is safe. So Tara expresses anger to Siri that she doesn’t show to her parents or friends—with them she plays the part of a “perfect child.” Stephanie overhears her daughter yelling at Siri and says, “She vents to Siri. She starts to talk but then becomes enraged.”
Stephanie wonders if this is “perhaps a good thing, certainly a more honest conversation” than Tara is having with others in her life. It’s a thought worth looking at more closely. It is surely positive for Tara to discover feelings that she censors for other audiences. But talking to Siri leaves Tara vulnerable. She may get the idea that her feelings are something that people cannot handle. She may persist in her current idea that pretend perfection is all other people want from her or can accept from her. Instead of learning that people can value how she really feels, Tara is learning that it is easier not to deal with people at all.
Reclaiming Conversation Page 36
