by Don Norman
Until recently, technology has been pretty much under control. Even as technology gained more intelligence, it was still an intelligence that could be understood. After all, people devised it, and people exerted control: starting, stopping, aiming, and directing.
No longer. Automation has taken over many tasks, some thankless—consider the automated equipment that keeps our sewers running properly—and some not so thankless—think of the automated teller machines that put many bank clerks out of work. These automated activities raise major issues for society. Important though this might be, however, my focus here is on those situations where the automation doesn’t quite take over, where people are left to pick up the pieces when the automation fails. This is where major stresses occur and where the major dangers, accidents, and deaths result.
Consider the automobile, which, as the New York Times notes, “has become a computer on wheels.” What is all that computer power being used for? Everything. Controlling the heating and air conditioning with separate controls for the driver and each passenger. Controlling the entertainment system, with a separate audio and video channel for each passenger, including high-definition display screens and surround sound. Communication systems for telephone, text messaging, e-mail. Navigation systems that tell you where you are, where you are going, what the traffic conditions are, where the closest restaurants, gas stations, hotels, and entertainment spots are located, and paying for road tolls, drive-through restaurants, and downloaded movies and music.
Much of the automation, of course, is used to control the car. Some things are completely automated, so the driver and passengers are completely unaware of them: the timing of critical events such as spark, valve opening and closing, fuel injection, engine cooling, power-assisted brakes and steering. Some of the automation, including braking and stability systems, is partially controllable and noticeable. Some of the technology interacts with the driver: navigational systems, cruise control, lane-keeping systems, even automatic parking. And this barely scratches the surface of what exists today, and what is planned for the future.
Precrash warning systems now use their forward-looking radar to predict when the automobile is apt to get into a crash, preparing themselves for the eventuality. Seats straighten up, seat belts tighten, and the brakes get ready. Some cars have television cameras that monitor the driver, and if the driver does not appear to be looking straight ahead, they warn the driver with lights and buzzers. If the driver still fails to respond, they apply the brakes automatically. Someday, we might imagine the following interchange at a court trial:
Prosecutor: “I now call the next witness. Mr. Automobile, is it your sworn testimony that just before the crash the defendant was not watching the road?”
Automobile: “Correct. He was looking to his right the whole time, even after I signaled and warned him of the danger.”
Prosecutor: “And what did the defendant try to do to you?”
Automobile: “He tried to erase my memory, but I have an encrypted, tamper-proof storage system.”
Your car will soon chat with neighboring cars, exchanging all sorts of interesting information. Cars will communicate with one another through wireless networks, technically called “ad hoc” networks because they form as needed, letting them warn one another about what’s down the road. Just as automobiles and trucks from the oncoming lane sometimes warn you of police vehicles by flashing their lights (or sending messages over their two-way radios and cell phones), future automobiles will tell oncoming autos about traffic and highway conditions, obstacles, collisions, bad weather, and all sorts of other things, some useful, some not, while simultaneously learning about what they can expect to encounter. Cars may even exchange more than that, including information the inhabitants might consider personal and private.
Gossiping cars. When two cars talk to one another, what do they talk about? Why, the weather, or traffic patterns, or how to negotiate the intersection they are both approaching at high speed. At least, that’s what the researchers are working on. But you can also bet that clever advertisers are thinking about the potential. Each billboard can have its own wireless network, broadcasting its wares to the car. Imagine a billboard or store checking with the navigation system to see just what the car’s destination is, perhaps to suggest a restaurant, hotel, or shopping center. What if it could take control of the navigation system, reprogramming it to instruct the driver to turn into the advertiser’s establishment? When the day comes that the steering is under the car’s control, the car might very well decide to take you to the restaurant of its choice, possibly even preordering your favorite food for you. “What,” the car might say to you, “you mean you don’t want your favorite food every day, every meal? Strange—why is it your favorite, then?”
What about an overload of advertisements or viruses inserted into the telephones, computers, and navigation system in the auto? Is this possible? Never underestimate the cleverness of advertisers, or mischief makers, or criminals. Once systems are networked together, it is amazing what unexpected events can transpire. The technology experts say it is not a matter of if but of when. It is always a race, and no matter what the good guys do, the bad guys always figure out a way to wreak havoc.
Hitting the Conference Circuit
Old MacDonald had a conference, ee-eye, ee-eye oh. Here a conference, there a conference, everywhere a conference.
One nice thing about academic life: there are always conferences in exotic locations. Florence in the summer, Hyderabad in the winter (but certainly never in the summer). Stanford in the spring, Daejon in the fall. There is a large industry in providing exotic conference centers.
The conferences and centers are not just for show: real work actually gets accomplished. The money sometimes comes from governmental granting agencies or foundations, sometimes from organizations such as the United Nations or the North Atlantic Treaty Organization, sometimes from private industry. Whatever the source of the funds, the grantors keep a strict eye out for results—positive, substantive results, like books, research reports, inventions, devices, desired breakthroughs. Of course, this guarantees that the call for papers will glorify the wonders that will soon result from these expected breakthroughs, and the resulting conference reports will be optimistic to a fault. The exceptions, of course, are those conferences run by the humanists and philosophers whose conference call will emphasize the approaching dangers from all these expected breakthroughs.
I find myself invited to events held by both sides—those expecting the technology of the future to free us and those expecting it to degrade and enslave us. My take is to deemphasize both claims. Technology will not set us free. It will never solve all the problems of humankind. Moreover, for all the problems that are solved, new ones will arise. Neither will the technology enslave us, at least not any more than it already does. Once we get used to it, the daily requirements of our technologies do not feel like enslavement. Mostly, they feel like improvements. Most people wash their hands several times a day, bathing frequently, and changing clothes daily. This wasn’t the case in earlier eras: is this enslavement? We prepare our meals with cooking and eating utensils made of complex materials, on stoves heated by electricity or gas, produced far away through advanced technological means and transported to our homes through complex networks of pipes and wires. Is this enslavement? Not to me.
I believe that the new technologies will mainly befuddle and confuse, frustrate and annoy us, even while delivering some of the promised benefits. After deployment, technologies always deliver benefits never contemplated by their designers and developers, along with problems and difficulties never conceived of.
There is many a cry that we should not release new technologies into the world until we have thought through all the benefits and disadvantages, carefully weighing the one against the other. Sounds good. Alas, it is quite impossible. The unexpected consequences of technologies always outweigh the expected ones, both the positive and the negative. And if the
y are unexpected, how can we plan for them?
Want to know what the future will be about? Keep an eye on those conferences. Nothing is going to happen in the future, not in technology anyway, without huge early-warning signs in the research laboratories of the world, all duly announced in the scientific journals, at a host of conferences, and through the establishment of centers for research. It usually takes a long time, decades, for ideas to move from conception to product, and that time is engaged in continual interaction with others pursuing similar ideas. Along the way, industrial forces take sufficient interest that the work starts to become commercialized, at which point it becomes invisible, hidden inside corporate research and development centers, carefully guarded.
Beyond those locked doors, however, there is the public world of conferences. And there a lot is going on, all under many labels: assistance, smart, and intelligent; ubiquitous computers, invisible ones, or perhaps ones still in the process of disappearing; ambient technologies intended to fit seamlessly into a life, but only if the life itself has been altered to make the seams invisible. Here are selected quotations from two conference announcements, just so you get the drift:
INTERACTION CHALLENGES FOR INTELLIGENT
ASSISTANTS, STANFORD UNIVERSITY, CA, USA
In an increasingly complex world, a new wave of intelligent artificial assistants has the potential to simplify and amplify our everyday personal and professional lives. These assistants will help us in mundane tasks from purchasing groceries to organizing meetings; in background tasks from providing reminders to monitoring our health; and in complex, open-ended tasks from writing a report to locating survivors in a collapsed building.
Some will offer tutelage or provide recommendations. Whether robotic embodiments or software processes, these assistive agents will help us manage our time and budgets, knowledge and workflow as they assist us in our homes, offices, cars, and public spaces.
_____________________________________________
INTERNATIONAL WORKSHOP ON ARTIFICIAL
INTELLIGENCE (AI) FOR HUMAN COMPUTING.
HYDERABAD, INDIA, JANUARY
Human Computing is about next-generation anticipatory interfaces that should be about humans, built for humans, and based on models of humans. They should go beyond the traditional keyboard and mouse to include natural, human-like interactive functions, including understanding and emulating behavioral and social signaling.
“Humanlike interactive functions”: the very phrase suggests secretive, mysterious nonbeings out to do our bidding. Socially assistive robots that will teach, entertain, and safeguard our children and entertain and safeguard our elderly (presumably, they no longer need teaching), ensuring that they take their medicine, don’t do dangerous activities, and if they fall down, either the robots will help them up or, at least, call for help.
Yes, the development of intelligent devices that can read our minds, cater to all our wishes (even before we are aware of them), and take care of the young, the elderly, the hospitalized, and even the rest of us, is a growing industry. Robots to talk with you, robots to cook for you, robots this and robots that. “Smart home” research projects show up all over the world.
The reality is a lot less than the dreams. Artificially intelligent agents are the mainstay of many a computer game, but that doesn’t count as real assistance in everyday lives. And yes, the technology behind the ever popular line of robotic vacuum cleaners can be expanded indefinitely to all tasks that simply require navigating over a designated area with sufficient thoroughness: cleaning a swimming pool, sweeping leaves from a yard or mowing the grass. The reality is that intelligent devices can indeed interact well with the physical world or other intelligent devices. It is when they must interact with real people that the difficulties begin.
Intelligent devices do work well in controlled settings where the task is well specified, such as washing clothes. They also are very effective in industrial settings, where not only is the task well specified, but the people who control and supervise them are well trained, spending hours learning how the devices work, often spending considerable time in simulators, where many of the failures can be experienced and appropriate responses learned. But there are profound differences between intelligent devices in industrial settings and the very same technologies in the home. First, the technologies are apt to be different, for although industry can afford to spend tens of thousands of dollars on automation, home owners are usually only willing to spend tens or hundreds of dollars. Second, in industrial settings, the people are extremely well trained, whereas in the home and in the automobile, they are relatively untrained. Third, in most industrial settings, when difficulties arise, there is considerable time before damage might be done. In the automobile, the time is measured in seconds.
Intelligent devices have made great progress in the control of automobiles, airplanes, and ships. They make sense for machines with fixed tasks. They work well in the world of computer agents, where they need only some intelligence and an image to display on the screen; real, physical bodies are not needed. They are successful in games and entertainment, controlling dolls, robotic pets, and characters in computer games. In these environments, not only do occasional misunderstandings and failures not matter, but they can add to the fun. In the world of entertainment, a well-executed failure can be even more satisfying than success.
Tools that rely on statistical inference are also widely popular and successful. Some online stores recommend books, movies, music, or even kitchen appliances by finding products that people similar to you in taste seem to like, then recommending those items to you. The system works reasonably well.
Despite the multiple conferences and the well-formed beliefs of scientists throughout the world, making devices that truly interact with us in useful ways is far beyond our capabilities. Why? Oh, for lots of reasons. Some are for physical reasons: devices—or robots—that can go up and down stairs, walk around the natural environment, and pick up, manipulate, and control real, naturally occurring objects are still far beyond our capabilities. Some reasons are lack of knowledge: the science of understanding human behavior is growing rapidly, but even so, what we do not know far exceeds what we do. Our ability to create natural interaction is very limited.
Cars That Drive Themselves, Houses That Clean
Themselves, Entertainment Systems That Decide
How You Shall Be Entertained
What’s next? Clearly, we are headed for cars that drive themselves; washing machines that determine the color and material of clothes, adjusting themselves automatically; cooking appliances that mix, heat, and stir, making the entire meal, after selecting the foods you should eat, having collaborated with the refrigerator and your medical records. Entertainment systems will select your music for you and prerecord television shows and movies they determine might be of interest, automatically deducting the charges from your bank account. Houses will take care of the temperature settings, of watering the lawn. Robot cleaners and lawn mowers will vacuum, mop, dust, and, of course, mow the lawn. Much of this has already happened. Most of this will soon happen.
Of all the areas of automation that impact our everyday lives, the most advanced is that of the automobile. So, let’s take a quick look at what is happening here, although the level of automation required to give the family vehicle full driving capability still remains far in the future. Some people estimate it will take twenty to fifty years, and chances are that this estimate will still be accurate no matter when you read this. In some circumstances, cars can—and already do—drive themselves.
How do we automate sensibly, controlling some parts of the driving experience but ensuring that drivers are kept alert and informed? “In the loop,” is the way this is described in aviation safety. How do we warn drivers who are about to change lanes that there is another vehicle in the way, or that there is an obstacle in the road or a car approaching from an intersecting side street, which is not yet visible to the driver?
What should be done when two cars arrive at an intersection on a collision course, and one car decides the best way to avoid a crash is to accelerate past the danger zone at the same time as the driver decides the best solution is to brake? Should the car ignore the pressure on the brake pedal and just accelerate? Should a car prevent its driver from changing lanes when another vehicle is in the other lane? Should the car prevent the driver from exceeding the speed limit, or from going slower than the minimum limit, or from getting too close to the car ahead? All of these questions and more face automobile engineers and designers today. In many of these situations, asking the driver what to do, or even giving the driver the relevant information, is seldom possible: there simply isn’t enough time.
Cars today can almost drive themselves. Take adaptive cruise control, which adjusts the auto’s speed according to its distance from the car in front. Add lane-keeping control and automatic toll-payment systems, and the car can drive itself, following the roadway and debiting the driver’s bank account. Lane keeping is not (yet) completely reliable, and in chapter 1, I discussed some of the problems with adaptive cruise control, but these systems will improve in reliability and be available at greatly lowered costs, so that they will eventually be installed in all car models. When cars start communicating among themselves (something that is already happening in experimental deployment), safety will increase even more. The technology does not have to be perfect to improve safety: human drivers are far from perfect.
Put all these components together, and, oops, we are training drivers to be inattentive. Their cars will be able to drive on highways for hours with little need for driver interaction: the driver could even fall asleep. This already happens in aviation: the automatic pilots are so good that pilots have indeed fallen asleep. A friend, a physicist who worked for a navy research laboratory, once told me that he was in an airplane doing experiments for the navy, flying over the ocean for several hours. When the tests were finished, his group called the cockpit to tell the pilots. There was no response, so they went forward to talk to them: the pilots were asleep.
