Research has shown that much of our daily activity is driven by deep, preconscious impulses and primitive-brain emotions rather than logical, conscious analysis and decision-making. In Descartes’ Error: Emotion, Reason, and the Human Brain (G.P. Putnam’s Sons), neurologist Antonio Damasio shows how mind, body, reason, and emotion all work as a single system rather than separate entities. In fact, reason is an outgrowth of emotion; it is crippled without an emotional foundation to drive our decisions.[69]
In another major work on the subject, Thinking, Fast and Slow (Farrar, Straus and Giroux), psychologist and economist Daniel Kahneman shows how behavior exhibits two sorts of consciousness at work, which he refers to as System 1 and System 2. System 1 is subconscious and emotionally driven; it works fast and automatically; it’s frequently engaged and relies on stereotype. System 2 is conscious, working slowly with purposeful effort; it is engaged less frequently (in comparison to System 1), employing logical calculation.[70] A related distinction from behavioral economics calls these systems “Automatic” and “Reflective.”[71]
There are other models exploring these different levels of consciousness, some of them breaking it down into more layers. One is Don Norman’s three-level model of “Visceral, Behavioral, and Reflective,” in which the first two are unconscious, and reflective is conscious.[72] Another, from the information sciences, is Marcia Bates’ quadrant model for information seeking, in which one axis is Active/Passive and another is Directed/Undirected.[73] I mention these models because they all provide equally useful perspectives on how much user action occurs without the same level of awareness or conscious thought. Don Norman stresses that “design must take place at all levels.”[74]
Here, I offer a simple model influenced by these others. Instead of presenting stages or levels, this model shows how these states of mind are not cleanly separated but instead function on a spectrum of conscious attention between Explicit and Tacit, as depicted in Figure 5-1.
Explicit is related to terms from other models such as conscious, deliberate, reflective, and System 2. It’s a state in which we are reflecting, self-aware, and consciously thinking through our actions or considering the meanings in our environment.
Figure 5-1. Explicit and tacit, along a spectrum
Tacit is related to other terms, such as unconscious, intuitive, automatic, and System 1. It’s a state in which we take action implicitly without reflection, driven by unconscious impulse, resonance with the environment, and by habit or convention.
Again, these are not binary categories. They are part of a gradual continuum; and we can’t count on someone being in just one side or another for everything they’re doing—although satisficing through the loop-of-least-resistance principle means that people do as much as possible as far toward “Tacit” in this spectrum as they can. We evolved to conserve energy, and making conscious, explicit decisions burns a lot of fuel. In fact, research has shown we can suffer mightily from something called decision fatigue.[75] Our ability to make decisions deteriorates after too much deliberation, causing us to make worse or more impulsive decisions from the more tacit level of consciousness, even when we should be explicitly concentrating.
This spectrum also aligns with the Explicit/Tacit model for knowledge first described by Hungarian chemist and philosopher Michael Polanyi in the mid-twentieth century. Tacit knowledge is that which is hard or impossible to communicate through explicit means, such as written instructions. In Polanyi’s terms, “We know more than we can tell.”[76] Riding a bicycle can’t be learned from a book; the body has to do the physical work of learning. A softball pitcher can’t explain how she pitches in explicit terms; the knowledge is essentially embodied in the act of pitching. However, even tasks that aren’t so physical are often tacitly driven. Language itself is eventually used tacitly after we learn it to the point of fluency.
Context depends heavily on what level of conscious attention is being demanded of the agent (perceiver, user, person, and so on). Anything that doesn’t accurately fit one’s unconscious, tacit habits and conventions must be explicitly attended to and learned, or else it runs the risk of tricking the perceiver into an unintended consequence.
When we drive on a road at night, we assume by habit and convention that the road has enough friction to keep us safely moving forward, unless we detect something about the road’s surface that would indicate otherwise. That’s why the phenomenon called “black ice” is so dangerous: it appears to be one sort of structure, when it actually is the opposite: a slick of frictionless surface that doesn’t reflect enough light to alert us to its presence, showing us only the dark asphalt underneath. When we see a variation in the road, we attend to it, slow down, drive more carefully. When we don’t, we act tacitly, unconsciously.
As designers of environments, one of the biggest risks we run is putting black ice in front of people; we inadvertently trick them into thinking the environment affords one thing when it actually affords something else, possibly to the user’s detriment. Facebook’s Beacon created a wormhole of information, leaking personal actions into actively published feeds. By being too subtle (and in some ways, just confusing) about how the system indicated its actions, Beacon created a sort of black ice that wasn’t perceived for what it would actually do. The few users who carefully paid explicit attention to it noticed what it was doing; but we can’t expect users to do the digital equivalent of checking every inch of road for ice.
Environmental Control
Consciousness is something we think of as internal to the individual, but the external environment plays a powerful role in consciousness and behavior, as well. This idea was formulated in 1926 by psychologist Kurt Lewin, who created what is now called Lewin’s Equation: B=f(P,E). It’s a heuristic formula (rather than a mathematical equation) that states, “Behavior is a function of the person and his or her environment.” It was controversial at the time it was published because it emphasized the environment of the person in the moment of perception over the learned experience of the past. It has since become a foundational idea in the field of social psychology.[77] Although this equation predates the work of ecological psychology theorist James J. Gibson by quite a few years, the spirit of it is in line with Gibson’s idea of the coupled relationship between the perceiver and the environment.
To a significant degree, context controls conduct. We like to think we actively decide our every action; isn’t that what free will is all about? Yet, it so happens that the environment’s structural constraints determine much of our daily behavior.
This doesn’t mean that we have no agency whatsoever; a perceiver detects information in the environment and then has the ability to decide what to do about it, controlling the perceiver’s motion.[78] But, environmental information is central to the very origin of the whole perceptual system, and it still exerts its structural pressures on our every act. Moreover, given that our cognition is bound up in action, our environment’s constraints can shape how we think, as well. Certainly, we modern humans are able to control many of our behaviors and teach our bodies new ways to respond to environmental information, but we’re doing it on top of a core organism—from limbs to limbic system—that was formed by the environment in which we evolved.[79]
Nature is not the only force exerting this pressure. Technologies also alter our perception of our bodies and their abilities. In experiments during which full-sized adults were put into an apparatus where they perceived themselves as having a virtual child’s body, the adults started to perceive the structures around them as a child might, including whether they could fit through openings or climb onto surfaces. The new body and the environment strongly influenced the adults’ choices and behaviors, and even their emotions.[80] In a comparison between a room suited for adults and one more suited to children, the study found that “you see the world bigger, have more childlike attributes, and prefer a [child-suited] environment rather than an adult one.”[81] The adults’ brains didn’t change; they were just e
stimating the world using a different sort of body than they were used to.
Some might argue that these adaptations mean the brain’s “schema” or internal representation of the body has plasticity, meaning that it can adapt and change.[82] But an embodied argument could be more straightforward: just as stairs have intrinsic information that doesn’t require a mental model for understanding their use, the brain doesn’t need a representation of the body and its capabilities. It already has an actual body present, so the body can act as its own image. Thus, if the brain is given information that tricks it into thinking it has a different body, it uses the new one instead. If the “trick” is convincing enough that, even when moving and calibrating, the body can continue to believe the trick, it continues behaving accordingly.
A version of this adaptation affects us in all areas of life. The environment’s information is soaked up by our bodies and changes our habits. Even our devices and applications affect our behavior, because new or different powers give us a different sense of what we can and can’t do. When a desktop application starts remembering what documents we had open after we quit, we start opening the app to get back to those documents rather than hunting down the files on our hard drives. Gmail gave users a big Archive button and a more powerful way to search emails, and many users stopped sweating over filing away messages in folders. When our phones began remembering phone numbers as names, we stopped memorizing everyone’s digits.
Our brains might not need a map or schema because all the information is right there in the environment. Of course, past experience is part of this dynamic as well, and we will look at that as part of memory and learning; but we tend to underestimate how much present information can shape our understanding and action.
This “nudging” effect is explored in the field of behavioral economics, which is largely about how environment influences behavior in complex cultural systems. One study showed that in the United States, when citizens have to “opt in” to be an organ donor, only about one-third do so. But in Austria, 99 percent are donors, partly because their government enlists all citizens in the program by default, giving them instead the choice to opt out. In both situations, people have a choice, but because of satisficing, this nudge in one direction versus another makes a huge difference.[83] The environment makes the initial, hard decision for them. Behavioral economists call these sorts of policy structures choice architecture. But this isn’t architecture of stone or steel; it’s architecture of rules and communications made of language.
We see environmental control in action in software environments, too. In the airport scenario in Chapter 1, when I assumed that my coworkers could see my travel schedule in my work calendar, it was because the affording information available in the interface didn’t specify that other people couldn’t see what I could see. The structures evident in the calendar app exerted control over what I perceived to be “my calendar.”
Likewise, when Facebook users were taken by surprise by Beacon publishing private information to public feeds, it was because the environment’s information didn’t adequately specify what was going on behind the scenes. Software can all too easily trick our perception into assuming our environment has a particular stable structure that isn’t really all that stable or universal.
When watching people use gadgets and software, we need to remember that the way they’re making use of their context is largely being determined by the structures available to them. Often, I have heard e-commerce clients complain that their customers are using the online shopping cart improperly, as a sort of wish-list, even when the site provides a separate wish-list function. Though when you look at the environment neutrally as a cluster of environmental structures, it becomes clear that Add to Cart is usually a much easier and quicker function to find and use than Add to Wish-List—the button tends to be more prominent, more available, and the “Cart” itself is always represented somewhere (normally as a concrete metaphor with a picture of a cart) regardless of where the user is shopping. Why wouldn’t the user make use of such an available, straightforward environmental structure over a less-available abstraction?
Part of what we will continue to explore is how these semantic constructs—whether the “choice architecture” of civic policy or the “information architecture” of software—aren’t merely metaphorical architecture, but real structure that exerts nudges and constraints on our behavior, similar to anything in the natural or built environment.
Memory, and Learning the Environment
Through all of this talk of perceiving environments and how the environment exerts control over our cognition, one might wonder, “yeah but what about memory? Don’t we remember things about the environment? And, isn’t that a sort of image or representation we store in our brains? If we’re supposed to be designing context, doesn’t it need to account for what people remember from previous experience?”
In short, the answer is yes; memory is a crucial part of how people experience context. However, the complicating factor is, no, we can’t count on stable, fixed memory of our users in what we design. Because this is such a big issue for context, we should spend some time looking at it more thoroughly.
What is Memory?
The idea of “memory” is so deeply ingrained in our language and culture that it’s a bit of a shock to learn that there is no universally accepted science or model for how it works.[84] The way we retrieve knowledge from ourselves is still, in its details, largely unknown and the subject of much scientific research and debate.
The prevailing idea of memory is the storage metaphor. We assume memory must be a place in our heads—like a sort of database or file cabinet—where our brains store experiences and then pull them out when needed. Until about 20 or so years ago, even cognitive science assumed this to be accurate but has since acknowledged that memory is much more complicated.[85]
Still, the storage metaphor is the way we conventionally talk about memory, even though it’s terribly misleading. If our brains actually stored everything away like cans of soup in a cupboard, we should be much better at remembering than we actually are. Memory is untrustworthy, and seems to hang onto only certain things and not others, often with little apparent rhyme or reason. In one study from 2005, people in the United Kingdom were asked if they’d seen closed-circuit television footage of a well-publicized bus bombing. Eighty-four percent of the participants said they had—some of them providing elaborate details in response to questions—even though no such footage existed.[86] More recent research has shown that even those who we popularly think of as having “photographic memory” (actually called highly superior autobiographical memory) are nearly as unreliable as those considered to have normal memory.[87]
Of course, we know that we can recall some sort of information from our past, using neurochemical activity that makes it possible for our nervous systems to retain a kind of information about our environment and past experience.[88] Yet, in spite of all that modern science has at its disposal, “human memory remains a stunning enigma.”[89]
The question is, what do we need to know about how memory works to design appropriately for it, especially when it comes to the prior experience people bring to context?
Types of Memory
From traditional cognitive science, there are many different models for how memory works, most of them variations on similar themes. Figure 5-2 presents a diagram showing one version.
Figure 5-2. Various types of memory, related to the disciplines that tend to study them (source: Wikipedia)[90]
Such models have been built up over the years, based on the patterns researchers see in test-subjects’ behaviors, and in the little we can learn from watching energy and blood moving in their brains. A model like this can mislead us into thinking there are distinct areas of the brain that perform each of these functions. In actuality, it’s not so clear-cut.
An Embodied Perspective on Memory
Embodied cognition theorists tend to question a lot of the re
ceived wisdom about memory. J.J. Gibson criticized the idea of memory as a “muddle”—a sort of “catchall of past experience” that lacks real evidence. He argues against the theory that what we experience in the present is mostly assembled from memories of the past: “the doctrine that all awareness is memory except that of the present moment of time must be abandoned.”[91] Elsewhere, he points out that even the assumption that there is a clear distinction between present and past experience is somewhat of a fiction; perceiving and remembering are just two ways of looking at the same dynamic.[92]
Louise Barrett’s Beyond the Brain: How Body and Environment Shape Animal and Human Minds (Princeton University Press) shows how many seminal studies on memory—for example, Piaget’s “A not B” test regarding infant memory—have since been undermined by newer research that accounts for an embodied dynamic.[93] Barrett explains that “memory is not a ‘thing’ that an animal either does or doesn’t have inside its head, but a property of the whole animal-environment nexus; or, to put it another way, it is the means by which we can coordinate our behavior in ways that make it similar to our past experiences.”[94] From this perspective, memory is really accumulated impressions from environmental perception. It’s not something that begins in the mind, where some computer-like entity is recording sensory perception for later retrieval, or processing symbols and categories; rather it’s built up from what our bodies-plus-brains retain from our ongoing activity in the perception-action loop.
Understanding Context Page 9
