Understanding Context

by Andrew Hinton

  Other times, it’s clear that what we are using is an object, but it conflates not with place but with our bodies. When we use a desktop computer with a mouse, our cursor becomes an extension of the arm as much as our finger is an extension of our hands-on touch screens. Our bodies have an appetite for environmental extension, always working to find equilibrium that allows us to be more tacit than explicit, more “ready-to-hand” than otherwise.

  Digital objects are also physical objects with digital information driving some of their affordances and agency. We are increasingly surrounded by these, in our burgeoning Internet of Things. When we perceive these objects and interact with them, we’re tapping into ancient cognitive functions to comprehend what they’re up to in relation to ourselves.

  But, we don’t always know if an object has agency or not, and they’re often harder to comprehend than any natural creature. One wouldn’t expect one’s computer to whisper secrets to one’s friends about buying a private item from Amazon or cranking up a game on Kongregate during business hours. The information readily perceived by the user was not clearly indicative of those behaviors. Yet, when Beacon posted news items on behalf of Facebook users, it was doing just that as a digital agent making a decision not directly controlled by the human agent, and without clearly informing its intentions and rules of cause and effect.

  How well we perceive context in digitally affected environments is often a matter of how well the environment clarifies what is an object that is detached or attached, what sort of object it is, whether it has agency of its own, and what rules it follows.

  Smartphones “at-hand”

  Recent research into how we use our smartphones shows that we tend to use them—in Heidegger’s terms—as “ready-to-hand” objects, extensions of ourselves.

  Figure 6-3. An honest bit of graffiti in the San Francisco Mission District (photo by Jeff Elder)[136]

  In one study, over 50 percent of smartphone owners kept their devices close to them when going to bed at night.[137] In another study, for well over 80 percent of days tracked, the smartphone was the first and last computing device used by participants—bookending their days more pervasively than laptop and desktop computers. Generally, users “always” kept their phones with them, defaulting to desktop and laptop devices only “when absolutely necessary.” The study concluded that “the phone is emerging as a primary computing device for some users, rather than as a peripheral to the PC.”

  Of course, for many populations in the world—those who missed out on the personal-computer revolution—phones are their first and only computing devices. And according to a World Bank report, “About three-quarters of the world now have easier access to a mobile phone than a bank account, electricity, or clean water.”[138]

  Mobile phone devices have many characteristics that, taken together, make them a sort of “perfect storm” object for extending our cognitive abilities—and the dimensions of our personal context. Here are some of those characteristics:

  For over a century, the telephone has been one of our most intimate modes of communication, allowing us to whisper to each other—lips to ears—in private conversations.

  Mobile phones are small, portable, and more easily treated as “ready-to-hand” extensions of ourselves than other, larger devices.

  They’re always connected, often through multiple networks.

  Phones are more likely to be geolocation capable, providing extended-cognitive situational awareness.

  Because we carry phones on our person, they can communicate with us through touch—vibrating our skin, asking for attention.

  Considering all these factors, it makes sense that adoption of mobile phones is outstripping all other devices by leaps and bounds.[139]

  Layout

  Layout means “the persisting arrangement of surfaces relative to one another and to the ground.”[140] It’s the invariant relationship of elements such as objects and other features in a particular setting.[141] Each arrangement provides a particular set of affordances that differs from some other layout; one layout involving trees might afford one sort of behaviors for a squirrel and different behaviors for a bird.[142]

  Perceiving layout depends on both perception and action. We tend to use the word “layout” for static two-dimensional artifacts such as newspapers or posters, which have pictures and text. However, our perception of layout evolved in three dimensions, which we come to understand by moving through an area and seeing things from different angles, touching them, smelling, and hearing. The layouts we see in two-dimensional surfaces such as paper and screens are representational, artificial layouts. They borrow some properties of the three-dimensional environment but are not as physically information-rich.

  Layout affects the efficacy of action and an environment’s understandability. The layout of a kitchen in a restaurant can make or break its success—you don’t want wait staff bumping into the sous-chef, or the baker’s oven opening over the head of the saucier. Likewise, an airport ideally has a layout that clearly informs us about the rules and expectations of the space.

  Related to layout is clutter, which are objects that obscure a clear view of the ground and the sky for a given perceiver.[143] Yet, when one of those objects becomes the subject of the perceiver’s attention, it’s no longer perceived as clutter but as an affording object in its own right. A stone on the ground is part of the clutter that obscures a full view of the ground; but the stone can also be an object that affords picking up and using as a tool or placing as part of a wall. A room might be full of furniture that is clutter to the person trying to get as quickly as possible from one end of the room to the next; but when the same person is tired from all that jumping about, a chair affords respite.

  Sometimes, a cluttering object acts as a barrier, which hinders further movement. A barrier’s affordance is that it blocks our movement. Remember, all affordances are just affordances, not anti-affordances—whether an affordance is positive or negative for the perceiver is a matter of context.

  What is clutter or not is a nested property of the perceiver’s current experience. Again, this isn’t a permanent categorical hierarchy. Contextually, anything can change its role in the layout depending on current behavioral needs.

  Digital Layout

  Digital interfaces and linked environments likewise have these layout concerns. Some are about efficiency of action, such as with guidelines like Fitts’ law, which concerns the relationship between size of target and distance to target. Some are about comprehending the relationships of the elements on the screen and between different “places” in the software environment. Users must understand the difference between an underlying surface and what is an attached or detached object and what these things allow the user to do: Can I move a document into a folder? Can I edit or erase a string of text? Is a drop-shadow under an object an indication of its movability, its press-ability, its priority in the visual hierarchy, or is it just decoration? These treatments are all clues as to the relationship of a particular entity to other entities, in an overall layout.

  Clutter is a factor for digital interfaces, as well. We often hear users say “there’s so much clutter” in an interface. Yet, everything in an interface was put there by someone for some reason, whether warranted or not. One user’s trash is another user’s treasure. One shopper’s clutter is a marketer’s sale promotion insert.

  Events

  An event is a change in the invariant structures of the environment, such as a change in substance, object, or layout.[144] Gibson offers a thorough classification of terrestrial events, which we won’t cover fully, but it includes three major categories:

  Changes in the layout of surfaces (and by extension, objects, and so on)

  Changes in color and texture of surfaces

  Changes in the existence of surfaces[145]

  Examples of these could include things such as a ball being thrown from the pitcher’s mound to home plate; the movement of a rabbit from “in range�
� for catching to “out of range” down a hole; or the changes observed in substances as they transition, such as from ice to water or as soft clay hardens. Changes in existence can be due to burning in fire, or decay, or in an animal that was once not in existence being born and now in the environment as a detached object.

  Like other structural elements of the environment, events can be nested and interdependent.[146] A place can change from “cool shade” to “hot and bright” if a tree (an attached object) falls (another event, changing the tree to a detached object). Most natural events contain so many small events that it’s impossible to identify them all. We perceive these complex events in a compound way, not having to keep cognitive track of every observable change. We don’t have to perceive every single raindrop to know that it’s raining.

  Events are related to a psychological phenomenon called change blindness. This occurs when something in the environment changes without our noticing it. We can easily overlook events if they’re not perceived as part of what is relevant to our current action. In a well-known video demonstrating this effect, the viewer is asked to keep track of how many times a group passes a basketball between its members. Most viewers don’t notice the weird, costumed “dancing bear” among them. Facebook’s Beacon presented only a small indicator of its activity that was easily overlooked in the busy events happening on a kinetic site such as Kongregate. These change-blindness issues are even more of a danger in screens and displays, because the information available for pickup is so limited compared to a three-dimensional, object-laden environment.

  Affordances and Laws for Events

  Events can have affordances of their own. A campfire can afford keeping us warm; rain affords watering our crops. These events are, of course, nested in particular contexts; the campfire can also burn our bodies, and rain can flood our fields.

  Perception evolved in a world in which events tend to follow natural laws that can be learned. The motion of water can be variant as a substance, but its behaviors are invariant in that water will always behave like water. The sun moves across the sky through the day, but it does so every day, following an invariant pattern.

  Events and Time

  We actually perceive time as events rather than as abstractions such as seconds or minutes.[147] We feel time passing because things are changing around us. Until only a century or so ago, most people still thought of “noon” as when the sun was directly overhead. Even though clocks were already fairly common, they were thought of as approximations within a given locality, not synchronized with any global standard. It was the rise of railroads that brought absolute, standardized time to the general public; railways couldn’t have one town’s noon be different from another’s; otherwise, trains would never be on time anywhere, or (at worst) they might collide at junctions.[148]

  We now live in an artificially mechanized sort of “time.” Our culture regulates the events around us to the point at which our surroundings tick to the same rhythm as the clock, no longer tied to the relatively fluid event of the sun rising and setting. Even when it’s a mechanical structure, though, it’s the environment that creates information marking time, not the abstract measurement of seconds or hours.

  This is why time can feel so relative to us, no matter how many seconds are measured.[149] A bad and boring 90-minute movie feels interminable; an exciting, quality 90-minute movie feels like it flies by. In digital design, we might want to judge the efficacy of a website, for example, by how long it takes a user to get from one page to another, either in terms of seconds or mouse clicks—but we find the user perceives the passage of that time differently depending on context. User perception is important to consider when looking at the results of analytics and other performance measurements. Ten clicks might be fine, if the user is getting value out of each one (and feels like she’s getting where she needs to go); three clicks can feel like forever if the user is floundering in confusion.

  Digital Events

  There are many sorts of events we perceive in digital interfaces. There are rich-interaction transitions (windows zooming in and out; swiped objects disappearing in a puff of animated smoke) that make clear where simulated objects have gone or where they came from. There are also system processes, including backing up a file or establishing a wireless connection. These can be initiated by us or initiated by the system’s own digital agency.

  Users often struggle to comprehend the events that occur in software environments, where cause-and-effect does not have to follow natural laws. This can be especially true when the system is doing something on its own, outside of our immediate perception. The more complex our digital infrastructure becomes, the greater role these invisible, automated events play in our environment.

  Place

  Place is the last element we will cover in this summary. It’s especially important to context because so much of context has to do with how we perceive our environment as organized into places. Here are the key factors that describe place:[150]

  A place is a location or region in the environment that has a particular combination of features that are learned by an animal. An animal experiences its habitat as being made up of places.

  Unlike an object, in nature, a place has no definite boundaries, which might overlap in a nested fashion depending on the current action of the perceiver, and other factors. Sometimes, however, there are in artificial boundaries like fences, walls, or city blocks. These are layout structures that nudge us into perceiving them as places, because of how they constrain and channel action. It’s still up to the perceiver to distinguish a city block or the inside of a building as a place, but the environment exerts great control over that perception.

  Places are nested—smaller ones are nested within larger ones, and they can overlap or be seen differently from various points of view, depending on the particular actions and perceptual motivations of the perceiver.

  Movement in the environment is movement among places, which can be reached via one or more paths (which themselves go through other places). Even a path through the woods can be experienced as a place—it’s a layout in the environment that affords motion between other places. Yet, the path is meaningful because of the places it connects.

  We don’t perceive “space.” Instead, we perceive places. Space is a mathematical concept, not an embodied one. Space is undifferentiated; but just by perceiving and moving within an area, the perceiver’s presence is already coming to grips with the area as one or more places.

  Place-learning is “learning the affordances of places, and learning to distinguish among them.” It is an especially important kind of learning for all animals, including humans. As we saw earlier, place has a function in how our extended cognition learns and recalls information, such as when people had more accurate recollection when staying in the same room.

  Place-learning and orienting oneself among places is what we call wayfinding—knowing where one is in relation to the entire environment, or at least the places that matter to the agent.

  Places persist from the point of view of the perceiver—they cannot be displaced the way objects can. Our perception counts on places being where they are, from one day to the next.

  The persistence of places and the way we learn them is especially important to point out. A place can be altered, such as by fire or a destructive storm, but it is still in the same location in relation to other places. Place-learning depends on this persistent situated quality.

  Digital Places

  This issue of persistence is especially important when we look at software environments. Our perceptual system relies on the persistent properties of place, and can be confounded by environments that don’t meet those expectations. Yet, software can create the useful illusion of persistent places and then obliterate or move them or change them fundamentally in ways that cannot occur through the natural laws that govern physical information.

  A website can feel like a place, but it can also feel as though it has places withi
n it. Likewise, a portal that gathers articles from many sites can be its own place, even if all the content is from elsewhere. An application can be a place nested in an operating system. A document is an object until it’s opened and, in a sense, “dwelled in,” and feels like a place again. Likewise, digital technology can change the way we experience physical places, such as when we use something like a “geo fence” to teach a smartphone to present home-related information when it’s near our home address.

  Places, and “placemaking,” will figure centrally in our exploration as we continue through semantic information, digital information, and the vast systems of language we inhabit. For humans, it turns out that place is as much about how we communicate as what we physically perceive.