Understanding Context

by Andrew Hinton

  Part VI takes a few steps toward perspectives, principles, and techniques that can help us consider context in our design work. It discusses what composition is, and the materials we use for composing context. It also covers how users make sense of their world through narrative, and offers some considerations for how we do research, analysis, and modeling that can build on the understanding we’ve explored in prior chapters.

  Figure VI.1. A modern subway map provides a composition of semantic information—an abstracted model, disconnected from the literal shape of streets above-ground; it’s an early example of infrastructure that allows people to navigate by label more than physical structures[376]

  * * *

  [376] Wikimedia Commons: http://commons.wikimedia.org/wiki/File:Milano_Subway_map.svg

  Chapter 19. Arrangement and Substance

  A poem is a little machine of words.

  —WILLIAM CARLOS WILLIAMS

  Composition in Other Disciplines

  WHY COMPOSITION? TO COMPOSE something is to arrange it, to relate all its parts, and to determine its substance. It’s a great way of thinking about how we handle context, because context is all about how we understand relationships between elements. Composition is an important concept in just about every field of human creation, which makes it a useful way to think about cross-channel, ambient environments that can involve so many different kinds of media and information. Let’s take a look at some of them:

  Writing

  The combination of elements such as tone, theme, setting, and voice within a structure, such as the classical essay framework of “Introduction—Body—Conclusion,” or the way the narrative arc of a plot gives structure to a novel. At a more granular level, composition has to do with sentence structure and parts of speech. Poems, too, have compositional elements—from the established forms of sonnets and sestinas to the smaller formal techniques of free-verse line-breaking and semantic improvisation.

  Music

  For music, composition is its “musical form.” It’s the structure or plan of a piece of music: its arrangement of repetitions and differences and structural patterns. What makes a Rondo different from a Sonata, and what makes them both different from a live improvised jazz performance, is the structural decisions made manifest in their compositions. Composition is such a central idea in music that we colloquially refer to the piece of music itself as a “composition.”

  Graphic arts and photography

  In any sort of picture, composition has to do with the artist’s structural choices of perspective and framing as well as the formal arrangement of the picture’s subject or subjects. It determines what is put in the frame and what is left out, and how the things pictured relate to one another within the frame of what the viewer can see.

  Film

  For movie-making, composition inherits everything from photography and writing and (usually) music, but adds the complexities that come with stringing many pictures together quickly over time. The way one shot makes sense (or not) when juxtaposed with another shot is a composition issue. For example, in a filmed conversation between two people, each should be facing a consistently distinct direction toward the other party; otherwise, it’s hard to track who is talking with whom. Films also use establishing shots to situate the action—a brief view of a cityscape or the outside of a restaurant, before focusing on a conversation. Beyond specific shots, composition also has to do with how the film is edited together, scene to scene, and how one scene sets context for the action in the next as well as how all the scenes come together to create a narrative whole. Film is the twentieth century’s most advanced medium for tackling the challenges of placemaking and sensemaking that come from de-contextualized information: in film, there’s no longer a single stage in a theater to hold context together. The film must artificially establish the invariant container of each scene. (Software can still learn much from the way the “grammar” of film constructs context.)

  Architecture

  For the built environment, composition refers to the arrangement of essential architectural elements, starting fundamentally as a single surface and on through various combinations, into meaningful components: rooms, corridors, windows, stairways, vestibules, courtyards, and so on. The way these are chosen, arranged, and connected are what determine the nature of the environment; they make the difference between an apartment building and a corporate campus, a parliamentary chamber and a nightclub. Architect Louis Kahn famously remarked that an architectural plan is a “society of rooms” and explained how that society is “knit together with the elements of connection.”[377] The connections are what define the relationships between the rooms and what give them much of their meaning and utility, and vice versa.

  Qualities of Composition

  There are common qualities and issues that we see across all the aforementioned sorts of the composition:

  Composition is about relationships that make up the whole

  One can’t consider the composition of a thing without considering its totality and how the whole is greater than the sum of its particulars. Context is about how elements relate to one another, which puts composition right at its center.

  Composition is nested

  In all the examples from other disciplines, the arrangement of elements creates nested relationships. A picture begins with its boundaries as its outer shell. It contains representations or formal abstractions, some of which might continue to contain one another, and so on. One approach to built-environment architecture is to start with the primitive foundation of a mere surface and then add complexity outward, while simultaneously starting with the outer shell of the surveyed building site and designing structure inward from that setting. Film has a nested grammar for orienting the viewer, such as providing establishing shots before close-ups, situating details within the larger story arc or mise-enscène. It makes sense that these fields would compose their artifacts in this nested manner—it’s how we, as terrestrial creatures, comprehend everything.

  Composition requires (and creates) structure

  Without structure, nothing has composition; putting elements in relation to one another creates structures in the environment, and vice versa, since creating structure is always composition of one sort or another. It’s important to point this out because otherwise, we tend to forget the next point...

  Composition is not neutral

  There’s no way to abdicate responsibility when composing something, because saying yes to a structural choice means saying no to the alternatives. Inclusion is also exclusion. You can’t have a door in every spot of a wall, or else there’s no longer a wall. You can’t show everything in a painting; you can show only what the frame will contain. And, you can’t label something with every possible bit of relevant language, because it eventually ceases to mean anything in particular. For something to have any affordance or semantic function, it needs invariant structural qualities.

  Composition makes an argument

  Because structure isn’t neutral, to compose something is to make an argument about what something should be and how it should work. As information architect Andy Fitzgerald has wonderfully put it, in a talk about digital design, built architecture is “rhetoric for spaces.”[378] As with maps, every composition has an agenda, whether it’s intentional or not. A Rothko color-field painting makes an argument that it can be made of shades of color and nothing else; Quentin Tarantino’s Pulp Fiction makes an argument that it can disrupt linear time but still have a coherent narrative; and Facebook’s policy against multiple accounts makes an argument that having a single profile with one’s real name is “good” and the alternative is not. Composition, done well, makes manifest an opinion of what relationships should exist, and which should be primary, secondary, and so on. And that argument is what creates the conditions under which an agent is trying to understand context.

  Composition means we have to make decisions about what things are, what they mean, and how they relate to one another,
all coming from an understanding of how people will perceive the environment.

  As Film Does to Stage Plays...

  In semantically dominant environments, in which so much of the structure is created with language, the meaning of a given structure can shift radically based on the context of the perceiver. In one of the classic examples of how context works, early twentieth-century Russian filmmaker Lev Kuleshov created a short film showing the face of a famous Russian actor of the time, intercut with unrelated scenes: a bowl of soup, the body of a child displayed at its funeral, a beautiful woman sitting on a divan.

  In the motion-picture version, it appears the actor is reacting uniquely to each scene: hunger, grief, lust. The still-image representation presented in Figure 19-1 doesn’t provide the full effect; in the film itself, the actor is in motion—sitting still for the camera, but breathing (to the point of sighing) and showing subtle facial reactions to whatever he is witnessing.

  Of course, the trick here is that the clip is showing exactly the same few seconds of facial expression across all three contexts.

  Kuleshov created this strip out of unrelated found footage to illustrate how the cinematic composition technique called “montage” can evoke narrative for an audience, even if there isn’t any actual intended narrative in the individual pieces. Composition can bring implicit meaning to information that the pieces of information themselves do not necessarily contain. Although film displays images captured from the interaction of light and physical information, its physical information is only that captured light, frozen in two-dimensional frames. It lacks the intrinsic perceptual information we have in nonmediated life. If we were in the room with this actor, watching a scene on stage, we would see the entire situation. And that’s what very early dramatic films were: captures of stage plays. Film came into its own as an art form by breaking ties with the “real” situation, and cutting that reality up into slices, rearranging it to make new possibilities of meaning.

  Figure 19-1. Frames from each section of the Kuleshov film[379]

  As for places in our environment: semantic information has always given us the capability to change the meaning of a place. From cave paintings to murals and political broadsides, we’ve transformed surfaces into radiators of meaning that alter our experience of a physical place.

  Digital technology does to that semantic-information ability what film did to staged narrative: it makes it possible for us to edit, intercut, and splice what place is to its inhabitants.

  Something to Walk On

  The more heavily an environment relies on semantic information for its structure and meaning, the more it requires us as designers to pay explicit attention to careful definitions in its composition, which requires stable points of reference. In Digital Ground: Architecture, Pervasive Computing, and Environmental Knowing (MIT Press), Malcolm McCullough argues for what the book’s title names—a grounded experience in the digital age: “Not all is flux. Much as a river needs banks unless it is to spread aimlessly like a swamp, the flow of information needs meaningful contexts. Even in an age in which distance has been annihilated, location still matters.”[380]

  McCullough attends to the challenges of “digital ground” from an interaction-design perspective, but because of the ambient and social qualities of how “technology accumulates locally,” he positions this work “more closely into alignment with the concerns of architecture.”[381] Later, he builds on that point: “The role of computing has changed. Information technology has become ambient social infrastructure. This allies it with architecture. No longer just made of objects, computing now consists of situations.”[382] Those situations are dynamic, perceived-and-acted, and contextual. For people to interact effectively, the architectural composition that grounds those interactions must be situationally sound—it must make sense as an environment.

  Even when considering plain old written text, it turns out that the physical context of an actual book has an effect on how understandable and learnable the semantic “content” might be. People using paper begin to really learn the material more quickly, over time, than those who are trying to fully understand it through an e-book interface.[383] One study showed that recall of a novel’s plot was significantly poorer on a Kindle device than on paper.[384] Just like walking from one room to another loses the tacit connective tissue of the physical surroundings that helped form a “memory,” not having actual pages in a physically navigable book can take valuable information out of the equation.

  What is true of objects can be true of places. When digital environments such as Twitter or Facebook continually innovate, grow, and change their “feature set” to please the marketplace, they often change fundamental rules in their architectures that alter the invariant qualities that attracted and kept users to begin with. What felt like a comfortable home-away-from-home can suddenly become more exclusive and cold, or noisy and fragmented.

  Or, it can become something else entirely—a confusing panopticon of swirling signifiers, where the language one uses in the environment—such as “liking” a tweet or post—morphs into a more complicated lever with invisible connections and rules. Even at Harry Potter’s Hogwarts School of Witchcraft and Wizardry, where the staircases would move around to connect different entrances in the school’s towers, the patterns were at least learnable, and the towers didn’t change into city parks. But when Twitter announced that it might change to an algorithm-driven feed—rather than the “everything” feed that has grounded the experience for seven years—many of its citizens were dismayed, because that move would fundamentally change what Twitter is to them.[385]

  The foundational concern for shaping context is determining what McCullough calls the “banks of the river” as the stable, invariant structures of our environment. The individual objects of interaction certainly contribute to the overall situational context. But they’re given much of their meaning by how they are nested within the broad, invariant structures of a whole architecture.

  Consider the challenges of Google’s latest social infrastructure. After the privacy debacles of Google Buzz (Chapter 12), Google rallied by creating Google Plus—an even more ambitious, comprehensive social platform, which has been slowly integrated into all of Google’s platforms over time. Having learned a tough lesson with Buzz, Google has tried to go above and beyond the call of duty in making sure users understand the labels and rules of the new environment.

  For example, Google Plus launched with a “Circles” metaphor used to organize contacts. In some ways, it’s a brilliant use of the label: it borrows from the idea of “social circle” and shares some of our natural social-circle behaviors—we can create connections with people by multiple facets such as shared interests or family connections.

  But in other ways, Circles are not like the social circles of non-Plus life. Our real relationships emerge “bottom up” through social activity, and only sometimes have any “top-down” definition. Natural social connections also usually lack hard boundaries, and our proxemics tend to be fluid over time, closer to one group for a while, but drifting into more closeness with a different group later.

  Although it’s possible to create circles that we name based on “close friends” versus “acquaintances,” Plus doesn’t know when someone in one of those circles goes from “close” to “not very close at all”—we’d have to consciously move them from one circle to another. Plus, the circles are presented to us as equally sized, separate structures, until we select one, as in Figure 19-2.

  Figure 19-2. A slice of the Google Plus Circles interface

  Still, Google has worked hard to teach the users of Plus about how the invariant structures and rules of Plus work.

  There is one challenge that Plus faces: the more complex the environment, the harder it is to keep coherent and invariant structures across so many different contexts, such as the channels, services, and devices connected to Plus. For example, in desktop browsers, the Plus site makes it clear when I’m reading within
the place of a specific Circle. So, when I’m catching up with family in my Family Circle (which I defined), any post I create is limited to that Circle. It makes intuitively physical sense—just as if I were in a room with my family and listening to them, I’d expect what I said to be in that room, with those people.

  Contrast the desktop with the architecture of the Mobile iOS version of Plus (Figure 19-3), for which the rules change. A post I make while reading inside a Circle defaults to Your Circles—something easy to miss, especially if I’ve learned the environment from the desktop. Instead, I must go through multiple steps to change my post to be in the Circle that I already think I am “in.”

  The composition of Google Plus—the arrangement and relationships between its parts—does a much better job of making sense than Buzz did, but just this one wayward structure disrupts its coherence. This breaks one of Resmini and Rosati’s heuristics for information architecture: consistency, the capability of an information architecture model “to suit the purposes, the contexts, and the people it is designed for (internal consistency) and to maintain the same logic along different media, environments, and times in which it acts (external consistency).”[386] Consistency isn’t about just the details, but really about the coherence of meaning from one context to the next. It’s another way of attending to invariance for environmental design.