Understanding Context

by Andrew Hinton

  Symbols

  Symbols are different, in that they are signifiers that could mean anything, but have specific meaning only because of conventional usage in a given culture. We know what they mean only by experiencing their use in a system of other symbols and contexts, such as the “heart” symbol in Figure 8-6.

  Figure 8-6. A gesture forming the symbol of a “heart” for “liking” something in the visual field of Google Glass (from Google’s 2013 patent)[179]

  Back to our example of the stairs: I can point at a stairway in answer to the question, “Where are the stairs?” and be performing an indexical sign. Or I might see a picture that iconically represents the form of stairs, giving a clue that stairs can be found near the picture.

  But if I say the word “stairs,” there is nothing in that phonetic sound that is intrinsically connected to an actual stairway; it has no uniquely bound connection other than through general agreement. Other languages don’t have to use the same words used in English; they have their own conventional combinations of different sounds, as shown in Figure 8-7.

  Figure 8-7. The symbolic written signifiers for stairs, from English, Greek, Korean, and Urdu, respectively (captured from Google Translate)

  The essence of a symbolic sign is that the signifier is essentially arbitrary. Any sound could have ended up as our term for stairs. Even within the same language, we have synonyms: “steps” or “stairway” or “flight,” as in, “I ran up 10 flights to get here!” That’s not to say that language is just generated randomly, like rolling dice. Etymologies often point to onomatopoeic origins, imitating the sounds of nature, such as “buzz” or “zap” or even “scream.” But most symbolic language is unmoored even from these tenuous literal connections.

  The Superpowers of Symbols

  Symbols are what make language so terrifically powerful: they can be variables that we can use with great flexibility to create new sorts of meaning. Here are just a few of the superpowers symbols give us:

  Evocative expression

  The symbolic mode gives us the capability to use words in evocative, novel ways as similes and metaphors. This can make communication much more efficient: if I say I saw a truck “jackknifed” on the freeway, it conveys a lot of meaning about the truck’s folded state that I would have otherwise needed to explain in detail. Symbols can also help us be more descriptive with language: I can describe an encounter with an attorney by saying, “That guy negotiates like a bulldog. I hate it that I got ripped off, but I swear I felt like I’d brought a banana to a knife fight.” Of course, there were no actual animals, ripping, fighting, fruit, or sharp objects involved in this situation. If all those words were confined to their more literal meanings, such a sentence would be impossible, and its replacement would be far less expressive.

  Categories

  Because they’re often used to point at physical objects, categories can still have an indexical flavor, but they’re freed from the tethers of specific reference. This allows us to combine concepts with other concepts. The word “mango” can mean the type of fruit rather than a specific mango I’m holding in my hand.

  Categories have allowed us to organize the world around us to a scale that would’ve been impossible otherwise. The categorical label of an item makes it possible for us to find it more easily, but it can also change the way we perceive and understand that item, for good or ill.

  Concepts

  Symbols also allow us to work with concepts that are not categories of physical referent-objects but are new things that exist only in the realm of language and ideas. This might be the most super of our symbolic superpowers; we can create whole new environments of language-based things that we can use to solve problems, imagine new possibilities, and establish social agreements, boundaries, and expectations. In other words, symbols are the means by which we create new contexts out of nothing but language.

  Reification

  When we reify, we take an abstraction and treat it as if it were singular and concrete. For example, we often speak of “the economy” as if it were a monolithic object we could go and touch, measure, and understand in all its dimensions. But the economy is actually made up of many millions of transactions that don’t necessarily have anything to do with one another. We do the same with concepts such as “nature,” “the media,” or “society.”

  Reification is a sort of cognitive quirk that allows us to understand, communicate about, structure, and act upon the world with categories and symbols rather than having to ponder every scattered facet every time. It’s partly responsible for our ability to have language at all.

  Still, like so many of our cognitive superpowers, it can also have a downside as a cognitive fallacy (called, you guessed it, the reification fallacy). We often confuse symbols or models we have created as the reality they represent. Treating the “economy” like it’s a singular thing can lead us to assume one part of it, such as the stock market, is the whole, to the detriment of all the other factors that contribute to a healthy economy, such as infrastructure, education, and healthcare. Likewise, treating “design” as a single entity with solid boundaries results in endless debates among designers online and elsewhere.

  Or, take the Facebook Beacon example: here was a situation in which the idea of “friend” was reified by Facebook to mean just one narrowly defined entity. (In fact, reification is a term of art in computer science for doing pretty much this very thing.) In fact, “friend” is not unlike the idea of “the economy” or “work”—when you try pinning it down with any precision, it dissolves under your fingers, because “friend” is an abstraction we use as shorthand to describe many different sorts of relationships in many different contexts. But defining that abstraction so narrowly resulted in collapsing the contexts of many real-life relationship dimensions to a single point.

  Reification is inevitable; it’s a core dynamic in how humans make meaning. So the trick is to pay attention to where it happens and ensure that it’s establishing meaning in the way we need it to.

  Signification Conflation

  The modes of signification help us understand how the artifacts we design can mean something to users. But in everyday life, we and our users conflate these modes all the time. We use language so naturally, with so little conscious thought, that we tend to use symbolic signs as if they were indexical or iconic.

  In actual experience, we tend to reify symbols as real things because it’s quicker and easier. This is crucial to remember when we think about how we experience context: people make do in the world by satisficing. We reach for whatever tool, object, method, or mode of understanding that will get the job done with the least effort. This isn’t laziness so much as an evolutionary feature of all natural things.

  This is why scholars until the sixteenth century tended to think of words as copies of what they signified; it seems obvious in the same way that it seems obvious that the sun orbits the earth. In daily life, we talk about a “sunrise”; we don’t say, “Let’s go out to the beach and watch the earth turn!” It’s only through thinking objectively about the whole system that we understand how something is actually working, rather than how we immediately perceive it. But that’s a lot of work. In the systems we design, it’s the designer’s responsibility to do that work as much as possible, so the user doesn’t have to. We want our users to be able to say “sunrise,” even if it doesn’t accurately reflect the “business rules” of how the solar system actually works. It wasn’t the users’ responsibility to comprehend the complexities of Beacon; and it wasn’t my responsibility as a user of Google Calendar to comprehend that others in my company could see some parts of my calendar but not others. It was the system’s responsibility to make its environmental invariants clear to the user.

  Understanding signification can help us work through everyday design issues, because digital interfaces are essentially language constructs, wherein one of the most important tasks is to disambiguate the meaning of symbols. For example, in the version
of the Zimbra email interface presented in Figure 8-8, it can be hard to determine which of the red X icons one should click, if trying to do so in a hurry.

  Figure 8-8. A version of the Zimbra email interface, showing an example of an email detail view

  If you look at the interface closely enough and consciously ponder the layout and labels, it’s not too hard to figure out. The nested position of each X icon gives clues to its meaning, but that’s only when we pay explicit, deliberate attention. But in the heat of the workday, I found myself deleting or closing when I meant to decline an invitation, causing confusion between me and coworkers.

  What was especially frustrating was that I didn’t get better at this over time, I actually got worse. Why?

  My cognition’s “loop of least resistance” bends toward needing to learn an environment to the point at which I can do most or all of the actions tacitly, especially if it’s an environment I use a lot. Yet, this interface has invariant structures that look too much like one another, keeping me from being able to just mindlessly click the correct object. This environment resists learning, keeping the user in a frustrating oscillation between tacit and explicit action.

  In nature, things can masquerade as other things, but it’s done for deception, allowing prey to better hide from predators, or for predators to fool their prey. Most flowers are actually flowers, not fly traps; most tree branches on the ground are wooden, not a snake. Our perception-action loop works best, and with the least confusion and explicit effort, when the environment allows us to conflate signifiers without having to solve referential puzzles.

  Language Is Contextual

  The contextual nature of language is something that makes semantic information fundamentally different from physical information. Language works entirely because of the context of what is said in relation to everything around it. We understand what a word means because we understand its relationship to its sentence, or the surface on which it is inscribed, or the character of the person talking. And, just as with our early diagrams of contextual relationships in Chapter 1, everything “around” a word also depends on context for its own meaning. We define words with other words or pictures of the things they name.

  Shouting “fire” in a crowded theater is not the same as shouting “fire” during a military engagement. The phrase “Meet me on the bridge,” spoken by Star Trek’s Captain Kirk has a particular meaning involving space adventures and ship commands; but if spoken by Jimmy Stewart’s character in the movie It’s a Wonderful Life, “meet me on the bridge” would involve dark emotions and whimsical angels earning their wings.

  Language isn’t just about words or pictures. It consumes much of what it touches, including many of the objects and surfaces in our environment. Take, for example, a simple object like the name tag in Figure 8-9.

  If you see my name on a sticker affixed to my shirt, you can surmise that name is labeling me, not because of any intrinsic connection between the name and myself but through other means. There seems to be a direct indexical relationship between the written name “Andrew” and myself, because the name is “on me.” It’s one of the most straightforward ways we use language in everyday life. Yet, even that seemingly direct connection is constructed of many other elements in our surroundings:

  Figure 8-9. Hello, my name is Andrew

  The sticker is a common object for displaying one’s name at particular kinds of social functions at which strangers tend to meet and are expected to call one another by name.

  The sticker’s layout fits a convention in which “Hello my name is” is pre-printed on the object, and participants write in their names (or the host runs the stickers through a printer, if neatness and RSVPs are high priorities).

  If the “name” part of the sticker said “cantaloupe” and not “Andrew,” it would be ironic, because cultural convention dictates that name tags show people-names, and most people-names aren’t lowercase categories of melon.

  At some gatherings, the cultural norm is that only newcomers wear temporary sticker name tags, which adds more context to the meaning of the object. Even these cultural norms are largely established through language use and then embodied in active ritual.

  Here’s the bottom line: a name written on a person’s surface does not always mean that is the person’s name. Wearing a T-shirt that says only “Bob Dylan” on it would more likely be a minimalist concert souvenir. The words-on-person construct needs a lot more context in order to even mean something as simple as “this is my name.”

  Yet, every time I see someone with a name tag, I don’t have to calculate all of this explicitly. It’s a learned feature of my environment. The name-tag object is an invariant cultural convention, learned in a system of other signifiers, just like language itself.

  The Facebook “Like” is an example of a decontextualized gesture that lacks the clues I mentioned for the name tag. It’s a blunt instrument that can mean almost anything depending on what is being liked, by whom, at what time, and with what commentary. Yet, complicating it with more choices would certainly constrain user expression more than help it. In that sense, the “Like” works the way any language does.

  As we will see in other examples, much of the human-made environment makes sense only because of the language that stitches its meanings together. Our physical-information environment is contextually transformed by the way we communicate about it.

  * * *

  [172] Chandler, Daniel. Semiotics: The Basics. New York: Taylor & Francis, 2007:71.

  [173] Wikimedia Commons: http://bit.ly/1xay75J

  [174] For clarity’s sake, we are skipping a lot of background here about the different schools of thought behind signification, particularly the difference between Saussure’s dyadic approach and Pierce’s triadic approach, and plenty of other important details. Beyond our scope here, these are still valuable details that I recommend curious readers investigate, because they help us think more rigorously how we design for meaning.

  [175] http://www.aiga.org/symbol-signs/

  [176] Wikimedia Commons: http://bit.ly/1yufIn2

  [177] Wikimedia Commons: http://commons.wikimedia.org/wiki/File:Unclesamwantyou.jpg

  [178] Photo by author.

  [179] https://www.google.com/patents/US8558759

  Chapter 9. Language as Infrastructure

  If I feel physically as if the top of my head were taken off, I know that is poetry.

  —EMILY DICKINSON

  Language and the Body

  LANGUAGE IS PHYSICAL. WHEN we speak, we’re using our bodies to breathe and create the sound vibrations for articulation, not to mention gesturing and “body language.” When we write, we add to the environment physical information that we assume a reader will interpret.

  Broca, the French physician for whom the language center of the brain is named (Broca’s area), argued that we have “not a memory of words, but a memory for the movements necessary for articulating words.”[180] We don’t recall language-expression as a disembodied set of abstracted concepts; it is rehearsed bodily action, eventually internalized over time. Recent research has demonstrated Broca’s argument: the more scientists observe the neural mechanisms at work when we use language, the more they find that language and bodily action are not separate systems, as once assumed, but part of a single connected system.[181] Even when we read silently, our bodies are firing neurons that we use when reading aloud.[182] This subvocalization has been used by NASA in new technologies with which users can give commands without having to literally say them.[183]

  There is mounting evidence that sophisticated, symbolic language has been with humans since before we were Homo sapiens and that it has been a factor in shaping the evolution of our species. Patients with brain injuries to the main language-related areas of the brain are often still able to relearn language, an ability requiring extreme redundancy in brain structures that takes millions of years of evolution to develop.[184] Anthropologists have established a strong connection b
etween language and the sophistication of complex tools and weapons; and they have discovered evidence of such tools from many thousands of years before Homo sapiens emerged 200 thousand years ago.[185]

  Like those ancient tools, language is something we add to our environment to extend our abilities. But, as established earlier, the meaning of language depends on context and convention. As ecological psychologists Sabrina Golonka and Andrew Wilson explain, “Conventions can change and so can the meaning of words; language is much less stable than [physical-information] perception.”[186] We will keep coming back to this unstable nature of symbolic language, because it’s central to how this kind of information adds ambiguity to context.