Pixels and Place

by Kate O'Neill

  Still, no doubt there are digital art projects and “galleries” that merit the designation of being curated, but for general purposes, the pictures most of us are adding to our Pinterest accounts are only connected to the core idea of curation in the loosest possible way.

  I don’t find that troubling, and neither should you. After all, re-pinning pictures of sidewalk cafés in the Left Bank to my “Paris” board in Pinterest is no more removed from art museum curation than browsing on the web is from idly walking through a shopping center with no intent to buy.

  When Online Goes Offline

  It hasn’t happened as much yet, but some metaphors have gone the other way: Ideas about how we experience digital life have started to trickle back over into a more physical/spatial usage. The connected dimension of our digital lives is separate enough from our physical lives that we understand the metaphor of “disconnecting” ourselves from the digital stream so that we can fully experience the physical side. The metaphorical notion of “offline” as the opposite counterpart to “online” has colored our understanding of context. We talk about “going offline” or taking a discussion “offline.”

  People occasionally speak hashtags aloud and use other internet memes colloquially. I hear people say “hashtag-first-world-problem” in informal social settings, and occasionally hear other spoken variants of internet cultural memes. Similarly, I hear people jokingly say “Delete!” when referring to something they want to stop.

  URLs appear everywhere, on ads, in stores, in the dentist office; and social icons show up everywhere, too, although this is often a rather funny faux pas. The icon in print, without the context of the username or profile of the given social network, is fairly useless except as a general reminder to a would-be customer or fan that the entity exists on that social network, and the customer or fan should search them out to engage with them. As difficult as it is to acquire interested prospective customers and fans, that’s a lot to ask, and it’s not likely to happen in most cases.

  It’s increasingly common to see hashtags promoted as part of user generated content campaigns within brand advertising, such as #ShareACoke. And in turn, these can become promoted trends, often used in offline advertising such as billboards, bringing the convergence of this online content with its offline context and online promotion full circle and then some.

  As yet, though, the metaphors of the digital world haven’t made as much impact on physical placemaking. The physical world, though—the world we’ve all known all our lives—has been consciously and unconsciously woven into every dimension of online space.

  Maps: Mapping Physical Place Versus Maps of Experience: Empathy Maps, Journey Maps

  Journey maps, strategy maps, empathy maps: We seem to rely a great deal on the notion of maps for guidance. Perhaps that’s understandable given their history of guiding us through space. But clearly our relationship with maps extends beyond the cartographic and into the abstract conceptual.

  Even maps that describe the physical world differ in their approach to fidelity to spatial relationships versus system relationships. Try navigating the New York City subway using a street map and you may well find it challenging, whereas the subway system map removes the context that isn’t needed—the street grid and above-ground landmarks—and places emphasis upon what matters in the moment for the subway rider. But likewise, if you try to navigate the streets of New York City on foot using a subway map, you’ll be at a loss for necessarily detail.

  A successful map shows the relationships between points in a system using only the context layer that matters. It’s a great example of what I call “relevance as a form of respect”: Any more detail than needed could bog down the viewer or detract from the clarity of the relationships that matter.

  So the scope of, say, a strategy map is only the relationships between elements needed to successfully understand the strategy. It doesn’t need the details that matter at the tactical implementation level; those would only confuse matters.

  Breadcrumbs

  There’s another interesting metaphor used in digital that is borrowed from the physical world, and it relates to this discussion: breadcrumbs.

  What we’re referring to culturally when we talk about breadcrumbs is the fairytale “don’t get lost in the woods” sense. Which is also a pretty relevant idea when you think about someone navigating an unfamiliar website. The further and deeper they get into it, the more helpful it can be to have a way to trace their way back out. Yet we also know that digital experience is not a linear footpath, and it does not look like a carefully laid out journey map.

  Those journey maps of our websites and mobile apps and digital presences and so on are really just flattened abstractions of a complex relationship of ideas, anyway, with access points we don’t always plan for; and they are often presented in language that represents our thinking, not the visitor’s.

  What breadcrumbs can help us do is to be mindful of the humanistic element of context and how a user is coming into that experience, and what additional context of physical space or of background or of desire or confusion or whatever else is going on potentially in that person’s real life as they interact with your flattened abstraction of how you want the thing to go down.

  ***

  A customer journey map is an abstraction of the encounters and interactions a customer has with a brand—from awareness through some desired post-purchase state, such as referral, renewal, or retirement from the process. It’s a figurative overlay of digital (and sometimes physical) experience, presented through the physical metaphor of linear progress through a journey. That journey is presented in an idealized sequence because to try to account for the variations on every customer’s experience would render the map useless.

  Speaking of maps, it’s also interesting that when we look at GIS data, we’re looking at a literal overlay of data and place.

  When you have sensors connected to machinery, or even to discrete parts of machinery, you have the potential for vast amounts of data that can be used predictively to determine when equipment might break down, but with visualization on a map, you also get geographical patterns of usage. Departments of Transportation (DOTs) in states like Iowa have equipped their snowplows with sensors and communication equipment, and they departments can keep tabs on the fleet relative to weather forecasts so that they can be most prepared, effective, and efficient with salt application and snow removal.39 They’ve even made that data publicly accessible in their Track a Plow website, which lets drivers know which roads have most recently been cleared.

  Image source: http://www.transportationmatters.iowadot.gov/

  There are many ways maps can guide us through experience, and in our exploration of integrated experiences, they route together several relevant concepts: directions and navigation, as we discussed earlier in “Traffic, Cost, and the Nuance of Metaphor,” GPS, as discussed in “Senses in Place; Sense and Technology,” and how we give directions, as we’ll discuss in “Humanlike Nuances.”

  And because of what maps demonstrate about context and layered experience of place, they also lead us perfectly into the next discussion: an examination of how augmented reality affects our experience of place.

  CHAPTER EIGHT

  Reality, Virtual Reality, Augmented Reality

  Nowhere do the layers of pixels and place come together more than in the realms of augmented reality and virtual reality. The two are not really the same, but they deserve consideration together because they both pertain to the sensory experience of a “place” that doesn’t exist without the aid of technology.

  Augmented reality and virtual reality both pertain to the sensory experience of a “place” that doesn’t exist without the aid of technology.

  Augmented reality is where you can still see your natural surroundings but you are also interacting through a screen layer, often transposed through a camera view of the surroundings. In other words, you might be holding your phone up in front of you, looking at y
our living room, and on your phone screen you see your living room with a chair you’re thinking about buying.

  Pokémon GO is augmented reality. IKEA has an augmented reality app for its catalog. Google Glass was an augmented reality wearable device.

  What distinguishes virtual reality from augmented reality, primarily, is how immersive the experience is. At the moment, virtual reality is mostly experienced through goggles or a headset, sometimes with attached earpieces for accompanying audio, and is presented with alternate surroundings. Most VR experiences focus on immersion of sight and hearing, but some also involve gloves or other hand-related wearables that deal with your sense of touch, using vibrations and other haptic feedback. But augmented reality is basically a screen layer over the physical world, and it can be achieved in a wide range of ways.

  Google is even experimenting with blended virtual and augmented. It has the potential to create amazingly rich experiences. In some cases, they may be experiences that we as users/customers/visitors can feel more in control of. In the meantime, there are already some worthwhile integrations of the physical and digital experience through both augmented reality and virtual reality, which we’ll examine in the following sections.

  Experiencing Place with Augmentation

  We’ve established that the meaning and experience of a place is at least partly dependent on sensory experience. Since augmentation affects our sensory perception (at least what we see, and maybe other senses in the future), then to what extent is the meaning of that place altered by the augmentation?

  How does the experience of a place change when it’s augmented? What characteristics does a place take on, and how does it affect the meaning of that place?

  The experience of a place through augmentation changes the sense of the place. Augmentation fundamentally changes that sense and potentially also changes the function of the place by adding a functional or descriptive layer over top of the physical layer. That can be very helpful in interpretation, museums or monuments.

  Augmentation is starting to be used a great deal in retail, museums, and games (see the next section on “Augmented Reality Games”). It’s also growing within branding and advertising, mainly through apps like Blippar, bringing ads into a physical experience.

  Augmented Place and Community

  You may be connected to other people who are likewise experiencing the surroundings in an augmented way, but you may be disconnected physically.

  On the other hand, augmented reality is typically experienced through a camera lens or viewfinder that does not obscure surroundings, so as opposed to virtual reality, the user of the technology is not disconnected from other people who are not connected through the same technology.

  Augmented Place and Identity

  As you become part of the place for other people, your presence may be augmented as well. If you are able to understand what you might have in common with people in your vicinity, and they may be better able to understand what you might have in common with them, then your augmented identity overlaps the augmented experience of the place. There are, of course, privacy considerations and concerns with augmenting experience to include personal information, and the full impact of those considerations is just coming to light.

  Augmented Place and Culture

  Augmentation, in a sense, enhances the metadata of place, surfaces otherwise invisible connections. This can amplify cultural and historical artifacts and connect people who are viewing it in to others in dimensional ways across time and space.

  As I wrote in an earlier section, “Chapter Five - The Meaning of Place,” when we talk about meaning, we are looking for significance: “In thinking about what place means, we are looking for clues about what a place conveys into context, what signifiers it brings with it.”

  Sometimes significance is subtle, and it may take something external to make us realize what is significant about a place. When our perception of place is augmented by a layer of technology, the significance may be brought to the surface. The augmentation can expose a layer of significance and connectedness of surroundings that it might have taken us a long time to find.

  If we were to experience a park, for example, with an augmented reality app that can identify types of trees, wildflowers, rocks, and other nature, might it surprise us with how much biodiversity surrounds us? Might we be struck by that, and be more likely to feel protective of it?

  Or say we experience a retail store with the assistance of an augmented reality app or device. AR technology could be very helpful in guiding shoppers to the items they’re looking for, or offering them alternatives based on their known history and patterns as well as inventory and availability. If the technology helps us discover a product we love, might it increase our loyalty to the store where we found the product?

  Augmented Reality Games

  As I write this, Pokémon GO is the biggest thing taking over the internet. The internet is awash in opinions about every aspect of the game, its growing player base, and its social implications.

  In the first few days after it was released, there were stories about people complaining of sore calves because they did so much walking. Pokémon GO did what several generations of wearable fitness trackers failed to do: It got people outside and moving around.

  It’s no surprise that a gamification approach to getting people to move dramatically amplifies the effectiveness. But that probably wasn’t exactly Niantic’s strategy with Pokémon GO, either. Their strategy was presumably to build on the platform that had already been being built with Ingress and other apps and wrap a popular game around it. Pokémon GO just fit.

  What’s more interesting about it, though, is the sweeping changes it brings with it: new marketing models, opportunities with augmented reality, location-based marketing, and assorted issues with data privacy and security. The most interesting things about Pokémon GO have nothing to do with the game itself and everything to do with how different things are becoming and are continuing to be.

  Connected Experiences Bring New Marketing Models

  The marketing models are poised to be totally different now that an online interaction can be credibly and consistently traced to offline visits in stores. See the McDonald’s deal with Pokémon GO to make all three thousand of its Japanese stores “gyms” in the game. The full details of their deal haven’t been disclosed, but one option this presents is an incredible opportunity for cost-per-visit modeling.

  Connected Experiences and Social Interaction

  The social experiences are different with augmented reality, when interacting with a digital experience doesn’t automatically mean being oblivious to the world around you. (Although obviously it still can. See, for example, the guys who fell off a cliff while playing Pokémon GO, or the person who drove into a cop car.)

  But since you can engage with the game through a camera view of what’s ahead of you, it’s actually possible to walk and play and still be at least somewhat connected to your surroundings.

  Augmented Reality Games and the Experience of Place

  It’s also interesting how AR gaming differs from what came before it. A history of gaming is beyond the scope of this book, but even glimpsing at some of the dominant trends in the industry shows a progression that centers increasingly on the experience of immersion in place. From the top-down view of Pac-Man and the side-scrolling view of Mario Bros., first-person perspective games like Doom and Halo have emerged. These latter games create an expectation of being immersed in the game’s world, which increasingly feature rich textured landscapes.

  And with augmented games, as opposed to a static escapist experience, we are offered an even more immersive experience that integrates our natural environment into a built partially artificial reality.

  Connected Experiences . . . and Your Business Strategy?

  This is only the beginning of what’s to come.

  People have been laughing at the businesses who are developing Pokémon GO strategies, but honestly, eve
n they are a little late to the opportunity. The gold rush happened during the first two weeks, even if the game’s popularity continues to grow. And if your business is still laughing, you’re missing out on time to think about how augmented reality and connected experiences stand to change the status quo.

  I’m not saying to rush out and do something specific to Pokémon GO that has no alignment with your customers’ motivations or your brand. This is a call for strategic action about a macro trend, not mindless reaction to a micro trend. Trying to capitalize on the trend without strategy will probably come across as an attempt to manipulate the moment, but there’s enough transformation taking place that there will be a relevant, meaningful way to make these opportunities align with your brand and your customers. Your job is to try to catch it.

  Augmented Reality Shopping Experiences