There are other, even more grandiose ambitions for the rendition of all solitary things. Companies such as Qualcomm, Intel, and ARM are developing tiny, always-on, low-power computer vision modules that can be added to any device, such as your phone or refrigerator, or any surface. A Qualcomm executive says that appliances and toys can know what’s going on around them: “A doll could detect when a child’s face turns toward it.”17
Consider “smart skin,” developed by brilliant university scientists and now poised for commercial elaboration. Initially valued for its ability to monitor and diagnosis health conditions from Parkinson’s disease to sleep disorders, smart skin is now hailed for its promise of ultra-unobtrusive ubiquity. Researchers at Georgia Tech developed a version of “smart skin” that sucks energy from radio waves and other energy sources, eliminating the need for batteries. Smart skin, described as “the ultimate sensing tool that could potentially allow for the mass implementation of perpetual wireless networks,”18 can cognize, sense, analyze, wirelessly communicate, and “modify parameters” using simple radio frequency (RFID) technology.19 As in the case of Paradiso’s “sensor tape,” the researchers stress that it can also “be applied everywhere” to “monitor, sense, and interact with the world around us in a perpetual way, thus significantly enhancing ambient intelligence,” all of it as inconspicuous as a “decal sticker.” They suggest, for example, the shelves of grocery stores, where revenue opportunities are plentiful.20
Rendition has become a surveillance capitalist project shaped by its imperatives and directed toward its objectives. In the composition of the shadow text, rendition is Step One: the concrete operationalization of the “original sin of simple robbery” that defined this market project from the start. Google rendered the Earth, its streets, and its dwelling places, bypassing our consent and defying our protests. Facebook rendered the social network and its limitless details for the sake of the company’s behavioral futures markets. Now the ubiquitous apparatus is the means to the ubiquitous rendition of human experience. We have seen the urgency with which surveillance capitalists pursue the elimination of “friction” as a critical success factor in supply operations. The prediction imperative makes boundaries and borders intolerable, and surveillance capitalists will do almost anything to eliminate them. This pursuit transforms “connection” into a commercial imperative and transforms individual autonomy into a threat to surveillance revenues.
Surveillance capitalism’s rendition practices overwhelm any sensible discussion of “opt in” and “opt out.” There are no more fig leaves. The euphemisms of consent can no longer divert attention from the bare facts: under surveillance capitalism, rendition is typically unauthorized, unilateral, gluttonous, secret, and brazen. These characteristics summarize the asymmetries of power that put the “surveillance” in surveillance capitalism. They also highlight a harsh truth: it is difficult to be where rendition is not. As industries far beyond the technology sector are lured by surveillance profits, the ferocity of the race to find and render experience as data has turned rendition into a global project of surveillance capital.
This chapter and the next survey a range of rendition activities in the pursuit of economies of scope. The remainder of this chapter concentrates on extension, the first dimension of scope, as rendition operations move into the real world, seizing fresh unexpected chunks of human experience. Extension wants every corner and crevice, every utterance and gesture on the path to dispossession. All that is moist and alive must hand over its facts. There can be no shadow, no darkness. The unknown is intolerable. The solitary is forbidden. Later, in Chapter 9, we move into the depth dimension. The net is cast wide over the waters of daily life, but there are also submarines exploring the depths in search of new sources of surplus prized for their rare predictive powers: your personality, emotions, and endorphins. The examples in these chapters are not intended to be exhaustive but rather to illustrate the seriousness of purpose, tenacity, and subterfuge with which surveillance capitalists pursue their search for new aspects of human experience that can be monetized as certainty.
In this pursuit, we are necessarily involved in citing specific actors, products, and techniques, knowing that the details of persons and companies are constantly churning. Firms will be bought and sold, fail or succeed; people will come and go. Specific technologies, products, and techniques will be abandoned, refined, and surpassed. When they fall, new ones will take their place, as long as surveillance capitalism is allowed to flourish. Velocity and churn have been critical to surveillance capitalism’s success, and we cannot allow the constant movement to inhibit our determination to grasp the “laws of motion” that command this roiling landscape. It is the pattern and its purpose that we want to grasp.
II. Body Rendition
The rendition of your body begins quite simply with your phone. Even when your city is not “smart” or owned and operated by Google, market players with an interest in your behavior know how to find your body.21 For all of the elaborate ways in which surveillance capitalists labor to render reality as behavior for surplus, the simplest and most profound is their ability to know exactly where you are all the time. Your body is reimagined as a behaving object to be tracked and calculated for indexing and search. Most smartphone apps demand access to your location even when it’s not necessary for the service they provide, simply because the answer to this question is so lucrative.
Location data can be extracted from “geotags” created when your smartphone automatically embeds your identity and location in photos and videos. Retailers use “geofencing” to demarcate a geographical area and send alerts to smartphones within those parameters: “Come here now!” “Buy this here!” “An offer, just for you!”22 Download the Starbucks app, and then leave your house if you want to see this in action. As one marketing consultancy advises, “Mobile advertising, the ultimate form of geo-targeting, is the holy grail of advertising.”23 “Tips and tricks” for location-based marketing are generously offered by a firm that specializes in mobile advertising: “It allows you to tap into people’s compulsive nature by encouraging impulse buys with the notifications you send out.… It also allows you to gain insight on your current customers by reading what they’re saying on Yelp and Facebook.…”24
Another mobile marketing firm recommends “life pattern marketing” based on techniques derived from military intelligence known as “patterns of life analysis.” These involve gathering location and other data from phones, satellites, vehicles, and sensors to assemble intelligence on the daily behavioral patterns of a “person of interest” in order to predict future behavior. Marketers are exhorted to “map the daily patterns” of a “target audience” in order to “intercept people in their daily routines with brand and promotional messages.” As the firm emphasizes, “The psychological power of the perception of ubiquity is profound. Life Pattern Marketing forms a powerful psychological imprint on consumers.”25
You can shut down the GPS locator on your phone, but most people do not, both because they rely on its utilities and because they are ignorant of its operations. According to Pew Research, 74 percent of US smartphone owners in 2013 used apps that required location data, and 90 percent did so in 2015—that’s about 153 million people, more than those who listen to music or watch video on their phones.26 Surveillance capitalism’s reliance on secret operations means that most of us simply do not and cannot know the extent to which our phone doubles as a tracking device for corporate surveillance.
A study by researchers at Carnegie Mellon University proves the point efficiently.27 During a three-week period, twenty-three participants were continuously informed of the number of apps accessing their location information and the total number of accesses in a given period. They were flabbergasted by the sheer volume of the onslaught as they each variously learned that their locations were accessed 4,182 times, 5,398 times, 356 times, and so on, over a 14-day period—all for the sake of advertisers, insurers, retailers, marketing firm
s, mortgage companies, and anyone else who pays to play in these behavioral markets.28 As one participant summed it up, “It felt like I’m being followed by my own phone. It’s scary.”29 Fifty-eight percent of participants subsequently restricted the permissions granted to their mobile apps.
Unsurprisingly, Google represents the vanguard of location-based tracking. A 2016 affidavit from law-enforcement officials seeking a search warrant for a California bank robber made plain why Google location data are unparalleled: “Google collects and retains location data from Android-enabled mobile devices. Google collects this data whenever one of their services is activated and/or whenever there is an event on the mobile device such as a phone call, text messages, internet access, or email access.” The officials on the case requested location information from Google because it offers far more detail than even the phone companies can provide. The location systems in Android combine cell-tower data with GPS, Wi-Fi networks, and other information culled from photos, videos, and other sources: “That lets Android pinpoint users to a single building, rather than a city block.”30 In November 2017 Quartz investigative reporters discovered that since early 2017, Android phones had been collecting location information by triangulating the nearest cell towers, even when location services were disabled, no apps were running, and no carrier SIM card was installed in the phone. The information was used to manage Google’s “push” notifications and messages sent to users on their Android phones, enabling the company to track “whether an individual with an Android phone or running Google apps has set foot in a specific store, and use that to target the advertising a user subsequently sees.”31
Google’s location history system is a product of the corporation’s global mapping operations. Though active for over a decade, it was unveiled to the public only in 2015 as “Your Timeline,” a feature that “allows you to visualize your real-world routines.”32 The corporation calculated that any negative reaction to the volume and persistence of tracking revealed by Timeline would be mitigated by the value of users’ active contributions to their own stocks of behavioral surplus as they fine-tune the information, add relevant photos, insert comments, and so forth. This was represented as an individual’s investment in personalized services such as Google Now so that it might more effectively comb your e-mail and apps in order to push relevant traffic and weather updates, notifications, suggestions, and reminders. Location data is the quid pro quo for these services.
This transaction is greased by the usual promises of privacy and control: “Your Timeline is private and visible only to you; and you control the locations you choose to keep.” But Google uses your location data to target ads; indeed, these are among the most significant sources of surplus in Google’s advertising markets with a direct impact on click rates. The standard account from Google and other surveillance capitalists is that behavioral surplus is retained only as meta-data, which are then aggregated across large numbers of individual users. We are told that it’s not possible to identify individuals from these large-scale amalgamations. However, with as little as three bits of data easily culled from the public record—birth date, zip code, and sex—reidentification science has demonstrated its ability to de-anonymize meta-data with “disturbing ease.”33 In a summary of this research, legal scholar Paul Ohm writes that “re-identification makes all our secrets fundamentally easier to discover and reveal. Our enemies will find it easier to connect us to facts that they can use to blackmail, harass, defame, frame or discriminate against us.… This mistake pervades nearly every information privacy law.” Regarding the massive caches of supposedly anonymous behavior surplus, Ohm calls them “databases of ruin.”34
When it comes to location data, the situation is just as bad. In 2013 a group of MIT and Harvard computer scientists demonstrated that because individuals tend to have idiosyncratic mobility signatures, any analyst with the right tools can easily extract the mobility pattern of a specific individual within a large anonymized data set of location meta-data. Another research team demonstrated that data collected by seemingly “innocuous” smartphone-embedded sensors, such as accelerometers, gyroscopes, and magnetometers, can be used to infer “an ever-growing range of human activities and even moods.” Their work also shows that these sensor data can be used “to obtain sensitive information about specific users from anonymized datasets.”35
Companies are putting these surveillance capabilities to work. Broadcom produced a “global navigation satellite system” in a chip that combines satellite communications with the sensors in your cell phone to create a “positioning engine” that can find your location even if you are not connected to a network, including your location in a building, how many steps you’ve taken, in what direction, at what altitude. All this depends solely on one factor, says a company vice president: “the device in your hand.”36 Princeton computer scientists Arvind Narayanan and Edward Felten summed it up this way: “There is no known effective method to anonymize location data, and no evidence that it’s meaningfully achievable.”37
Even without “de-anonymization,” location meta-data constitute an unrivaled concentration of knowledge within private firms and an extraordinary advantage in the division of learning. In 2016 Chinese search engine Baidu, often referred to as the Google of China, announced that its “Big Data Lab” uses location data from its 600 million users to track and predict the dynamics of the Chinese economy. The company built an “employment index” for the national economy as well as a “consumption index.” It also touted its ability to generate quite-specific predictions such as Apple’s second-quarter earnings in China that year. “To the best of our knowledge,” Baidu researchers wrote, “we are the first to measure the second-largest economy by mining such unprecedentedly large scale and fine granular spatial-temporal data.”38
As powerful as location data are, wearable technologies and their applications are another significant proving ground in the act of body rendition.39 One 2017 report describes a new generation of wearables “armed with more sensors and smarter algorithms… focused on biometric monitoring… and… body parts as conduits for data collection.…” These complex sensors can access “environmental context… smells… emotional state.…”40 Google has developed internet-enabled fabrics, claiming that it aims to bring inductive yarns to every garment and fabric on Earth. “If you can weave the sensor into the textile, as a material,” explains project leader Ivan Poupyrev, “you’re moving away from the electronics. You’re making the basic materials of the world around us interactive.” A partnership with Levi Strauss has already yielded “interactive denim,” including a jacket first brought to market in September 2017. The material is described as able to “infer behavior” in order to be “interactive yet authentic.”41 The jacket contains sensors that can “see” through the fabric to detect and decipher gestures as subtle as the twitch of your finger.
There is a numbing repetition of MacKay’s themes throughout the literature on wearables. Just as he insisted that telemetric devices must operate “outside the awareness” of “unrestrained animals,” today’s developers stress that wearables must be “unobtrusive” to avoid raising alarm. They are to be “continuous,” “pervasive,” and, crucially, “low cost” in order to achieve economies of scope.42 The digital marketing firm Ovum forecasts 650 million wearables by 2020, nearly double the number used in 2016, and its research suggests that growth is largely driven by the lure of surveillance revenues. Mobile advertisers, they report, see wearables as “a source of very granular data insights and also new types of behavioral and usage data. Wearables of the future will have the ability to capture a wide array of data related to a user’s contextual activity, health and emotional state. This information can be used to enhance and tailor both products and marketing messages to a very high degree.…”43
Health care is an especially active proving ground for wearable sensored technologies, a development that’s particularly pernicious given the more-innocent origins of this idea. When
telemetry first shifted from MacKay’s gaggles, flocks, and herds to the human animal, one of its first applications was as a means to oversee the vulnerable in the form of push-button pendants for elderly people alone in their homes. In 2002, the year that a still-secret surveillance capitalism achieved its first breakthroughs, a review of “wireless telemedicine” stressed the value of home monitoring for the elderly and for the expansion of health services in remote areas. As in the case of the Aware Home, a diagram of the proposed digital architecture for such home-monitoring services features only three parties: a closed loop that exclusively links a patient at home, her hospital’s servers, and her physician.44 There are no extra parties imagined in either of these designs, no companies capturing your behavior, no behemoth tech firms with their porous platforms and proprietary servers transforming your life into surplus so they can make book on what you will want next and enable their customers to sell it to you first.
Before the birth and spread of surveillance capitalism, it was possible to imagine digital representations of your body as an enrichment of the intimate relationships between a patient and a trusted doctor, a mother and her child, elderly parents and their adult children. As surveillance capitalism overwhelms the digital milieu, that vision has been made ridiculous. Both the Aware Home and the telemedicine design assume that all behavioral data are reinvested in service to the human being who is the subject of these arrangements, providing serenity, trust, and dignity: a chance for real knowledge and empowerment.
Many articles on health monitoring continue to emphasize its utility for the elderly, but the conversation has decisively moved on from this earlier state of grace. Some researchers anticipate the fusion of “smart cities” and what’s now called “m-health” to produce “smart health,” defined as “the provision of health services by using the context-aware network and sensing infrastructure of smart cities.”45 Toward that end, there are now reliable sensors for rendering an increasing range of physiological processes as behavioral data, including body temperature, heart rate, brain activity, muscle motion, blood pressure, sweat rate, energy expenditure, and body and limb motion. There are sensors that can render audio, visual, and physiological data during postsurgical patient recovery and rehabilitation. A flexible, sensored textile patch has been developed that can render breathing, hand movements, swallowing, and walking as behavioral data. In other applications, “wearable micromachined sensors” provide “accurate biomechanical analysis” as you walk or run, and a “body area network” records and analyzes walking and running “under extreme conditions.”46
The Age of Surveillance Capitalism Page 30
