Niantic itself is like a tiny probe rising from the immensity of Google’s mapping capabilities, surplus flows, means of production, and vast server farms as it constructs and tests the prototype of a global means of behavior modification owned and operated by surveillance capitalism. Niantic discovered that in the rapture of engaging competitive social play, the dreaded friction of individual will voluntarily gives way to game protocols that set the conditions for “natural selection.” In this way the game automatically elicits and breeds the specific behaviors sought by the high rollers in Niantic’s behavioral futures markets. With this second game board in motion, the players in the real game vie for proximity to the wake of cash that follows each smiling member of the herd.
In the end we recognize that the probe was designed to explore the next frontier: the means of behavioral modification. The game about the game is, in fact, an experimental facsimile of surveillance capitalism’s design for our future. It follows the prediction imperative to its logical conclusion, in which data about us in scale and scope combine with actuation mechanisms that align our behavior with a new market cosmos. All the flows of surplus from all the spaces, all the things, all the bodies, all the laughter, and all the tears are finally aimed at this triumph of certain outcomes and the revenue that it can unleash.
IV. What Were the Means of Behavioral Modification?
The new global means of behavioral modification that we see under construction at Facebook and Niantic represent a new regressive age of autonomous capital and heteronomous individuals, when the very possibilities of democratic flourishing and human fulfillment depend upon the reverse. This unprecedented state of affairs rises above debates about the Common Rule. It goes to the heart of our allegiance to the ideals of a democratic society, with full knowledge of the challenges that burden those ideals.
What has been forgotten here is that the Common Rule was the product of a similar challenge to principles of individual autonomy and democratic fidelity. It was one result of a deeply contested struggle in which democratically minded public officials joined with social activists, scholars, and litigators to resist the design, development, and deployment of behavioral modification as a mode of governmental power. It was not long ago that US society mobilized to resist, regulate, and control the means of behavioral modification, and it is this history that we can now draw upon to rediscover our bearings and rouse our awareness.
In 1971 the Senate Subcommittee on Constitutional Rights, led by North Carolina Senator Sam Ervin and including luminaries from across the political spectrum such as Edward Kennedy, Birch Bayh, Robert Byrd, and Strom Thurmond, undertook what would become a multiyear investigation into “a variety of programs designed to predict, control, and modify human behavior.” Ervin was a conservative Democrat and constitutional expert who became an unlikely civil liberties hero for his defense of democracy during the Watergate crisis as chair of the Senate Watergate Committee. In this case the Subcommittee on Constitutional Rights would subject the principles and applications of behavior modification to intense constitutional scrutiny for the first time, questioning and ultimately rejecting the use of behavioral modification as an extension of state power.
The Senate investigation was triggered by a growing sense of public alarm at the spread of psychological techniques for behavior control. There were many points of origin, but most salient was the influence of the cold war and the range of psychological techniques and programs for behavior modification that it had bred. The Korean War had publicized communist “brainwashing” techniques that, according to then newly appointed CIA Director Allen Dulles, had reduced US prisoners of war to a state of robotic passivity, in which the victim’s brain “becomes a phonograph playing a disc put on its spindle by an outside genius over which it has no control.”52 America’s enemies appeared to be on the verge of mastering the art and science of “mind control” with psychological and pharmacological methods unknown to the US military. There were reports of Chinese and Soviet achievements in the remote alteration of a subject’s mental capacities and the elimination of his “free will.”53 Dulles committed the agency to rapid research in and development of “mind control” capabilities, from “de-patterning” and “rewiring” an individual to shaping an entire country’s attitudes and actions.54
Thus began a morbidly fascinating and often bizarre chapter in the history of American spy craft.55 Much of the new work was conducted in the context of the CIA’s highly classified MKUltra project, which was tasked with “research and development of chemical, biological, and radiological materials capable for employment in clandestine operations to control human behavior.” According to testimony in the 1975 Senate investigation of covert CIA Foreign and Military Intelligence operations, a 1963 Inspector General’s report on MKUltra noted several reasons for the program’s secrecy, but chief among them was the fact that behavior modification was seen as illegitimate. “Research in the manipulation of human behavior is considered by many authorities in medicine and related fields to be professionally unethical, therefore the reputation of professional participants in the MKUltra program are on occasion in jeopardy,” the report began. It also noted that many of the program’s activities were illegal, violated the rights and interests of US citizens, and would alienate public opinion.56
Key to our interests is the growth and elaboration of behavioral modification as an extension of political power. To this end, CIA “demand” summoned an ever-more-audacious supply of behavioral-modification research and practical applications from academic psychologists. Scientists from the fields of medicine and psychology set out to demystify Chinese brainwashing techniques, reinterpreting them through the established frameworks of behavior modification.
Their research concluded that “mind control” was better understood as a complex system of conditioning based on unpredictable schedules of reinforcement, consistent with B. F. Skinner’s important discoveries on operant conditioning. According to Harvard historian Rebecca Lemov, the “mind control” researchers had a powerful effect on the CIA and other branches of the military. The notion that “human material was changeable”—that one’s personality, identity, awareness, and capacity for self-determining behavior could be crushed, eliminated, and replaced by external control—incited a new sense of panic and vulnerability: “If indeed the world was rife with threats to the inner as much as the outer man, then experts in these realms were needed more than ever. Many good and well-meaning professors—self-described or de facto human engineers—participated in the CIA’s programs to bring about slow or rapid change in the minds and behavior of people.”57
By the time the senators on the Constitutional Rights Subcommittee convened in 1971, the migration of behavior-modification practices from military to civilian applications was well underway. Behavior-modification techniques had dispersed from government-funded (typically CIA) psychology labs and military psyops to a range of institutional applications, each driven by a mission to reengineer the defective personalities of captive individuals in settings that offered “total control” or close to it, including prisons, psychiatric wards, classrooms, institutions for the mentally challenged, schools for the autistic, and factories.
The subcommittee was emboldened to act when public concern bubbled over into outrage at the mainstreaming of these behavior-modification programs. Historian of psychology Alexandra Rutherford observes that Skinnerian behavior-modification practices expanded rapidly during the 1960s and 1970s, achieving some “remarkable successes” but also exposing practitioners to the scrutiny of an often-hostile public. A number of journalistic accounts raised alarms about the zealousness with which behavior-modification techniques were applied and the sense that they debased their subjects, violated ethical considerations, and infringed on fundamental civil liberties.58
Another factor was the 1971 publication of B. F. Skinner’s incendiary social meditation Beyond Freedom & Dignity. Skinner prescribed a future based on behavioral control, reje
cting the very idea of freedom (as well as every tenet of a liberal society) and cast the notion of human dignity as an accident of self-serving narcissism. Skinner imagined a pervasive “technology of behavior” that would one day enable the application of behavior-modification methods across entire human populations.
The ensuing storm of controversy made Beyond Freedom & Dignity an international best seller. “Skinner’s science of human behavior, being quite vacuous, is as congenial to the libertarian as to the fascist,” wrote Noam Chomsky in a widely read review of the book. “It would be not absurd but grotesque to argue that since circumstances can be arranged under which behavior is quite predictable—as in a prison, for example, or… concentration camp.… therefore there need be no concern for the freedom and dignity of ‘autonomous man.’”59 (In the mid-1970s graduate department at Harvard where I studied and Skinner professed, many students referred to the book as Toward Slavery and Humiliation.)
From the first lines of the preface of the subcommittee’s 1974 report, authored by Senator Ervin, it should be evident to any twenty-first-century captive of surveillance capitalism that US society has undergone a social discontinuity more profound than the mere passage of decades suggests. It is worth reading Ervin’s own words, to grasp the passion with which he located the subcommittee’s work at the heart of the Enlightenment project, pledging to defend the liberal ideals of freedom and dignity:
When the founding fathers established our constitutional system of government, they based it on their fundamental belief in the sanctity of the individual.… They understood that self-determination is the source of individuality, and individuality is the mainstay of freedom.… Recently, however, technology has begun to develop new methods of behavior control capable of altering not just an individual’s actions but his very personality and manner of thinking… the behavioral technology being developed in the United States today touches upon the most basic sources of individuality and the very core of personal freedom… the most serious threat… is the power this technology gives one man to impose his views and values on another.… Concepts of freedom, privacy and self-determination inherently conflict with programs designed to control not just physical freedom, but the source of free thought as well.… The question becomes even more acute when these programs are conducted, as they are today, in the absence of strict controls. As disturbing as behavior modification may be on a theoretical level, the unchecked growth of the practical technology of behavior control is cause for even greater concern.”60
The report’s critique of behavior modification has unique relevance for our time. It begins by asking a question that we must also ask: “How did they get away with it?” Their answer invokes the “exceptionalism” of that era. Just as surveillance capitalism was initially able to root and flourish under the protection of a so-called “war against terror” and the compulsion for certainty that it stirred, in the middle of the twentieth century the means of behavioral modification migrated from the lab to the world at large primarily under the cover of cold-war anxieties. Later, the behavior-change professionals of the 1960s and 1970s were summoned into civilian practice by a society turned fearful after years of urban riots, political protests, and rising levels of crime and “delinquency.” The senators reasoned that calls for “law and order” had motivated the search for “immediate and efficient means to control violence and other forms of anti-social behavior. The interest in controlling violence replaced more time-consuming attempts to understand its sources.”
With so many behavior-modification programs aimed at involuntary populations in state prisons and mental institutions, the senators recognized that the means of behavioral modification had to be reckoned with as a form of state power and questioned the government’s constitutional right to “control” citizens’ behavior and mentation. In its survey of government agencies, the subcommittee found “a wide variety of behavior modification techniques… presently employed in the United States under the auspices of the federal government” and observed that “with the rapid proliferation of behavior modification techniques, it is all the more disturbing that few real efforts have been made to consider the basic issues of individual freedom involved and… fundamental conflicts between individual rights and behavior technology.”61
The senators reserved their most vivid rebukes for what they considered as the two most extreme and pernicious behavior-modification techniques. The first was psychosurgery. The second was “electrophysiology,” defined as “the use of mechanical devices to control various aspects of human behavior.” The report notes with special horror the example of “devices” designed to be worn by a subject “constantly to monitor and control his behavior through a computer” and to “prevent a suspected behavior from occurring.”
The First Amendment, the subcommittee argued, “must equally protect the individual’s right to generate ideas,” and the right to privacy should protect citizens from intrusions into their thoughts, behavior, personality, and identity lest these concepts “become meaningless.” It was in this context that Skinnerian behavioral engineering was singled out for critical examination: “A major segment of the emerging behavior control technology is concerned with conditioning, through which various forms of persuasion are used to stimulate certain types of behaviors while suppressing others.”62
In anticipation of future gamification techniques as means of behavioral modification, the subcommittee report also noted with apprehension more “benign” approaches that relied on “positive reinforcement,” from “gold-star incentives” to elaborate reward systems, in order “to restructure personality through artificially applied techniques.” The generalized obsession with controlling violence had also produced methods of “behavior prediction” that “raise profound questions with respect to due process, privacy, and individual liberties.” A psychologist writing in the American Psychological Association’s journal Monitor in 1974 sounded the alert, warning colleagues who touted their ability to “control behavior” that they were now “viewed with increasing suspicion, if not revulsion, and threatened with restriction… The social control of behavior control is underway.”63
The subcommittee’s work had enduring consequences. Not only did prisoners’ and patients’ rights groups gain momentum in their efforts to end the behavioral oppression suffered in public institutions, but psychologists also began to discuss the need to professionalize their discipline with clear ethical standards, accreditation procedures, training programs, and career ladders.64 The National Research Act, passed in 1974, stipulated the creation of institutional review boards and laid the foundation for the evolution and institutionalization of the Common Rule for the ethical treatment of human subjects, from which Facebook famously held itself exempt. That same year, Congress established the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. When the commission published its findings five years later in the “Belmont Report,” it became the professional standard for imposing ethical guidelines on all federally funded research with human subjects in the US.65
The clamorous rights consciousness of the 1970s drove behavior modification from civilian life, or at least dimmed its star. A Federal Bureau of Prisons official recommended that program leaders avoid using “the term ‘behavioral modification’ but to talk about positive reward and reinforcements for the type of behavior we are attempting to instill.” Another said, “We’re doing what we always did… but to call it ‘behavior modification’ just makes things more difficult.”66 Skinner’s 1976 “primer,” titled About Behaviorism, motivated by what he believed were public misconceptions stirred by the wave of harsh reaction to Beyond Freedom & Dignity, failed to capture much public attention. According to Skinner’s biographer, “the battle had climaxed.” The public had made Beyond Freedom & Dignity a bestseller “but had just as surely rejected Skinner’s argument that there were cultural matters more important than preserving and extending individual freedom.”67
Unknown to the senators, scholars, rights activists, litigators, and many other citizens who stood against the antidemocratic incursions of the behavioral engineering vision, these methods had not died. The project would resurface in a wholly unexpected incarnation as a creature of the market, its unprecedented digital capabilities, scale, and scope now flourishing under the flag of surveillance capitalism. During the same years that US democratic forces combined to resist behavior modification as a form of state power, the energies of capitalist counterinsurgency were already at work within society. The corporation was to enjoy the rights of personhood but be free of democratic obligations, legal constraints, moral calculations, and social considerations. Certainly in the US case, a weakened state in which elected officials depend upon corporate wealth in every election cycle has shown little appetite to contest behavior modification as a market project, let alone to stand for the moral imperatives of the autonomous individual.
The Age of Surveillance Capitalism Page 40
