by Nick Yee
CHAPTER 9 INTROVERTED ELVES, CONSCIENTIOUS GNOMES, AND THE QUEST FOR BIG DATA
Whether it’s guilds or raids, online gamers constantly face sudden and stressful decisions. And the decisions they make would seem to be very character revealing. Gaining access to such in-game information, however, is another story. When World of Warcraft launched in 2004, Blizzard, the game’s developer, allowed players and third-party developers to modify interface elements via a scripting language. Of course, players could not make their characters indestructible or create a sword that killed anything it touched, but these addons modified the game interface in interesting and important ways. Some modifications added new buttons and charts on players’ screens; others worked silently in the background. There were healing helpers that automatically targeted and cured players who had been poisoned. There were combat add-ons that tracked the average damage output of each team member. There were quest helpers that plotted the most efficient route to complete your current quests. And there were auction helpers that identified and snagged good deals for you in the auction house. None of these add-ons changed the rules or mechanisms of game play, but they provided visibility into the large number of variables being tracked by the game. In order to work, these add-ons had to access Blizzard’s server-side data. And this is how the balance of data accessibility began to shift: for the first time, a game company allowed public access to server-side game variables.
What data researchers could access, however, was severely limited. For example, combat helpers could track data only from players in their group. Add-ons could access data only if the character was logged into the game. And add-ons could never access data outside the server the character belonged to. So in theory, everyone’s combat data were accessible, but in practice, an add-on had access to only a tiny slice of those data.
In 2005, three research scientists at the Palo Alto Research Center—Nicolas Ducheneaut, Bob Moore, and Eric Nickell—had created a research group studying social interaction and communities in online games called PlayOn. They had created a census add-on tool for World of Warcraft. The tool used the “/who” command in the game to return a list of characters who fit certain search parameters. This in-game command let players search for active characters on their server by using a name, name fragment, character class, or character level. Among other things, the command allowed players to search for potential teammates nearby. Instead of searching for a specific character, the PARC census tool returned a list of all active characters on the server. The parameters you could use in searches were also data that the command returned. Thus, the census tool gathered seven variables from every character currently logged on: name, level, class, race, geographical zone, name of guild (if any), and whether the character was currently grouped with other players. The PARC team gathered data from five separate servers at fifteen-minute intervals, twenty-four hours a day, seven days a week. Altogether, the census tool recorded data related to over two hundred thousand unique characters.
In the spring of 2005, after the PARC PlayOn group had accumulated about three months of data, they began looking for a summer intern to help with data analysis. It was a perfect fit for me. I had been playing World of Warcraft since its launch, had a few years’ experience analyzing large data sets from online gamers, and lived eight minutes away. At the time, we four felt that the data set would make a perfect summer internship project. Even though the data set gathered was vast, there were only seven variables. Our collective intuition that summer was that the interesting findings from the data set would be exhausted in less than three months (the length of my internship), giving us time to write and submit a paper to a conference.
Two years and four papers later, we at last ran out of interesting findings. After the summer internship, I worked part-time at PARC as a research assistant for another two years. We hadn’t originally realized that a valuable eighth variable, time, was also being tracked in our growing data warehouse. Not only did we have snapshot data of the seven variables returned by the “/who” command, but because we tracked the servers continuously, we could reconstruct longitudinal profiles of individual gamers—calculating, for example, the speed at which they leveled up. With this metric in hand, we could examine whether a preference for grouping or being in a guild affected a character’s rate of advancement.
In addition, the variable of time allowed us to construct social network graphs of guilds. Every time we saw two characters from the same guild in the same zone (outside the major cities, of course) at the same time, we assumed that they were working together. We would increase their connection weight by 1. Using data spanning thirty days allowed us to create network graphs of guilds—the frequency of interactions between members of that guild. These graphs allowed us to identify the best-connected characters in each guild—the information brokers who bridged different cliques—as well as to quantify how cohesive or fragmented a guild was. In one of our papers, we explored whether we could predict a guild’s survival in six months based on its current guild metrics. We found that several of the top predictors were related to diversity. Certainly, a large guild is more likely to survive than a small guild, but what also mattered is having cohesive cliques spread across the range of levels. Because players are constantly getting tired of the game and quitting, the key to a guild’s success is in being able to fill those vacant positions rapidly. A guild with only high-level players has difficulty dealing with the turnover, whereas the presence of several cohorts leveling up guarantees long-term guild stability. A vacant slot can be filled with someone who is already familiar with the guild’s culture and leaders.1
The Magic Box
This early work in World of Warcraft data—even with just eight variables—made me realize how special virtual worlds are for social science research. The game can record everything you say, everything you do, and everyone you’ve talked to and interacted with, no matter where you are in the virtual world. There is nothing near this degree of surveillance in the physical world.
But a more important and subtle point is that many behaviors are already instrumented in a virtual world. Even if we could record every person’s behaviors in the physical world, tracking everyone with a video camera, we would still need additional algorithms or manual coding to interpret the recorded behaviors. In an online game, the ambiguity is reduced. Attacking or healing in a game requires specific commands and buttons. In the physical world, someone would need to decode behavior manually: Was that friendly wrestling or a vicious assault?
And all this tracking happens continuously, whenever a player logs into a virtual world. When we run an experiment in a laboratory or ask people to fill out a personality survey, we get snapshot data—how they behaved or what they thought in one moment in time. In a virtual world, we can generate rich longitudinal profiles of how people behave over weeks and months. There are two other important differences between virtual worlds and our current tools such as laboratories and surveys. First, we’re not asking people how they might behave in a hypothetical situation or to self-report their preferences. Instead of taking their accounts at face value, we can track their actions directly. Second, unlike being in a psychology lab or filling out a survey, there is no researcher scrutinizing the participant’s every action. Most players are not conscious of the game’s unobtrusive tracking system because it occurs entirely behind the scenes, silently.
There is nothing in the standard toolset of psychology that even approaches the ability of virtual worlds to gather data on human behavior. Instead of being limited by our ability to bring a few dozen (coerced or bribed) undergraduate students into a physical lab room, we now have access to longitudinal behavioral data from hundreds of thousands of people. And although online gamers are not fully representative of the general population, they are much more diverse than undergraduate students from introductory psychology classes.
But even in 2006 with the PARC PlayOn data and four published papers, we were still quite far from this
vision. By relying on serverside data collected with our census add-on, we had information only about characters, not players. We knew nothing about the gender, age, or personality of the players behind the characters. Even though we knew that most players had multiple characters, there was no way to determine that player-character mapping from the server-side data alone. There was also no easy way to survey the players we were tracking because we didn’t have their contact information, and it would have been too time-consuming (and suspicion-provoking) to approach them individually in the game. We were close, yet still so far.
Dividing Nature
Although we frequently see portrayals of psychologists on TV and in movies, what those actors do on the screen bears little resemblance to what research psychologists actually do in their work. Portrayals of psychologists tend to focus on clinical psychology and psychiatry—talking to people on couches or prescribing pills to treat mental conditions. Magazines and Facebook apps focus on personality tests. Such portrayals imply that psychology is a science of divination. The psychologist on TV asks the female protagonist about her day and then magically figures out that she has secretly been in love with her adopted brother all her life. And the conceit of personality tests is that taking a few minutes filling out a few innocuous questions somehow lets a magazine know you better than yourself.
In my second-year psychology sequence in college, our professor told us that personality psychology is the closest we would get to what we likely, but incorrectly, thought psychology was about. Personality psychologists certainly conduct studies and run lots of statistics, but at the heart of the research area is the question of how we define and measure personality—the set of interesting individual differences that remain stable over time. Dividing nature is always a contentious business. How do we categorize ethnicities in a census? What are the appropriate checkboxes for sexual orientation? The same is true for personality. It is not hard to see how different psychologists might develop a fondness for certain personality assessment tools over others. This might be because they played a role in the development of a certain taxonomy or their adviser had a preference for one scale over another. Of course, researchers may have empirical reasons for preferring one scale, but not every researcher will agree with those reasons. And there is no grand arbiter of personality scales, so for many decades, psychologists freely developed and published their own personality scales.
Books appeared that collated these disparate scales for easier reference. One popular compilation from 1991 that many psychology researchers have on their reference shelves is Measures of Personality and Social Psychological Attitudes. As a sampling of the book’s contents, there are eleven different scales for self-esteem, ten different scales for shyness and anxiety, twenty-one different scales for depression and loneliness, and twenty-nine different scales for alienation. The California Psychological Inventory divided personality into thirty-three factors; the Personality Factor Questionnaire divided it into sixteen; the Jackson Personality Inventory divided it into fifteen. Oliver John and Sanjay Srivastava, personality psychologists at UC Berkeley and the University of Oregon, respectively, described this earlier era of personality psychology as a “Babel of concepts and scales” in which researchers were “faced with a bewildering array of personality scales . . . with little guidance and no overall rationale at hand.”2
A solution emerged in the mid-1980s that began to unify the field of personality psychology. To be more accurate, this approach re-emerged when research from the 1930s came back into vogue after a paradigm shift. The solution hinged on the lexical hypothesis, the assumption that everyday language already captures the most salient personality traits. Any socially relevant trait would find a place in the vocabulary of the people speaking that language. Thus psychologists turned to dictionaries and English texts to extract hundreds of personality-relevant terms. They created large inventories and asked thousands of study participants to rate how well each trait described them. The psychologists’ task then was to understand how these terms clustered statistically. By the early 1990s, multiple studies had confirmed that this multitude of English adjectives and phrases could be cleanly divided into five factors. Reliable scales were developed for these factors, and additional studies have shown that they are cross-culturally valid. These personality factors are known as the Big Five, and this personality framework is the current gold standard in personality psychology research.3
These five factors form the acronym OCEAN. Openness to Experience measures a person’s intellectual curiosity, appreciation for art, creativity, and preference for novelty. People with high scores on Openness are more likely to enjoy going to museums and joining in philosophical discussions and to have unconventional ideas and beliefs. People with low scores on Openness are more practical and down-to-earth and more likely to be conventional and traditional. Conscientiousness measures self-discipline, organization, planning, and a sense of duty. People who score high on this factor are usually prepared, plan things in advance, and pay attention to details. People who score low on this factor tend to be spontaneous, don’t mind a bit of chaos in their lives, and may be perceived by others as disorganized. Extraversion measures activity level and the desire to seek out stimulation in social settings. People who score high on this factor enjoy large crowds and being the center of attention, and they have no trouble starting a conversation with strangers. People who score low on this factor avoid social situations, are quiet and reserved, and in general keep in the background. Agreeableness measures compassion and cooperation. People who score high on this factor sympathize with others’ feelings, take time to help others, and are interested in other people’s problems. People who score low on this factor tend to be more self-interested, competitive, even antagonistic, and in general suspicious and untrusting of others. Neuroticism measures emotional stability and the tendency to experience negative emotions. People who score high on this factor are vulnerable to stress, anxiety, and depression. They are easily upset. People who score low on this factor tend to be calm, emotionally stable, and relaxed.
Ever since the Big Five emerged and became standardized, many researchers have explored how these personality traits are expressed in everyday life. It turns out that personality assessments by complete strangers even after a brief interaction are moderately accurate. Furthermore, people tend to use the same cues to infer personality traits. For example, in a study that videotaped strangers getting acquainted, researchers found that people with high Extraversion spoke louder and with more enthusiasm and were more expressive with their gestures. As another example, people with high Conscientiousness wore more formal clothing and were less likely to use rapid body movements. These studies of face-to-face interactions led other researchers to wonder whether personality can be inferred from the spaces we inhabit. Personality psychologist Sam Gosling and his colleagues at the University of Texas examined how personality is expressed in people’s bedrooms and offices. They found that people with high Conscientiousness had well-lit, neat, and well-organized bedrooms. And people with high Openness to Experience had more varied books and magazines.4
These findings also extend to online interactions. Moderately accurate personality impressions can be formed based on an individual’s personal website, Facebook profile, email messages, blog posts, and even email address—the smallest slice of online identity expression possible. For example, in terms of a person’s blog posts, people with high Agreeableness were more likely to use words related to family and happy emotions (for example, happy, joy). And people with high Conscientiousness were more likely to use words related to achievement. Thus, the Big Five isn’t just a theoretical framework of personality. These personality traits are also readily expressed when we interact with other people and the world around us. This means that the behavioral traces we leave behind—the blog posts we make, the way we speak, or how often we gesture—are cues that can be used to infer our personalities.5
But does personality get expre
ssed in virtual worlds? Given that the average online gamer spends more than twenty hours a week in virtual worlds in which all their actions are tracked, there should be a wealth of digital behavioral cues to find. On the other hand, virtual worlds are different from everyday life precisely in that players are actively encouraged to not be themselves. When people write emails for work or to their friends, their words and identities are rooted in a shared physical reality, and thus it makes sense that their personalities are expressed. You’re writing an email as yourself, not as a Night Elf shadow priest. But when we’re in a fantasy world where people are in nonhuman bodies doing nonhuman things, are all bets off? In a world in which you can be a gnomish priest resurrecting the dead using magical light rays, does the fantasy break the linkage between personality and behavior?
Quelling Pandora’s Box
In 2007, the balance between data access and third parties changed yet again. Blizzard released the Armory for World of Warcraft, a website that allows you to look up any active character playing the game. For each character, the Armory catalogs thousands of statistics and achievements. And this list has kept growing with the updates over the years. Currently, the Armory provides over 3,500 variables for each active character, updated daily. These publicly accessible variables cover a broad range of behaviors in World of Warcraft: progress through high-level dungeons, combat skill specializations, number of vanity pets owned, frequency and nature of deaths (for example, by drowning or by falling from high places), and even the exact number of virtual hugs given over a character’s career. It is an unbelievable treasure trove of data.
