Technically Wrong

by Sara Wachter-Boettcher

  We hold technology in our pockets. We tell it our secrets. We rely on it to sustain relationships. It’s the first thing many of us interact with in the morning, and the last thing we look at at night. Technology isn’t just pervasive. It’s personal.

  But it’s when things get personal that change is possible. In a 2009 Gallup poll, researchers found that respondents who said they knew a gay person were 40 percent more likely to think that same-sex relationships should be legal, and 64 percent more likely to think that gay marriage would not change society for the worse, than those who reported not knowing any gay people. The implication is clear: exposure to difference changes perspective, and increases tolerance.1

  That’s why it matters so much that marginalized groups are validated within our interfaces. Because if technology has the power to connect the world, as technologists so often proclaim, then it also has the power to make the world a more inclusive place, simply by building interfaces that reflect all its users. When the needs of trans people are made explicitly visible within an interface, everyone who uses that interface gets a subtle reminder that trans people exist. The more visible trans people are online, the more a part of our communities they become—and the harder it is for them to be treated like a bogeyman and subjected to “bathroom bills.”

  Inclusion also saves lives. In early 2017, researchers from Johns Hopkins and Harvard published a study analyzing more than 750,000 responses from the Youth Risk Behavior Surveillance System, an annual study conducted by the Centers for Disease Control and Prevention. The data, which was gathered from surveys that American high-school students completed between 1999 and 2015, revealed that after a state legalized same-sex marriage, as thirty-two states did between 2004 and 2015, its rate of teen suicide attempts dropped by an average of more than 7 percent. Among LGBTQ students, who report suicide attempts at a staggering four times the rate of straight students, the decrease was even larger: 14 percent fewer reported that they’d attempted suicide after gay marriage passed in their state than before. In the fifteen states that did not legalize same-sex marriage in that time frame, rates stayed the same. According to the study’s authors, if this reduction holds true for all US students, we’ll see 134,000 fewer teen suicide attempts per year, now that marriage equality is national law.2

  The study wasn’t designed to assign causation, so the authors are careful to note that they can’t say for sure whether the legalization of gay marriage directly caused the decreases. But it certainly appears that way. “Stigma is one of the most frequently hypothesized risk factors for explaining sexual orientation disparities in suicide outcomes,” wrote Mark L. Hatzenbuehler of Columbia University in an editorial accompanying the study in JAMA Pediatrics.3 But most past research had focused on the effects of “individual and interpersonal” stigmas, rather than the “broader, structural forms of stigma” that this research pointed to: laws and cultural norms.

  Changing form fields won’t change laws. But the more our daily interactions and tasks happen in digital spaces, the more power those spaces hold over cultural norms. Every form field, every default setting, every push notification, affects people. Every detail can add to the culture we want—can make people a little safer, a little calmer, a little more hopeful.

  . . .

  I’m sitting in Yerba Buena Gardens, a park in the heart of San Francisco’s SoMa neighborhood. It’s a Friday in early spring, just after 5 p.m., and that quintessentially Californian golden light is bouncing off the waterfall-style fountain in front of me. Twentysomethings hop into Ubers headed for happy hours and yoga classes. A trio plops down at a table outside the tea shop next to me to talk about user engagement rates and daily active users. Young men pour out of the Game Developers Conference at the Moscone Center next door, lanyards still around their necks, making plans for an evening out. Everything feels easy—if you avoid looking at the woman with a stroller and a cardboard sign on the corner, or the man sleeping in an alcove over on Howard Street, or the fact that all your friends are leaving this place because their salaries, even the ones inflated by tech jobs, just don’t stack up in a city where people make cash offers at open houses and the average two-bedroom apartment rents for more than $5,000 a month.

  About a mile southwest, at Uber’s headquarters, another scandal is brewing: a tool called “Greyball,” used to systematically mislead authorities in markets where the service was banned or under investigation, has just been reported in the New York Times.4 Across the street, at Twitter, stock prices fell more than 10 percent in a single month, and the company is scrambling.5 And thirty miles south, in Menlo Park, Facebook has just started rolling out its solution to fake news: stories shared on Facebook that have been debunked by third-party, nonpartisan fact-checking organizations have begun being marked with a red caution icon and the word “Disputed”—a label that’s already being disputed itself, with some calling it censorship and others calling it too milquetoast for news that’s demonstrably false.6

  Unrest is brewing at the big tech companies. I don’t feel bad for them. Tech has spent too long being treated as a marvel—getting a free pass on ethics by promising us convenience wrapped in a slickly designed package. It’s time to turn on the pressure, and keep it on until things change. Send customer service complaints. Tell the media. Write your congressperson. Support an alternate product.

  If we don’t, there’s real danger ahead. Tech companies are now engaging in all manner of civic life—from Mark Zuckerberg’s vision for a journalism industry utterly intertwined with Facebook, to the downright terrifying work of Trump adviser Peter Thiel’s data-mining company Palantir, which received $41 million from Immigration and Customs Enforcement to build software that tracks and deports immigrants (and could, of course, be turned on any of us).7 If tech companies can’t get the basics right—if they can’t stop themselves from designing “yellowface” photo filters, or pushing cutesy language at people in crisis, or creating photo-tagging systems that fail black users—why should we trust them to provide solutions to massive societal problems?

  We’ll only be successful in ridding tech of its excesses and oversights if we first embrace a new way of seeing the digital tools we rely on—not as a wonder, or even as a villain, but rather as a series of choices that designers and technologists have made. Many of them seem small: what a button says, where a data set comes from. But each of those choices reinforces beliefs about the world, and the people in it.

  It’s up to us to demand that those choices be made differently—not because we want to see technology fail, but rather because we want it to succeed, on terms that work for all of us. After all, most of us don’t hate tech. We love it. It’s time we demand that it love us back.

  Acknowledgments

  Thank you to my editorial team: Alane Salierno Mason, for emailing me out of the blue and encouraging me to do this; and Ashley Patrick, for patiently answering my endless questions. Many thanks to my copy editor, Stephanie Hiebert, for bringing clarity and thoroughness to the manuscript (and for coining the delightful phrase “tenuous legitimacy at best,” which is how I plan to describe myself from now on).

  Endless gratitude to those who read early drafts: my love, William Bolton; and my friends Marie Connelly, Katel LeDû, Ethan Marcotte, and Mary Rohrdanz. I owe all of you wine, doughnuts, and hugs.

  This book wouldn’t have been possible without Eric Meyer. Collaborating with you on Design for Real Life changed the course of my career.

  Thanks to those I spoke with during the writing process: Erin Abler, Jacky Alciné, Libby Bawcombe, Sally Jane Black, Anil Dash, Maggie Delano, Veronica Erb, Sorelle Friedler, Aimee Gonzalez-Cameron, Lena Groeger, Sydette Harry, Dan Hon, Kate Kiefer Lee, Safiya Noble, Sally Rooney, Grace Sparapani, Kaya Thomas, Indi Young, and a whole host of wonderful people who shared their stories with me in confidence. I am also incredibly grateful to my friends Steve Fisher, Jason Santa Maria, and Matt Sutter for providing design help.

  Thank you to the friends
who gave feedback on all kinds of details, and who were there for me as I moaned and griped through this process—especially all the members of Camp Contentment, the Male Tears Club, Pizza Club, and the Ladies Anti-Fascist Friends Society. Cat-heart-eyes emoji for days.

  Thank you to anyone I missed. I hope you forgive my terrible memory.

  And, finally, thank you to everyone striving to make tech fairer, kinder, and more humane. I know we can do it.

  Notes

  Chapter 1: Welcome to the Machine

  1. Andrew Perrin, “One-Fifth of Americans Report Going Online ‘Almost Constantly,’” Pew Research Center, December 8, 2015, http://www.pewresearch.org/fact-tank/2015/12/08/one-fifth-of-americans-report-going-online-almost-constantly.

  2. Oliver Wheaton, “Gym’s Computer Assumed This Woman Was a Man Because She Is a Doctor,” Metro, March 18, 2015, http://metro.co.uk/2015/03/18/gyms-computer-assumed-this-woman-was-a-man-because-she-is-a-doctor-5110391.

  3. For an overview of the study’s findings, see Pam Belluck, “Hey Siri, Can I Rely on You in a Crisis? Not Always, a Study Finds,” March 14, 2016, New York Times, http://well.blogs.nytimes.com/2016/03/14/hey-siri-can-i-rely-on-you-in-a-crisis-not-always-a-study-finds. The full study can be found in Adam S. Miner et al., “Smartphone-Based Conversational Agents and Responses to Questions about Mental Health, Interpersonal Violence, and Physical Health,” JAMA Internal Medicine 176, no. 5 (2016): 619–25, http://jamanetwork.com/journals/jamainternalmedicine/article-abstract/2500043.

  4. Anil Dash, “There Is No Technology Industry,” Humane Tech, August 19, 2016, https://medium.com/humane-tech/there-is-no-technology-industry-44774dfb3ed7.

  5. Safiya Noble, “Challenging the Algorithms of Oppression” (talk, Personal Democracy Forum, June 10, 2016), YouTube, June 15, 2016, https://www.youtube.com/watch?v=iRVZozEEWlE.

  Chapter 2: Culture Misfit

  1. Throughout the book, anecdotes that I relate without attribution were told to me in confidence—in person, by phone, or via email—and therefore remain anonymous. Some names have been changed to maintain that anonymity.

  2. Amélie Lamont, “Not a Black Chair,” Medium, March 14, 2016, https://medium.com/@amelielamont/not-a-black-chair-8a8e7e2b9140#.gpk28w3fp.

  3. Erica Joy, “The Other Side of Diversity,” Medium, November 4, 2014, https://medium.com/this-is-hard/the-other-side-of-diversity-1bb3de2f053e#.xn7th3cbt.

  4. Megan Rose Dickey, “Apple’s Tim Cook on Latest Diversity Numbers: There’s ‘a Lot More Work to be Done,’” TechCrunch, August 13, 2015, https://techcrunch.com/2015/08/13/apples-tim-cook-on-latest-diversity-numbers-theres-a-lot-more-work-to-be-done.

  5. Maxine Williams, “Facebook Diversity Update: Positive Hiring Trends Show Progress,” Facebook Newsroom, July 14, 2016, http://newsroom.fb.com/news/2016/07/facebook-diversity-update-positive-hiring-trends-show-progress.

  6. Nancy Lee, “Focusing on Diversity,” The Keyword (blog), June 30, 2016, https://www.blog.google/topics/diversity/focusing-on-diversity30.

  7. Apple, “Inclusion & Diversity,” accessed October 15, 2016, http://www.apple.com/diversity.

  8. Google, “Diversity,” accessed October 2016, https://www.google.com/diversity.

  9. Airbnb, “Employee Diversity & Belonging: 2016 Assessment,” October 25, 2016, http://blog.airbnb.com/employee-diversity-belonging-2016-assesment.

  10. Williams, “Facebook Diversity Update.”

  11. Kaya Thomas, “The Diverse Talent Pool Exists. Facebook Just Isn’t Hiring Us,” Fusion, July 15, 2016, http://fusion.net/story/325940/the-diverse-talent-pool-exists.

  12. Elizabeth Weise and Jessica Guynn, “Tech Jobs: Minorities Have Degrees, but Don’t Get Hired,” USA Today, October 12, 2014, http://www.usatoday.com/story/tech/2014/10/12/silicon-valley-diversity-tech-hiring-computer-science-graduates-african-american-hispanic/14684211.

  13. Thomas, “Diverse Talent Pool Exists.”

  14. Sarah Cooper, “Honest Diversity in Tech Report,” Cooper Review (blog), accessed November 18, 2016, http://thecooperreview.com/diversity-in-tech-report.

  15. Ellen Huet, “Facebook’s Hiring Process Hinders Its Effort to Create a Diverse Workforce,” Bloomberg, January 9, 2017, https://www.bloomberg.com/news/articles/2017-01-09/facebook-s-hiring-process-hinders-its-effort-to-create-a-diverse-workforce.

  Chapter 3: Normal People

  1. Maggie Delano, “I Tried Tracking My Period and It Was Even Worse Than I Could Have Imagined,” Medium, February 23, 2015, https://medium.com/@maggied/i-tried-tracking-my-period-and-it-was-even-worse-than-i-could-have-imagined-bb46f869f45.

  2. Glow, “About Glow,” Wayback Machine, September 21, 2013, https://web.archive.org/web/20130921143302/https://www.glowing.com/about.

  3. Kia Kokalitcheva, “Glow Brings in $17M in New Funding, Puts Big Data to Work for Women’s Health,” VentureBeat, October 2, 2014, http://venturebeat.com/2014/10/02/glow-brings-in-17m-in-new-funding-as-puts-big-data-to-task-with-fertility-challenges.

  4. Glow, “About Glow,” Wayback Machine, March 27, 2014, https://web.archive.org/web/20140327011628/https://glowing.com/about.

  5. Erin Abler, Twitter post, January 31, 2017 (6:12 p.m.), https://twitter.com/erinabler/status/826614200114016256.

  6. Michael M. Grynbaum, “New York’s Cabbies Like Credit Cards? Go Figure,” New York Times, November 7, 2009, http://www.nytimes.com/2009/11/08/nyregion/08taxi.html.

  7. Lena Groeger, “Set It and Forget It: How Default Settings Rule the World,” ProPublica, July 27, 2016, https://www.propublica.org/article/set-it-and-forget-it-how-default-settings-rule-the-world.

  8. Madeline Messer, “I’m a 12-Year-Old Girl. Why Don’t the Characters in My Apps Look Like Me?” Washington Post, March 4, 2015, https://www.washingtonpost.com/posteverything/wp/2015/03/04/im-a-12-year-old-girl-why-dont-the-characters-in-my-apps-look-like-me.

  9. Adrienne LaFrance, “Why Do So Many Digital Assistants Have Feminine Names?” Atlantic, March 30, 2016, http://www.theatlantic.com/technology/archive/2016/03/why-do-so-many-digital-assistants-have-feminine-names/475884.

  10. For a whole ton of examples, see Neha Prakash, “Snapchat Faces an Outcry against ‘Whitewashing’ Filters,” Mashable, May 16, 2016, http://mashable.com/2016/05/16/snapchat-whitewashing/#pLqUZA7KJuqs.

  11. Jessica Nordell, “Stop Giving Digital Assistants Female Voices,” New Republic, June 23, 2016, https://newrepublic.com/article/134560/stop-giving-digital-assistants-female-voices.

  12. Todd Rose, The End of Average: How to Succeed in a World That Values Sameness (New York: HarperCollins, 2016).

  13. Libby Bawcombe, “Designing New Products with Empathy: 50 Stress Cases to Consider,” Design at NPR, August 16, 2016, https://npr.design/designing-news-products-with-empathy-50-stress-cases-to-consider-61f068a939eb. © 2016 National Public Radio, Inc. Used with the permission of NPR. Any unauthorized duplication is strictly prohibited.

  14. Ibid.

  15. Libby Bawcombe, email to the author, December 10, 2016.

  16. Indi Young, “Describing Personas,” Medium, March 15, 2016, https://medium.com/@indiyoung/describing-personas-af992e3fc527#.vw2pgmvm5.

  17. Shonda Rhimes, “You Are Not Alone,” Medium, March 16, 2015, https://medium.com/thelist/you-are-not-alone-69c1a10515ab#.r9gaxhcei.

  Chapter 4: Select One

  1. Shane Creepingbear, Twitter post, October 13, 2014 (7:43 p.m.), https://twitter.com/Creepingbear/status/521853766803673088.

  2. Shane Creepingbear, “The Removal of American Indians from Facebook,” Last Real Indians, accessed November 3, 2016, http://lastrealindians.com/the-removal-of-american-indians-from-facebook-by-shane-creepingbear.

  3. Chris Cox, Facebook post, October 1, 2014, https://www.facebook.com/chris.cox/posts/10101301777354543.

  4. Chris Matyszczyk, “Batman’s Fight to Get on Facebook,” CNET, March 8, 2009, https://www.cnet.com/news/batmans-fight-to-get-on-facebook.

  5. Justin Osofsky and Todd Gage, “Community Support FYI: Improving the Names Process on
Facebook,” Facebook Newsroom, December 15, 2015, http://newsroom.fb.com/news/2015/12/community-support-fyi-improving-the-names-process-on-facebook.

  6. Jens Manuel Krogstad and D’Vera Cohn, “U.S. Census Looking at Big Changes in How It Asks about Race and Ethnicity,” Pew Research Center, March 14, 2014, http://www.pewresearch.org/fact-tank/2014/03/14/u-s-census-looking-at-big-changes-in-how-it-asks-about-race-and-ethnicity.

  7. Pew Research Center, “Multiracial in America: Proud, Diverse and Growing in Numbers,” June 11, 2015, http://www.pewsocialtrends.org/2015/06/11/multiracial-in-america.

  8. Jamelle Bouie, “What Pundits Keep Getting Wrong about Donald Trump and the Working Class,” Slate, May 5, 2016, http://www.slate.com/articles/news_and_politics/politics/2016/05/what_pundits_keep_getting_wrong_about_donald_trump_and_the_working_class.html.

  9. Renee Stepler and Anna Brown, “Statistical Portrait of Hispanics in the United States,” Pew Research Center, April 19, 2016, http://www.pewhispanic.org/2016/04/19/statistical-portrait-of-hispanics-in-the-united-states.

  10. Jeffrey S. Passel and D’Vera Cohn, “U.S. Population Projections: 2005–2050,” Pew Research Center, February 11, 2008, http://www.pewhispanic.org/2008/02/11/us-population-projections-2005-2050.

  11. Andrew R. Flores et al., “How Many Adults Identify as Transgender in the United States?” Williams Institute, UCLA School of Law, June 2016, http://williamsinstitute.law.ucla.edu/wp-content/uploads/How-Many-Adults-Identify-as-Transgender-in-the-United-States.pdf.