I'm Feeling Lucky: The Confessions of Google Employee Number 59

by Douglas Edwards

  Eventually I realized that the answer, as always, was in the data. In the spring of 2000, I had Max start quantifying how many emails per day he received and responded to. I plotted the data on a chart and estimated Max's maximum capacity, then extrapolated how many Max-equivalents we would need to answer all email within forty-eight hours of receiving it. The data clearly showed that we would require at least one more Max as well as better tools for managing the email itself.

  Reluctantly, and under pressure from Omid, eStaff gave us a green light to hire another customer service representative (CSR) and to buy an electronic customer-relationship-management (CRM) system. I immediately began the search for both, a task that took on increased urgency that summer, when Max moved into his new role helping advertisers generate better returns from their AdWords campaigns. The sales team nicknamed him "The Maximizer."

  I had tried contractors even before Max left, but the generic office temps who showed up couldn't grasp the subtleties of search protocols or master the intricacies of our epistolary style. Max trolled Stanford's campus networks and landed a couple of prize candidates, including Anna Linderum, an international student who donated her time because we couldn't legally pay her, and Rob Rakove, a poly sci major. They quickly mastered the art of providing intelligent answers to befuddled users.

  Meanwhile, after five months of rejecting applicants, I hired Denise Griffin in October 2000 to fill the role of full-time CSR. Surprisingly few candidates had the temperament and writing skills necessary to represent Google with the grace and grammar the job required. Denise had the requisite ability, a Berkeley degree, and community service experience that Sergey found appealing.

  The day Denise started we were three thousand emails behind. Issues kept popping up like the methane fires dotting the grass-covered landfill down the street. A porn star desperately needed us to remove her home phone number from our index. History buffs claiming Israel had sunk a U.S. warship in 1967 objected to being classified as a hate group in our directory. Turks were outraged by an ad about genocide targeted to the keyword "Turkey." And those were the minor complaints. We were reaching the limits of what our Outlook software could handle, and user questions were becoming more complex and time-consuming to answer.

  Our email output slowed from Sergey's remembered (and perhaps unconsciously optimized) personal response rate of one per minute to a more languid pace of one every three minutes. Of course, Denise and Rob couldn't pull answers out of their heads as Sergey had, but in his eyes that didn't excuse lower productivity. Then Cindy found some week-old unanswered emails that could have resulted in negative press.

  "The situation is getting out of control," she warned me. "You need to get it in hand."

  You know that dream where you're trying to run away from some menacing figure and your legs turn into melted marshmallows? I felt my feet mired in the sticky mess of text that was our user base asking for help.

  I went back to the only solution that ever worked. I argued the numbers with Salar and convinced him to support a push for another CSR for our group. Omid put in a separate request for a service person for advertising clients. With two additional staffers, we would be able to make progress for at least a couple of months, until the volume of mail doubled again. Prompted by Salar, Sergey agreed to let us hire another CSR. One CSR, that is, to handle both the user support and advertising jobs. It wouldn't be enough, but it was better than nothing.

  Since so much incoming mail was not in English, I sent out a note asking Googlers to list their language abilities in hopes of tapping in-house translation skills.

  The languages spoken by staff included Arabic, Chinese, Hindi, Indonesian, Japanese, Hebrew, Romanian, and Swedish, but that didn't help me. Engineers didn't have the time or inclination to sift through foreign-language spam all day looking for urgent messages. We needed better technology to sort the mail, store common responses, and send bulk replies, but choosing the right customer-relationship-management (CRM) software required engineering skill to evaluate the efficiency and security of the code we'd be installing. Engineering project manager Mieke Bloomfield agreed to help me separate the good bits from the bad.

  Mieke and I quickly learned that Google's trivial needs didn't impress big CRM firms. They focused on companies managing extensive sales operations and providing product support for thousands of paying customers. We didn't need to identify potential big spenders, and we had no desire to create buyer profiles. We just wanted a tool that could track basic metrics around volume and response rates. And we didn't want to spend a ton of money. Unfortunately, it was a seller's market.

  Kana, which Wall Street dubbed the leader in the CRM space, had just split its stock at an adjusted high of about fifteen hundred dollars a share. Wall Street believed CRM tools were as essential to the web economy as shovels at a gold rush, and their sellers priced them accordingly. The cheapest solution from Kana's rival eGain cost more than a hundred thousand dollars. Larry and Sergey would have preferred to write our replies longhand with quill pens rather than sink that amount into someone else's code. Nor were the vendors willing to bend very far to accommodate a small startup like ours. Any features we wanted to add could only be considered during their regular development cycle and would be incorporated only if they were beneficial for their other, more significant clients.

  So we looked at second-tier firms that offered stripped-down versions we could strip down even further. After evaluating a half dozen, we settled on a year-old company I'll call "Miasma" that offered what we needed: a simple solution to routing mail, a way to store a list of responses, and support for some foreign languages. Miasma proposed to give us five seats (that is, individual user licenses) plus a one-year maintenance contract and installation at a total cost of thirty-five thousand dollars, plus travel expenses for their technicians to come and plug everything in. I thought that was relatively reasonable.

  "The technology's okay, I guess," Sergey told me after seeing a live demo, "but they should pay us to implement it. They need reference clients, and we're growing very quickly. They could learn a lot from watching how our usage changes. And make sure to get two extra licenses for Larry and me. They should throw those in for free because it doesn't cost them anything and we'd be good test users."

  Miasma wouldn't pay us, but they did cut their price almost in half, including five seat licenses and the cost of traveling out to install their product, contingent on our willingness to act as a reference for them (evidently Sergey's logic was not entirely unconvincing). We scheduled an install date and awaited the arrival of the technology that would simplify our lives.

  Miasma's techs worked industriously to get the software up and running while training us to write rules for routing mail to the proper queues and for forwarding messages to our staff "experts" for translation or sales follow-up. Everything was going according to plan, and though we were falling even further behind during the transition period, I was confident we had put into place a scalable solution to a growing problem. We said goodbye to the install team, certain that we would be caught up on our email in no time.

  The problems at first were minor. Our engineering "experts" couldn't read our rich-text-formatted attachments on their Linux-based machines, so they couldn't help us answer technical questions. Some of the routing rules were unpredictable, and everyone who contacted us received multiple automated responses. Then, one day, a help request I sent to Miasma's own corporate mail system bounced back as undeliverable. That seemed a touch foreboding.

  Miasma's staffers were eager to help when we finally got in touch with them, and we continued plugging in patches as needs arose. They fixed reporting so we could see how far behind we were and gave us access to the system remotely via the Internet. We added RAM to the server sitting under Denise's desk and learned how to restart it when it froze, as it did frequently. Though Miasma improved our organization of email, it ran slowly and required repeated manual intervention. With the new technology completely i
nstalled, we weren't responding to users any more quickly than we had been before.

  We fell further and further behind. I checked the queue one day and saw we had more than ten thousand unanswered emails. Miasma's tech staff came back to tweak and tune and reboot the server, but—after months of fits and starts—we gave up. We declared email bankruptcy, archived the unanswered mail, and started over with a blank slate. We were able to normalize our system after a few months, but we never approached the level of rapid response we had envisioned. Each week as I reported our anemic numbers, I felt the pressure to make things work better.

  If Max had been able to answer x number of emails without Miasma, Salar asked, shouldn't Denise and Rob be able to answer some multiple of x emails with the new tool in place? Why hire more reps if we weren't getting everything out of the people we had? I knew the problem wasn't the people answering the email, it was the increase in complex queries and foreign-language messages and the built-in limitations of the software we were using.

  I had put myself in a precarious position. My chosen vendor's product had failed to improve throughput and instead hampered our ability to maintain the level we had achieved previously. It didn't matter that Larry and Sergey had liked the software enough to consider using it for their personal email. It didn't matter that budget constraints precluded a more established supplier. It didn't matter that the vendor seemed committed to whacking the moles as they popped their problematic heads out to taunt us. We were leaking productivity, and it was my mess to clean up.

  Denise and Rob worked diligently over long hours to clear our backlog, but if they answered easy questions, the response time for more technical questions grew too long. If they focused on technical questions, overall response rates dropped, because more time was needed to find the answers. Foreign-language email just languished in limbo.

  This went on for a full year, with issues popping up and emails flying back and forth to Miasma tech support in India. Finally Miasma announced they would deliver a full upgrade to their software to fix all our problems and make rainbows shine across our network and unicorns dance on our desks. First, though, we needed to walk through the fire of a major assault on user support.

  Post Apocalypse

  It was November 2000 when I first learned that Google was buying another company. The acquisition, code-named "Yogi," was an online archive of Usenet posts known as Deja News. We would announce the deal the following February.

  If you're a Usenet aficionado, you'll probably take issue with what I say about it here, so why not skip the next paragraph and avoid the heartburn? Of course you won't. If you like Usenet, you live for heartburn.

  In a nutshell, Usenet is a computer network that preceded the World Wide Web. Founded in 1980, it provided a place for academics and scientists to share information with colleagues by posting messages in newsgroups on an electronic bulletin board. Newsgroups were divided into subject categories such as "comp." for computers or "sci." for science or "rec." for recreation. The name coming after the period indicated the subgroup, as in "sci.research.AIDS." Over time, Usenet devolved from its noble purpose to reflect the common concerns and issues of our times, with groups like "rec.arts.movies.slasher" and an explosion in binary files, which contained encoded software, music, and images since reposted on websites requiring a credit card and proof of age. Moreover, the nature of the dialogue on Usenet changed from dry academic discussions to heated polemics on politics and religion and a multitude of other contentious issues, giving birth to such terms as "flame mail" and "trolling."

  Deja News was home to a continuously updated archive of five hundred million of these posts going back to 1995, including such classics as the announcement of AltaVista's launch and the first mention of Google. Unfortunately, Deja News could no longer afford to maintain the service. In fact, it couldn't even provide access to all the data it had archived. Desperate, it came to Google, seeking a way to keep their data from sinking forever into obscurity. Recognizing the value of the content, Larry and Sergey threw them a lifeline, offering to take the archive, clean it up, make it more searchable, and host it going forward. Google already had plans to launch its own Usenet site at groups.google.com, so the timing was fortuitous. Still, it was an act of mercy and everyone involved knew it.

  The handoff happened quickly—too quickly for Google to do much more than launch a stopgap service based on a separate, recently acquired archive of Usenet posts, while our engineers organized the Deja data and built a better system to deliver it. The interim site wouldn't contain all the posts back to 1995 as Deja's had: it would only offer posts dating back a year. Users wouldn't be able to browse through different groups (though they could search them) or post new messages. Most significantly, they wouldn't be able to "nuke" or remove posts they had already written, even if they had deleted them previously (Deja had suppressed display of deleted comments, but never actually expunged them from the old database). Some Deja users were about to rediscover the offensive and embarrassing notes they had written while angry or drunk, thought twice about, and destroyed. Or at least believed they had destroyed.

  I could smell the crap clouds gathering.

  I drafted copy for Deja's former homepage and an FAQ explaining that Google was engaged in "brute force mud-wrestling with gigabytes of unruly data" to reintroduce a new and improved Usenet archive. I made it clear the effort was ongoing and things would get better soon.

  On Monday, February 12, 2001, the old Deja.com went away and Google's interim site went live. Within seconds, outrage overflowed from clogged limbic systems across the network and flooded my inbox. User support began responding with the soothing language I had supplied, acknowledging that we had "received a number of questions and comments" and letting our angry customers know that we understood "the inconvenience that this has caused." Cindy assured me the tone was perfect. I had to agree, and as we were both professional wordsmiths, I assumed that would be the end of it.

  It wasn't. Deja's stung fans reacted as if we'd snatched honeycomb out of their hive with a big hairy bear claw. They swarmed us. Their emails overloaded our fragile CRM system all Monday afternoon and just kept coming. We couldn't answer the specialized questions that related only to Usenet, and our generic responses just agitated users. Our mailboxes filled with mud and fire.

  Larry and Sergey were surprised by Deja's ungrateful users. We had rescued a valuable Internet resource from the ash heap of history at our own expense and committed to launching an improved archive with access to far more data than Deja had ever offered. "What's wrong with these people?" they wondered. They wouldn't have to go far to find out; within days, disgruntled Usenetters were literally knocking on Google's front door to complain.

  "OK, you guys are in damage control mode, act like it!!!" screamed one user. "So far your attitude is real smug." He went on to compare us to Firestone, whose fatally flawed tires were the subject of a safety recall, though as far as I know no one died from lack of access to three-year-old posts in rec.arts.sf.starwars.

  Wayne Rosing, our new head of engineering, shook off the assault, saying, "What matters is whether we're doing the right thing, and if people don't understand that now, they will eventually come to understand it."

  It was a lesson that would shape Google's attitude toward the public from that point on. Sure, we had upset people with MentalPlex, but at least some of us conceded their kvetching might have had cause. With Deja, we were clearly on the side of the angels. The public just didn't get it. Even when we worked our asses off, spent our own cash, and tried to do something good for them, they bellowed and ranted, bitched and moaned. Since users were being so unreasonable, we could safely ignore their complaints. That suited our founders just fine—they always went with their guts anyway.

  I've been asked if Larry and Sergey were truly brilliant. I can't speak to their IQs, but I saw with my own eyes that their vision burned so brightly it scorched everything that stood in its way. The truth was so obvious that th
ey felt no need for the niceties of polite society when bringing their ideas to life. Why slow down to explain when the value of what they were doing was so self-evident that people would eventually see it for themselves?

  That attitude was both Google's strength and its Achilles' heel. From launching a better search engine in an overcrowded field to running unscreened text in AdWords, the success of controversial ideas gave momentum to the conviction that initial public opinion was often irrelevant.

  By the end of the year we proved our intent had always been honorable, rolling out not just the features users had clamored for but an archive that extended back twenty years instead of just the five Deja had offered.* This was almost single-handedly the work of Michael Schmitt, an engineer who took it upon himself to conduct a global search to track down tapes and CD backups of the earliest Usenet posts and the hardware that could read them. He recovered for posterity the first Usenet mention of Microsoft, Tim Berners-Lee's first posted reference to the "World Wide Web," and Marc Andreessen's public disclosure of the Mosaic web browser that would become Netscape.

  "If there were justice in the world," wrote a formerly disappointed Usenetter, "you guys would be rich and Bill Gates would be standing in line waiting for watery soup." Not that Larry and Sergey needed affirmation that they had made the right call, but still, it was nice to hear.

  Chapter 15

  Managers in Hot Tubs and in Hot Water

  A MONTH AFTER WE bought Deja, a hundred and forty Googlers packed up overnight bags, boarded a fleet of buses, and headed for the hills. It was time for Google's annual ski trip.

  The ritual started when Google was just eight people and Larry very cautiously drove a rented van to Lake Tahoe while Sergey, Craig, Ray, and Harry killed time playing logic games in the back and Heather struggled to stay awake. The group saved $2.50 a day by designating Larry the only driver, which was a given anyway because Larry wasn't about to put his life in anyone else's hands.