Tubes: A Journey to the Center of the Internet

by Andrew Blum

  An economic geographer would describe all this as a “a business cluster.” Silicon Valley’s unique combination of talent, expertise, and money has created an atmosphere of astounding innovation—as well as what the local venture capitalist John Doerr once described as the “greatest legal accumulation of wealth in human history.” Indeed, this place, perhaps more than anywhere else in the world, oozes with a collective belief in the limitless potential of technology, and that technology’s potential to turn into limitless money. The aspiration in the air is palpable.

  Yet there seems to be a fundamental irony to all this. Among the computer’s greatest contributions to humanity has undoubtedly been its ability to connect people in different places. Perhaps more than any other technology in history, the Internet has made distance less relevant—it’s made the world smaller, as the saying goes. As the MIT sociologist Sherry Turkle describes, “A ‘place’ used to comprise a physical space and the people within it.” But the ubiquity of the Internet makes that no longer true. “What is a place if those who are physically present have their attention on the absent?” she wonders. “The Internet is more than old wine in new bottles; now we can always be elsewhere.” We feel the consequences of this every day—the disconnection that comes as a consequence of connections, as if in a zero-sum game. And yet that isn’t the only truth about the network—especially not in Silicon Valley. Undergirding our ability to be everywhere is a more permanent thicket of connections, both social and technical. We can only talk about being connected as a state of mind, because we take the physical connections that allow it as a given.

  But the evolution of those connections is very specific and has occurred in very specific places—Palo Alto especially. Whatever alchemy goes on there doesn’t, or perhaps can’t, happen across a wire. At this intensity, connection is an unabashedly physical process. When I lived there, the faithful who fill the cafés always reminded me of priests in Rome, fingering smartphones rather than rosary beads, but similarly sticking close, for reasons both practical and spiritual, to the center of power. They are all there to connect: the gambling venture capitalists, the Stanford engineers, the lawyers and MBAs, and the start-up junkies who smell the future like bloodhounds. The same is true when we start talking about the actual wires.

  Palo Alto is only thirty-five miles from San Francisco, but on the day I drove down it was twenty-five degrees warmer, a dry heat thick with the fragrance of eucalyptus. I was meeting two of those Valley stalwarts for lunch at a café on University Avenue, Palo Alto’s main drag. Afterward, we would visit the Palo Alto Internet Exchange—one of the Internet’s most important places of connection, past and present.

  Jay Adelson and Eric Troyer sat at a sidewalk table, looking out toward the passing crowd, jovial with a couple beers on a Thursday afternoon. They are old friends, onetime roommates, former colleagues, and among the most knowledgeable two people anywhere about how—and, more important, where—Internet networks connect to one another. Troyer calls himself a “recovering network engineer,” a moniker that both maintains and deflects his geek cred. With his close-cropped gray-flecked hair and wraparound sunglasses, he gave off a relaxed surfer-dude vibe, like the net-geek version of Anderson Cooper. “ET,” as he’s known in the networking community, works for Equinix, a company that operates “colocation” facilities around the world.

  Adelson hired him there; in fact, Adelson founded Equinix, building it up from a loose concept in 1998 to a billion-dollar publicly traded company, before leaving in 2005. He is the Silicon Valley archetype: an entrepreneur with a gift not only for seeing the future but convincing others to meet him there. He maintained a reputation as a boy wonder, but he was a few weeks shy of his fortieth birthday, and a few months past his most recent job as CEO of Digg, a web service that allows readers to express their approval or disapproval of an online article or blog post, or photograph of a talking cat. Adelson’s departure from Digg was widely reported to be contentious, but he seemed relaxed, wearing jeans and an untucked dark button-down shirt, his trademark bangs hanging over an angular face, like a teenager. He had been using his post-Digg sabbatical to take guitar lessons, move into a new $3 million house, spend time with his three kids, and weigh his options for the future—mulling a third act in what had already been a very successful Silicon Valley career. It was the first act that interested me: when he helped solve the problem of MAE-East, and in the process raised the Equinix flag above what are today the Internet’s most important choke points.

  “You want to hear the whole story?” Adelson said, already gearing up, digging into a chicken Caesar salad. “It was an interesting period, a real transitional point for the Internet.” It all happened fast, at the height of the dot-com boom. At the end of 1996, Adelson had a job at Netcom, one of Silicon Valley’s first commercial Internet providers. In contrast to the Virginia-based companies focused on big corporate clients, Netcom’s bread and butter was “nerds in withdrawal,” recent castoffs from university computer science departments desperate to “extend their addiction” to the Internet. Netcom had started out connecting its customers through the academic backbone, even though doing so was in clear violation of its “acceptable use policy.” This side door to the Internet had been fine to service the needs of a handful of quiet programmers, but when things took off, it became untenable. So at great expense Netcom leased a data line from its Bay Area headquarters back to Tysons Corner, to join the scrum of networks at MAE-East. Adelson was shocked at what he found there. “It was an old boys club. If you weren’t a telecom company, and you weren’t controlling the fiber in the ground, you were at a profound disadvantage. They’d tell us, ‘We’re out of capacity.’ But you’d never know if it was a conflict of interest”—if they wanted the business for themselves.

  For the Internet to grow, it had to be freed from metered interconnections, carrier interference, and the clogged switches that the National Science Foundation had unwittingly codified with the creation of the network access points. Networks had to be able to connect with as little friction as possible. “We’d post, ‘It should be a free Internet! It’s unfair for these exchange points to be owned by telecom companies!’” Adelson recalled about the angry debate, played out on the email lists and message boards of the networking community. Because how open was the Internet really if a single company effectively had a velvet rope strung across the door?

  Adelson, all of twenty-six at the time, had already distinguished himself as a different kind of networking guy—an Internetworker, you might say. Networking predominantly attracted people who preferred spending time with machines rather than other people. “In order to be proficient in Internet technology at this time, you kind of have to be weird,” Adelson explained. He was—a little. He’d been playing obsessively with computers since he was a kid, hanging out on hackers’ message boards and spending long hours in the lab in college. But he also studied film at Boston University and had acquired the wheeling-dealing, fast-talking poise of a Hollywood producer. His skill was getting people to connect—as well as computers.

  The Internetworking world is still surprisingly small, but it was tiny then, and Adelson attracted the attention of an engineer named Brian Reid at Digital Equipment Corporation, one of Silicon Valley’s oldest and most venerable computer companies (now part of the giant Hewlett-Packard—another company born and headquartered in Palo Alto). Digital had a node on the ARPANET nearly from the beginning, but it wasn’t until 1991 that it began hosting a crucial private Internet link—a wire strung across the room connecting two of the pre-MAE-East era’s biggest regional networks, Alternet and BARRnet. It was originally set up in a spirit of community service—“for the good of the Internet,” as the engineers like to say. But as the Internet grew, Digital recognized another benefit: the link gave them a front-row seat to a key Internet intersection. They were like traffic experts with offices overlooking Times Square. And it was getting messy out there.

  Digital was particularly sen
sitive to the failures of MAE-East, because it designed and manufactured the “GIGAswitch/FDDI,” the router at its heart that couldn’t keep up with demand. To continue to grow, there needed to be a new way for networks to connect to one another that eliminated the congestion problem. Reid had the simple idea that networks should connect directly, literally plugging one router into another, rather than all plugging into a single shared machine as at MAE-East and the other network access points. Most of the networks had already moved lots of equipment into the buildings, which were overflowing because of it. They needed a better environment—a more suitable piece of real estate than a concrete bunker carved out of a parking garage—and one that could accommodate all the direct interconnections. Reid also imagined that the revenue model would change: the connections would be “untariffed,” meaning Digital wouldn’t charge by the amount of traffic. Instead it would charge rent, both for the obvious square footage of the “cage” in which customers kept their equipment, but also for the far subtler (and skinnier) piece of air that each wire traversed to connect to another company’s cage. At MAE-East this would have been commercial suicide, like a restaurant giving away food; but Digital thought it could make a business charging for the table. And the risk was worth it, particularly if it would help grow the Internet—and sell more of Digital’s machines.

  Digital put in a few million dollars of internal funding and a spare bit of office space: the basement of 529 Bryant Street, constructed in the 1920s as a telephone switching office. In technical terms it would be “carrier neutral,” meaning Digital wouldn’t be competing with its customers, as at the network access points. And it would be built within a “class A data center,” a space specially designed for computer and networking equipment. Reid christened it the “Palo Alto Internet Exchange,” or PAIX. All he needed then was somebody to run the place—somebody who knew networking. Somebody with some vision.

  For Adelson, Digital’s job offer was a funny thing. “I remember thinking, ‘Digital?!’” he recalled. “All my friends were going to dot-coms and they were going to make millions as 50 percent owners of their start-ups, and I’m being recruited by a thirty-year-old company! But I was an Internet nerd, and Digital had this nerd cred.” Besides, that wasn’t exactly how it worked out.

  The exterior of 529 Bryant Street was impeccably maintained, its walls a match for the sandstone structures on Stanford’s famous quad. Elaborate bas relief engravings surrounded the entrance, as if it were some lost London bank. Brass letters spelled out “PAIX” beside the door. We stepped into the small lobby, hidden from the street by tinted windows. “Oh. My. God,” Adelson said, when his eyes adjusted to the relative dark. “That is really, really cool.” In front of him was a big red and black “E”—Equinix’s logo. Adelson hadn’t been inside this building since the bad day in 1998 when the handshake deal that was going to make this the nascent Equinix’s first location fell apart, and the place was sold out from under them for $75 million. But only a few weeks before our visit, a dozen years later, Equinix—without Adelson—had finally gotten the PAIX, as a spoil of its purchase of a key competitor, Switch & Data, for $689 million in cash and stock. For Adelson, the Equinix logo on the wall was the symbol of an earlier wrong finally righted—and proof that his vision for the Internet was correct.

  Two technicians greeted us, both of whom had worked in the building since those early days. In Internet time, multiple epochs had passed; but amid the hugs and back slaps the span of time felt reassuringly human. The newborn babies were still hardly grown. “I was going to ask what you been doing these last ten years, but of course I know!” Felix Reyes, one of the techs, said to Adelson. “It’s good to see you! A lot has changed here, a lot of corporate politics, a lot of growth. But still kicking it!” That seemed like an understatement: the building was on its fourth owner; the Internet was transformed, and had transformed everything.

  “It is a long time ago in Internet years,” Adelson said.

  Reyes was wearing a brand-new Equinix polo shirt, black with the red logo, and Adelson flicked at it. “I don’t have any schwag!” he complained. “I always had the technicians’ staff shirts.”

  “We’ll get you one,” Reyes said. “We have samples of all the shirts over the years.”

  “This place has changed hands so many times, and morphed again and again, but the reality is the same exact service has been taking place here since its inception,” Troyer said.

  We headed down a staircase behind the security desk toward the basement, where the first equipment was installed, early in 1997. By the end of that year, the Palo Alto Internet Exchange had grown to be the most important building of its kind on the planet. It no longer claimed that title, but it still remains high on a short list of the Internet’s most important places: a key nodal point where networks connect to one another. The building in Milwaukee was the Internet equivalent of a small regional airport, with just one or two airlines flying to a couple big regional hubs; but the Palo Alto Internet Exchange is like San Francisco International, or even bigger—a “major global connectivity hub,” in the words of Rich Miller, a key industry observer. The building all around us was the aggregate manifestation of those connections. It provides the real estate to satisfy a basic economic and technical desire: it is cheaper and easier to connect two networks directly than to rely on a third network to do it for you. The PAIX is the depot: a convenient central point to string a cable from one router to another. And in particular, it’s a popular place for the undersea cables linking Asia and North America to install their network POPs, or “points of presence.” This is the place that makes “connect” a physical word.

  From a modest foyer at the foot of the stairs, I could see tight rows of cages stretching far off into the half-light, like the stacks of a library. Each was about the size of a cubicle and rented by a single network, which installs its equipment and starts arranging connections to other networks—looking to literally string a wire. Originally, the companies that owned the long-distance fiber-optic lines came to the building to be close to the local and regional Internet service providers that brought the Internet to homes and business—the “eyeball” networks, they’re called. These were the physical network owners. But soon the “content providers”—a Facebook or YouTube today, but back then a Yahoo!, an electronic greeting card company, or a pornography site—wanted to be close as well, to improve the connections to their eyeballs. “I remember Filo and Yang from Yahoo! coming through here, and thinking, ‘Who are these clowns?’” Adelson said about Yahoo!’s billionaire cofounders. But as the Internet evolved, eventually everybody showed up, from virtually everywhere, over a hundred networks in all. Today, there are big content players like Microsoft, Facebook, and Google; eyeballs like Cox, AT&T, Verizon, and Time Warner; and then the global telecoms large and small, with a particularly strong showing from the Pacific Rim—everyone from Singapore Telecommunications to Swisscom to Telecom New Zealand to Qatar Telecom to Bell Canada, coming in on the transpacific cables or the big backbones crossing the United States. Like a throbbing world capital, the PAIX thrived on its own diversity.

  As we stepped into the dimly lit corridor lined with cages, in front of us was an enormous cardboard box the size of a shower stall. Inside was a brand-new router, the most powerful model made by Cisco, one of the industry leaders, with a six-figure base price. Only the largest websites, corporations, or telecom carriers would have enough traffic to justify a beast like this. Finding it sitting there waiting to be set up was like seeing a brand-new 747 parked idle on the airport tarmac. But what made it special wasn’t just the volume of data it could move but the number of different directions it could move it. In that sense, to shift the analogy, the big router was more like the traffic roundabout at the meeting point of 160 highways—160 being the number of individual “ports” it accommodated, each with a processor that handled the communication with another router, like a two-way street. It was vastly more powerful than the old Catalyst
or GIGAswitch boxes used in Tysons Corner. But even more remarkably, it represented the needs of just one network, rather than being at the center of many. It wasn’t the singular machine at the building’s heart, but one of hundreds all connected to one another.

  Those connections are always physical and social, made of wires and relationships. They depend on the human network of network engineers. Early in his career, Troyer spent his fair share of time sitting on the floor of one of these cages, wrestling with a broken router. But more recently his job was to be more of a social director, encouraging networks to connect with each other—with Equinix collecting a monthly fee when they did. What surprised me was how personal that process was. Troyer knew the network engineers; he was friends with them on Facebook and made sure to buy them beers. The Internet is built on connections between networks agreed on with a handshake and consummated with the plugging in of a yellow fiber-optic cable. Technically the connections happening here could happen across any distance. But there’s a profound efficiency in doing it directly, in plugging my box into your box, in an exponentially repeating pattern.

  Walking through the PAIX is a lesson in the “network effect,” the phenomenon by which something becomes vastly more useful the more people use it—leading more people to use it. In Palo Alto, the more and larger Internet players moved into the building, the more and larger players wanted to be there, seemingly ad infinitum, up against the laws of physics—and the Palo Alto city council. All this equipment needs backup electrical generators, in case of a blackout; and generators need vast quantities of stored diesel fuel—more than any of the neighbors would prefer. “We ran this facility right to the max of what it could handle,” Adelson recalled.