by Andrew Blum
Troyer had long been sympathetic to my strange quest. He saw my excitement at coming to this small room that seemed to anchor the whole building—and with it much of the Internet—into the soft planet. “The whole idea of this building is that data can go in, and data can go out,” he soliloquized. “It’s the meet-me point where the Internet physically comes together to connect, so that it can become seamless and transparent to the end user. Where you’re standing here, this happens to be the largest concentration of providers on one single campus in the US.” Among the places where Internet networks connect, this was among the biggest—the nexus of nexus. Hot and still. I could smell it: it smelled like dirt.
I cracked a wide smile at the thought—at what a singular piece of Internet this was. And then Troyer knocked me down. As the sign on the door said, this was Fiber Vault 1. On the other side of the building was Fiber Vault 2. There were all the other buildings like it on the campus, each with its own multiple fiber vaults. This was the place, but so was that. And that. And that. The Internet was here, there, and everywhere.
We walked the length of the building back to the red silo, into the man-trap and out into the lobby. I slid my visitor’s badge through the slot to the guard behind his bulletproof windows. We leaned on the single door that didn’t look like anything and stepped out of the cool, dark building and into the hot, bright Virginia day.
Troyer raised his hand to the sky and moaned, “Ahh! Giant fiery orb!”
4
* * *
The Whole Internet
That evening in Washington, staying at my sister’s house, I told my eight-year-old niece about what I’d seen at Equinix. She’s an instant-messaging, YouTube-watching, video-chatting, iPad-swiping member of her generation: a digital native. And like most eight-year-olds, she’s difficult to impress. “I saw the Internet!” I told her. “Or at least a very significant piece of it.” I was accustomed to adults furrowing their brows at statements like this, skeptical that the Internet’s physical reality could ever be so legible. But she didn’t think it was strange at all. If you believe the Internet is magic, then it’s hard to grasp its physical reality. But if, like her, you’ve never known a world without it, why shouldn’t the Internet really be out there, something you could touch? It seemed to me that childlike wonder was a good way of looking at this world; it transformed everyday structures into monuments. And truly and fairly—even by rational adult standards—Equinix Ashburn was the Internet to a far greater degree than most anyplace else on earth could claim. The Internet is an expansive, near-infinite thing; but it’s also astonishingly intimate. How reductive could I be—both in my imaginings of the Internet and my experience of its physical reality? What were the limits of precision?
Throughout this book I have been capitalizing “Internet,” treating it as a proper noun. This increasingly goes against convention. If in the early days of the Internet, it was universally recognized as a unique thing and therefore deserving of its big letter, over time that novelty has been lost. As the website Wired.com explained when it switched away from the big “I,” way back in 2004: “In the case of internet, web and net, a change in our house style was necessary to put into perspective what the internet is: another medium for delivering and receiving information.”
But that’s not how I see it. Not exclusively, at least. Because as soon as I began to engage with the Internet’s physical presence—its places—it came into focus as a singular thing, however unusual and amorphous. By holding on to the proper noun, I’m also holding on to the idea that the Internet is demonstrably there, once you know where to look. And I don’t mean only hidden away behind locked doors in unmarked buildings, but everywhere—in the wires circling the block and the towers of the skyline. This isn’t to say I’m unaware of this idea’s limits, and unwilling to acknowledge the way the networks slip from sight. Seeing the Internet like this demands a certain amount of imagination (occasionally crossing the line to hallucination). The writer Christine Smallwood is on to something when she points out that “the history of the Internet is a history of metaphors about the Internet, all stumbling around this dilemma: How do we talk to each other about an invisible god?” She weighs the relative merits of describing the Internet as a Tootsie Roll, a hot tub, a highway, or a plane, before finally acknowledging how ugly the Internet—the real Internet, the one I’ve been visiting—actually is. “I wish,” she concludes, “the Internet looked like Matt Damon, or like lines of light written by an invisible hand in the night sky.” And so she finds our old friend again: the amorphous blob, the infinite universe, vast, uncontainable, and expanding. The poets’ metaphors all nestle together under the same starry night.
But comedians, I’ve noticed, tend to go in the other direction, toward “the Internet” as a single machine. On the episode of the television cartoon South Park called “Over Logging,” the squat, obnoxious little characters faced a particularly extreme case of a familiar dilemma: the Internet breaks, everywhere. First they try to figure out if this is really happening, but “there’s no Internet to find out there’s no Internet!” a character deadpans. Soon enough “the Internet” itself appears on-screen, in the form of a machine the size of a house, looking suspiciously like a giant version of a home Linksys router, the blue one with the black front and little rabbit ear antennae in the back, lit by klieg lights in its underground bunker. Government agents in dark sunglasses are trying their best to fix it—at one point playing John Williams’s famous five-tone motif from Close Encounters of the Third Kind at it, like a benediction. Eventually, one of the boys finds the solution: he climbs the aircraft-carrier-like steel ramp that leads up to the giant machine, goes around the back of it, unplugs it from an enormous socket, and plugs it back in. Salvation! The “flashing yellow light is steady green now!” a little guy cheers. Peace prevails.
The British sitcom The IT Crowd took the same joke to the opposite size extreme: the Internet wasn’t a single big machine but a tiny little one. As an office prank, two of the IT guys convince a gullible colleague that “the Internet” is inside a black steel box hardly the size of a shoe, with a single red LED. It normally lives at the top of Big Ben—“that’s where you get the best reception”—but with the permission of “the Elders of the Internet” they’ve been able to borrow it for the day, so she can use it for an office presentation. “This is the Internet?” she asks, incredulously. “The whole Internet? Is it heavy?” Her colleagues guffaw at her. “That’s a bit of a silly question. The Internet doesn’t weigh anything.”
Watching these clips on YouTube, I felt a shudder of embarrassment. I seemed to be on a wild goose chase, looking for a world that few believed existed. But if only they really knew! The Internet isn’t a little steel box, of course—not in toto, at least. But that isn’t to say that there weren’t a few steel boxes of vast importance (and that occasionally might need to be unplugged and plugged back in again). Sometimes the center of the Internet—or, at least, a center—is even more particular than a single building. So where were the biggest and most important boxes? And who were these “Elders of the Internet” calling the shots, assuming they existed at all?
The answer to both questions lay within the intimate world of “Internet exchanges.” The terminology can be confusing, but for the most part the place where Internet networks meet is known as an Internet exchange, often abbreviated as “IX.” The PAIX has this in its name; so too does Equinix, if more subtly. However, where things begin to get more slippery is that an “IX” might refer to either the brick-and-mortar building where networks connect to one another, or the institutions that facilitate that connection, whose equipment is often spread out among multiple buildings in a city. The important distinction is that an Internet exchange need not be a piece of real estate; it could be an organization. But there is still a physical thing—often with a single machine at its heart.
The rationale for an Internet exchange is straightforward, and not very different from the founding
principle of MAE-East: get your packets to their destination as directly and cheaply as possible, by increasing the number of possible paths. If Ashburn serves this purpose on a global scale, there’s also a need for smaller regional hubs. As the Internet itself has grown, that need has increased dramatically. Many engineers use the airport analogy: in addition to the handful of global megahubs, there are hundreds of regional hubs, which exist to capture and redistribute as much of the traffic within their area as is practical. But as with the airlines, the smaller nodes of this hub-and-spoke system are always pressured by the tendency toward global consolidation. As Internet networks (or airlines) merge, the big hubs become even bigger—sometimes with a significant loss of efficiency.
In Minnesota, the local network engineers refer to this as the “Chicago problem.” Two small competing Internet service providers in rural Minnesota might find themselves sending and receiving all their data to and from Chicago, by buying capacity on the paths of one of the big nationwide backbones, like Level 3 or Verizon. But—as with a hub airport—the path of least resistance doesn’t always make a lot of sense. In the simplest example, an email from the first network to the second network across town would travel to Chicago and back. Visiting the University of Minnesota’s website, while sitting in Minneapolis, would entail a digital trip across state lines. But if you had a local Internet exchange, you could connect the two (or more) networks directly, often for only the cost of the equipment. The thing is, that might not even be worth the effort given the low cost of getting traffic to Chicago, and the low volume of traffic between those two particular networks. Sometimes it is easier to fly through Atlanta. But if that local traffic were to increase—and it always does—there’s a point at which the elegance of interconnecting all of them, literally cutting Chicago out of the loop, is unmistakable.
In places that hang on to the Internet by a narrower thread, that threshold is more easily crossed, and keeping the traffic local is essential. Until recently, for example, the landlocked central African country of Rwanda depended entirely on satellite Internet connections, which were expensive and slow. If the few local ISPs weren’t careful, an email heading across the capital of Kigali might end up having to make two 45,000-mile round-trips to space. In 2004, the RwandaIX opened to solve the problem, speeding up access to the local pieces of the Internet, and saving the expensive international bandwidth for traffic that was, in fact, international. And indeed it was this same idea that inspired the creation of what are now the biggest IXs globally.
In the mid-1990s, Internet networks everywhere didn’t have a “Chicago problem”; they had a “Tysons Corner problem.” The traffic all went through MAE-East. The dozens of Internet exchanges now distributed around the world serve exactly this purpose. They range from behemoths like JPNAP in Tokyo, which posts astonishingly high traffic numbers but primarily serves intra-Japan communication; to the decidedly smaller Yellowstone Regional Internet Exchange, YRIX, which links together seven networks in Montana and Wyoming (curing them of their “Denver problem”). There’s the MIX in Milan, the SIX in Seattle, the TORIX in Toronto, MadIX in Madison, Wisconsin, and—the solution to Minnesota’s Chicago problem—the MICE, the Midwest Internet Cooperative Exchange. The vast majority of exchanges exist out of sight, often run as cooperative side projects for the “good of the Internet,” and, despite their efforts at outreach, are known and appreciated only by the handful of network engineers who craft the routes across them.
But the largest Internet exchanges are different beasts altogether. Their participants aren’t public-spirited groups of network engineers, but the Internet’s biggest players globally. They are large, professional operations, with marketing departments and teams of engineers. The router manufacturers curry their favor, like sneaker companies courting the best athletes. And they are intensely competitive with one another, jockeying for the title of “world’s biggest”—often by finding new ways of measuring. The two most used criteria are the amount of traffic passing through the exchange (both the peak at a given instant, or on average), and the number of networks that connect across it. In the United States, exchanges tend to be smaller; mainly because Equinix has been so successful in allowing networks to connect directly to each other. The big IXs, in contrast, rely on a centralized machine, or “switching fabric.” The three biggest are all European: the Deutscher Commercial Internet Exchange, or DE-CIX, in Frankfurt; the Amsterdam Internet Exchange, or AMS-IX; and the London Internet Exchange, or LINX. Each has a live traffic graph on its website, along with a running tally of member networks. These three are an order of magnitude larger than the next tier—with the exception of the Moscow Internet Exchange, which has been breaking away from the pack. Watching their traffic statistics daily has the feel of a horse race, with one pulling away for a few weeks before another catches up, to the cheers of an invisible crowd. For months I tracked them closely, looking for changes and trends. And I quizzed network engineers and industry observers about which of the exchanges was qualitatively most important. “Well, Frankfurt’s just huge,” Alan Maudlin, TeleGeography’s analyst, had gushed to me about DE-CIX. “People have so much bandwidth going into bandwidth that it’s just amazing.” But Amsterdam was a close second and had been bigger for longer. And London, while posting lower numbers, trumpeted its “private” links, which moved much of its traffic off the exchange itself and onto direct connections, as in Ashburn.
But regardless of which was the single biggest, the idea of these big exchange points transfixed me. When I first set out in search of the Internet, I expected to find a loose arrangement of little pieces; it was all supposed to be distributed, amorphous, nearly invisible. I hadn’t actually expected anything as grand and specific as a single booming box at the Internet’s “center.” That sounded more like science fiction. Or satire. But that’s exactly what these big Internet exchanges were—except uncelebrated, under the radar, and somewhat oddly arranged, seemingly bypassing some world capitals while colonizing others. Their geography was peculiar: Why was Frankfurt big but not Paris? Tokyo but not Beijing? Did Germans spend more time online than the French? Or was it that the cities adhere to more fixed geographical patterns? Maudlin pointed out that Spain isn’t a hub, and never will be. “It’s a peninsula,” he said. Geography was destiny, even on the Internet. Especially on the Internet.
But aside from looking at them like an analyst, measuring their size and commercial significance, I was curious about their physical reality. If geography mattered as much as it seemed to, that implied that these places worked on a smaller, more specific scale—a building or a box. To acknowledge that at all was to pull the Internet fully into the physical world. And once it was there I wanted to see it, touch it, ponder its corporeal presence. What were these big Internet exchanges like? A little black steel box with a single blinking light? A giant insectlike construction beneath klieg lights and behind barbed wire? The way Maudlin talked about DE-CIX in Frankfurt, it seemed like its “core” would be a sight to see—the Internet tourist’s equivalent of the Grand Canyon or Niagara Falls or something else really big, certainly something worth crossing the ocean for. It was the Internet’s busiest box. Surely that made it worthy of scrutiny, if not reverie. What might it say to me?
But there was also the security issue. These big exchanges made me nervous. Wasn’t it dangerous for things to be concentrated like that? Or more to the point, was I being dangerous in seeking them out—with the intention of telling the tale? Certainly the existence of choke points like these contradicted the conventional wisdom about the Internet’s redundancy. And the more I thought about this, the more paranoid I became.
Then, not long after my visit to Ashburn, arriving back in New York by plane, I had what you might call a brush with the law. As we taxied into the gate, the pilot came on the intercom and instructed us all, without further explanation, to stay in our seats. Two NYPD detectives, straight out of Law & Order central casting in loose-fitting suits, marched up the
aisle, trailed by a uniformed patrolman. Everyone shot up to attention, pointing their chins over the tops of the seat backs.
At that moment, I thought they might be coming for me. The day before I had toured a particularly sensitive piece of Internet infrastructure, a building in downtown Miami known as the “NAP of the Americas.” It serves as the Ashburn for Latin America, but it is also acknowledged to be a key interconnection place for military communications—a point that was hinted at on my tour. I had visited the building with permission and transparency about my project, but that didn’t fully assuage my paranoia. The map of the Internet in my head had filled out. Was it possible, was there even any chance, that I now knew too much, without even knowing it?
With each row the cops passed toward the back of the plane the odds increased that they were coming for me. My wife and baby were with me, and the drama quickly played out in my head, the clichés cribbed from a hundred television shows: the handcuffs, the shouts (“I love you, call my lawyer!”), the outstretched arms. The cut to commercial.
But it wasn’t me they wanted. It was the guy two rows back in a Mets cap, gray sweatshirt, and fifth-grader’s backpack. He was silent as they led him off the plane. But he didn’t look especially surprised. He did look like he’d had better days.
A little later I was interviewing a pair of veteran builders of the Internet’s physical infrastructure at the offices of the private equity firm bankrolling their newest project. It was a lush place high above Manhattan, with thick carpeting and Impressionist paintings on the wood-paneled walls, and they were readily answering my questions about the important pieces of the Internet, and how their new piece would fit in among them. But I must have pushed a little too far.
The senior partner, in a double-breasted suit with a silk handkerchief in the pocket, interrupted his more loquacious colleague and glared at me across the oak conference table. “Let me ask you this,” he said. “Are we creating through this book a road map for terrorists? By identifying the ‘monuments,’ as you refer to them, if they are known and damaged and destroyed, it’s not just one building that goes down, it’s the entire country that goes down, and is that a wise thing to be broadcasting to the world?” It was a startling accusation. Could my search for the physical infrastructure of the Internet be dangerous? Might the Internet be hidden for a reason?
