Tubes: A Journey to the Center of the Internet

by Andrew Blum

  First, I sat down with Arnold Nipper, DE-CIX’s founder, chief technology officer, and something of a father to the German Internet. He looked the part, dressed like an esteemed computer science professor in a work shirt and jeans, his smartphone and BMW keys set on the conference table in front of him. With twenty-five years of experience explaining computer networking to the rest of us, he spoke slowly and precisely, his English accented a bit like Sean Connery.

  In 1989, Nipper established the first Internet connection for the University of Karlsruhe, a technology powerhouse, and later was a lead developer of Germany’s national academic network. When the commercial Internet came into existence in the early 1990s, Nipper became chief technology officer of one of Germany’s first ISPs, Xlink, where he faced a familiar headache: MAE-East. “Every packet had to go across really expensive international links to the NSFNET backbone,” Nipper said, between sips of an espresso. In 1995, Xlink teamed up with two other early ISPs, EUnet in Dortmund (home of another prominent university computer science department) and MAZ in Hamburg, intending to take the transatlantic crossing out of the loop by linking their networks together on German soil.

  The Deutscher Commercial Internet Exchange was founded with a ten-megabit interconnection—about 1/100,000th its current capacity—and a hub installed on the second floor of an old post office building near the center of Frankfurt. This first incarnation of DE-CIX was far from grand, but it changed everything. For the first time, Germany had its own Internet—its own network of networks. “With the invention of DE-CIX we had only to cross national links,” Nipper said.

  But why Frankfurt? Nipper admits the decision to put the hub here—a decision that today exerts a sunlike gravity on the geography of the whole Internet—was somewhat unconsidered. Mostly it was because the city was at the rough geographic center of those first three participants, a quality from which it’s often benefited. And it helped that Frankfurt was the traditional center of German telecommunications and, of course, Europe’s financial capital. Yet it would be wrong to conclude that in this case the Internet followed finance—or followed Deutsche Telecom. That first hub wasn’t planned for growth, or at least not the rocket ship that was about to take off. The Internet’s expansion was ad hoc, as always.

  In the decade and a half since its founding, a lot has changed, not only at DE-CIX but across Europe—and it’s Europe’s changes that have primarily driven DE-CIX’s growth. As Internet use in the former Soviet republics catches up with the West, DE-CIX has aggressively pitched ISPs there, offering connections to networks from around the world, for less money than London or Ashburn and with a richer mix of global carriers to boot, particularly from the Middle East and Asia. It’s not that the former republics have so much to say to Frankfurt; it’s that Frankfurt, in large part thanks to DE-CIX, is the easiest place for them to hear from the rest of the world. The long-distance infrastructure is here: the major European fiber routes converge along the Main, leveraging the same geographic centrality that makes Frankfurt’s airport one of Europe’s largest hubs, and has always made Frankfurt a hub. But mostly the networks are following the economic truth that it’s cheaper and more reliable to “tether” yourself into a big exchange than rely on someone else to bring everything back for you. That truth is self-perpetuating.

  For example, Qatar Telecom, based in the tiny Persian Gulf state, has established beachheads in a familiar list of places around the world: Ashburn, Palo Alto, Singapore, London, Frankfurt, and Amsterdam. Its network carries both voice and private data services for corporations, but when it came to public Internet traffic, undoubtedly the easiest thing would have been to buy “transit” at its doorstep in Qatar from one of the big international providers—perhaps Tata, the Indian conglomerate, which has robust links to the Persian Gulf. But that would have meant leaving to someone else the business of bringing the Internet there. Instead, Qatar Telecom has installed its own networking equipment in those major exchange points and leased its own private fiber-optic circuits back to the Gulf. No wonder the exchanges are competitive with one another. They’re all lobbying for more peering—but really they’re hoping that “more” really means “here.”

  In Frankfurt, there was still the matter of seeing what that “here” looked like. After lunch, Nipper drove us east along the river in his little station wagon, onto a narrow street in a densely packed neighborhood of sturdy old warehouses. DE-CIX’s primary “core” is in a building operated by a colocation company named Interxion, a European competitor of Equinix’s. It opened in 1998, with DE-CIX as one of its first tenants (and still the only exchange). In contrast with the suburban expanse of Virginia, its parking lot was narrow and tidy, with cobbled pavement and manicured bushes, surrounded by a cluster of low, white buildings studded with security cameras. We squeezed in beside a gunmetal-gray van, its rear doors swung open to reveal a rack of tools, “Fibernetworks” painted on its side. Nearby, a red-hard-hatted worker rode a jackhammer into the ground, and a pair of technicians were busy taking apart an automatic doorway.

  “You can see business is healthy,” Nipper said, nodding at the construction. I asked if he was a big customer. “We are an important customer,” he corrected, coolly. They’re the honey trap, and treated well because of it; many Internet exchanges often won’t have to pay rent for their own equipment. We were waiting for a guard to escort us, and Nipper thought the security rigmarole was overkill. “It’s only a telecommunications hub,” he said. “Data going through here is transient, you see. It’s not like a disaster recovery center for a bank, where data is stored—that really has to be secure. Even if this completely fails, it will of course cause an impact on the Internet, but perhaps no email will be lost, just your browser will hang for a second or so, then everything is rerouted.” The DE-CIX core, for its part, is designed to “fail-over” to its backup location across town within ten milliseconds. Easy come, easy go.

  When our security guard finally arrived, we raced to keep up with her as she crossed the parking lot. Nipper buzzed us into the bare lobby of the data center with his keycard and then swiped us again into a second anteroom, all white walls and fluorescent lights. There we came upon a bit of hubbub. A construction worker in blue coveralls was stuck inside the glass cylinder of the man-trap, like a squid in a test tube. The fingerprint scanner wasn’t recognizing his dirty hands and had locked him in, to the jeers of his coworkers on either side of the glass. Compared to Equinix, it all felt less cyberrific, more Teutonic prison. When it was our turn, the guard barked into her radio, and both sides of the man-trap snapped wide open. A loud klaxon sounded, and she hustled us through the door.

  Inside, the equipment cages weren’t cages at all, but full enclosures of beige steel that reached up toward the high, open ceiling, lined with the familiar tracks of yellow fiber-optic cables. Each space was labeled with a numeric code divided by a decimal point—no signs or names anywhere. In Europe it’s typical for power and cooling to run underneath the floor, whereas in the United States it often runs up above. When we came to DE-CIX’s space, the guard handed Nipper a key on a green rubber bracelet, as if we were visiting a safe-deposit box at a bank. The room had the size and character of an airport bathroom, all tidy and beige. As in Ashburn, the roar of the machines was deafening, and Nipper shouted to be heard.

  “One of our principles is to make everything as simple as possible, but not simpler,” he said, quoting Einstein. DE-CIX’s “typology,” in engineering terms, is its own design. The connections from each of the nearly four hundred networks that exchange traffic here are aggregated together—“muxed up”—into a handful of fiber-optic cables. A “fiber protection device” then acts like a two-way valve, directing the flow of data between DE-CIX’s two core switches, the active and the “hot standby,” here and across town. The job of the core is to direct incoming traffic out the correct door toward its destination. The majority of the light travels through the active path to the live core, but 5 percent of the signal
is refracted toward the backup, which is always up and running.

  “All these boxes have been communicating. If one link fails, it tells all of the other boxes to switch over at the same time, and they do it within ten milliseconds,” Nipper said. He tests the system four times a year, switching between the two cores during the quiet hours early on a Wednesday morning. Despite its international customers, traffic across DE-CIX plots a cresting wave, rising throughout the day and peaking midevening, German time, as everyone settles in at home with their web videos and shopping. As Nipper explained all this the guard watched us carefully from the end of the aisle, like a stalker in a grocery store.

  Nipper saved the core itself—the crown jewels—for last, working the key on the green bracelet into the lock on the cabinet and then opening it with a playful flourish. I skipped a breath as I took it in: a black machine in a standard-sized rack; yellow fiber-optic cables sprouting from it like spaghetti from a pasta maker; dozens of busily blinking LEDs; a printed white label that read CORE1.DE-CIX.NET; a plaque that said MLX-32.

  As airhead Jen on The IT Crowd asked: “This is the Internet? The whole Internet?” As machines go, I confess that it looked a lot like all the Internet’s other machines. I had tried to prepare myself for this—for the possibility of banality, of an apparently unremarkable black box. This was like visiting Gettysburg: it’s just a bunch of fields. The object in front of me was true and tangible, if unequivocally abstract; material, yet unknowable. I knew from Austin that this very machine was among the Internet’s most important—the center of one of the biggest Internet exchanges—but it wore that significance discreetly. Its meaning had to come from inside me.

  I was reminded of an extraordinary scene in Henry Adams’s strange third-person autobiography, The Education of Henry Adams, when he describes his visit to the Great Exposition of 1900 in Paris. There he saw a miraculous new technology: a “dynamo,” or electrical generator. It’s a breathtaking confrontation with technology. Taken at face value “the dynamo itself was but an ingenious channel for conveying somewhere the heat latent in a few tons of poor coal hidden in a dirty engine-house carefully kept out of sight,” he writes. Yet the dynamo meant everything to Adams: it became “a symbol of infinity.” Standing beside it he felt its “moral force, much as the early Christians felt the cross.” He continues: “The planet itself seemed less impressive, in its old-fashioned, deliberate, annual or daily revolution, than this huge wheel, revolving within arm’s length at some vertiginous speed, and barely murmuring—scarcely humming an audible warning to stand a hair’s-breadth further for respect of power.” But it wasn’t the machine’s mystery or power that terrified Adams most. It was how clearly it signified a “break of continuity,” as he puts it. The dynamo declared that his life had now been lived in two different ages, the ancient and the modern. It made the world new.

  I felt the same way about this machine at the center of the Internet. I believe in the transformative power of the network. But I’ve always been at a loss for the physical symbols of that transformation. The Internet lacks monuments. The screen is an empty vessel, an absence not a presence. From the standpoint of a user, the object through which the Internet arrives is totally flexible—an iPhone, a BlackBerry, a laptop, or a television. But the DE-CIX was my dynamo—a symbol of infinity, pulsing with eight hundred billion bits of data per second. (Eight hundred billion!) It was louder, if smaller, than Adams’s, and not on display in any grand hall but hidden behind half a dozen locked doors. But it was similarly a sign of the new millennium, something that made tangible the changes in society. I had come a long way from the squirrel in the backyard—and not only in miles traveled but in the movement from the edge of the network to the center, and in my understanding of the virtual world.

  The guard tapped her foot impatiently. Nipper and I stepped around the back of the machine, where powerful fans blew away the heat generated by its efforts at directing all those bits, those fragments of each of us. The hot wind prickled my eyes, and I teared slightly. Then Nipper locked the cage, and we hustled out the doors.

  In the car back to the DE-CIX office, Nipper asked if I was satisfied. Was it a good tour? I was looking for the real amid the merely virtual—something realer than pixels and bits—and I found that. Yet I was nagged by the thought that this machine in particular wasn’t so different from so many other machines, which only seemed to reiterate the inconsequentialness of its unique reality. I believed in the importance of that particular box among others, but I felt far out on a limb in doing so. There were other boxes in the world, certainly. But equally, there was a world inside this box. I was far from done with the Internet. The essence of what I was after was the unique intersection of a place and a technology: a singular box in a singular city that stood as a physical crossroads of our virtual world. A bald fact of geography. Nipper and I drove along beneath the towering cranes of the port lands.

  Back at the office, Orlowski fished around in a supply closet before emerging with my prize: a black T-shirt with large yellow writing on the front that said I PEERING. Then, out of another cardboard box, he produced a black Windbreaker, with a small DE-CIX logo on the breast. “Wear this in Amsterdam,” Orlowski said with a wink. “And send my best to Job.” Their competition extended to swag.

  That evening in my hotel room I quickly typed up my notes and copied the audio files from my digital recorder onto my laptop. The hotel room desk faced the window. It had begun to rain and the rush-hour streets were noisy with traffic. A tram clattered by. Then, for added protection, and to soothe a lingering bit of paranoia, I copied everything to an online backup service, which I had learned was physically located in a data warehouse in Virginia, deliberately close to Ashburn. On my screen, I watched the status bar grow as the big audio files made their way toward it. I had a good idea which way the bits went. I was shading in the map.

  After the stern grayness of Frankfurt, Amsterdam was a relief. I stepped off the tram into the middle of a busy spring evening on the Rembrandtplein, one of the living-room-like plazas that dot the center of the city. Pretty couples clattered by on heavy black bikes, coats floating open like wings. Forget the clichés of hash and whores; Amsterdam’s liberalness seemed far deeper, more considered. Walking the quiet side streets lining the canals I looked through uncovered windows into living rooms lit by stylish modern chandeliers and filled with books. It reminded me of home in Brooklyn—or Breuckelen, as the Dutch called it, with our similarly tightly packed, stoep-fronted town houses.

  In his book The Island at the Center of the World, Russell Shorto argues that this Dutch spirit is deep in the DNA of New York, and of America. There is, he writes, a shared “cultural sensibility that included a frank acceptance of differences and a belief that individual achievement matters more than birthright.” It felt strange to apply these terms to the Internet—to think of it as something other than stateless, fluid, even postnational. The glass rectangles of our screens and the browser windows within them have had a flattening effect on the world greater than the presence of any McDonald’s. Online, political borders are mostly invisible, smoothed over by the corporate triumvirate of Google, Apple, and Microsoft. But Amsterdam would begin to make the case otherwise. As it turned out, the Dutch Internet was very Dutch.

  From early on, the Amsterdam Internet Exchange was heralded by the government as a “third harbor” for the Netherlands—a place for the bits, in the way that Rotterdam is a place for ships and Schiphol for planes. The Dutch saw the Internet as only the latest in a five-hundred-year lineage of technologies that could be exploited for national gain. “In the Netherlands, forts, canals, bridges, roads, and ports have always been first of military importance, and later very useful for trade,” a 1997 op-ed argued. “The logistics of bits in the Netherlands will need a place of their own, in anticipation of catching a substantial proportion of the hundreds of billions of dollars at stake in the future world of Internet commerce.” History had already proved the model:
“Access to the open sea was in the time of the East India Company a decisive factor for success.... Providing access to the digital arteries of the global network will be decisive today.” If Frankfurt had the luck of being in the geographic center of Europe, Amsterdam would have the pluck to make itself one of the logical centers of the Internet. If there was a broader lesson in this, it was the need for governments to invest in infrastructure—and then get out of the way. Throughout its history, the Internet has needed help getting going, only to benefit enormously when left alone.

  The first ingredient in Amsterdam and everywhere was (and remains) fiber. In 1998, the Netherlands passed a law requiring all landowners to provide a right-of-way for private networks to lay fiber-optic cables—a right that was previously reserved for KPN, the national telephone company. Going one step further, the law stipulated that any company wishing to dig had to announce its intention and allow others to lay their own fiber and share the cost of construction. The intent was partly to keep the streets from being dug up repeatedly. But more important, the policy would eviscerate the old monopolies.

  The results were successful, nearly comically so. I visited Kees Neggers, the director of SURFnet, the Dutch academic computer network, and a key player in the Internet’s development in the Netherlands, at his office in a tower above the Utrecht railway station. From his bookshelf he pulled a bound report and opened it to a page of photographs taken at the moment of all that digging. In one, dozens of multicolored conduits bulged out of the soft ground of a polder, spreading out like a split whale on a beach. Another showed dozens of conduits pouring out of the doorway of an Amsterdam town house. Lain out on the cobblestone street, waiting to be buried, they filled a full lane of traffic. The colors—orange, red, green, blue, gray—indicated different owners; each contained hundreds of strands of fiber. It was an absurd abundance, and still is. “They shared the cost of digging along the dykes, toward the Amsterdam Internet Exchange, and that instantly provided an opportunity for all to connect—and to connect to each other very cheaply,” Neggers said. “And then it grew like mushrooms.”