Where Wizards Stay Up Late

by Matthew Lyon

  Cerf and others argued that TCP/IP couldn’t have been invented anywhere but in the collaborative research world, which was precisely what made it so successful, while a camel like OSI couldn’t have been invented anywhere but in a thousand committees. Perhaps most important, the Defense Department had already announced its choice of TCP and IP as the protocols that would run on military computers.

  ISO meetings, which were often held overseas in the 1980s, were occasionally painful experiences for people like Cerf and Postel. They attended them only to feel like King Canute yelling at the incoming tide. “I was the guy who was forever writing the counter-paper,” Cerf recalled.

  If anyone could claim credit for having worked tirelessly to promote TCP/IP, it was Cerf. The magic of the Internet was that its computers used a very simple communications protocol. And the magic of Vint Cerf, a colleague once remarked, was that he cajoled and negotiated and urged user communities into adopting it.

  While at MCI in 1983, building what was to become MCI Mail, Cerf tried to get IBM, Digital, and Hewlett-Packard to support TCP/IP, but they refused and adopted OSI instead. Digital, in particular, had invested a great deal of money in its DECNET network, based on OSI. TCP/IP, they argued, was “a research thing.” Cerf was disappointed and a little irked. “They said they weren’t gonna make products out of it. So I had to build MCI Mail out of a dog’s breakfast of protocols.” Cerf patched together MCI Mail from existing protocols that were being used internally by Digital and IBM, and developed a few more specifically for MCI Mail. “I understood why they took the position they did, but it still bugged me.”

  The Switch

  On January 1, 1983, the ARPANET was to make its official transition to TCP/IP. Every ARPANET user was supposed to have made the switch from the Network Control Protocol to TCP/IP. On that day, the protocol that had governed the ARPANET would be mothballed, so that only those machines running the new protocols could communicate over the network. Some sites that hadn’t made the transition yet pleaded their case to Postel or his colleague Dan Lynch, or, to Bob Kahn, who was overseeing the transition, and usually won a grace period. But by the spring of 1983, either you had made the conversion or your machine fell off the network.

  As milestones go, the transition to TCP/IP was perhaps the most important event that would take place in the development of the Internet for years to come. After TCP/IP was installed, the network could branch anywhere; the protocols made the transmission of data from one network to another a trivial task. “To borrow a phrase,” Cerf said, “now it could go where no network had gone before.” An impressive array of networks now existed—from the ARPANET to TELENET to Cyclades. There were so many, in fact, that in an attempt to impose some order, Jon Postel issued an RFC assigning numbers to the networks.

  In 1983 the Defense Communications Agency decided that the ARPANET had grown large enough that security was now a concern. The agency split the network into two parts: the MILNET, for sites carrying nonclassified military information, and the ARPANET for the computer research community. Before the split, there were 113 nodes in the combined network. Afterward, 45 nodes remained with the ARPANET, and the rest went to MILNET. Administratively and operationally there were two different networks, but with gateways connecting them users couldn’t tell. The old ARPANET had become a full-fledged Internet.

  In 1988, five years after the 1983 ARPANET transition to TCP/IP, the ISO finally produced standards for Open Systems Interconnection, and the U.S. Government immediately adopted the rival OSI protocols as its official standard. It appeared that OSI might prevail over TCP/IP. In Europe, where national governments decree the standards, it seemed an article of faith that OSI was the solution.

  On the other hand, an American culture of the Internet was growing exponentially, and its foundation was TCP/IP. And while governments throughout Europe were anointing OSI, something of an underground movement sprang up at European universities to implement TCP/IP.

  One key development in determining the outcome between TCP/IP and OSI turned out to be the popularity of the UNIX operating system, which had been developed at AT&T’s Bell Laboratories in 1969.

  Programmers liked UNIX for two primary reasons: Its flexibility let them tailor it to whatever program they were working on, and it was “portable,” meaning it could be made to work on many different computers. In the late 1970s, programmers at Berkeley developed their own brand of UNIX, and seeded the computer science community with it. Berkeley UNIX eventually became a fixture at universities and research institutions all over the world. Around 1981, Bill Joy, a UNIX hacker at Berkeley, got ARPA funding to write TCP/IP into a version of Berkeley UNIX. BBN had already written a version of UNIX with TCP/IP, but Joy didn’t like it and decided to do it his own way.

  Then, in 1982, Joy joined a couple of Stanford Business School graduates who were starting a new company to build and sell powerful “workstations,” computers that were of an order of magnitude more powerful than personal computers. Joy was brought in as the UNIX expert. They called their company Sun (for Stanford University Network) Microsystems. The first Sun machines were shipped with the Berkeley version of UNIX, complete with TCP/IP. Berkeley UNIX with TCP/IP would be crucial to the growth of the Internet. When Sun included network software as part of every machine it sold and didn’t charge separately for it, networking exploded.

  It further mushroomed because of Ethernet.

  While packet radio and SATNET sparked the thinking about a conceptual framework for internetworking, they were largely experimental. Ethernet—the local area network designed by Bob Metcalfe and his colleagues at Xerox PARC back in 1973—was a practical solution to the problem of how to tie computers together, either on a campus or at a company. Xerox began selling Ethernet as a commercial product in 1980. At around the same time, Bob Taylor’s division at Xerox PARC gave a grant to major research universities in the form of Ethernet equipment, powerful computers, and laser printers. It amounted to millions of dollars worth of hardware. Then a small networking company called Ungermann-Bass sold Ethernet as a connection between terminals and host computers. And Metcalfe started his own company, 3Com, to sell Ethernet for commercial computers, including Sun machines.

  Throughout the early 1980s, local area networks were the rage. Every university hooked its workstations to local area networks. Rather than connect to a single large computer, universities wanted to connect their entire local area network—or LAN—to the ARPANET.

  Ethernet made this possible. Ethernets were simple and, compared to the 50-kilobit lines of the ARPANET, they were tremendously powerful. Their rapid growth in the university and research community pushed the demand for network interconnection. If your whole university was not connected to the ARPANET, CSNET gave you a way to connect one computer at your university to the ARPANET. But it was Ethernet that created a huge networking constituency.

  At major research universities there would be a network of hundreds of computers that could all talk to each other over an Ethernet network. To send traffic from an Ethernet in say, San Diego, to another Ethernet in Buffalo, you sent it through the ARPANET hub. In this way, the ARPANET was the centerpiece of what was called the ARPA Internet. And through the first half of the 1980s, the ARPA Internet resembled a star, with various networks surrounding the ARPANET at the center.

  Perhaps what TCP/IP had to recommend it most was the fact that it was unerringly “open.” Its entire design was an open process, following a path first blazed by Steve Crocker and the Network Working Group and continuing into the Internet. The ARPANET, and later the Internet, grew as much from the free availability of software and documentation as from anything else. (By contrast, Digital Equipment’s DECNET was a proprietary network.) The Internet also supported a wide range of network technologies. Although the satellite and packet-radio networks had finite lifetimes, they helped open developers’ eyes to the need to handle a multitude of different networks.

  Reforming e-mail

&nbs
p; The TCP and IP standards weren’t the only major renovation to networking in the early 1980s. For years, every e-mail program written for the ARPANET had depended on the original file-transfer protocol to serve as its barge for schlepping the mail back and forth. It may have been a neat hack to attach the mail commands to the file-transfer protocol at first, but the processing of e-mail had grown more complicated. In a message to his colleagues in the MsgGroup mailing list one day in late August 1982, Postel said, “If you really go look at the FTP spec, you will see that the mail commands are really some sort of wart.” Postel and a lot of others felt it was time to build a completely separate transfer mechanism for mail.

  Since the network was undergoing massive rearrangement anyway with the switch to TCP/IP, this seemed an appropriate time to bring out the new standard. Postel and his colleagues called it the simple mail transfer protocol (SMTP). It clarified existing practices, while adding a few new control features.

  At the same time, the growth of the network gave rise to a new problem. “When we got to about two thousand hosts, that’s when things really started to come apart,” said Craig Partridge, a programmer at BBN. “Instead of having one big mainframe with twenty thousand people on it, suddenly we were getting inundated with individual machines.” Every host machine had a given name, “and everyone wanted to be named Frodo,” Partridge recalled.

  Sorting out the Frodos of the Internet wasn’t unlike sorting out the Joneses of Cleveland or the Smiths of Smithville. Where one lived, precisely, was important in differentiating who one was. For years, sorting this out was among the most troublesome, messiest issues for the Internet, until at last a group chiseled out a workable scheme, called the domain name system, or DNS.

  The core of the DNS team was Jon Postel and Paul Mockapetris at ISI, and BBN’s Craig Partridge. They spent three months working out the details of the new addressing scheme and in November 1983 came forward with two RFCs describing the domain name system. “DNS was a very significant change in the way we thought about the system being organized,” said Postel. “Tree-branching” was the guiding metaphor. Each address would have a hierarchical structure. From the trunk to the branches, and outward to the leaves, every address would include levels of information representing, in progression, a smaller, more specific part of the network address.

  But that sparked a debate about the sequence of the hierarchy; what should come first or last. Postel and others finally decided on a specific-to-general addressing scheme. The Internet community also argued back and forth over what to name the domains, delaying any implementation for about a year. It was asserted by some, unconvincingly, that domain names should reflect specific funding sources—MIT, DARPA, for example. Eventually, a committee agreed on seven “top-level” domains: edu, com, gov, mil, net, org, and int. Now there could be seven Frodos: a computer named Frodo at a university (edu), one at a government site (gov), a company (com), a military site (mil), a nonprofit organization (org), a network service provider (net), or an international treaty entity (int).

  DARPA began pressuring people to adopt DNS addresses in 1985. In January 1986 a grand summit meeting took place on the West Coast, bringing together representatives of all the major networks. By the time the summit was over, everyone had agreed that yes, they really believed in the DNS concept. “And yes, here was how we were going to make it work,” Partridge recalled, “And yes, we have the technology to make it all fly.”

  Pulling the Plug

  The first hint Cerf got that the Internet was going to be embraced by a world outside the scientific and academic communities came in 1989, when he walked on to the exhibition floor at Interop, a trade show started by Dan Lynch in 1986 to promote interconnectivity through TCP/IP. In its first couple of years, Interop was attended by a few hundred hardcore networking people. By 1989 the show was teeming with men and women in business attire. “It was an epiphany to walk into Interop and see the major money being spent on exhibitions with huge demonstrations set up,” Cerf said. “I realized, oh my God, people are spending serious money on this.” The exhibitors had names like Novell, Synoptics, and Network General. “We started looking at the network statistics and realized we had a rocket on our hands.” For years Cerf had seen the Internet as a successful, satisfying experiment. Occasionally he had hoped the Internet might reach a wider world of users. Now here was evidence that it was doing just that.

  By this time, virtually everyone was using TCP/IP. And there was an ever-increasing infrastructure built upon TCP/IP in Europe. TCP/IP was so widespread and so many people depended on it, that taking it down and starting over seemed unthinkable. By virtue of its quiet momentum, TCP/IP had prevailed over the official OSI standard. Its success provided an object lesson in technology and how it advances. “Standards should be discovered, not decreed,” said one computer scientist in the TCP/IP faction. Seldom has it worked any other way.

  By the late 1980s the Internet was no longer a star with the ARPANET its center; it was a mesh, much like the ARPANET itself. The NSFNET program had democratized networks as even CSNET hadn’t. Now anyone on a college campus with an Internet connection could become an Internet user. The NSFNET was fast becoming the Internet’s spine, running on lines that were more than twenty-five times faster than ARPANET lines. Users now had a choice between connecting to the ARPANET or to the NSFNET backbone. Many chose the latter, not only for its speed but because it was so much easier to connect to.

  As the 1990s approached, the number of computers in the world that were connected to one another via the NSFNET far outstripped the number of computers connected to one another via the ARPANET. The ARPANET was now just one of hundreds of ARPA Internet networks, and a dinosaur, unable to evolve as quickly as the rest of the Internet.

  Bob Kahn, DARPA’s sole remaining champion of networking, had left the agency in 1985 to form the Corporation for National Research Initiatives, a nonprofit company whose charter was to foster research and development for a “national information infrastructure.” The people now running DARPA weren’t particularly interested in networking. In their view, all the interesting problems had been solved. Moreover, the agency was distracted by President Ronald Reagan’s Star Wars program.

  The ARPANET itself, which cost ARPA $14 million a year to run, looked arthritic next to the higher-speed NSFNET. DARPA management decided the ARPANET had outlived its usefulness. It was time to shut it down.

  Mark Pullen, a DARPA program manager who now ran the networking project, was given the task of decommissioning the ARPANET. Exactly who gave the order from within DARPA’s higher reaches was never made quite clear. “No one wanted to be the ghoul that turned off the ARPANET,” Pullen said, “so I became the source of the policy.” Pullen’s plan was to pull sites off the ARPANET and put them on the NSFNET backbone.

  It was hard telling Bob Kahn about the plan to decommission the network. Kahn had hired Pullen, and now Pullen played the executioner. “I had a sense he might feel I was turning off his greatest achievement,” Pullen said. “The one that seemed to hurt him worse was when I turned off the old SATNET.” SATNET was slow and expensive and antiquated. “No doubt he must have felt it was his very own child. For valid reasons. But after he thought about it, he agreed I was doing the right thing.” (As it turned out, the money DARPA saved by turning off the ARPANET helped fund Kahn’s new project.)

  One by one, Pullen turned off the IMPs and TIPs that still lay at the heart of the original network. There was a certain sadness in its demise that called to mind the scene from Arthur C. Clarke’s 2001: A Space Odyssey where the fictional fifth-generation computer HAL is threatening its mission and has to be dismantled circuit by circuit. As HAL gradually loses its “mind,” it makes pathetic appeals for its “life” to Dave, the astronaut, who is doing the dismantling.

  In the case of the ARPANET, the network died but its pieces lived on. “It wasn’t all that different from the breakup of Ma Bell,” Pullen recalled. “It involved locating clusters of
ARPANET sites and finding someone to take them over.” In most cases, Pullen transferred each ARPANET site to one of the regional networks, and eased the transition by subsidizing the cost for a while. With the exception of two sites that went on to the MILNET, all the sites went to one or another of the regional networks. “I never had anyone object all that loudly,” Pullen said. “I think they all knew the time had come.” One site at a time, Pullen found new homes for them. Where there wasn’t a home, DARPA and NSF helped create one. Several ARPANET sites in Southern California quickly formed their own regional network and called it Los Nettos; it was run by Danny Cohen and Jon Postel. The IMPs themselves were powered down, uncabled, and shipped away. Most were simply junked. Others went into service on the MILNET. The Computer Museum in Boston got one, and Len Klein-rock put IMP Number One on display for visitors at UCLA. The last IMP to go was at the University of Maryland. By coincidence, Trusted Information Systems, a company in Maryland where Steve Crocker now worked, was connected to that IMP. Crocker had been there at the birth and he was there at the death.

  By the end of 1989, the ARPANET was gone. The NSFNET and the regional networks it had spawned became the principal backbone. That year, to mark both the ARPANET’s twentieth anniversary and its passing, UCLA sponsored a symposium and called it “Act One.”

  In his speech, Danny Cohen found a source of inspiration, and he said this:

  “In the beginning ARPA created the ARPANET.

  “And the ARPANET was without form and void.

  “And darkness was upon the deep.

  “And the spirit of ARPA moved upon the face of the network and ARPA said, ‘Let there be a protocol,’ and there was a protocol. And ARPA saw that it was good.

  “And ARPA said, ‘Let there be more protocols,’and it was so. And ARPA saw that it was good.