Where Wizards Stay Up Late

by Matthew Lyon

  Communicating by computer and using remote resources were still cumbersome processes. For the most part, the Net remained a user-hostile environment, requiring relatively sophisticated programming knowledge and an understanding of the diverse systems running on the hosts. Demand was growing among users for “higher-level” application programs aimed at helping users tap into the variety of resources now available. The file-transfer and Telnet programs existed, but the user community wanted more tools, such as common editors and accounting schemes.

  SRI’s Network Information Center estimated the number of users at about two thousand. But a newly formed users’ interest group, called USING, was convinced there was a gap between the design of the network resources and the needs of the people trying to use those resources. Envisioning itself as a lobby group, a consumers’union even, USING began immediately to draw up plans and recommendations for improving the delivery of computer services over the ARPANET.

  But DARPA saw no need to share authority with a tiny self-appointed watchdog group made up of people the agency viewed as passengers on its experimental vehicle. The initiative died after about nine months with a terse memo from a DARPA program manager named Craig Fields, warning the group that it had overstepped its bounds. With neither funding nor official support for their effort forthcoming, members put USING into a state of suspended animation from which it never emerged.

  Other problems developed for DARPA as the profile of the network began to rise. Like the USING insurgency, most were relatively minor affairs. But together they illustrated the ongoing tensions related to DARPA’s stewardship of the network. One area of tension had to do with DARPA’s Pentagon masters. IPTO in particular managed to steer clear of the most blatantly military research. But while the Illinois students were wrong about the ILLIAC IV being used for simulated bombing missions against North Vietnam, there were plans to use it for nuclear attack scenarios against the Soviet Union. Similarly, researchers of all sorts used seismic information stored on the Computer Corporation of America (CCA) database server, information that was being collected to support Pentagon projects involving underground atomic testing.

  In the late 1960s, growing political unrest—both violent and nonviolent—caught the U.S. military by surprise. Army intelligence knew all about Prague, Berlin, and Moscow, but now the Pentagon was contemplating Newark, Detroit, and Chicago. The Army gathered information from dozens of U.S. cities on the location of police and fire stations, hospitals, and so forth. Someone in the Pentagon thought it would be a good idea to keep track of local troublemakers as well.

  In 1972 public outcry erupted over the Army’s information gathering, and the order went out for the files to be destroyed immediately. But three years later, allegations surfaced that Army intelligence officers had instead used the ARPANET to move the files to a new location. When the story broke, the fact that something like the ARPANET existed was news to most Americans. That the story was reported in the most draconian, cloak-and-dagger terms only added to the stormy reaction. The result was a Senate investigation in which DARPA was called upon to explain how it was using the ARPANET.

  DARPA eventually proved that the Army files had not moved on the ARPANET by reviewing hundreds of rolls of Teletype printouts that had been stored in a dusty crawl space at BBN. DARPA was vindicated, but a perceived entanglement with the Army’s clandestine operations was the last thing the ARPANET needed.

  Changes at DARPA

  Discussions about how DARPA would ultimately divest operational responsibility for the network had already begun around 1971. DARPA had set out to link the core processing capabilities in America’s top computer science research centers, and as far as the agency was now concerned, it had accomplished that. Its mission was research. It wasn’t supposed to be in the business of operating a network. Now that the system was up and running, it was becoming a drain on other priorities. It was time for DARPA to shed the service provider’s role.

  Handling the transition was a touchy matter. The ARPANET was now a valuable tool, and Roberts’s goal was to ensure its continued development. He commissioned several studies to help determine the best option. The best route, it seemed, was to maintain a networking research effort, but sell off the network itself to a private contractor. But sell to whom? The market for data-communications networks was still largely uncharted, and the big communications companies remained as skeptical as ever about the DARPA technology.

  When Roberts contacted AT&T to see if it wanted to take over the ARPANET, AT&T formed a committee of corporate and Bell Labs staff and studied the idea for months. AT&T could have owned the network as a monopoly service, but in the end declined. “They finally concluded that the packet technology was incompatible with the AT&T network,” Roberts said.

  Others were not so blind to the prospects of computer networking. In July 1972 three engineers left BBN to form a company that would market a commercial network called Packet Communications Incorporated. And BBN itself had spoken to Roberts about buying the network and setting up a subsidiary company to operate it. Small, specialized, commercial carriers like these were the obvious solution to DARPA’s problem.

  But soon Roberts had a new problem. In early 1973, BBN recruited him to run a new subsidiary called TELENET (not to be confused with Telnet, the program for remote log-ins), which would market a private packet-switching service. Now unable to recommend a sale by the government to TELENET, Roberts arranged to have the ARPANET transferred temporarily to the Defense Communications Agency—the same agency that Baran had refused to let build his network ten years earlier. The generals and majors and captains were still only slightly more receptive to the idea of a packet-switched network than AT&T, but Roberts expected this to be only an interim arrangement.

  Having decided to accept the post at TELENET, Roberts now had the job of finding a successor. DARPA’s Information Processing Techniques Office, however, no longer held the appeal it once had for those in academe. Roberts approached a couple of his principal investigators at universities, but people who had active research programs did not want to leave them. Others were concerned about the salary cuts they would have to take to come to the DARPA office.

  When Licklider heard of the trouble Roberts was having, he offered to return if Roberts needed him. Roberts knew it was just a gesture on Licklider’s part. Lick was now happily entrenched back at MIT. But after six months of searching, Roberts decided he didn’t have a choice. When Roberts called Lick’s office at MIT, he was told that Lick was on a walking tour in England. Roberts tracked him down in the middle of Wales and asked him if he was serious about taking the job. Licklider said yes. “I never saw Larry so overjoyed as when he finally got his replacement, because he was ready to go,” recalled a colleague of Roberts. “I think some of his delight was also the fact that Lick was going to take it, because he liked him. Everybody did.”

  One of the first problems Lick faced upon his return was an awkward one involving BBN, his onetime employer. BBN was refusing to release the IMP source code—the original operating program written by the IMP Guys five years earlier.

  BBN’s apparent impulse to control every aspect of the network created a certain tension from the beginning. Len Kleinrock and his group at UCLA’s Network Measurement Center had found it particularly frustrating. The center’s job was to find problems in the network, but when they did, BBN refused to help. “Whenever we found a software bug or inefficiency, we’d alert Heart specifically, and the response we got was typically almost a stiff arm,” said Kleinrock. “He’d say, ‘Look, the network is working, I’ve got a network to keep up and running and we’ll put your comment in the queue.’ We couldn’t fix it ourselves because we didn’t have the source code.”

  The intellectual property issue finally boiled over when the engineers who left BBN to start their own company made a pointed request for their former employer to turn over the IMP source code. When BBN refused, they appealed to DARPA. While keeping source code prop
rietary is usually the prerogative of the company that develops it, the IMP source code was different, for it had been developed by BBN with federal funds. Moreover, BBN was in the midst of starting up the TELENET subsidiary, which would be competing with the company started by the engineers. There was some concern at DARPA that if anyone in Congress or the press corps latched onto the fact that BBN’s subsidiary and no one else had access to the Defense Department–sponsored IMP technology, the agency could have a major problem on its hands.

  Frank Heart and Dave Walden at BBN argued that since the source code was changed frequently to improve performance or fix bugs, the company was uncomfortable with distributing software that would become obsolete. The company held its ground.

  Steve Crocker, by then a DARPA program manager who oversaw most of BBN’s contracts, took charge of the situation. He had control of about $6 million worth of work a year at BBN, about a quarter of the firm’s gross revenues. “I seriously considered moving all of the work we were supporting at BBN to other places because we couldn’t reach closure with them on the data rights to the IMP code,” he said. And he let BBN know what he was thinking.

  Lick had had a long association with BBN, and he had high regard for people there, but he was dismayed by their posture. Reaction throughout the computer community was much the same. Finally, in direct response to Crocker’s threat, BBN agreed to provide the code to whoever asked for it, charging a nominal handling fee. “This was just an early version of much more serious intellectual property rights issues that emerged across the industry over the next few decades,” Crocker said.

  With Bob Kahn’s help, Licklider also finished the job of spinning off the network to the Defense Communications Agency. Roberts’s earlier probes into selling off the network notwithstanding, federal rules required that a resource as rich as the ARPANET couldn’t just be sold to an outside party until the Defense Department determined whether it had a need for the Net in Defense. The agency eventually decided it did. In the summer of 1975,DCA took over the network management job from DARPA. Now DCA set operational policy for the network. DCA decided such things as where and when new nodes would be installed and what the configuration of data lines should be. And BBN retained the contract for network operations, which meant the company carried out the decisions made by DCA. Shortly after the transition had been made, Lick returned to MIT, where he would spend the remainder of his career.

  Soon everyone at BBN noticed an increasing number of forms that had to be filled out, even for the smallest of jobs. The DCA bureaucracy rattled many of the university people, too. “The agency generated a blizzard of memoranda from colonels and generals about things you were and weren’t allowed to do,” recalled Brian Reid. A few months after DCA took over the ARPANET, several of the graduate students in Stanford’s computer science department came to a Halloween party dressed as colonels.

  Relieved of the day-to-day management chores, DARPA was now able to concentrate on developing the new CATENET protocols. By 1975, Yogen Dalal, a Stanford graduate student, had polished the Transmission-Control Protocol from Cerf and Kahn’s 1974 paper into a set of concrete specifications. The TCP specification was sent to three separate places for concurrent implementation: BBN, Cerf’s computer lab at Stanford, and University College, London.

  At about the same time, Kahn and Steve Crocker began talking to Cerf about taking a job as a DARPA program manager in Washington. “I flew there and landed in a big snowstorm and thought, I don’t want to live in this part of the country,” Cerf recalled. “So I said no. But the real reason was I was afraid I wouldn’t do a very good job. I thought it would be a very visible position and if I screwed up, everybody would know.”

  A year later, Kahn and Crocker tried again. This time Cerf accepted. “I was getting a little tired of being so fragmented at Stanford. I couldn’t get any research done. So I thought, Why not go to DARPA and have a bigger impact, because instead of the small little budgets I got at Stanford for my work, at DARPA I’d have an opportunity to have a bigger impact with more money to spend.”

  As a program manager, Cerf was given responsibility for the packet radio, packet satellite, and research programs in what was now simply called the ARPA Internet. Cerf also continued working intensively on refining the TCP specification. A milestone occurred in October 1977, when Cerf and Kahn and a dozen or so others demonstrated the first three-network system with packet radio, the ARPANET, and SATNET, all functioning in concert. Messages traveled from the San Francisco Bay area through a packet-radio net, then the ARPANET, and then a dedicated satellite link to London, back across the packet-satellite network and across the ARPANET again, and finally to the University of Southern California’s Information Sciences Institute (ISI) in Marina del Rey. The packets traveled 94,000 miles without dropping a single bit.

  During a break from a meeting Cerf chaired at ISI to discuss TCP in early 1978, Cerf, Postel, and Danny Cohen, a colleague of Postel’s at ISI, got into a discussion in a hallway. “We were drawing diagrams on a big piece of cardboard that we leaned up against the wall in the hallway,” Postel recalled. When the meeting resumed, the trio presented an idea to the group: break off the piece of the Transmission-Control Protocol that deals with routing packets and form a separate Internet Protocol, or IP.

  After the split, TCP would be responsible for breaking up messages into datagrams, reassembling them at the other end, detecting errors, resending anything that got lost, and putting packets back in the right order. The Internet Protocol, or IP, would be responsible for routing individual datagrams.

  “I remember having a general guideline about what went into IP versus what was in TCP,” Postel recalled. “The rule was ‘Do the gateways need this information in order to move the packet?’If not, then that information does not go in IP.” This twist on the protocol was inspired by a group of engineers from the Xerox Corporation’s Palo Alto Research Center who had attended the Stanford seminar. The Xerox team had solved similar issues on a smaller, proprietary scale by defining a family of protocols called the PARC Universal Packet, or PUP. Once Postel decided that creating a separate protocol was the right thing to do, he set about making sure it got done. With a clean separation of the protocols, it was now possible to build fast and relatively inexpensive gateways, which would in turn fuel the growth of internetworking. By 1978, TCP had officially become TCP/IP.

  ETHERNET

  In 1973, just when Cerf and Kahn had begun collaborating on the concept of internetworking, Bob Metcalfe at Xerox PARC was inventing the technological underpinnings for a new kind of network. Called a short-distance, or local-area, network, Metcalfe’s network would connect computers not in different cities but in different rooms.

  Metcalfe had received his undergraduate degrees in electrical engineering and management from MIT and enrolled at Harvard for graduate school. But he hated Harvard immediately. “Harvard is full of old-money people,” he said. “MIT is full of no money. It was a class thing.”

  Metcalfe took a job at MIT, where he felt more comfortable. The Institute was about to join the ARPANET, and he was assigned to build the interface between MIT’s PDP-10 and its IMP. Harvard had a PDP-10, too, and Metcalfe offered to make a duplicate interface to give to Harvard. But the networking people at Harvard declined the offer. “They said they couldn’t possibly let a graduate student do something that important,” said Metcalfe. Harvard officials decided to have BBN do it. BBN in turn gave the job to its graduate student in residence, Ben Barker, who recruited John McQuillan, a fellow Harvard graduate student, to help him.

  Although enrolled at Harvard, Metcalfe stayed at MIT to work on his dissertation, an examination of the ARPANET. When Metcalfe submitted the work, Harvard rejected the finished dissertation as insufficiently theoretical (too much engineering and not enough science, he was told). The rejection was embarrassing for Metcalfe, who had just taken a job at Xerox Corporation’s Palo Alto Research Center after persuading his wife to leave her job to joi
n him. He went to Xerox PARC anyway and began looking for a more theoretical topic for his thesis.

  Then in 1972, while on DARPA-related business for PARC, Metcalfe stayed at the Washington home of his friend Steve Crocker. Crocker had gathered together some of the best technical people from the early sites to assist new sites and called these people “network facilitators.” Metcalfe, who had become something of an ARPANET expert at MIT, was one of them.

  Crocker had just been visited by Norm Abramson, the primary architect of the ALOHA network at the University of Hawaii. On the night of Metcalfe’s visit, Crocker left one of Abramson’s papers about the ALOHA network out. Metcalfe picked it up and read it before going to sleep. The paper kept him up much of the night. “The math in the paper was not only familiar but infuriating,” he said, “because they made the typical inaccurate assumptions that people make so that their models work.”

  Metcalfe’s chance encounter with Abramson’s paper was to change his life. “I set out to make a new model for the ALOHA system.” Within a few weeks, he was on a Xerox-funded trip to the University of Hawaii. He stayed a month, and before coming home had added an extensive analysis of the ALOHA system to his thesis. It was just the theoretical boost the dissertation needed. When he resubmitted the work, it was accepted.