Cuckoo's Egg

by Clifford Stoll

  Now, Dave and I could guess Roy’s reaction to our problem: “Why the hell did you leave our doors wide open?”

  Our boss’s reaction might be predictable, but how should we react? Dave’s first thought was to disable the suspect account and forget about it. I felt we ought to send a nastygram to whoever was breaking in, telling him to stay away or we’d call his parents. After all, if someone was breaking in, it was bound to be some student from down on campus.

  But we weren’t certain that someone was breaking into our system. It might explain some of our accounting problems—someone learns the system manager’s password, connects to our machine, creates a new account, and tampers with the accounting system. But why would someone use a new account if they already had access to the system manager account?

  Our boss never wanted to hear bad news, but we swallowed hard and called a lunchtime meeting. We had no clear proof of a hacker, just circumstantial pointers, extrapolated from trivial accounting errors. If there was a break-in, we didn’t know how far it extended, nor who was doing it. Roy Kerth blasted us. “Why are you wasting my time? You don’t know anything and you haven’t proven a whit. Go back and find out. Show me proof.”

  So how do you find a hacker? I figured it was simple: just watch for anyone using Sventek’s accounts, and try to trace their connection.

  I spent Thursday watching people log into the computer. I wrote a program to beep my terminal whenever someone connected to the Unix computer. I couldn’t see what each user was doing, but I could see their names. Every couple minutes my terminal beeped, and I’d see who had logged in. A few were friends, astronomers working on research papers or graduate students plugging away on dissertations. Most accounts belonged to strangers, and I wondered how I could tell which connection might be a hacker.

  At 12:33 on Thursday afternoon, Sventek logged in. I felt a rush of adrenaline and then a complete letdown when he disappeared within a minute. Where was he? The only pointer left for me was the identifier of his terminal: he had used terminal port tt23.

  Sitting behind a computer terminal, fingers resting on his keyboard, someone was connecting into our lab. My Unix computer gave him the address of port tt23.

  Well, that’s a start. My problem was to figure out which physical wires corresponded to the logical name tt23.

  Terminals from our laboratory and modems from dial-in telephones are all assigned “tt” labels, while network connections show up as “nt.” I figured that the guy must be either from our laboratory or dialing in on a phone line over a modem.

  For a few seconds, I’d sensed a hesitant feeler into our computer. Theoretically, it must be possible to trace the path from computer to human. Someone must be at the far end of that connection.

  It would take six months to track that path, but my first step was to trace the connection out of the building. I suspected a dial-in modem, connected from some telephone line, but it conceivably might be someone at the laboratory. Over the years, well over five hundred terminals had been wired in, and only Paul Murray kept track. With luck, our homegrown hardware connections were documented better than the home-brew accounting software.

  Paul’s a reclusive hardware technician who hides in thickets of telephone wire. I found him behind a panel of electronics, connecting some particle detector to the lab-wide ethernet system. Ethernets are electronic pipelines connecting hundreds of small computers. A few miles of orange ethernet cable snaked through our lab, and Paul knew every inch of it.

  Cursing me for surprising him in the middle of soldering a wire, he refused to give me any help until I proved that I had a legitimate need to know. Aw, hell. Hardware technicians don’t understand software problems, and software jockeys know nothing about hardware.

  Years of ham radio had taught me to solder, so Paul and I had at least one common denominator. I picked up his spare soldering iron and earned his grudging respect after a few minutes of burning my fingers and squinting. Finally, he disentangled himself from the ethernet cables and showed me around the LBL communications switchyard.

  In this roomful of wires, the telephones, intercoms, radios, and computers were all interconnected by a tangle of cables, wires, optical fibers, and patch panels. The suspicious port tt23 entered this room and a secondary computer switched it to one of a thousand possible terminals. Anyone dialing into the lab would be randomly assigned to a Unix port. The next time I saw a suspicious character, I’d have to run over to the switchyard and unwind the connection by probing the switching computer. If he disappeared before I disentangled the connection, well, tough. And even if I did succeed, I’d only be able to point to a pair of wires entering the laboratory. I’d still be a long way from the hacker.

  By lucky accident, though, the noontime connection had left some footprints behind. Paul had been collecting statistics on how many people used the switchyard. By chance he had recorded the port numbers of each connection for the past month. Since I knew the time when Sventek was active on port tt23, we could figure out where he came from. The printout of the statistics showed a one-minute 1200-baud connection had taken place at 12:33.

  1200 baud, huh? That says something. The baud rate measures the speed that data flows through a line. And 1,200 baud means 120 characters per second—a few pages of text every minute.

  Dial-up modems over telephone lines run at 1200 baud. Any lab employee here on the hill would run at high speed: 9600 or 19,200 baud. Only someone calling through a modem would let their data dribble out a 1200-baud soda straw. And the anonymity and convenience of these dial-in lines are most inviting to strangers. So pieces were beginning to fit together. I couldn’t prove that we had a hacker in the system, but someone dialed into our lab and used Sventek’s account.

  Still, the 1200-baud connection was hardly proof that a hacker entered our system. An incomplete trace, especially one that went no farther than my building, would never convince my boss that something was up, something weird. I needed to find incontrovertible evidence of a hacker. But how?

  Roy Kerth had shown me the high-energy particle detectors attached to the Bevatron: they find jillions of subatomic interactions, and 99.99 percent are explainable by the laws of physics. Spending your time exploring each particle trail will lead you to conclude that all the particles obey known physics, and there’s nothing left to discover. Alternatively, you could throw away all the explainable interactions, and only worry about those that don’t quite satisfy the canonical rules.

  Astronomers, distant cousins of high-energy physicists, work along similar lines. Most stars are boring. Advances come from studying the weirdies—the quasars, the pulsars, the gravitational lenses—that don’t seem to fit into the models that you’ve grown up with. Knowing cratering statistics on the planet Mercury tells you how often the planet was bombarded in the early solar system. But study the few craters intersected by scarps and ridges and you’ll learn how the planet shrank as it cooled during its first billion years. Collect raw data and throw away the expected. What remains challenges your theories.

  Well, let’s apply this way of thinking to watching someone visiting my computer. I’ve got a terminal on my desk, and can borrow a couple others. Suppose I just watched the traffic coming into the computer center. There’s about five hundred lines entering the system. Most of these lines run at 9600 baud, or around one hundred fifty words per second. If half the lines are used at any time, I’d have to read well over ten thousand pages every minute. Right. No way could I monitor that kind of traffic on my terminal.

  But the high speed lines come from people at LBL. We’d already traced one suspicious connection to a 1200-baud line. There are fewer of them (we can’t afford too many incoming phone lines), and they’re slower. Fifty lines at 1200 baud might generate a hundred pages a minute, still far too fast to watch on the screen of my terminal. I might not be able to watch fifty people running at once, but maybe I could print out all their interactive sessions, and read the piles of paper at my leisure. A paper
printout would provide hard proof of someone messing around; if we found nothing suspicious, we could drop the whole project.

  I’d record everything that happened during each 1200-baud connection. This would be technically challenging—since I didn’t know which line the hacker was calling, I’d have to monitor four dozen. More worrisome was the ethical problem of monitoring our communications. Did we have the right to watch the traffic running through our lines?

  My sweetheart, Martha, was just finishing law school. Over a deep-dish pizza, we talked about the implications of someone breaking into a computer. I wondered how much trouble I’d be in by listening to incoming traffic.

  “Look,” she mumbled, burning the roof of her mouth on the vulcanized mozzarella. “You’re not the government, so you don’t need a search warrant. The worst it would be is invasion of privacy. And people dialing up a computer probably have no right to insist that the system’s owner not look over their shoulder. So I don’t see why you can’t.”

  So with a clear conscience, I started building a monitoring system. We had fifty 1200-baud lines, and a hacker might be using any of them. I had no equipment designed to record the traffic.

  But there’s an easy way to record a hacker’s activity. Modify the Unix operating system so that whenever a suspicious person logged in, the system records all the keystrokes. This was tempting, because I only had to add some lines of code to the Unix daemon software.

  The daemons themselves are just programs that copy data from the outside world into the operating system—the eyes and ears of Unix. (The ancient Greek daemons were inferior divinities, midway between gods and men. In that sense, my daemons are midway between the god-like operating system and the world of terminals and disks.)

  I could split the daemon’s output like a T-joint in a pipe, so the hacker’s keystrokes would simultaneously go to both the operating system and a printer. Software solutions are simple and elegant.

  “Muck with the daemons at your own risk,” Dave Cleveland said. “Just respect their timing needs.”

  Wayne also warned me, “Look, if you goof up, you’ll break the system for sure. It will turn the system into molasses, and there’s no way you’ll follow everything that happens. Just wait till you see the system console print out ‘Panic kernel mode interrupt’—don’t come crying on my shoulder!”

  Dave chipped in, “Hey, if your hacker has any Unix experience, he’s bound to notice a change in the daemons.”

  That convinced me. A sharp systems person would notice that we’d changed the operating system. The moment the hacker knew someone was watching him, he’d trash our databases and scram. Our wiretaps had to be completely undetectable, even to an omnipotent super-user. Silent, invisible monitors to trap the hacker’s activity.

  Maybe just tape recording the telephone lines would work, but tape recorders didn’t feel right, too much of a kludge. We’d have to play them back, and couldn’t watch the keystrokes until long after a hacker had disconnected. Finally, where would I find fifty tape recorders?

  About the only other place to watch our traffic was in between the modems and the computers. The modems converted the tones of a telephone into electronic pulses, palatable to our computers and the daemons in their operating systems. These modem lines appeared as flat, twenty-five conductor wires, snaking underneath the switchyard’s false floor. A printer or personal computer could be wired to each of these lines, recording every keystroke that came through.

  A kludge? Yes. Workable? Maybe.

  All we’d need are fifty teletypes, printers, and portable computers. The first few were easy to get—just ask at the lab’s supplies desk. Dave, Wayne, and the rest of the systems group grudgingly lent their portable terminals. By late Friday afternoon, we’d hooked up a dozen monitors down in the switchyard. The other thirty or forty monitors would show up after the laboratory was deserted. I walked from office to office, liberating personal computers from secretaries’ desks. There’d be hell to pay on Monday, but it’s easier to give an apology than get permission.

  Strewn with four dozen obsolete teletypes and portable terminals, the floor looked like a computer engineer’s nightmare. I slept in the middle, nursing the printers and computers. Each was grabbing data from a different line, and whenever someone dialed our system, I’d wake up to the chatter of typing. Every half hour, one of the monitors would run out of paper or disk space, so I’d have to roll over and reload.

  Saturday morning, Roy Kerth shook me awake. “Well, where’s your hacker?”

  Still in my sleeping bag, I must have smelled like a goat. I blinked stupidly and mumbled something about looking at the fifty piles of paper.

  He snorted, “Well, before you start poking around those printouts, return those printers. You’ve been running around here like a maniac swiping equipment used by people who are getting work done. You’ve pissed off a dozen astronomers. Are you getting work done? No. Whaddya think this place is, your own personal sandbox?”

  Bleary-eyed, I dragged each printer back to its rightful owner. The first forty-nine showed nothing interesting. From the fiftieth trailed eighty feet of printout. During the night, someone had sneaked in through a hole in the operating system.

  For three hours, a hacker had strolled through my system, reading whatever he wished. Unknown to him, my 1200-baud Decwriter had saved his session on eighty feet of single-spaced computer paper. Here was every command he issued, every typing mistake, and every response from the computer.

  This printer monitored the line from Tymnet. I didn’t realize it, but a few of our 1200-baud lines weren’t dial-in modem lines. Rather, they came from Tymnet, a communications company that interconnected computers around the world.

  Back before divestment, the Bell system monopolized communications. AT&T was the only way to connect New York to Chicago. By using modems, the phone system could handle data, but the noise and expense of the long distance service made it unsuitable for computers. By the late ’70s, a few other companies dipped their toes in the water, offering specialized services like data phones. Tymnet created a network to interconnect computers in major cities.

  Tymnet’s idea was simple and elegant: create a digital communications backbone, let anyone connect to the backbone by making a local telephone call, then send the data to any computer on the network. Tymnet would compress dozens of users’ data into a few packets, and economically send these around the country. The system was immune to noise, and each user could run as fast as he wished. Customers saved money because they could access a distant computer by making a local call.

  To satisfy scientists around the country, LBL subscribed to Tymnet. When a researcher in Stonybrook, New York, wanted to connect to our computer, he dialed his local Tymnet number. Once his modem was connected to Tymnet, he just asked for LBL and worked as if he were in Berkeley. Physicists from far away loved the service, and we were delighted to find them spending their research dollars on our computers, rather than their home machines.

  Someone was breaking in, using the Tymnet line. Since Tymnet interconnected the whole country, our hacker might be anywhere.

  For the moment, though, I was fascinated not by where the hacker came from, but what he had done in three hours. My guess was right: Sventek’s account was being used to break into our Unix computer.

  Not just break in. This hacker was a super-user.

  The hacker had sneaked through a hole in our system to become a super-user—he’d never even logged into the system manager’s account. He was like a cuckoo bird.

  The cuckoo lays her eggs in other birds’ nests. She is a nesting parasite: some other bird will raise her young cuckoos. The survival of cuckoo chicks depends on the ignorance of other species.

  Our mysterious visitor laid an egg-program into our computer, letting the system hatch it and feed it privileges.

  That morning, the hacker wrote a short program to grab privileges. Normally, Unix won’t allow such a program to run, since it never gives p
rivileges beyond what a user is assigned. But run this program from a privileged account, and he’ll become privileged. His problem was to masquerade this special program—the cuckoo’s egg—so that it would be hatched by the system.

  Every five minutes, the Unix system executes its own program named atrun. In turn, atrun schedules other jobs and does routine housecleaning tasks. It runs in a privileged mode, with the full power and trust of the operating system behind it. Were a bogus atrun program substituted, it would be executed within five minutes, with full system privileges. For this reason, atrun sits in a protected area of the system, available only to the system manager. Nobody but the system manager has the license to tamper with atrun.

  Here was the Cuckoo’s nest: for five minutes, he would swap his egg for the system’s atrun program.

  For this attack, he needed to find a way to move his egg-program into the protected systems nest. The operating system’s barriers are built specifically to prevent this. Normal copy programs can’t bypass them; you can’t issue a command to “copy my program into systems space.”

  But there was a wildcard that we’d never noticed. Richard Stallman, a free-lance computer programmer, loudly proclaimed that information should be free. His software, which he gives away for free, is brilliantly conceived, elegantly written, and addictive.

  Over the past decade Stallman created a powerful editing program called Gnu-Emacs. But Gnu’s much more than just a text editor. It’s easy to customize to your personal preferences. It’s a foundation upon which other programs can be built. It even has its own mail facility built in. Naturally, our physicists demanded Gnu; with an eye to selling more computing cycles, we installed it happily.