Cuckoo's Egg

by Clifford Stoll

  PUBLISHED BY DOUBLEDAY

  a division of Bantam Doubleday Dell Publishing Group, Inc.

  666 Fifth Avenue, New York, New York 10103

  DOUBLEDAY and the portrayal of an anchor with a dolphin are trademarks of Doubleday, a division of Bantam Doubleday Dell Publishing Group, Inc.

  Library of Congress Cataloging-in-Publication Data

  Stoll, Clifford.

  The cuckoo’s egg : tracking a spy through the maze of computer espionage / by Clifford Stoll. — 1st ed.

  p. cm.

  1. Hess, Marcus. 2. Stoll, Clifford. 3. Espionage, Soviet—

  United States. 4. Espionage, Soviet—Germany (West)—

  Hannover. 5. Defense information, Classified—United States—Data

  bases. 6. Computer crimes—United States. 7. Computer crimes—

  Germany (West)—Hannover. I. Title.

  UB271.R92H477 1989

  364.1′68′0973—dc20 89-7808

  364.1′68′0973—dc20

  eISBN: 978-0-307-81942-0

  Copyright © 1989 by Clifford Stoll

  All Rights Reserved

  v3.1

  Contents

  Cover

  Title Page

  Copyright

  Acknowledgments

  Chapter 1

  Chapter 2

  Chapter 3

  Chapter 4

  Chapter 5

  Chapter 6

  Chapter 7

  Chapter 8

  Chapter 9

  Chapter 10

  Chapter 11

  Chapter 12

  Chapter 13

  Chapter 14

  Chapter 15

  Chapter 16

  Chapter 17

  Chapter 18

  Chapter 19

  Chapter 20

  Chapter 21

  Chapter 22

  Chapter 23

  Chapter 24

  Chapter 25

  Chapter 26

  Chapter 27

  Chapter 28

  Chapter 29

  Chapter 30

  Chapter 31

  Chapter 32

  Chapter 33

  Chapter 34

  Chapter 35

  Chapter 36

  Chapter 37

  Chapter 38

  Chapter 39

  Chapter 40

  Chapter 41

  Chapter 42

  Chapter 43

  Chapter 44

  Chapter 45

  Chapter 46

  Chapter 47

  Chapter 48

  Chapter 49

  Chapter 50

  Chapter 51

  Chapter 52

  Chapter 53

  Chapter 54

  Chapter 55

  Chapter 56

  Epilogue

  Bibliography

  About the Author

  How do you spread the word when a computer has a security hole? Some say nothing, fearing that telling people how to mix explosives will encourage them to make bombs. In this book I’ve explicitly described some of these security problems, realizing that people in black hats are already aware of them.

  I’ve tried to reconstruct this incident as I experienced it. My main sources are my logbooks and diaries, cross-checked by contacting others involved in this affair and comparing reports from others. A few people appear under aliases, several phone numbers are changed, and some conversations have been recounted from memory, but there’s no fictionalizing.

  For supporting me throughout the investigation and writing, thanks to my friends, colleagues, and family. Regina Wiggen has been my editorial mainstay; thanks also to Jochen Sperber, Jon Rochlis, Dean Chacon, Donald Alvarez, Laurie McPherson, Rich Muller, Gene Spafford, Andy Goldstein, and Guy Consolmagno.

  I posted a notice to several computer networks, asking for title suggestions. Several hundred people from around the world replied with zany ideas. My thanks to Karen Anderson in San Francisco and Nigel Roberts in Munich for the title and subtitle.

  Doubleday’s editors, David Gernert and Scott Ferguson, have helped me throughout. To them, as well as my agent, John Brockman, thanks for your continued encouragement and wise advice.

  To each of these people, I’m indebted; I owe most of them boxes of cookies as well.

  Lawrence Berkeley Laboratory supported me throughout this quest; the people of Smithsonian Astrophysical Observatory—especially Joe Schwarz and Steve Murray—have been most gracious and supportive while I’ve been writing this book. My deep thanks go to my friends at both institutes, and my hopes that I’ll now be able to return to astronomy.

  I was ten years old when Ernst Both of the Buffalo Museum of Science invited me to look through a telescope, opening up a universe of astronomy. I wonder if I’ll ever be able to thank him properly.

  I needn’t thank my sweetheart and wife, Martha Matthews. She’s been as much a part of writing this book as she was in the story. I love her with all my heart.

  —Cliff Stoll-Matthews

  cliff@cfa.harvard.edu

  Me, a wizard? Until a week ago, I was an astronomer, contentedly designing telescope optics. Looking back on it, I’d lived in an academic dreamland. All these years, never planning for the future, right up to the day my grant money ran out.

  Lucky for me that my laboratory recycled used astronomers. Instead of standing in the unemployment line, I found myself transferred from the Keck Observatory at the Lawrence Berkeley Lab, down to the computer center in the basement of the same building.

  Well, hell, I could fake enough computing to impress astronomers, and maybe pick it up fast enough that my co-workers wouldn’t catch on. Still, a computer wizard? Not me—I’m an astronomer.

  Now what? As I apathetically stared at my computer terminal, I still thought of planetary orbits and astrophysics. As new kid on the block, I had my choice of a cubicle with a window facing the Golden Gate Bridge, or an unventilated office with a wall of bookshelves. Swallowing my claustrophobia, I picked the office, hoping that nobody would notice when I slept under the desk. On either side were offices of two systems people, Wayne Graves and Dave Cleveland, the old hands of the system. I soon got to know my neighbors through their bickering.

  Viewing everyone as incompetent or lazy, Wayne was crossthreaded with the rest of the staff. Yet he knew the system thoroughly, from the disk driver software up to the microwave antennas. Wayne was weaned on Digital Equipment company’s Vax computers and would tolerate nothing less: not IBM, not Unix, not Macintoshes.

  Dave Cleveland, our serene Unix buddha, patiently listened to Wayne’s running stream of computer comparisons. A rare meeting didn’t have Wayne’s pitch, “Vaxes are the choice of scientists everywhere and helps build strong programs twelve ways.” Dave retorted, “Look, you keep your Vax addicts happy and I’ll handle the rest of the world.” Dave never gave him the satisfaction of getting riled, and Wayne’s complaints eventually trailed off to a mutter.

  Great. First day on the job, sandwiched between two characters who were already ruining my daydreams with their periodic disputes.

  At least nobody could complain about my appearance. I wore the standard Berkeley corporate uniform: grubby shirt, faded jeans, long hair, and cheap sneakers. Managers occasionally wore ties, but productivity went down on the days they did.

  Together, Wayne, Dave, and I were to run the computers as a lab-wide utility. We managed a dozen mainframe computers—giant workhorses for solving physics problems, together worth around six million dollars. The scientists using the computers were supposed to see a simple, powerful computing system, as reliable as the electric company. This meant keeping the machines running full time, around the clock. And just like the electric company, we charged for every cycle of computing that was used.

&n
bsp; Of four thousand laboratory employees, perhaps a quarter used the main computers. Each of these one thousand accounts were tallied daily, and ledgers kept inside the computer. With an hour of computing costing three hundred dollars, our bookkeeping had to be accurate, so we kept track of every page printed, every block of disk space, and every minute of processor time. A separate computer gathered these statistics and sent monthly bills to laboratory departments.

  And so it happened that on my second day at work, Dave wandered into my office, mumbling about a hiccup in the Unix accounting system. Someone must have used a few seconds of computing time without paying for it. The computer’s books didn’t quite balance; last month’s bills of $2,387 showed a 75-cent shortfall.

  Now, an error of a few thousand dollars is obvious and isn’t hard to find. But errors in the pennies column arise from deeply buried problems, so finding these bugs is a natural test for a budding software wizard. Dave said that I ought to think about it.

  “First-degree robbery, huh?” I responded.

  “Figure it out, Cliff, and you’ll amaze everyone,” Dave said.

  Well, this seemed like a fun toy, so I dug into the accounting program. I discovered our accounting software to be a patchwork of programs written by long-departed summer students. Somehow, the hodgepodge worked well enough to be ignored. Looking at the mixture of programs, I found the software in Assembler, Fortran, and Cobol, the most ancient of computer languages. Might as well have been classical Greek, Latin, and Sanskrit.

  As with most home-brew software, nobody had bothered to document our accounting system. Only a fool would poke around such a labyrinth without a map.

  Still, here was a plaything for the afternoon and a chance to explore the system. Dave showed me how the system recorded each time someone connected to the computer, logging the user’s name, and terminal. It timestamped each connection, recording which tasks the user executed, how many seconds of processor time he used, and when he disconnected.

  Dave explained that we had two independent accounting systems. The ordinary Unix accounting software just stored the timestamped records into a file. But to satisfy some bureaucrat, Dave had built a second accounting system which kept more detailed records of who was using the computer.

  Over the years, a succession of bored summer students had written programs to analyze all this accounting information. One program collected the data and stashed it into a file. A second program read that file and figured how much to charge for that session. Yet a third program collected all these charges and printed out bills to be mailed to each department. The last program added up all user charges and compared that total to the result from the computer’s internal accounting program. Two accounting files, kept in parallel by different programs, ought to give the same answer.

  For a year, these programs had run without a glitch, but weren’t quite perfect this week. The obvious suspect was round-off error. Probably each accounting entry was correct, but when added together, tenths of a penny differences built up until an error of 75 cents accumulated. I ought to be able to prove this either by analyzing how the programs worked, or by testing them with different data.

  Rather than trying to understand the code for each program, I wrote a short program to verify the data files. In a few minutes, I had checked the first program: indeed, it properly collected the accounting data. No problem with the first.

  The second program took me longer to figure out. In an hour I had slapped together enough makeshift code to prove that it actually worked. It just added up time intervals, then multiplied by how much we charge for computer time. So the 75-cent error didn’t come from this program.

  And the third program worked perfectly. It looked at a list of authorized users, found their laboratory accounts, and then printed out a bill. Round-off error? No, all of the programs kept track of money down to the hundredths of a penny. Strange. Where’s this 75-cent error coming from?

  Well, I’d invested a couple hours in trying to understand a trivial problem. I got stubborn: dammit, I’d stay there till midnight, if I had to.

  Several test programs later, I began actually to have confidence in the mishmash of locally built accounting programs. No question that the accounts didn’t balance, but the programs, though not bulletproof, weren’t dropping pennies. By now, I’d found the lists of authorized users, and figured out how the programs used the data structures to bill different departments. Around 7 P.M. my eye caught one user, Hunter. This guy didn’t have a valid billing address.

  Ha! Hunter used 75 cents of time in the past month, but nobody had paid for him.

  Here’s the source of our imbalance. Someone had screwed up when adding a user to our system. A trivial problem caused by a trivial error.

  Time to celebrate. While writing this first small triumph into the beginning pages of my notebook, Martha, my sweetheart, stopped by and we celebrated with late-night cappuccinos at Berkeley’s Cafe Roma.

  A real wizard would have solved the problem in a few minutes. For me, it was unknown territory, and finding my way around hadn’t been easy. As a consolation, I’d learned the accounting system and practiced a couple obsolete languages. Next day, I sent an electronic mail message to Dave, preening my feathers by pointing out the problem to him.

  Around noon, Dave stopped by to drop off a pile of manuals, and casually mentioned that he had never added a user named Hunter—it must have been one of the other system managers. Wayne’s curt response: “It wasn’t me. RTFM.” Most of his sentences ended with acronyms, this one meaning, “Read the fucking manual.”

  But I’d read the manuals. Operators weren’t supposed to add a new user without an account. At other computer centers, you just log into a privileged account and tell the system to add a new user. Since we also had to make several bookkeeping entries, we couldn’t run such a vanilla system. Ours was complex enough that we had special programs which automatically did the paperwork and the systems juggling.

  Checking around, I found that everyone agreed the automatic system was so superior that nobody would have manually added a new user. And the automatic system wouldn’t make this mistake.

  Well, I couldn’t figure out who had made this goof. Nobody knew Hunter, and there wasn’t an account set for him. So I erased the name from the system—when he complained, we could set him up properly.

  A day later, an obscure computer named Dockmaster sent us an electronic mail message. Its system manager claimed that someone from our laboratory had tried to break into his computer over the weekend.

  Dockmaster’s return address might have been anywhere, but signs pointed to Maryland. The e-mail had passed through a dozen other computers, and each had left a postmark.

  Dave answered the message with a noncommittal “We’ll look into it.” Uh, sure. We’d look when all our other problems disappeared.

  Our laboratory’s computers connect to thousands of other systems over a dozen networks. Any of our scientists can log into our computer, and then connect to a distant computer. Once connected, they can log into the distant computer by entering an account name and password. In principle, the only thing protecting the networked computer is the password, since account names are easy to figure out. (How do you find account names? Just use a phone book—most people use their names on computers.)

  Dockmaster’s electronic mail message was a curiosity, and Dave passed it to Wayne, attaching a question, “Who’s Dockmaster?” Wayne forwarded it to me with his guess—“Probably some bank.”

  Eventually, Wayne bounced the message to me. I guessed Dockmaster was some Navy shipyard. It wasn’t important, but it seemed worth spending a few minutes looking into.

  The message gave the date and time when someone on our Unix computer tried to log into Dockmaster’s computer. Since I’d just mucked around the accounting system, scrabbled around the files, looking for records from Saturday morning at 8:46. Again, the two accounting systems disagreed. The stock Unix accounting file showed a user, Sven
tek, logging in at 8:25, doing nothing for half an hour, and then disconnecting. No timestamped activity in between. Our home-brew software also recorded Sventek’s activity, but it showed him using the networks from 8:31 until 9:01 A.M.

  Jeez. Another accounting problem. The time stamps didn’t agree. One showed activity when the other account said everything was dormant.

  Other things seemed more pressing, so I dropped the problem. After wasting an afternoon chasing after some operator’s mistake, I wasn’t about to touch the accounting system again.

  Over lunch with Dave, I mentioned that Sventek was the only one connected when Dockmaster reported the break-in. He stared and said, ‘Joe Sventek? He’s in Cambridge. Cambridge, England. What’s he doing back?’ Turned out that Joe Sventek had been the laboratory’s Unix guru, a software wizard who built a dozen major programs over the past decade. Joe had left for England a year ago, leaving behind a glowing reputation throughout the California computer community.

  Dave couldn’t believe Joe was back in town, since none of Joe’s other friends had heard from him. ‘He must have entered our computer from some network,’ Dave said.

  ‘So you think Joe’s responsible for this problem?’ I asked Dave.

  “No way,” Dave replied. “Joe’s a hacker of the old school. A smart, quick, capable programmer. Not one of those punks that have tarnished the word ‘hacker.’ In any case, Sventek wouldn’t try to break into some Maryland computer. And if he did try, he’d succeed, without leaving any trace.”

  Curious: Joe Sventek’s been in England a year, yet he shows up early Saturday morning, tries to break into a Maryland computer, disconnects, and leaves behind an unbalanced accounting system. In the hallway I mention this to Wayne, who’s heard that Joe’s on vacation in England; he’s hiding out in the backwoods, far away from any computers. “Forget that message from Dockmaster. Sventek’s due to visit Berkeley RSN and he’ll clear it up.”

  RSN? Real Soon Now. Wayne’s way of saying, “I’m not sure when.”

  My worry wasn’t Sventek. It was the unbalanced accounts. Why were the two accounting systems keeping different times? And why was some activity logged in one file without showing up in the other?