by Bowden, Mark
5
The X-Men
HE AND OTHERS LIKE HIM, BORN WITH
GENETIC POTENTIAL FOR GREAT POWERS,
ARE KNOWN AS MUTANTS. THE WORLD OF
HUMANS FEARS THEM FOR BEING DIFFERENT
. . . AND HATES THEM FOR BEING GIFTED.
—The X-Men Chronicles
By mid-December, three weeks after Conficker first appeared, the worm had burrowed into well over a million computers worldwide. It had spread silently for six days before it began regularly trying to connect with its botmaster, who could have been hiding behind any of the 250 domian names the worm generated afresh each day. Such a large infection became a noisy presence on the Internet.
Yet still it had attracted no attention outside the Tribe. A select but widely scattered circle of computer security experts watched with mounting concern, mindful of what a botnet of that size could do. Beyond this group, word of a new and major botnet surfaced only at the website Ars Technica, a technology journal owned by Condé Nast with a very small readership. An article by Joel Hruska on December 2 called Conficker “a rising specter,” but the story was upbeat, suggesting that the various private security firms and academics monitoring it had the threat under control.
Like many of those in the field, Hruska took Conficker at face value. Since its only known use was a simple moneymaking scheme—to download the fraudware package at TrafficConverter.biz that had given the worm its name—he portrayed the worm as pedestrian. His post focused on the fact that it had spread even though, or more likely because, Microsoft had patched the vulnerability it exploited—the buffer overflow opportunity at Port 445.
“Microsoft appears to have dealt with the problem in a textbook fashion,” Hruska wrote. “. . . It would be extremely fascinating to see data on how a patch spreads throughout the Internet once released. . . . Events like this raise the question of whether or not Microsoft should have the capability to push critical security updates out to home users automatically, regardless of how AutoUpdate is configured. . . . How do you solve a security problem that’s caused by users refusing to update their machines?”
While the new worm was raising some interesting questions, it was not yet in the same league as Storm or Srizbi. As he saw it, the big commercial security firms like Symantec, F-Secure, I-Defense and others were dissecting it and figuring out how to contain it, with an eye toward offering remedial software for sale somewhere down the line.
The problem was already known to be bigger than that in the security community, but since there is no such thing as an agency charged with protecting the Internet for its own sake, concern about Conficker proceeded from a variety of narrower motives. The AV industry was worried about protecting its customers, but was also mindful that the growing lists of bots represented a potential gold mine of new customers—since the malware disabled security updates, each bot was a prime mark for remedial software (the botnet itself was a valuable list of unprotected computers). The telecoms folks were interested in protecting their vital networks from DDoS attacks. Microsoft wanted to safeguard its customers and reputation, while researchers like Phil Porras at SRI had more of a purely academic interest, figuring out what this latest wrinkle from the black hats meant.
That was the focus in Menlo Park. The work demanded aptitude, but also years of experience. It has been almost thirty years since computers became ubiquitous, and twenty since the Internet blinked to life, so the youngsters who were the first to embrace networks and operating systems are now middle-aged. The elders of the Tribe remember the old Altair 8080 kits, but most of its elite übergeeks today were the first generation to grow up with computers, and have absorbed an intuitive fluency with networks. Today they work for software companies, research labs, security firms, telecommunications companies, government, or Internet service organizations. Whatever the overarching agendas of their employers, these guys (and they were mostly men) were viscerally drawn to fighting Conficker. This was intellectual combat, pitting the best good-guy minds against the best bad-guy minds.
The dozen or so white hats who joined this fight, which would eventually include Phil Porras, the man who best under stood the worm, assembled in the orbit of T. J. Campana, who wielded Microsoft’s deep pockets and clout. Among them were Rick Wesson, a brash forty-two-year-old San Francisco entrepreneur who was CEO of his own Internet security firm, coauthor of some central Internet protocols, and owner of his own small Internet registrar; Rodney Joffe, the eldest member at fifty-five and a self-proclaimed “adult in the room,” a burly transplanted South African from Phoenix who was (among other things) security chief of Neustar, a telecommunications company that operated the .biz top-level domain and several Internet registries; Andre DiMino, a quiet, self-possessed New Jerseyite who worked for Bergen County law enforcement during the day, but by night was one of the founders of a unique nonprofit botnet-killing service called Shadowserver. Joining them were Paul Vixie, a dour and irascible geek who is one of the architects of the Internet and who was on the board of trustees of the American Registry for Internet Numbers; Andre “Dre” Ludwig, the youngest member at age twenty-eight, a self-taught computer security consultant in Alexandria, Virginia, with a big reputation and a forthright, confrontational style; John Crain, a transplanted Brit who lived in Long Beach, traveled the world for ICANN (the global nonproft agency that assigns domain names and IP addresses), and had a penchant for cowboy attire; and Chris Lee, a meticulous graduate student at Georgia Tech who would end up running the bulk of the sinkholing operation.
To varying degrees, as individuals, they had warned in speeches and articles about the ludicrous fragility of the Internet, as the global net grew and grew, and as society kept leaning on it more and more heavily. They were accustomed to being ignored. Some took it better than others, and remained hopeful. Some were more fatalistic, assuming that at some point the whole thing was going to crash. In their darkest moods, some of them—Vixie and Wesson come to mind—could be positively surly about it, like the engineer who tries to explain for the thousandth time that luck does not lessen risk, even when it seems to—You do realize, Mr. NASA Chief Engineer, that just because you keep getting away with these Space Shuttle launches, it doesn’t mean that sucker is any less likely to blow up on national TV.
In the case of the Internet, and those who understood it best, you could sense beneath the immediate concern a deeper frustration, the exasperation of someone who has spent every minute, every hour, every day of his life feeling smarter than everyone else, only to be dismissed as strange. Paul Vixie had a lecture called “Vixie’s Internet Rant,” which details the train of errors committed by the architects of the Internet, who designed it to be shared by like-minded, friendly colleagues, without a thought to what might happen when a billion strangers crashed the party.
At the 2005 DefCon 13 Convention, an annual hackers’ gathering, one of the few places in the world where geeks do not encounter the Glaze, here is Paul delivering his rant: a thickly built man in a black Spamarama T-shirt with a broad clean-shaven face, big glasses, crew-cut black hair, and heavy, dark eyebrows. He is speaking in a low, affectless monotone, his hands held stiffly at his sides. His speech is highly technical, weirdly more powerful for its peculiar delivery, and, if you are paying close attention, darkly humorous. It frames the annals of global interconnection as a perfect instance of historical folly, meeting all the criteria set down by the historian Barbara Tuchman in her book The March of Folly: it was the action of a group, not an individual; it consistently chose the “boneheaded” course over others that were obviously correct; and the chosen course was not just something discovered to be mistaken in retrospect, but something known to be stupid in its own time. Paul encouraged audience members to pick up the book, and offered to personally refund the purchase price if they did not find it alarmingly pertinent.
“What were they thinking?” Vixie asked. “Were they thinking?”
Government took it on the chin in this presentation,
for its lack of foresight and of oversight, for its inability to see the danger—and Paul would be the first to tell you that things haven’t changed. Indeed, as we will see, the government was notably absent from the effort against Conficker.
It is one of the peculiarities of modern times that as industrialized nations depend more and more on computer networks for everything, relatively little thought has been given to protecting them. The United States spends billions on its military, not just to protect its own borders, but to project force anywhere in the world on short notice. Yet the telecommunications networks that increasingly undergird every aspect of modern life, not to mention the military itself, are shockingly vulnerable to infiltration and sabotage, not just from pranksters and cybercriminals, but from the very nations the United States are most likely to confront as enemies.
“Private sector networks in the United States, networks operated by civilian U.S. government agencies, and unclassified U.S. military and intelligence agency networks increasingly are experiencing cyber intrusions and attacks,” said a U.S.-China Economic and Security Review Commission report to Congress that was published the same month Conficker appeared. “. . . Networks connected to the Internet are vulnerable even if protected with hardware and software firewalls and other security mechanisms. The government, military, businesses and economic institutions, key infrastructure elements, and the population at large of the United States are completely dependent on the Internet. Internet-connected networks operate the national electric grid and distribution systems for fuel. Municipal water treatment and waste treatment facilities are controlled through such systems. Other critical networks include the air traffic control system, the system linking the nation’s financial institutions, and the payment systems for Social Security and other government assistance on which many individuals and the overall economy depend. A successful attack on these Internet-connected networks could paralyze the United States [emphasis added].”
The ad hoc group that formed to combat Conficker reached out repeatedly to government agencies, including law enforcement, the military, the intelligence community, and every other agency you might expect to have an interest in protecting the computer networks of the nation (not to mention . . . the world). They eventually succeeded in getting reps from the alphabet soup—NSA, DOD, CIA, FBI, DHS, etc.,—to sign on as members of the private chat channel where they coordinated strategy, but throughout the effort the feds would remain lurkers; they logged in and listened, but rarely made a peep. Over four months in December 2008 and January, February, and March 2009, as Conficker assembled the largest botnet in the world, government, which would seem to have had the largest share of overarching responsibility, played a shockingly minor role. At first the übergeeks assumed the feds were constrained by the need for secrecy: you know, protecting official tactics and methods. Surely behind the scenes there was a sophisticated, well-funded clandestine official apparatus—everyone has seen the gleaming, dark glass and metal, see-everything/hear-everything sets Holly wood dusts off for its espionage blockbusters. What the anti-Conficker group discovered was deeply disillusioning. The real reason for the feds’ silence was . . . they had nothing to offer! They were in way over their heads.
So the battle was in the hands of this odd and uniquely talented collection of volunteers. Given the esoteric nature of the combat, it lent itself less to the analogies of earthbound warfare than to the fantastic. It called to mind DC Comics’ “Justice League of America,” or, better still, “X-Men,” because this was definitely more of a Marvel crowd. What were superheroes, after all, but those with special powers? Marvel’s creations were also invariably outsiders, not just special but mutant, a little bit off, defiantly antisocial, prone to sarcasm and cracking wise, suspicious of authority, both governmental and corporate (as T.J., Mr. Microsoft, would learn, to his chagrin). There is not one of the übergeeks involved who had not, at one time or another in his life, realized that he could run rings around the safeguards and defenses of most computer systems. The X-Men could make things happen that others could not. Knowledge empowered them: Rick Wesson commandeering the engineering computer system at Auburn as an undergrad to generate supercool fractal-based images, which he then copied on T-shirts and sold; T.J. providing free movies and music to his friends at FSU; another, who still wishes to go unnamed, who disassembled the mechanism for commercial online gaming so that he could play for free. This facility for computers and networks, being able to puzzle through the defenses of powerful systems, was very much like possessing a superpower—LADIES AND GENTLEMEN, THE TRUTH IS THAT MUTANTS ARE VERY REAL, AND THAT THEY ARE AMONG US. Even the standard cerebral, geeky civilian alter ego stereotype applied, since few of these guys were the least bit intimidating or commanding in person. They went about their day jobs as unassuming techies, men whose conversation was guaranteed to produce the Glaze, but out here in the cyberworld they were nothing less than the Anointed, the Guardians, the Special Ones: not just the ones capable of seeing the threat that no one else could see, but the only ones who could conceivably stop it—“We are the last line of defense at this point. . . . There are no others,” T.J. wrote to the group early on. “You all are the smartest people in the security industry. . . . If not us . . . who? If not now . . . when?”
They were psyched!
But we are getting ahead of ourselves . . .
In mid-December 2008, the chat channel was up, a private Listserv—the List!—where these real-life X-Men, Rick Wesson, Rodney Joffe, Andre DiMino, and the others, plotted strategy, shared insights, coordinated efforts, and kept up a running dialogue. Anything posted to the List was available to the entire group, and most of it dealt with the minutiae of technical analysis, code-breaking, sinkholing, etc. Much of it read like this:
MD5: 38c3d2efdd47b1034b1624490ce1f3f2
>> SHA1: c6c1ed21ea15c8648a985dbabc8341cf1e3aa21e
>
>> That’s the unpacked version and it was sent by VirusTotal on Monday.
Or like this:
> <
2.6 urllib user-agent:
But from time to time various members would use the List as a soapbox or sounding board, speculating, proposing, arguing, praising, lamenting, criticizing, sometimes with real eloquence. Mining these exchanges and ventings reveals a detailed chronicle of the effort, often minute by minute. So the history of this remarkable technological drama, which might be called the First Internet World War, and which took place almost entirely out of the public eye, unfolds as a series of missives, like an epistolary novel. Shades of Samuel Richardson!
As the threat mounted, working with the X-Men became a mark of status. Here was a band of warriors for the Internet, which is to say warriors for civilization. It sounds corny, but it was true. Most of the core members knew one another well: Paul Vixie wrote in one of his first postings to the List, “Whenever I’m added to a security list, I look around for [the usual suspects] and a bunch of other regulars, and if they’re not there yet I know they’ll be along in a few weeks. Sometimes I even nominate them myself just to cut down on the suspense.” Those who wished to join had to be vouched for by others already on the List, and some were turned away: “I feel like we are in high school,” wrote Rick Wesson. But there were only a few hundred people in the world capable of the work.
By the end of December, the X-Men were regularly pulling all-nighters, trying to stay one step ahead of the evil botmaster. T.J. was working until ten o’clock most nights in his office up in one of Microsoft’s Redmond sprockets. His boss would stop by, surprised to see him in so late.
“What are you doing?” he’d ask.
“Conficker.”
“Everything okay?”
“Well, the Internet’s melting. We’re just keeping it from melting completely.”
The bad guys behind C
onficker, its unknown botmaster, would prove to be worthy adversaries. They were villains in the truest sense, talented programmers bent on using their powers for evil. And the world war was about nothing less than the soul of the future, the soul of the new global mind. As for as the X-Men, what could be cooler than to be right in the middle of it, showing off your chops?
6
Digital Detectives
THIS MAY NOT BE MUCH OF A WORLD . . . MAY
NOT EVEN BE THE WORLD IT IS SUPPOSED TO
BE . . . BUT IT IS OUR WORLD NEVERTHELESS.
AND WE WILL FIGHT FOR IT.
—The Amazing X-Men
At the October 2008 botnet conference in Arlington, Virginia, the one where T. J. Campana unveiled Microsoft’s “out of band” emergency patch, he had handed a sheet of paper to Andre DiMino.
“Do you know anything about this?” he asked.
It was a printout of information about Gimmiv, the first piece of malware that used the Chinese kit to exploit the newly discovered Port 445 vulnerability. Andre didn’t recognize the strain offhand, but he was the right person to ask.
On a Monday morning ten years earlier, Andre had stumbled into the malware wars when he discovered that over the weekend someone had broken into the computer system he was managing for a small company in New Jersey. Andre has an undergraduate degree in electrical engineering with an emphasis in computer science, but most of what he knows about botnets he has taught himself. At forty-five, he is a tall, slender man whose dark hair is cropped close to his scalp. He is affable and quietly idealistic, and he has a selfless passion for his work. His day job is doing computer forensics for the Bergen County prosecutor, but the work that drives him most is done before a small array of computers in an upstairs bedroom of his suburban New Jersey home.
