by Kim Zetter
The market is “gray” only because the buyers and sellers are presumed to be the good guys, acting in the interest of public safety and national security. But one person’s national security tool can be another’s tool of oppression, and there’s no guarantee that a government that buys zero days won’t misuse them to spy on political opponents and activists or pass them to another government that will. Even if a government agency is using a zero day for a legitimate national security purpose, vulnerabilities sold on the gray market are not disclosed to vendors for patching, which leaves anyone who doesn’t know about them—including other government agencies and critical infrastructure owners in the buyer’s own country—open to attack should foreign adversaries or independent hackers discover the same security holes and exploit them.
The sale of exploits is legal and largely unregulated. Though export controls in the United States that govern the sale of conventional software would also prohibit the sale of exploits to countries like Iran and North Korea, exploits don’t come with a copyright notice identifying their maker or country of origin, so anyone selling to these markets would not likely be caught.
The price of zero days varies greatly, depending on the rarity of the vulnerability—systems that are more difficult to crack produce fewer holes—as well as the time and difficulty involved in finding a hole and developing an exploit for it, the ubiquity of the software it exploits, and the exclusivity of the sale. An exploit sold exclusively to one customer will naturally bring more than one that’s sold to many. Exploits that require more than a single vulnerability to provide the attacker root-level access to a machine also demand a higher price, as do ones that bypass antivirus and other security protections on a system without producing any side effects, such as crashing the browser or the machine or otherwise tipping off the computer owner that something is amiss.
A zero-day exploit for Adobe Reader can go for $5,000 or $30,000, while an exploit for the Mac OS can cost $50,000. But an exploit for Flash or Windows can jump to $100,000 or more because of the programs’ ubiquity in the marketplace. An exploit for Apple’s iOS can also go for $100,000 because the iPhone is more difficult to crack than competing mobile phones. Browser exploits that attack Firefox, Internet Explorer, and Chrome can sell for anywhere from $60,000 to more than $200,000, depending on their ability to bypass security protections the vendors have put in the software.1
Whatever the price on the gray market, however, it far surpasses in most cases what a seller can get from the white-market bounty programs. The Mozilla Foundation pays just $3,000 for bugs found in its Firefox browser and Thunderbird e-mail client, for example, while Microsoft, which was criticized for years for having no bug bounty program, began offering just $11,000 in 2013 for bugs found in the preview release of its new Internet Explorer 11 browser. The company, however, also now offers $100,000 for vulnerabilities that can help an attacker bypass the security protections in its software products, plus an additional $50,000 for a solution to fix it. Google usually pays just $500–$20,000 for bugs found in its Chrome browser and web properties, such as Gmail and YouTube, though it will pay $60,000 for some types of holes found in Chrome during an annual contest it sponsors. But while some vendors are making attempts to compete with the black market, they’re still no match, in most cases, for the price some governments will pay on the gray market. And Apple and Adobe still offer no bounty programs whatsoever to pay for bugs in software used by millions of people.
The gray market for zero days has been around for about a decade, but only recently has it emerged in its current, robust form. For many years it operated ad-hoc, with sales occurring only quietly in private between security firms and researchers and their government contacts. If someone wanted to sell an exploit but had no government contacts, it was difficult to sniff out a buyer.
Beginning in 2006, for example, one security firm sold several zero-day exploits to a contact at a large US defense firm, according to a former employee who worked there. The zero days, all browser exploits targeting security holes in Safari, Firefox, and Internet Explorer, sold for about $100,000 each. The security firm got $50,000 up front for each sale it made and $10,000 a month thereafter until the price was paid off—payments were spread out to discourage them from reselling the exploits to other buyers or disclosing them to the vendors for patching.
One of the first people to openly admit selling exploits to the government is security researcher Charlie Miller, a former NSA hacker who was recruited by the spy agency in 2000 after earning a PhD in mathematics from the University of Notre Dame. Miller worked for the intelligence agency about five years, initially cracking codes on its behalf before turning his skills to cracking computers—doing reconnaissance scans to map foreign networks and conduct “computer network exploitations against foreign targets,” according to his NSA-cleared résumé. CNE in spy-speak means hacking systems and networks to siphon data and intelligence. After leaving the NSA, Miller earned prominence in the security community for hunting zero-day bugs and creating exploits, not all of which he sold to the government. He was the first, with a colleague, to crack the security of the iPhone after its debut in 2007, and he’s a four-time winner of Pwn2Own, an annual hacking contest sponsored by HP TippingPoint that pays contestants for zero-day bugs found in specific software targets.
But in 2006, Miller was working for a small security firm, doing a little bug hunting on the side, when he sold a zero-day exploit to a US government contractor for $50,000. He sold the exploit to someone he knew from his days at the NSA, but says he has no idea where it went after the sale or how it was used. The contracts he signed for this and other exploits he sold never stipulated what the buyer could do with them. “I don’t know if he did anything good or bad with it; I do know that he worked for the US government,” Miller says. “They’re buying the intellectual property, you know? They can do whatever they want with it.”
Miller caused an uproar in 2007 when he published a paper about the zero-day market and admitted publicly that he sold exploits to the government.2 He wrote the paper because he wanted people to know the practice existed and to help other researchers navigate the pitfalls of the trade that he’d experienced. At the time, selling exploits was the security industry’s dirty little secret. Researchers occasionally discussed the practice among themselves, but no one was willing to talk about it openly. Miller soon learned why. Colleagues in the security community accused him of putting users at risk, and some called for his CISSP (certified information systems security professional) certification to be revoked for violating the industry’s code of ethics. “I talked about it … I got beat up for it. And I don’t talk about it anymore,” Miller says.3
But to him, it didn’t make sense to hand over bugs to vendors for free—only a couple of vendor bounty programs existed at the time, and they paid little for bugs and exploits. It was also a time when vendors were less likely to thank researchers for disclosing a hole than threaten them with a lawsuit or criminal prosecution for probing their system or software to discover it.
Miller abandoned the zero-day trade years ago—he now works on Twitter’s security team—but he still sees nothing wrong with selling zero days to the government and gets annoyed when people talk about the ethics of it. “No one gets mad that, you know, companies sell the government guns and tanks,” he says, noting that while US researchers are selling zero days to their government, Chinese and Russian hackers are doing the same for their governments. It’s better for the United States to pay top dollar for exploits, he says, than allow them to get into the hands of enemies.
“I don’t think it’s earth-shattering that researchers can sell exploits to the government,” Miller told me, “but I think people should … be aware that it happens. I’m OK with the government doing it out in the open.… I don’t know why they don’t just set up a public program [and say] ‘find a zero day, we’ll buy it.’ ”4
But in the years since Miller’s days on the exploit hunt, the g
ray-market demand for zero days has mushroomed, as evidenced by the fact that exploits that might have taken months to sell before now do so within days or weeks. A burgeoning ecosystem has emerged to meet the demand—populated by small firms whose primary business is bug hunting as well as by large defense contractors and staffing agencies that now employ teams of professional hackers dedicated to the task of creating exploits for governments. There are also more middlemen willing to broker exploit sales for independent sellers.
One such middleman is a South African security researcher based in Thailand who is known in the security community by his hacker handle “The Grugq.” The Grugq brokers exploit sales between his hacker friends and government contacts, pocketing a 15 percent commission per transaction. He only launched his business in 2011, but by 2012 sales were so good, he told a reporter he expected to make $1 million in commissions. A published photo of him taken at a Bangkok bar showed a satchel of cash at his feet, evidently payment from one of his sellers, though he later said the photo was just a joke.5
Most of the exploits he sold went to government buyers in the United States and Europe, he told Forbes, because they were willing to pay more than others. One Apple iOS exploit he sold to a US government contractor went for $250,000, though he later concluded he’d asked too little because the buyer was way too happy with the sale. He attributed his success to the professionalism he put into marketing the exploits and the support he gave to his clients. “You’re basically selling commercial software, like anything else,” he told Forbes. “It needs to be polished and come with documentation.”
But the really big trade in exploits these days is not done by middlemen and individual sellers like Miller and The Grugq, but by the security firms and defense contractors who have made the development and sale of exploits for government part of the new military industrial complex.
Although governments still produce their own exploits—the NSA employs teams for this—they also outsource to other firms because the demand for exploits has grown, as has the cost of producing them: two or three years ago, a single vulnerability was sufficient to gain root-level access to a machine. But today, it can take multiple ones to bypass security protections to achieve the same results.
Most of the companies involved in the trade are secretive about their work in this area, not only because it’s classified but because they don’t want to be targeted by activists who oppose the work or by adversaries who might hack them to steal their exploits. Because zero days can be used for both defending a system and attacking it, many of the companies also hide their offensive activity behind a cover of defensive work. US companies like Endgame Systems, Harris, Raytheon, Northrop Grumman, SAIC, Booz Allen Hamilton, and Lockheed Martin have all been in the exploit game to varying degrees. Companies in Europe include the boutique firms ReVuln in Malta, which creates exploits for industrial control systems, and VUPEN in France, which sells to law enforcement and intelligence agencies. Hacking Team in Italy and the Gamma Group in the UK both sell surveillance tools for law enforcement and intelligence agencies that use zero-day exploits to get installed.
The zero-day work of Endgame Systems, a Georgia-based firm, was a badly kept secret in the security community for years but wasn’t widely known outside of the community until 2011, when hackers with the Anonymous collective broke into servers belonging to another firm called HBGary Federal and dumped thousands of its e-mails online, including correspondence with executives at Endgame. The e-mails discussed Endgame’s exploit work as well as its efforts “to maintain a very low profile” on the advice of its government customers. The e-mails, which included PowerPoint presentations for prospective Endgame clients, described the company’s mission to enhance the “Information Operations capability of the United States intelligence and military organizations.” The head of Endgame’s board of directors is also the chief executive of In-Q-Tel, the CIA’s venture capital firm.
Publicly, Endgame was offering services to protect customers against viruses and botnets, while privately selling vulnerability and exploit packages containing information that could “lead to actionable intelligence for CNA efforts.” CNA, or computer network attacks, is military-speak for hacking that manipulates or destroys data or systems or retards or halts the performance of systems. The company launched in 2008 and its business prospects were so rosy that two years later it raised $30 million in venture capital, followed by $23 million in a subsequent round. In 2011, Endgame CEO Christopher Rouland told a local paper in Atlanta that the company’s revenue was “more than doubling yearly.”6
The stolen e-mails described three different packages Endgame offered, called Maui, Cayman, and Corsica. For $2.5 million a year, the Maui package provided buyers with a bundle of twenty-five zero-day exploits. The Cayman package, which cost $1.5 million, provided intelligence about millions of vulnerable machines worldwide already infected with botnet worms like Conficker and other malware. A sample map in the e-mails showed the location of vulnerable computers in the Russian Federation and a list of infected systems in key government offices and critical infrastructure facilities that included the IP address of each machine and the operating system it used. The list showed 249 infected machines at the Central Bank of the Russian Federation, and a handful of machines at the Ministry of Finance, the National Reserve Bank, the Novovoronezh Nuclear Power Plant, and the Achinsk Oil Refinery Plant. Endgame collected the data in part by setting up sinkholes to communicate with machines infected with Conficker—when the malware contacted the sinkhole, Endgame collected intelligence about the machine. A similar map for Venezuela showed the location of web servers in that country and the software running on them. Web servers, if breached and poorly configured, can often provide attackers access to back-end systems and databases. The systems on the list included servers for Corporación Andina de Fomento—a development bank that provides financing to eighteen member countries in Latin America, the Caribbean, and Europe—as well as Venezuela’s central budget office, the Office of the Presidency, the Ministry of Defense, and the Ministry of Foreign Affairs. After it was hacked, Endgame began telling reporters in 2012 that it was getting out of the exploit business, and in early 2014 it made a formal announcement to this effect.
While Endgame made a concerted effort to hide its exploit business, one company that’s positively garrulous about its role in the zero-day trade is VUPEN Security, based in Montpellier, France. VUPEN bills itself as a boutique security firm creating and selling exploits to intelligence agencies and law enforcement for offensive cyber security operations and lawful intercept missions. Originally launched in 2008 to protect government clients from zero-day attacks, the company began creating exploits for offensive operations two years later. In 2011, it earned $1.2 million in revenue, nearly 90 percent of which came from sales outside France. In 2013, it announced that it was opening an office in the United States.
VUPEN’s founder and CEO, Chaouki Bekrar, is a bold and cheeky sort who likes to rile critics on Twitter who think supplying exploits to governments is unethical. He also often challenges his secretive competitors to come clean about their own zero-day trade. “We are the only company in the world saying clearly that we are doing this stuff,” he says. “There are some companies in the US or in Europe, for example, doing this, but they are doing this undercover. But we have chosen to do it clearly, just because we want to be very transparent.”7
Where Endgame and others take pains to keep a low profile, Bekrar and his researchers regularly travel the security conference circuit, participating in contests like Pwn2Own, to increase the company’s profile. At the CanSecWest conference (an annual computer security conference in Canada) in 2012, where the Pwn2Own competition is held, Bekrar and a team of four of his researchers took first place wearing matching black hoodies with the company’s name on the back.
But VUPEN’s transparency goes only so far. Bekrar won’t discuss his background or answer other personal questions, deflecting attention to his co
mpany instead. “I’m just an actor. I want to talk about the movie,” he says. But when it comes to the company, he’s equally close-mouthed—he won’t say how many employees he has, just that the company is small, or reveal their last names.
VUPEN’s researchers devote all their time to finding zero-day vulnerabilities and developing exploits—both for already-known vulnerabilities as well as for zero days. Bekrar won’t say how many exploits they’ve sold since they began this part of their business, but says they discover hundreds of zero days a year. “We have zero days for everything,” he says. “We have almost everything for every operating system, for every browser, for every application if you want.”
How much of Bekrar’s boasting is true and how much is strategic marketing is unclear, but whatever the case, his tactics seem to be working. In 2012, several months after his team won the Pwn2Own contest, the NSA purchased a one-year subscription for VUPEN’s “Binary Analysis and Exploits (BAE)” service. The contract, released under a public records request, was heavily redacted and didn’t reveal the price paid for the subscription. But a business-consulting firm, which named VUPEN entrepreneurial company of the year in 2011, indicated the subscription runs about $100,000 a year. According to VUPEN’s website, the BAE service provides “highly technical reports for the most critical and significant vulnerabilities to understand their root cause, exploitability techniques, mitigations and both exploit-based and vulnerability-based attack detections.”8
VUPEN also offers a Threat Protection Program that provides detailed research on exclusive vulnerabilities discovered by its researchers to allow customers “to reduce their exposure to zero-day attacks,” according to a company brochure that got leaked to WikiLeaks.9 Both of these programs are described as if they’re meant to help customers defensively protect themselves from zero-day attacks—zero-day exploits can be used to test a system for its vulnerability to an attack—but the information provided in them can also be used to offensively attack other unpatched systems. The company’s Threat Protection Package even provides customers with ready-made exploits for attacking the vulnerabilities it reveals. And VUPEN has a third service for law enforcement and intelligence agencies that’s clearly designed solely for covertly attacking targeted machines to gain remote access to them. “Law enforcement agencies need the most advanced IT intrusion research and the most reliable attack tools to covertly and remotely gain access to computer systems,” Bekrar is quoted saying in the brochure. “Using previously unknown software vulnerabilities and exploits which bypass Antivirus products and modern operating system protections … could help investigators to successfully achieve this task.”
