Data and Goliath

by Bruce Schneier

  Technology is value neutral. You can use your phone to call 911 or to plan a bank robbery. There’s no technical difference between a government’s using a tool to identify criminals or using it to identify dissidents. There’s no technical difference between corporate and government uses. Legitimate corporate tools for blocking employees from e-mailing confidential data can be used by repressive governments for surveillance and censorship. Conversely, the same anti-censorship tools that Saudi and Iranian dissidents use to evade their governments can be used by criminals to distribute child porn. Encryption allows the good guys to communicate without being eavesdropped on by the bad guys, and also allows the bad guys to communicate without being eavesdropped on by the good guys. And the same facial recognition technology that Disney uses in its theme parks to pick out photos its patrons might want to buy as souvenirs can identify political protesters in China, and Occupy Wall Street protesters in New York.

  GOVERNMENTS SUBVERTING COMMERCIAL SYSTEMS

  So far, I have discussed how government surveillance piggybacks on corporate capabilities. While this is mostly true, government are not above forcing corporations to spy for them.

  Back in the early 1990s, the FBI started worrying about its ability to conduct telephone surveillance. The FBI could do it with the old analog phone switches: a laborious process involving alligator clips, wires, and a tape recorder. The problem was that digital switches didn’t work that way. Isolating individual connections was harder, and the FBI became concerned about the potential loss of its ability to wiretap. So it lobbied Congress hard and got a law passed in 1994 called the Communications Assistance for Law Enforcement Act, or CALEA, requiring telcos to re-engineer their digital switches to have eavesdropping capabilities built in.

  Fast-forward 20 years, and the FBI again wants the IT industry to make surveillance easier for itself. A lot of communications no longer happen over the telephone. They’re happening over chat. They’re happening over e-mail. They’re happening over Skype. The FBI is currently lobbying for a legislative upgrade to CALEA, one that covers all communications systems: all voice, video, and text systems, including World of Warcraft and that little chat window attached to your online Scrabble game.

  The FBI’s ultimate goal is government prohibition of truly secure communications. Valerie Caproni, the general counsel for the FBI, put it this way in 2010: “No one should be promising their customers that they will thumb their nose at a US court order. They can promise strong encryption. They just need to figure out how they can provide us plain text.” Translation: you can’t actually provide security for your customers.

  Depending on the system, doing what the FBI wants would range from easy to impossible. E-mail systems like Gmail are easy. The mail resides unencrypted on Google’s servers, and the company has an office full of people who respond to requests for access to individual accounts from governments all over the world. Encrypted chat programs like Off the Record are impossible to undermine; the chat sessions are encrypted on the conversants’ computers, and there’s no central node from which to eavesdrop. In those cases, the only way to satisfy the FBI’s demands would be to add a backdoor to the user software, which would render it insecure for everyone. I’ll talk about the stupidity of that idea in Chapter 11.

  As draconian as that measure would be, at least the discussion is happening in public. Much government control of corporate communications infrastructure occurs in secret, and we only hear about it occasionally.

  Lavabit was an e-mail service that offered more security privacy than the large corporate e-mail services most of us use. It was a small company, owned and operated by a programmer named Ladar Levison, and it was popular among the tech-savvy. It had half a million users, Edward Snowden amongst them.

  Soon after Snowden fled to Hong Kong in 2013, Levison received a National Security Letter demanding that the company turn over the master encryption key that protected all of Lavabit’s users—and then not tell any of its customers that they could be monitored. Levison fought this order in court, and when it became clear that he had lost, he shut down his service rather than deceive and compromise his customers.

  The moral is clear. If you run a business, and the FBI or the NSA wants to turn it into a mass surveillance tool, it believes that it is entitled to do so, solely on its own authority. The agency can force you to modify your system. It can do it all in secret and then force your business to keep that secret. Once it does that, you no longer control that part of your business. If you’re a large company, you can’t shut it down. You can’t realistically terminate part of your service. In a very real sense, it is not your business anymore. It has become an arm of the vast US surveillance apparatus, and if your interest conflicts with the agency’s, the agency wins. Your business has been commandeered.

  The only reason we know this story is that Levison ran his own company. He had no corporate masters. He had no shareholders. He was able to destroy his own business for moral reasons. Larger, more beholden companies would never do that. We must assume that every other computer company that received a similar demand has eventually complied.

  For example, we know that the US government convinced Skype—through bribery, coercion, threat, or legal compulsion—to make changes in how the program operates, to facilitate eavesdropping. We don’t know what the changes were, whether they happened before or after Microsoft bought Skype in 2011, or how they satisfied whatever the government demanded, but we know they happened.

  In 2008, the US government secretly threatened Yahoo with a $250,000-per-day fine, with the daily amount increasing rapidly if it didn’t join the NSA’s PRISM program and provide it with user data. And in 2004, the NSA paid RSA Security to make a backdoored random number generator a default in its crypto library.

  Other types of government commandeering are going on as well, behind the backs of the companies whose technologies are being subverted. Where the NSA doesn’t have agreements with companies to tap into their systems, it does its best to do so surreptitiously. For instance, not satisfied with the amount of data it receives from Google and Yahoo via PRISM, the NSA hacked into the trunk connections between both companies’ data centers, probably with the cooperation of their service provider Level 3 Communications. The angry response from one of Google’s security engineers, posted on his personal Google Plus page, was “fuck those guys.” Google has since encrypted those connections between its data centers in an effort to keep the NSA out. Yahoo claims to be doing the same.

  This isn’t the only example of the NSA hacking US technology companies. The agency creates fake Facebook pages to hack into people’s computers, and its TAO branch intercepts Cisco equipment during shipping to install hardware implants.

  We don’t know what sort of pressure the US government has put on the major Internet cloud providers to persuade them to give them access to user data, or what secret agreements those companies may have reached with the NSA. We do know the NSA’s BULLRUN program to subvert Internet cryptography, and the companion GCHQ program EDGEHILL, were successful against much of the security that’s common on the Internet. Did the NSA demand Google’s master encryption keys and force it to keep quiet about it, as it tried with Lavabit? Did its Tailored Access Operations group break into Google’s overseas servers and steal the keys, or intercept equipment intended for Google’s overseas data centers and install backdoors? Those are all documented NSA tactics. In the first case, Google would be prohibited by law from admitting it, in the second it wouldn’t want to, and in the third it would not even know about it. In general, we know that in the years immediately after 9/11, the US government received lots of willing cooperation from companies whose leaders believed they were being patriotic.

  I believe we’re going to see more bulk access to our data by the NSA, because of the type of data it wants. The NSA used to be able to get everything it wanted from Internet backbone companies and broadband providers. This became less true as encryption—specifically a kind c
alled SSL encryption—became more common. It will become even less true as more of the Internet becomes encrypted. To overcome this, the NSA needs to obtain bulk data from service providers, because they’re the ones with our data in plaintext, despite any encryption in transit. And to do that it needs to subvert the security protocols used by those sites to secure their data.

  Other countries are involved in similar skullduggery. It is widely believed that the Chinese government embeds the capability to eavesdrop into all networking equipment built and sold by its own company Huawei. And we have reason to suspect that British, Russian, Israeli, and French Internet products have also been backdoored by their governments.

  We don’t know whether governments attempt to surreptitiously insert backdoors into products of companies over which they have no direct political or legal control, but many computer security experts believe that is happening. Are there Chinese nationals working at major US software companies trying to make it easier for the Chinese government to hack that company’s products? French programmers? Israeli programmers? Or, at least, are they passing the source code back to their own country so they can find vulnerabilities more easily? Are there US agents inserting backdoors into computer chips designed and manufactured in Asia? We know they have employees secretly embedded in countries like China, Germany, and South Korea to aid in subverting computer and communications systems.

  Companies have responded to this situation with caveat-laden pseudo-assurances. At a 2013 technology conference, Google CEO Eric Schmidt tried to reassure the audience by saying that he was “pretty sure that information within Google is now safe from any government’s prying eyes.” A more accurate statement might be: “Your data is safe from governments, except for the ways we don’t know about and the ways we cannot tell you about.” That’s a lousy marketing pitch, but as long as the NSA is allowed to operate using secret court orders based on secret interpretations of secret law, it will never be any different.

  For most Internet companies, this isn’t a problem. The other thing Schmidt didn’t say is: “And, of course, we still have complete access to it all, and can sell it at will to whomever we want . . . and you have no recourse.” As long as these companies are already engaging in massive surveillance of their customers and users, it’s easier for them to comply with government demands and share the wealth with the NSA. And as long as governments keep demanding access and refrain from legislating protections, it’s easier to design systems to allow it. It’s a powerful feedback loop: the business model supports the government effort, and the government effort justifies the business model.

  7

  Political Liberty and Justice

  In 2013, the First Unitarian Church of Los Angeles sued the NSA over its domestic spying, claiming that its surveillance of church members’ telephone calling habits discouraged them from banding together to advocate for political causes. The church wasn’t just being paranoid. In the 1950s and 1960s, the FBI monitored its minister because of his politics. Today, the church is worried that people, both Americans and foreigners, will end up on watch lists because of their association with the church.

  Government surveillance is costly. Most obviously, it’s extraordinarily expensive: $72 billion a year in the US. But it’s also costly to our society, both domestically and internationally. Harvard law professor Yochai Benkler likens NSA surveillance to an autoimmune disease, because it attacks all of our other systems. It’s a good analogy.

  The biggest cost is liberty, and the risk is real enough that people across political ideologies are objecting to the sheer invasiveness and pervasiveness of the surveillance system. Even the politically conservative and probusiness Economist magazine argued, in a 2013 editorial about video surveillance, that it had gone too far: “This is where one of this newspaper’s strongly held beliefs that technological progress should generally be welcomed, not feared, runs up against an even deeper impulse, in favour of liberty. Freedom has to include some right to privacy: if every move you make is being chronicled, liberty is curtailed.”

  ACCUSATION BY DATA

  In the 17th century, the French statesman Cardinal Richelieu famously said, “Show me six lines written by the most honest man in the world, and I will find enough therein to hang him.” Lavrentiy Beria, head of Joseph Stalin’s secret police in the old Soviet Union, declared, “Show me the man, and I’ll show you the crime.” Both were saying the same thing: if you have enough data about someone, you can find sufficient evidence to find him guilty of something. It’s the reason many countries’ courts prohibit the police from engaging in “fishing expeditions.” It’s the reason the US Constitution specifically prohibits general warrants—documents that basically allow the police to search for anything. General warrants can be extremely abusive; they were used by the British in colonial America as a form of social control.

  Ubiquitous surveillance means that anyone could be convicted of lawbreaking, once the police set their minds to it. It is incredibly dangerous to live in a world where everything you do can be stored and brought forward as evidence against you at some later date. There is significant danger in allowing the police to dig into these large data sets and find “evidence” of wrongdoing, especially in a country like the US with so many vague and punitive laws, which give prosecutors discretion over whom to charge with what, and with overly broad material witness laws. This is especially true given the expansion of the legally loaded terms “terrorism,” to include conventional criminals, and “weapons of mass destruction,” to include almost anything, including a sawed-off shotgun. The US terminology is so broad that someone who donates $10 to Hamas’s humanitarian arm could be considered a terrorist.

  Surveillance puts us at risk of abuses by those in power, even if we’re doing nothing wrong at the time of surveillance. The definition of “wrong” is often arbitrary, and can quickly change. For example, in the US in the 1930s, being a Communist or Socialist was a bit of an intellectual fad, and not considered wrong among the educated classes. In the 1950s, that changed dramatically with the witch-hunts of Senator Joseph McCarthy, when many intelligent, principled American citizens found their careers destroyed once their political history was publicly disclosed. Is someone’s reading of Occupy, Tea Party, animal rights, or gun rights websites going to become evidence of subversion in five to ten years?

  This situation is exacerbated by the fact that we are generating so much data and storing it indefinitely. Those fishing expeditions can go into the past, finding things you might have done 10, 15, or 20 years ago . . . and counting. Today’s adults were able to move beyond their youthful indiscretions; today’s young people will not have that freedom. Their entire histories will be on the permanent record.

  Another harm of government surveillance is the way it leads to people’s being categorized and discriminated against. George Washington University law professor Daniel Solove calls the situation Kafkaesque. So much of this data is collected and used in secret, and we have no right to refute or even see the evidence against us. This will intensify as systems start using surveillance data to make decisions automatically.

  Surveillance data has been used to justify numerous penalties, from subjecting people to more intensive airport security to deporting them. In 2012, before his Los Angeles vacation, 26-year-old Irishman Leigh Van Bryan tweeted, “Free this week, for quick gossip/prep before I go and destroy America.” The US government had been surveilling the entire Twitter feed. Agents picked up Bryan’s message, correlated it with airplane passenger lists, and were waiting for him at the border when he arrived from Ireland. His comment wasn’t serious, but he was questioned for five hours and then sent back home. We know that bomb jokes in airports can get you detained; now it seems that you have to be careful making even vague promises of international rowdiness anywhere on the Internet.

  In 2013, a Hawaiian man posted a video on Facebook showing himself drinking and driving. Police arrested him for the crime; his defense was that it was a paro
dy and that no actual alcohol was consumed on the video.

  It’s worse in the UK. There, people have been jailed because of a racist tweet or a tasteless Facebook post. And it’s even more extreme in other countries, of course, where people are routinely arrested and tortured for things they’ve written online.

  Most alarming of all, the US military targets drone strikes partly based on their targets’ data. There are two types of drone targeting. The first is “targeted killing,” where a known individual is located by means of electronic or other surveillance. The second is “signature strikes,” where unidentified individuals are targeted on the basis of their behavior and personal characteristics: their apparent ages and genders, their location, what they appear to be doing. At the peak of drone operations in Pakistan in 2009 and 2010, half of all kills were signature strikes. We don’t have any information about how accurate the profiling was.

  This is wrong. We should be free to talk with our friends, or send a text message to a family member, or read a book or article, without having to worry about how it would look to the government: our government today, our government in five or ten years, or some other government. We shouldn’t have to worry about how our actions might be interpreted or misinterpreted, or how they could be used against us. We should not be subject to surveillance that is essentially indefinite.