The Raging 2020s

by Alec Ross

  In every era before our own, physical damage could be dealt only by physical force. But in the 21st century, militaries can deal a physical blow with a digital signal. With the right lines of code, you can cripple a nuclear reactor, destroy a munitions factory, or knock out power to an entire country. You can infiltrate the computer networks of your enemy, surveil their every move, and stop them from launching attacks your way. The digital warrior never needs to look up from their keyboard.

  Cyber weapons are not the only digital technology that is transforming national security. Artificial intelligence is also revolutionizing how militaries do battle and spy agencies conduct espionage. Using AI systems, governments can spot individuals in a crowd, locate facilities to attack, detect intrusions on a computer network, predict civil uprisings, and identify potentially violent extremists.

  While the United States can best any other country in the four physical domains—land, sea, air, and space—it faces a much more level playing field in cyberspace. The Pentagon has been slower to adopt national security technologies like artificial intelligence than certain other countries including China. One major reason is that the leading developers of these new digital tools are not members of the traditional military-industrial complex—companies like Lockheed Martin, Raytheon, and Northrop Grumman. Instead, they are in the technology industry.

  This is a major difference. Traditional military contractors have worked for decades in lockstep with the Pentagon. But the leaders in the tech industry were people like Chris Lynch, skeptical of government bloat and bureaucracy, whose hackles were raised by the ways in which the same technology they were making could be turned toward destructive ends.

  Lynch’s experience at the Pentagon began to change his mind. As we’ve seen in recent years, democracy is a fragile thing, especially in the face of cyberattacks we are not prepared for. Someone needed to do the work to help prepare for the worst—and Chris Lynch realized that he was made of the right stuff.

  In 2015, Lynch became director of the Defense Digital Service, and in 2019 he left the government to found his own company, Rebellion Defense. His company is one of several to emerge in recent years to provide countries with the technologies needed to protect national security in the 21st century. This industry includes a handful of big-name players like Microsoft and Palantir, as well as countless smaller companies that few people outside the national security world would know, like Rebellion Defense. This new breed of government contractor is changing the way the US and its allies conduct foreign policy around the globe, and they are upending the traditional relationship between sovereign nations and their defense businesses.

  Lynch’s overall aim with Rebellion Defense is to provide the US military with much-needed digital tools while also helping young technologists reach the same epiphany about public service that he did.

  His strategy is encapsulated in the name of the company.

  While leading the Defense Digital Service, Lynch fashioned the organization into something of a haven for Star Wars geeks. Team members gave projects names like “BOBA,” “AT-AT,” and “JEDI.” The group had an office in Augusta, Georgia, called Tatooine. Lynch’s going-away party was attended by a group of Pentagon staff dressed up as Star Wars characters. During our walk, he showed me a photo from the party that included him, his dad, General Paul Selva (who was at the time the nation’s second-highest-ranking uniformed officer), Chewbacca, and a fully robotic R2-D2.

  But what really set the tone for the group was the sign outside Lynch’s office, which read “Defense Digital Service, Rebel Alliance.” According to Lynch, the plaque was intended to signify to the team that they were the “rebels” bringing change to the bureaucracy that surrounded them. To continue the Star Wars analogy, the Defense Digital Service was Luke, Leia, and Han, and the department’s culture was Darth Vader and Emperor Palpatine.

  Lynch saw the idea of rebellion as the central theme of his tenure at the Pentagon, and he looks to bring the same mentality to his new defense business.

  The name Rebellion Defense “is a recognition that there’s something new that has to be created. Now more than ever, this will be the team that does it, and this will be the time that we do it,” he said. “It’s something where we’re the outsiders, but we’re part of the system as well, because defense is owned by all of us.”

  There is a certain irony in naming a defense technology company after the destroyers of the Death Star, and Lynch himself is quick to acknowledge it. Still, it has helped bridge the gap between Washington and Silicon Valley, he said. “Who doesn’t want to be part of the Rebel Alliance?”

  Lynch also acknowledged that in today’s world, the line between good and evil is much less cut-and-dry than it was a long, long time ago in a galaxy far, far away. Even compared to the Cold War era, the landscape that Rebellion and its counterparts navigate today contains many shades of gray. Arms controls are not nearly as clear-cut for digital technology as they are for old-fashioned ballistics, so modern defense companies have much more that they need to figure out on their own.

  Throughout the Cold War, the United States created a broad regulatory regime to limit the export of weapons and national security technologies to foreign countries. For example, the International Traffic in Arms Regulations (ITAR) prohibits non–US citizens from accessing technical data or physical materials for certain defense-related technologies. (As of August 2020, the US Munitions List was more than one hundred pages long.) Other measures—like the Arms Export Control Act and Export Administration Regulations—restricted the export and use of national technologies to foreign countries. The goal behind such standards is clear: to prevent national security technologies developed by the United States from winding up in enemy hands. The government could customize its arms control agreements to fit its relationships with different countries. The regulations for the United Kingdom differed from those with Turkey, which differed from those with Iran. During the Cold War, these laws cemented the lines between militaries of liberal democracies and communist states, and in the post–Cold War period, they have (mostly) prevented traditional weapons from falling into the hands of terrorists and adversarial nations.

  But the arms control regime of the 20th century largely does not apply to the national security technologies of the 21st century. One of the main reasons is that digital tools like AI are more difficult to categorize than traditional defense technologies.

  Fighter jets and warships are used for one thing: the projection and exercise of military power. But artificial intelligence is a general-purpose technology with both national security applications and completely benign commercial uses. A computer vision algorithm can be trained to spot enemy combatants on a battlefield, but it can also be used to tag friends in social media posts and power self-driving cars. AI takes on the values and intentions of its human masters. The same AI-enabled facial recognition technology that can identify known terrorism suspects can just as easily profile and track members of an ethnic minority. The technology is also imperfect. The accuracy of artificial intelligence depends on the quality of the data used to train it, and it is not always clear how the software reaches a particular conclusion. While you can tolerate a certain number of errors in a system that provides online shopping recommendations, the consequences of an error on the battlefield can be lethal.

  These are core concerns of Rebellion Defense’s work. It makes traditional IT and cybersecurity tools, but its bread and butter is AI. This includes software that can read text, classify images, analyze video, and process the enormous amount of information flooding into the Pentagon from every corner of the globe.

  However, when it comes to arms control, the “dual use” nature of artificial intelligence software creates a conundrum for US policy makers. Subject all AI systems to the same regulations that apply to nuclear warheads, and you stifle innovation and cripple the American tech industry. But leave them completely unregulated, and you could enable terrorists and enemy militaries t
o get their hands on powerful weapons of war, made in the USA.

  To effectively regulate the sale of AI and other emerging technologies, policy makers must first agree on which narrow applications pose a national security threat if they wind up in enemy hands. But today, the definition of “security sensitive” technology varies widely based on whom you ask, said MIT’s R. David Edelman, a former White House senior official who led policy making at the intersection of technology and national security.

  “That question about what’s really sensitive is a fundamental debate that is taking place … at the government level, which is not always informed by technology; at the industry level, which is certainly not always informed by government; and at the researcher level, which is sometimes not informed by either of them,” Edelman told me. “You’ve seen little blips where these communities get out of sync.”

  He continued, “If you were to go ask researchers what constitutes an AI technology, they would give you exactly as many answers as the number of researchers you asked, possibly plus five or six. The reality is that something like AI means everything and nothing.”

  The confusion is compounded by the lack of technical expertise in the halls of government. Today, the thirty students in my son’s high school class could compete in technology know-how with the 535 members of the US Congress. Informed policy decisions require informed policy makers, and most of the government is still interpreting the national security challenges of the 21st century through the lens of 20th-century technology.

  In January 2020, the US Commerce Department issued its first export control on an artificial intelligence system. The rule limits the sale of AI software that can automatically analyze geospatial imagery, ostensibly collected by military drones and satellites. While this is a significant step, it is important to recognize that the government did not exactly break new ground. Geospatial technology was already highly regulated. Companies could not sell imagery above a certain resolution, and both drones and satellites are themselves subject to ITAR and other export controls. Policy makers simply amended an old framework to accommodate a new technology.

  However, there are many applications of AI for which there is no precedent in the existing arms control regime. AI systems can help authoritarian regimes consolidate power within their own borders. While facial recognition and surveillance technology may not fit the traditional description of national security technologies, they are no less threatening to free and open societies. Yet Western companies have freely exported these technologies for years, Edelman said, and “I think most members of the American public and certainly a lot of public policymakers wish they hadn’t.” Beyond this gray area, there are algorithms that can be even more consequential. Today, companies are developing systems that can identify enemy combatants on the battlefield, power semiautonomous weapons, and coordinate drone swarms.

  Edelman explained that certain types of AI can be more easily weaponized than others and that “those are the sorts of implementations that it is entirely appropriate to regulate, and frankly, government’s a little bit behind the ball in identifying them.”

  US military leaders have begun to stress the importance of AI ethics, and in 2020 the Pentagon signed on to a set of five broad principles for the ethical application of the technology. However, these principles are vague, stating platitudes such as “personnel will exercise appropriate levels of judgment and care” when developing and using AI.

  As we have discussed, today’s geopolitical landscape is not as binary as it was during the Cold War, and countries cannot be classified as either “allies of democracy” or “allies of communism.” Political and economic models fall on a spectrum from open to closed, and national alliances are not as fixed as they once were.

  The members of the new cyber-military-industrial complex are navigating this new world largely on their own. This puts them in a unique position where they need to formulate clear principles for the types of technology they are willing to develop, and what goes too far. This is a lot to ask of a company, and they cannot handle it wholly on their own. The challenge is exacerbated when the technology executive is very young and may be a great engineer but does not have much experience in the world of geopolitics. There is a difference between intelligence and wisdom, and I have seen too many mistakes made by technology executives who are very intelligent but not yet wise.

  At the same time, when the technology sector has so much more expertise than the traditional defense sector, it is worth harnessing that expertise and ensuring that technology companies shoulder the responsibility for what they are making. As we will see, a system that allows companies to weigh in and even lead allows more informed innovation and implementation—and provides more checks and balances than a system where government decides and drives everything.

  For Rebellion Defense founder Chris Lynch, that sense of responsibility is a motivating force. “If you have strong opinions about national defense and security and the utilization of all these technologies that are ultimately going to change the world over the next fifty years, you have an obligation to show up at the table,” Lynch said. “You are providing the things that people need, and you’re helping craft the strategy, the policy, the implementation, and the execution of how those technologies will be used.”

  Like most other technologies, artificial intelligence is neither inherently good nor inherently bad. It takes on the values and intentions of its creators and users. Given the Pentagon’s lack of technological expertise, the companies that develop its AI tools have a responsibility to steer it in the right direction.

  “Every technology company has a responsibility to think through the implications of the things that they’re building,” Lynch told me. “We need to be thoughtful … with what we build and why we build it. We need to care because we are dictating defense. We are dictating how technology gets used.”

  At Rebellion Defense, employees meet once a month to discuss the types of projects and customers the company would refuse to take on. For example, Lynch said, the company has already determined it will not build domestic surveillance technology, nor will it aid US officials in rounding up undocumented immigrants. Lynch was reluctant to disclose Rebellion’s other lines in the sand, though he said the company has turned down multiple offers based on feedback from employees.

  Of course, when this decision-making lands with companies, there can be major differences in how it is applied.

  In September 2017, Google began working with the Pentagon on a broad artificial intelligence initiative called “Project Maven.” Google’s particular project sought to build AI software that could sift through the troves of footage collected each day by military drones. The system would save intelligence officers from the tedious task of analyzing the footage frame by frame. This is the sort of geospatial analysis software that would fall under the government’s January 2020 export controls.

  Within months, Google employees began protesting the project, which they argued would help the Pentagon better target its drone strikes. In April 2018, some 3,100 employees signed a letter demanding that Google stop participating “in the business of war.” Soon after, Google declined to renew its contract with the Pentagon.

  Chris Lynch disagreed with that decision. Google may have seen it as its own line in the sand. But as Lynch saw it, Google gave up an opportunity to directly influence how the Pentagon uses artificial intelligence. Instead, the contract went to Anduril Industries, a defense technology company founded by Palmer Luckey, a controversial libertarian in his twenties who helped invent the Oculus Rift virtual reality headset.

  Anduril was contracted to build an AI-powered sensor network that would provide troops with a virtual view of the front lines. The sensors would be mounted on drones, fixed towers, and troops themselves, and used to identify potential targets and direct autonomous military vehicles into combat. The software helps troops in the field make real-time operational decisions. It might not directly decide who lives and who d
ies, but it will significantly influence how troops arrive at that answer.

  Anduril applied different principles to its work than Google or Rebellion Defense would have. The company went on to build a similar AI-sensor network to help US Customs and Border Protection to coordinate operations along the US-Mexico border. When asked in 2018 whether there were any Pentagon projects Anduril would turn down, Luckey said, “that’s not really totally up to us. We are working with the U.S. government.”

  That said, Anduril CEO Brian Schimpf told me there is one thing the company would not do: they would not build systems that execute “lethal force” without a human in the loop. In other words, Anduril will not create robots that can kill on their own accord.

  “This is a military decision-making responsibility—it can’t be outsourced to a machine,” he said. “Everything else is one of these questions where I think it’s mostly a matter of the controls on how the technology is employed. There are very few other technology areas that I think have those sort of bright lines.”

  Schimpf thinks it is the responsibility of military leaders to set those controls, and he trusts them to make the right call in the end. “Any of these [applications] that are too out there, they eventually get shut down, they eventually get stopped. The US system may take a while, but it is quite robust to keeping a lot of these overreaches in check.”

  That all three companies end up resolving on different principles in the development of AI might seem worrying—but that is also part of the debate that needs to happen with such new technologies. There are not clear-cut ethical answers at the outset. But realistically, the Pentagon does not have much of a choice as to whether to develop its artificial intelligence capabilities. China and Russia are investing heavily in military AI, and the national security of the US and its allies will suffer if they do not do the same. Russian president Vladimir Putin remarked to a group of students, “Artificial Intelligence is the future not only of Russia but of all mankind,” and added that “whoever becomes the leader in this sphere will become the ruler of the world.”