Editor’s note: The following is based on an article by the author recently published in Defense & Security Analysis, entitled “Deterrence in the Age of Artificial Intelligence & Autonomy: A paradigm shift in nuclear deterrence theory and practice?”
A growing number of great powers are investing political capital and financial resources in developing the field of artificial intelligence technology and AI-enhanced autonomous weapons systems, seeking to derive the maximum potential military benefits—at a tactical, operational, and strategic level—these systems offer. The likely ubiquity of these new classes of advanced capabilities—and the incentives for militaries to adopt them—on the future battlefield is fast becoming a foregone certainty. There is little research that indicates how existing concepts of escalation, nuclear terrorism, and classical deterrence theories might apply (or be tested) in the digital age—increasingly defined by developments in AI and autonomy—where perfect information and rational decision making cannot be assumed.
How might the rise of these capabilities weaken or strengthen deterrence? How might non-human agents’ introduction into a crisis or conflict between nuclear powers affect deterrence, escalation, and strategic stability? Are existing theories of deterrence still applicable in the age of AI and autonomy?
Rethinking Deterrence Strategy in the Digital Age
In light of changes to the geopolitical and technological landscapes, along with new security threats and domains (nonstate actors, gray-zone conflict, space, and cyberspace), political scientists have conceptualized “four waves” of deterrence theorizing. The fourth wave followed the end of the Cold War and continues to the present day, coinciding with the broader features of the “second nuclear age”: multipolarity, asymmetric threats, nonstate actors (especially rogue nations and terrorists), and advanced strategic (nuclear and non-nuclear) weapons.
Like research on cyber deterrence, early scholarship on deterrence theory and practice in the digital age has been predominately grounded in classical deterrence approaches—associated with the earlier waves in international relations deterrence theorizing that were rooted in known hierarchical relationships between actors and the principle of mutually assured destruction. But deterrence in the digital age might be better conceptualized within the nascent “fifth wave” of modern deterrence, representing a conceptual break from previous waves of classical deterrence theorizing (or post-classical deterrence), and nonhuman agents into deterrence. Any discussion surrounding emerging technology such as AI comes with an important caveat. Since we have yet to see how AI might influence deterrence, escalation, strategic stability, and crisis management in the real world—notwithstanding the valuable insights from experimental wargaming—the discourse is largely a theoretical and speculative endeavor.
The post–Cold War literature is rich in scholarship on how technologically complex nuclear systems can cause technical (and human-related) accidents and false alarms, which are considered particularly escalatory where one side lacks confidence in its retaliatory (or second-strike) capacity. During the Cold War, the perennial fear that an action or signal misinterpreted by the other side—in the context of uncertainty and incomplete information associated with modern warfare—could trigger nuclear pre-emption is a useful point of departure to consider AI and autonomy.
Accidental nuclear war—a nuclear confrontation without a deliberate and properly informed decision to use nuclear weapons on the part of the nuclear-armed state(s) involved—could be caused by a variety of accidents. Most often these encompass a combination of human error, human-machine interaction failure, and procedural or organizational factors. Moreover, despite paying lip service to Machiavelli’s fortuna (the role of uncertainty in international affairs), decision-makers underestimate the importance and frequency of accidents and randomness in these interactions.
Similar to historical cases where human-machine interactions have caused or compounded accidents involving complex weapon systems, AI-enhanced systems operating at higher speeds, increased levels of sophistication, and compressed decision-making timeframes will likely further reduce the scope for de-escalating situations and contribute to future mishaps. The rapid proliferation and ubiquity of advanced technologies like offensive cyber, hypersonic weapons, and AI and autonomous weapons will make it increasingly difficult for states to mitigate this vulnerability without simultaneously improving their ability to strike first, thereby undermining the survivability of others’ strategic forces.
Unraveling Deterrence in the Field: Supersizing Counterforce Capabilities
The size, mobility, hardening, and relatively hidden features of great powers’ nuclear arsenals ensured states’ ability to withstand the first strike and deliver a retaliatory second strike, constituting the core pillars of Cold War–era nuclear deterrence. Like other disruptive technologies associated with the information revolution—particularly big-data analytics, robotics, quantum computing, nanotechnology, and cyber capabilities—advances in AI and autonomy threaten to upend this fragile arrangement in several ways.
Hunting for Nuclear Weapons
The integration of AI, machine learning, and big-data analytics can dramatically improve militaries’ ability to locate, track, target, and destroy a rival’s nuclear-deterrent forces—especially nuclear-armed submarines and mobile missile forces—and without the need to deploy nuclear weapons. AI-enabled capabilities that increase the vulnerability of second-strike capabilities (or are perceived to do so) heighten uncertainty and undermine deterrence—even if the state in possession of these capabilities did not intend to use them. In short, the capabilities AI might enhance (cyber weapons, drones, precision-strike missiles, and hypersonic weapons), together with the ones it might enable (intelligence, surveillance, and reconnaissance, automatic target recognition, and autonomous sensor platforms), could make hunting for mobile nuclear arsenals faster, cheaper, and more effective than before.
AI-enabled Cyber Threats to Nuclear Command-and-Control Systems
Today, it is thought possible that a cyberattack (e.g., spoofing, hacking, manipulation, and digital jamming) could infiltrate a nuclear weapons system, threaten the integrity of its communications, and ultimately (and possibly unbeknown to its target) gain control of its—possibly dual-use—command and control systems. For instance, a nonstate third-party hacker might interfere with or sabotage nuclear command-and-control systems, spoof or otherwise compromise early warning systems (or components of the nuclear firing chain), or in a worst-case scenario, trigger an accidental nuclear launch. Advances in AI could also exacerbate this cybersecurity challenge by enabling improvements to the cyber offense. AI-augmented cyber tools’ machine speed could enable an attacker to exploit a narrow window of opportunity to penetrate an adversary’s cyber defenses or use advanced persistent threat tools to find new vulnerabilities faster and easier than before.
Drone Swarming under the Nuclear Shadow
Drones (especially micro-drones) used in swarms are conceptually well suited to conduct preemptive attacks and nuclear ISR missions against an adversary’s nuclear mobile missile launchers, ballistic missile submarines, and their enabling facilities (e.g., early-warning systems, antennas, sensors, and air intakes). In short, the ability of drone swarming technology infused with future iterations of AI and machine learning—mining expanded and dispersed data pools— to locate, track, and target strategic missiles (e.g., mobile ICBM launchers in underground silos and onboard stealth aircraft or submarines) is set to grow. Notwithstanding the remaining technical challenges—especially the demand for power—swarms of robotic systems fused with AI and machine-learning techniques may presage a powerful interplay of increased range, accuracy, mass, coordination, intelligence, and speed in a future conflict.
Automating Strategic Decisions: Double-Edged Sword for Deterrence?
On the one hand, future AI-augmented command-and-control support tools may overcome many of the shortcomings inherent to human strategic decision making during wartime (e.g., susceptibility to invest in sunk costs, skewed risk judgment, heuristics, and groupthink) with potentially stabilizing effects. Further, faster and more reliable AI applications could also enable commanders to make more informed decisions during a crisis, improve the safety and reliability of nuclear support systems, strengthen the cyber defenses of command-and-control networks, enhance battlefield situational awareness, and reduce the risk of human error caused by fatigue and repetitive tasks. On the other hand, AI systems that allow commanders to predict the potential production, commissioning, deployment, and ultimately launch of nuclear weapons by adversaries will likely lead to unpredictable system behavior and outcomes, which in extremis could undermine first-strike stability—the premise of mutually assured destruction—making nuclear wars winnable.
Today, the potential tactical and operational impact of AI is qualitatively axiomatic. Its effect at a strategic level remains, however, uncertain. AI systems that are programmed to pursue tactical and operational advantages aggressively, for example, might misperceive (or ignore) an adversary’s bid to signal to resolve (i.e., to de-escalate a situation) as a prelude to an imminent attack. These dynamics would increase the risks of inadvertent escalation and first-strike instability. If commanders decide to delegate greater authority to inherently inflexible AI systems, the dehumanization of future defense planning will undermine stability by significantly inhibiting induction.
Human induction—the ability to form general rules from specific pieces of information—is a crucial aspect of defense planning, primarily to manage situations that require high levels of visual and moral judgment and reasoning. Unwarranted confidence in and reliance on machines—known as “automation bias”—in the pre-delegation of the use of force during a crisis or conflict, let alone during nuclear brinksmanship, might inadvertently compromise states’ ability to control escalation.
Under crisis and conflict conditions, AI’s deterrent effect is predicated on the perceived risks associated with a particular capability it enables or enhances. With higher uncertainty, deploying AI-augmented capabilities in a crisis might encourage an adversary to act more cautiously and, in turn, bolster stability. Counterintuitively, therefore, states may view the expanded automation of their nuclear command-and-control systems as a way to manage escalation and strengthen deterrence, signaling to an adversary that any attack—or the threat of one—might trigger nuclear escalation.
Because of the difficulty of demonstrating a posture like this before a crisis or conflict, this implicit threat—akin to the Dr. Strangelove doomsday machine farce (or parable)—may equally worsen crisis instability. Moreover, the confusion and uncertainty that would result from mixing various (and potentially unknown) levels of human-machine interactions, along with AI reacting to events—such as signaling and low-level conflict—in nonhuman ways (using force where a human commander would not have) and at machine speed, could dramatically increase inadvertent risk.
The recent defeat of a human pilot by an AI system in a DARPA-hosted Alpha Dogfight Challenge demonstrated how AI’s performing in complex physics in a dynamic (albeit virtual) environment could compress the observe, orient, decide, and act (OODA) decision-making loop and apply nonconventional tactics in a high-stakes game of human-to-machine chicken.
Deterrence in “Human vs. Machine” Interactions
Recent experimental wargaming hosted by the RAND Corporation explored the effects of mixing various levels of humans and machine configurations on escalatory dynamics—signaling, decision making, and de-escalation—during a crisis revealed some interesting preliminary findings. The wargames’ tentative findings demonstrated that where high levels of autonomy coincide with primarily human decision making (or “humans on the loop”), escalation risk is generally lower.
This hypothesis was attributed to the fact that human involvement in decisions allowed more time to de-escalate (e.g., devise off-ramps) and that humans are likely to better understand signaling (e.g., seeking resolution, issuing a deterrent threat, indicating desire to de-escalate, or reassuring) compared to an AI.
Conversely—and this is perhaps most speculative—when decisions are primarily made by machines and combined with high levels of autonomy (or “humans out of the loop”), escalation risk is higher—but because of the lower human risk, the perceived costs of miscalculation are lower. The removal of human decision making and judgment from a crisis, and less risk to human life, would reduce the traditional risks associated with accidents in human-machine interactions.
The total absence of a normative deterrence framework—in particular, to signal to resolve to an adversary while simultaneously seeking to de-escalate a situation—may compress (or remove entirely) the various rungs of the inherently psychological escalation ladder. It may increase inadvertent escalation risks and complicate de-escalation and conflict termination—especially in asymmetric dyads where incentives to strike preemptively to achieve escalation dominance exist.
The potentially escalatory effects of AI’s tactical optimization programming would likely be compounded by differences in adversaries’ goal setting (an AI’s priorities, value alignment, control, and off-ramps), command-and-control organization (centralized vs. decentralized), and the configuration of their human-machine interactions. Specifically, machine decision making—designed to exploit the tactical and operational advantages in a situation—may lack the “theory of the mind” in an a priori situation with humans’ interaction. Not only would machines need to understand human commanders and human adversaries, but they must also interpret an adversary AI’s signaling and behavior.
An AI algorithm optimized to pursue pre-programmed goals might misinterpret an adversary simultaneously signaling its desire to seek resolution to avoid conflict or de-escalate a situation. Absent reliable means to attribute an actor’s intentions, AI systems may convey undesirable and unintended signals (by human commanders) to the enemy, thus complicating the delicate balance between an actor’s willingness to escalate a situation as a last resort and keeping the option open to step back from the brink. Simply put, the risks of inaction are great.
Given the potentially transformative effects of AI and autonomy augmentation on a range of (nuclear and non-nuclear) strategic technologies, rethinking existing assumptions, theories, and permutations of deterrence—premised on human rationality, perceptions, and nuanced signaling—is now needed. Success in these efforts will require all stakeholders to be convinced of the need and potential mutual benefits of taking steps toward establishing a coherent governance architecture to institutionalize and internalize new norms and to ensure compliance with the design and deployment of AI and autonomy in the military sphere. A reticence to engage on these issues, or worse, downplaying the potential risks associated with AI and autonomy would make it more challenging to alter incentives to enhance strategic stability and shape deterrence and escalation as the technology matures and the military use of these systems inevitably increases.
Dr. James Johnson is an assistant professor in the School of Law and Government at Dublin City University and a fellow with the Modern War Institute at West Point. Dr. Johnson was previously a postdoctoral research fellow at the James Martin Center for Nonproliferation Studies in Monterey, California. He the author of The US-China Military & Defense Relationship during the Obama Presidency. His latest book is titled Artificial Intelligence & the Future of Warfare: USA, China, and Strategic Stability. Follow him on Twitter: @James_SJohnson.
The views expressed are those of the author(s) and do not reflect the official position of the United States Military Academy, Department of the Army, or Department of Defense.
Image credit: Airman 1st Class Joel Pfiester, US Air Force