Legal Accountability for AI-Driven Autonomous Weapons
As algorithms begin to make decisions that determine who lives and who dies on the battlefield, the rise of AI-driven autonomous weapon systems (AWS) is forcing a re-examination of some of the most basic principles of international humanitarian law (IHL). This is especially so as recent breakthroughs in AI have fueled AWS proliferation among major powers, with China and Russia in particular pouring increasing resources into innovations like swarming drones. Despite supporters pointing to AWS’s potential for increased precision, decreased human error, and fewer military casualties, its introduction into armed conflicts raises pressing questions about compliance with IHL and the capacity to hold parties responsible for ensuing harms.
The Legality of Autonomous Weapons
Although the debate is relatively new and the term “autonomous weapons” frequently conjures up images of futuristic robots controlled by AI, the underlying concept is by no means novel. Long before today’s debates on algorithmic accountability and lethal AI, battlefields were already shaped by autonomous weapons like drifting naval mines, torpedoes, and cruise missiles designed to strike targets without real-time human input, and regulating such weapons sparked heated debates among military planners, legislators, and legal experts.
Take mines for example. They are among the oldest autonomous weapons, operating via simple stimulus-response mechanisms (e.g., pressure or magnetic triggers) without human oversight, yet are subject to stringent regulations. The 1997 Ottawa Convention prohibits anti-personnel mines because they are indiscriminate, meaning they cannot distinguish between civilians and combatants and continue to kill thousands long after conflicts are over. Anti-vehicle mines are still permitted on the battlefield, but only if they actually follow IHL rules, although the example of the Russo-Ukrainian War shows how easily rules can be disregarded.
The legality of autonomous weapons under IHL ultimately hinges on their compliance with the core principles set out in the 1949 Geneva Conventions and their 1977 Additional Protocol I: distinction; proportionality; precaution; and the Martens Clause. But once AI enters the picture, these long-standing rules run into a level of complexity they were never designed to handle, creating an entirely new set of accountability challenges.
This tension is most acute in the emergence of lethal autonomous weapons systems (LAWS). The International Committee of the Red Cross (ICRC) defines LAWS as weapons that can search for, detect, identify, select, and attack targets without meaningful human intervention, placing autonomy at the core of the targeting cycle rather than merely in ancillary functions. Once a machine rather than a human exercises the judgment traditionally required to distinguish civilian from combatant, or to weigh collateral damage against military advantage, or to cancel an attack when circumstances change, the foundational assumptions of IHL are thrown into question.
At the center of these debates is the “accountability gap,” a critical deficiency in which existing legal frameworks fail to attribute responsibility for infractions committed by autonomous systems. IHL fundamentally depends on human judgment to guarantee essential principles, such as proportionality, thereby facilitating accountability via individual or State liability. LAWS, however, bring layers of complexity with their inscrutable algorithms and unpredictable adaptations, making it highly problematic to prove intent or negligence among developers, military leaders, or deployers. In addition to depriving victims of justice, this absence also undermines the preventive power of law, allowing foreseeable injuries that are not explicitly intended to go unpunished.
The Background of AI-Driven Autonomous Weapons Systems
Autonomy is usually divided into three stages. Semi-autonomous systems are at one end of the spectrum. In these systems, operators retain control over critical actions like firing, while AI handles navigation or target suggestions. For example, in drone operations, strikes must be authorized by operators. In supervised autonomy, operators can intervene or override the weapon at any time, while fully autonomous systems use pre-programmed rules and AI to adapt to changing situations and can find and engage targets on their own.
However, not all systems fit into such neat categories. Loitering munitions like the Israeli Harop and Turkish Kargu-2 remain airborne above designated locations for extended periods, using AI to detect and engage targets once predefined conditions are met, blurring the line between supervised (or semi-autonomous) operation and full autonomy. Most of these technologies have already been used in conflicts, as drones been deployed in Pakistan, Azerbaijan, or in Ukraine. Drone swarms will go even further as fleets are programmed to self-coordinate, overwhelming defenses and dynamically adjusting formations without direct human input.
Applicable International Legal Framework
On the surface, AWS offer distinct military advantages like reducing troop exposure, enabling rapid, precise data processing, and potentially lowering collateral damage. However, these potential advantages are accompanied by serious ethical and legal challenges. For example, AI’s built-in unpredictability becomes especially problematic in the chaos of urban warfare as it may not adapt quickly to changing situations, and may mistake civilians or civilian objects for enemies, leading to unintended IHL violations and escalation. Algorithmic biases in training data can exacerbate discrimination, and machine learning’s “black box” nature obscures decision-making processes and complicates post-incident evaluations.
As with any other weapon system, compliance with IHL is the essential yardstick for AWS. Existing IHL rules were, however, developed for human operators, meaning the absence of direct human judgment pushes AWS operators to their limits, making genuine compliance with principles of distinction, proportionality, and precaution extraordinarily difficult. There is little doubt that AWS will fundamentally transform warfare, but its development and deployment must face rigorous scrutiny to prevent technological progress from eroding long-established legal and ethical obligations.
These strains appear most clearly in the three core principles. The principle of distinction forbids attacks that cannot reliably distinguish military objectives from civilians and civilian objects, which, for AWS, requires a degree of consistent, real-time discrimination that today’s technology still struggles to achieve.
Proportionality raises similar difficulties. It requires that the expected civilian harm not be excessive relative to the military advantage the attack is intended to deliver, a judgment that calls for weighing real-world context, human lives, and battlefield necessity against one another. In this respect, AWS pose a problem because AI relies on preset metrics instead of the complex context-aware reasoning that humans use to assess excessive harm in real time.
Precaution requires constant effort to lower risks in order to protect civilians, for instance, by verifying targets and choosing less harmful means whenever feasible. AWS might need programming safeguards or human intervention, but fully autonomous systems may never be able to respond as well as a person to new information, such as an unexpected civilian presence. Ultimately, these difficulties stem from a deeper issue: IHL requires countries to ensure that AWS follow these principles; however, the law’s human-centric assumptions highlight potential shortcomings in addressing machine autonomy, as they are reliant on how people understand them.
Although IHL contains no express ban on autonomy, one may argue that its core obligations implicitly require “meaningful human control” (MHC), particularly when invoking the Martens Clause to protect civilians and combatants in cases not covered by specific rules. Supported by the ICRC and other human rights organizations, MHC asserts that humans must maintain oversight over crucial functions, such as target selection and engagement, to ensure adherence to IHL principles. The Clause provides a safeguard by requiring that weapons and warfare practices comply with the principles of humanity and with the dictates of public conscience, even absent specific legal prohibitions.
For AWS, the Martens Clause implies that systems lacking MHC or risking indiscriminate harm can be deemed unlawful if they violate humane standards. The Clause provides a moral and legal benchmark that underscores the need for AWS to align with ethical norms and reinforces IHL’s adaptability to emerging technologies. Discussions at the Convention on Certain Conventional Weapons (CCW) since 2013 have highlighted these challenges, and the UN’s Group of Governmental Experts (GGE) has been deliberating the adequacy of existing IHL and whether it needs to be improved. Certain countries, including the United States, assert that existing regulations are sufficient, provided that AWS are developed with IHL considerations in accordance with their mandates for legal assessments of new weaponry. Meanwhile, various stakeholders, particularly from the Global South, advocate for more explicit norms, as they are aware that proliferation may intensify disparities in warfare.
Obstacles to Accountability
A core concern with autonomous weapons is the accountability gap. When a machine selects and attacks a target independently, current legal rules struggle to hold any specific person responsible for the resulting harm. The problem runs deeper than procedure, as IHL and International Criminal Law were built around human decisions. Thus, when a critical choice is made by AI-enabled system instead of a person, it becomes extraordinarily difficult to attribute violations to commanders, operators, or programmers. For example, who is accountable if an AWS erroneously targets people owing to a defective pattern recognition algorithm? The programmers and software engineers who developed the AI? If one is involved, the company employing them? The commander who deployed the AWS? Or the State that sanctioned its utilization? As the war in Gaza demonstrates, relying on AI-enabled targeting is in itself highly problematic, let alone when both targeting and execution depend on it. When accuracy is unreliable and human supervision is minimal, the result is large-scale civilian deaths.
As delineated by the Rome Statute of the International Criminal Court (ICC), criminal culpability under ICL requires the demonstration of mens rea and actus reus, which become ambiguous in the context of AWS. Programmers and software engineers might argue that they could not anticipate certain combat failures, especially in “black box” AI systems employing deep learning, whose internal mechanisms are not fully comprehensible to their developers. Commanders may well cite the idea of command responsibility; however, this presupposes effective control over subordinates, which is scarcely applicable to independently operating autonomous weapons. Indeed, as command responsibility was designed for hierarchical human structures, it fails when applied to AWS, potentially resulting in a lack of accountability for foreseen but inadvertent infractions.
State responsibility offers another avenue for accountability, but it is far from straightforward. When autonomous systems act unpredictably without specific human direction, it becomes difficult to pin those actions to a State under the Articles on State Responsibility. Civil liability runs into similar problems. Victims seeking compensation through war-tort claims or human rights bodies often face State immunity barriers. Even when cases proceed, it can be nearly impossible to access the evidence needed, especially when the system’s architecture and training data are classified.
Since the dispersal of responsibility among humans and AI generates a void that may promote imprudent use, it is easy to see that ultimately these problems erode the efficacy of IHL. Addressing this necessitates re-evaluating accountability beyond human-centered approaches, which can be done through hybrid frameworks that require traceability and collective liability. Additionally, implementing mandatory pre-deployment testing and certification for AWS should be considered to ensure systems meet IHL standards, and reduce the risk of violations.
Potential Mechanisms for Ensuring Accountability
Addressing the accountability gap in autonomous weapons calls for fixes as creative as the technology itself. Current international legislation must be modified to include new safeguards. A fundamental proposal is the necessity for MHC, using design standards that require human veto points or establish clear operational limitations, thereby prohibiting entirely autonomous decisions. Design standards should apply to all operational settings to ensure consistent safeguards against fully autonomous decisions, but they could incorporate risk-based levels of certainty, mandating higher confidence thresholds for AI outputs in complex or high-stakes environments. This hybrid approach, reflected in recent UN resolutions, would enhance MHC’s effectiveness without leaving gaps in lower-risk scenarios while maintaining human agency as a safeguard against algorithmic inaccuracies. However, the Gaza war already pointed to MHC’s serious limitations. Israeli Defense Force operators were reportedly approving AI-generated targets in an average of 20 seconds with minimal scrutiny, leading to a 10% error rate and thousands of civilian deaths. This underscores that while MHC may very well be effective in smaller operations, in large-scale and/or drawn-out conflicts where the speed and volume of AI decisions overwhelm operators, it is unlikely that rushed oversight will prevent IHL violations.
Against this background, IHL’s command responsibility framework provides a key accountability mechanism for AWS. It holds commanders liable if they knew or should have known that a system under their control was likely to commit violations such as misidentifying civilians or breaching proportionality, but failed to take reasonable steps to prevent those violations. Under the framework, deploying an AWS with foreseeable and serious defects in its targeting algorithms can therefore trigger war crime liability. This approach pushes commanders to scrutinize system reliability, monitor performance, and step in when risks emerge, helping close the accountability gap without requiring new treaties and keeping IHL’s core principles intact in an era of algorithmic warfare.
Yet another key step would be to strengthen traceability and explainability in AI design. Mandating “black box” algorithms to have logging capabilities or interpretable models would enable countries to support post-incident inquiries, in turn permitting courts to evaluate whether harms were foreseeable and attributable. Procedural accountability frameworks may also be of use. Responsibility is distributed among programmers, manufacturers, and commanders according to risk allocation, similar to product liability in domestic law but tailored for armed conflict. This could extend to State-level remedies such as expanding the jurisdiction of international tribunals like the ICC to handle AWS-related war crimes through doctrines of superior responsibility, or enabling victims’ claims via war torts in national courts.
Drawing on the CCW deliberations, two practical steps stand out: mandatory legal reviews for AWS development; and the establishment of an international body to monitor compliance. If implemented, these measures would clarify who is responsible when things go wrong and would push all States towards the same minimum standards. However, during the rapidly unfolding AI arms race, obstacles persist in reaching consensus, chiefly among these obstacles are major military powers (e.g., the United States, China, and Russia) resisting binding restrictions that risk constraining their technological edge. Yet even as major powers resist binding global rules, they can still lead by example. Robust national policies like the U.S. Department of Defense’s ethical AI directives can set a high bar and nudge others to encourage wider adoption. Taken together, these measures would strengthen IHL and would keep legal safeguards from falling behind the technology they are meant to govern.
Conclusion
The rapid rise of AI-driven autonomous weapons is putting real pressure on the core IHL principles of distinction, proportionality, and precaution. Additionally, accountability is still hard to pin down. Our existing criminal liability systems were never designed to deal with actions carried out by machines. Further, the opacity of modern AI makes it even harder to trace who is responsible for errors, and thus secure justice for victims. These gaps undermine both deterrence and enforcement, revealing how the Geneva Conventions and the Rome Statute fall short when applied to systems that make targeting decisions on their own. As AWS shift from hypothetical to operational, the call for binding international rules to stop the unchecked automation of lethal force and to preserve meaningful human judgment in life-and-death decisions has become impossible to ignore.
To uphold the rule of law, States and international bodies need to insist on three concrete measures: (1) meaningful human oversight at critical moments; (2) traceable and transparent AI-enabled decisions; and (3) clear rules that hold everyone in the chain—developers, commanders, and operators—accountable when things go wrong. Over 120 countries have endorsed a new international treaty on AWS, and the UN Secretary-General António Guterres has reiterated calls for a global ban. Policymakers should advance discussions under the UN’s Convention on Certain Conventional Weapons towards a binding instrument that prohibits completely autonomous weapons.
In reality, however, the major powers’ opposition to AWS regulation renders the likelihood of agreeing on such an instrument slim to none. Indeed, short of a fundamental shift in the strategic calculus of the UN Security Council’s permanent members, the GGE is highly unlikely to produce a legally binding protocol by its 2026 deadline. Meanwhile, with each passing year of inaction, AWS move closer to routine deployment, relentlessly gutting the very possibility of accountability and dragging the rule of law in armed conflict toward a precipice from which it may never fully recover.
***
Dr Gerald Mako is a Research Affiliate at the Cambridge Central Asia Forum at Cambridge University.
The views expressed are those of the author, and do not necessarily reflect the official position of the United States Military Academy, Department of the Army, or Department of Defense.
Articles of War is a forum for professionals to share opinions and cultivate ideas. Articles of War does not screen articles to fit a particular editorial agenda, nor endorse or advocate material that is published. Authorship does not indicate affiliation with Articles of War, the Lieber Institute, or the United States Military Academy West Point.
Photo credit: U.S. Air Force
