Submission to the United Nations Secretary-General on Autonomous Weapons Systems

Human Rights Watch appreciates the opportunity to submit its views and recommendations for consideration by the United Nations secretary-general in response to Resolution 78/241 on “Lethal autonomous weapons systems” adopted by the UN General Assembly on 22 December 2023. This historic resolution asks the UN secretary-general to seek the views of countries and other stakeholders on “ways to address the challenges and concerns raised” by such weapons systems “from humanitarian, legal, security, technological and ethical perspectives.”

This submission briefly summarizes our work on this issue, outlines specific challenges and concerns raised by autonomous weapons systems, and elaborates on ways to address these challenges and concerns through a legally binding instrument.

I. Background

Human Rights Watch is an international nongovernmental organization (NGO) in consultative status with the United Nations Economic and Social Council. We conduct research and advocacy to uphold human dignity and promote human rights and international human rights law across the globe. For more than 30 years, Human Rights Watch has documented and advocated for the prevention of civilian harm and human suffering caused by a range of arms, including landmines, cluster munitions, incendiary weapons, chemical weapons, and explosive weapons used in populated areas. We work to advance humanitarian disarmament, an approach that aims to prevent and remediate arms-inflicted human suffering and environmental harm through the establishment and implementation of norms.

In October 2012, Human Rights Watch co-founded the Campaign to Stop Killer Robots with six other NGOs working in the field of humanitarian disarmament.[1] The coalition, which is now comprised of more than 270 NGOs in 70 countries, advocates for the negotiation and adoption of an international treaty to prohibit and restrict autonomous weapons systems. On behalf of Human Rights Watch, Mary Wareham served as founding coordinator of the campaign from 2012 to 2021.

II. Challenges and Concerns Raised by Autonomous Weapons Systems

This submission is based on and informed by our years of research and advocacy on this issue. Since 2012, Human Rights Watch has published more than two dozen reports on autonomous weapons systems, most in conjunction with the International Human Rights Clinic (IHRC) at Harvard Law School. This research has explored the numerous serious ethical, moral, legal, accountability, and security challenges and concerns raised by weapons systems that select and engage targets based on sensor processing rather than human inputs.[2]

In November 2012, Human Rights Watch and IHRC released “Losing Humanity: The Case against Killer Robots,” the first major civil society report to examine the dangers of removing human control from the use of force.[3] This report – and later ones – found that allowing machines to select and attack targets without further human intervention would be incompatible with fundamental provisions of international humanitarian law, including the principles of distinction and proportionality.[4] Autonomous weapons systems would find it difficult to distinguish between combatants and civilians, particularly when the former commingle with the latter, because they could not interpret subtle cues. In addition, they would lack the human judgment to determine whether the expected civilian harm was excessive in relation to the anticipated military advantage in rapidly changing case-by-case situations.  

Human Rights Watch and IHRC have detailed the significant hurdles to assigning personal accountability to the actions undertaken by autonomous weapons systems under both criminal and civil law.[5] Accountability is essential to deter future unlawful acts, punish past ones, and recognize victims’ suffering. In both armed conflict and law enforcement operations, there is an accountability gap for the harm caused by autonomous weapons systems. It is legally challenging and arguably unfair to hold human operators criminally responsible for the actions of autonomous weapons systems if they could not predict or control those actions. There are a range of further obstacles to holding weapons manufacturers liable under civil law.

Autonomous weapons systems would also contravene basic principles of humanity and the dictates of public conscience established by the Martens Clause under international humanitarian law.[6] The principles of humanity require humanity (including compassion) and respect for human life and dignity, neither of which autonomous weapons can express because they are inanimate objects. In addition, removing human control from the use of force crosses a moral red line for many and thus runs counter to their public conscience. Thousands of scientists and artificial intelligence experts, more than 24 Nobel Peace Laureates, and more than 160 religious leaders and organizations of various denominations have demanded a ban on autonomous weapons systems.

Autonomous weapons systems raise serious concerns under international human rights law because they are likely to be used in law enforcement operations as well as situations of armed conflict. Human Rights Watch welcomed the first UN report on autonomous weapons systems presented to the Human Rights Council in May 2013 by the UN Special Rapporteur on extrajudicial, summary, or arbitrary executions Professor Christof Heyns of South Africa.[7] The seminal report recommended an immediate moratorium on what Heyns called “lethal autonomous robotics,” weapons systems that would select and engage targets without further human intervention.[8]

Research by Human Rights Watch and IHRC has found that autonomous weapons systems raise concerns under the foundational rights to life and to remedy and the principle of dignity.[9] Under the right not to be arbitrarily deprived of life in international human rights law, force may only be applied if it is necessary, a last resort, and proportionate. Weapons that operate without meaningful human control face challenges complying with all three parts of that test. A machine could find it difficult to determine if it was necessary to use force because as an inanimate object, it could not read subtle cues in people to determine whether they are true threats. The machines would lack human judgment to weigh the proportionality of an attack. While a human law enforcement officer may be able to avoid force by negotiating with a human who was perceived as a threat and defusing a situation, an autonomous weapon system would be unable to do this, and humans would be less likely to surrender to a machine. Autonomous weapons systems raise additional concerns, discussed above, about an accountability gap, which present challenges for the right to remedy.

Autonomous weapons systems would undermine the principle of dignity, a legal and moral concept, which implies that everyone has a worth deserving of respect. As inanimate objects, machines cannot comprehend or understand the value of human life or the significance of its loss. Allowing them to make life-and-death determinations thus strips people who are being targeted of their human dignity. In the process of determining whom to kill, autonomous weapons systems boil human targets down to data points. As Heyns noted: “To allow machines to determine when and where to use force against humans is to reduce those humans to objects…. They become zeros and ones in the digital scopes of weapons which are programmed in advance to release force without the ability to consider whether there is no other way out, without a sufficient level of deliberate human choice about the matter.”[10] Recent efforts to hardcode a threshold for civilian harm into automated systems have been inadequate and provide no constraint.

Security concerns noted by Human Rights Watch and IHRC include the risk of an arms race, the threat of autonomous weapons systems reaching the hands of states or non-state actors with no regard for international law, and a lowering of the threshold to war.[11] Because autonomous weapons systems would have the power to make complex determinations in less structured environments, their speed could lead armed conflicts to spiral rapidly out of control. Their use could foster crisis instability and conflict escalation.[12]

III. Ways to Address the Challenges and Concerns Raised by Autonomous Weapons Systems

A Legally Binding Instrument

As a member of the Stop Killer Robots campaign, Human Rights Watch endorses its call for the urgent negotiation and adoption of a legally binding instrument to prohibit and regulate autonomous weapons systems.[13] Autonomous weapons systems are a grave problem that can affect any country in the world so clear, strong, and global rules are essential. Those rules should be legally binding to promote compliance among states that join the treaty. Experience shows that a legally binding instrument can also influence states not party, and even non-state armed groups through norm-building and stigmatization of the most problematic weapons.

Only new international law will suffice to deal with the dangers raised by autonomous weapons systems.[14] Measures such as a voluntary code of conduct would only pave the way for a future of automated killing. Voluntary commitments such as the 2023 US political declarations aimed at ensuring responsible use of weapons systems that incorporate Artificial Intelligence (AI) capabilities are completely insufficient and provide no restraint.

As Human Rights Watch and IHRC have reported, relevant precedent for a legally binding instrument can be found in Protocol IV to the Convention on Conventional Weapons, which preemptively bans blinding laser weapons.[15] Indeed, threats to the principles of humanity and dictates of public conscience, as well as notions of abhorrence and social unacceptability, helped drive countries to ban blinding lasers through the protocol adopted in 1995. While blinding lasers are a narrower class of weapons than autonomous weapons systems, the parallels show that drawing the line on problematic emerging technologies through prohibitions is feasible and effective.

A legally binding instrument should be accompanied by national legislation and other measures to implement and enforce the treaty’s provisions at the domestic level. We agree with the International Committee of the Red Cross (ICRC) recommendation that the instrument “should require States Parties to take all appropriate legal, administrative and other measures, including the imposition of penal sanctions, to prevent or suppress any activity prohibited to States Parties under the instrument undertaken by persons or on territory under their jurisdiction or control.”[16]

Essential Treaty Elements

Human Rights Watch, IHRC, and others have outlined the essential elements for an international treaty on autonomous weapons systems, following precent provided in previous disarmament treaties, international human rights instruments, and international humanitarian law, which all offer models for the proposed provisions.[17]

A legally binding instrument should apply to all weapons systems that select and engage targets based on sensor processing, rather than human inputs. While the treaty’s restrictions will focus on a narrower group of systems, this broad scope will help future-proof the treaty and ensure that no systems escape review. The new treaty should include: 1) a general obligation to maintain meaningful human control over the use of force; 2) prohibit weapons systems that autonomously select and engage targets and by their nature pose fundamental moral and legal problems; and 3) include specific positive obligations that aim to ensure that meaningful human control is maintained in the use of all other systems that select and engage targets.

The concept of meaningful human control is fundamental to such an instrument because most of the concerns arising from autonomous weapons systems are attributable to the lack of such human control.[18] The concept of meaningful human control should comprise a combination of components, such as, but not necessarily limited to: 1) Decision-making components, for example, the ability to understand how the system works; 2) Technological components, including predictability and reliability, and 3) Operational components, notably restrictions on time and space in which the system operates.

A new treaty should prohibit the development, production, and use of systems that inherently lack meaningful human control over the use of force given the links between loss of control and the challenges and concerns discussed above. The treaty should also prohibit the development, production, and use of autonomous weapons systems that target people in order to prevent the use of weapons systems that strip people of their dignity, dehumanize the use of force, or lead to discrimination. It should cover weapons that always rely on data, like weight, heat, or sound, to select human targets. These prohibitions would help protect civilians and other non-combatants in armed conflict, and reduce infringements of human rights during law enforcement operations. They should apply “under any circumstances” to ensure that the provisions cover times of peace and war.

The treaty should also include regulations (positive obligations) to ensure all other autonomous weapons systems are never used without meaningful human control. It should outline affirmative steps states parties would need to take to cover systems that are not inherently unacceptable but still have the potential to be used to select and engage targets without meaningful human control.

There are other types of positive obligations common to international humanitarian and human rights law that may be useful to include in a legally binding instrument on autonomous weapons systems. For example, reporting requirements would promote transparency and facilitate monitoring. Verification and compliance mechanisms could help prevent treaty violations. Regular meetings of states parties would provide opportunities to review the treaty’s status and operation, identify gaps in implementation, and set goals for the future.

The Way Forward

In terms of negotiating fora, the Convention on Conventional Weapons (CCW) has run its course after providing a forum for useful discussions and the development of support for a legally binding instrument over the years. After more than a decade, it is clear that negotiations of a new instrument in the CCW are impossible. It is time to step outside of that forum to one that can aim higher, move faster, and be more inclusive of countries that are not party to the CCW as well as of international organizations and civil society. Disarmament precedent shows that stand-alone and UN General Assembly-initiated processes are viable options in which committed, like-minded states, in partnership with other stakeholders, can produce strong treaties in 15 months or less.[19]

The world is approaching a tipping point on this topic as support for negotiating a legally binding instrument on autonomous weapons systems reaches unprecedented levels.[20] The Stop Killer Robots campaign’s Automated Decision Research project identifies more than 110 countries that have expressed their desire through national and group statements for a new international treaty on autonomous weapons systems.[21] Human Rights Watch supports the joint call issued on October 5 by UN Secretary-General António Guterres and ICRC President Mirjana Spoljaric for UN member states to negotiate a new international treaty by 2026 to ban lethal autonomous weapons systems.

Human Rights Watch affirms our strong commitment to work with urgency and with all interested stakeholders for an international legal instrument to ban and regulate autonomous weapons systems. We are grateful for the opportunity to share the above views and recommendations on ways to address this grave threat to humanity.