Russia has developed an unmanned vehicle armed with a machine gun and a grenade launcher to defend missile launchers.

Russia has developed an unmanned vehicle armed with a machine gun and a grenade launcher to defend missile launchers. Russian Ministry of Defense

We’re Losing Our Chance to Regulate Killer Robots

After four years of nearly fruitless debate, nations are gathering once more to talk AI and autonomous weapons.

Scores of countries are gathering at the United Nations this week to discuss lethal autonomous weapon systems – essentially, robots that would pick their own targets. This marks their fourth year of debate with little to show for it; the group does not even have a shared working definition of “autonomous weapon.” Meanwhile, the technology of autonomy and artificial intelligence is racing forward.

When the countries last met, in April 2016, DeepMind’s AlphaGo had just beaten world champion Go player Lee Sedol in a head-to-head match — an unprecedented feat for a computer. But just a few weeks ago, DeepMind published a paper on its new AlphaGo Zero, which taught itself the game without human-supplied training data and, after a mere three days of self-play, defeated the older program in 100 straight games. Between those two events, the world’s countries held no substantive meetings on autonomous weapons ­— excepting only last year’s decision to bump the discussions up one rank in the diplomatic hierarchy.

A consortium of over 60 non-governmental organizations has called for an urgent ban on the development, production, and use of fully autonomous weapons, seeking to halt such work before it begins in earnest. Yet at this stage a legally binding treaty is almost inconceivable. The UN forum that the nations are using, the awkwardly named Convention on Certain Conventional Weapons, operates by consensus — meaning that although 19 nations have said they would back a ban, any one of the other 105 can veto it. Even advocates of a ban agree that the diplomatic process is “faltering financially, losing focus [and] lacks a goal.”

Four years ago, the first diplomatic discussions on autonomous weapons seemed more promising, with a sense that countries were ahead of the curve. Today, even as the public grows increasingly aware of the issues, and as self-driving cars pop up frequently in the daily news, energy for a ban seems to be waning. Notably, one recent open letter by AI and robotics company founders did not call for a ban. Rather, it simply asked the UN to “protect us from all these dangers.” Even as more people become aware of the problem, what to do about it seems less and less clear.

There are many reasons why a ban seems unlikely. The technology that would enable autonomous weapons is already ubiquitous. A reasonably competent programmer could build a DIY killer robot in their garage. Militaries are likely to see autonomy as highly useful, as it will give them the ability to operate machines with faster-than-human reaction times and in environments that lack communications, such as undersea. The risk to innocent civilians is unclear – it is certainly possible to envision circumstances in which self-targeting weapons would be better at some tasks than people. And the most difficult problem of all is that autonomy advances incrementally, meaning there may be no clear bright line between the weapon systems of today and the fully autonomous weapons of the future.

So if not a ban, then what?

There has been some halting diplomatic progress over the past few years in exploring the role of humans in using lethal force in war. This idea is expressed in different ways, with some calling for “meaningful human control” and others suggesting “appropriate human judgment” is necessary. Nevertheless, there is growing interest in better understanding whether there is some irreducible place for humans in lethal force decisions. The 2016 diplomatic meetings concluded with states agreeing to continue to discuss “appropriate human involvement with regard to lethal force,” a compromise term. None of these terms are defined, but they express a general sentiment toward keeping humans involved in lethal force decisions on the battlefield. There are good reasons to do so.

Machine intelligence today can perform well at tasks like object recognition, but lack the ability to understand context. Machines can identify the objects in a scene, but not knit them together into a coherent story. They can tell whether a person is holding a gun, but not whether the person is likely to be an insurgent or a farmer defending his property, which could depend on other surrounding clues. These limitations point to the value of human cognition, perhaps used in concert with machine intelligence. 

Over time, as artificial intelligence advances, machines may be able to overcome these and other limitations. Asking where human judgment is needed in war gets to more fundamental questions, though: Even if we had all of the technology we could imagine, what decisions would we want humans to make in war, and why? Are there some tasks in war that humans ought to do, not because machines cannot do them, but because machines should not? Some decisions in war have no clear right answer, such as weighing how many civilian deaths are acceptable in accomplishing some military objective. It may be preferable to know that humans have made these decisions, which involve weighing the value of human life, even if someday machines may be capable of doing so. Similarly, human moral responsibility for killing can be a check on the worst horrors of war. What would the consequences be if no one felt responsible for the killing that occurred in war?

Technology is rapidly bringing us to a fundamental threshold in humanity’s relationship with war. The institutions that humanity has for dealing with this challenge – slow, cumbersome diplomatic processes like the Convention on Certain Conventional Weapons – are imperfect. Nevertheless, the CCW is the best forum that exists for nations to come together to address this issue. If countries are going to get ahead of the technology, they’ll need to move faster.

NEXT STORY: Taking Putin's Word For It

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.