A Predator B unmanned aircraft lands after a mission at the Naval Air Station, Tuesday, Nov. 8, 2011, in Corpus Christi, Texas.

A Predator B unmanned aircraft lands after a mission at the Naval Air Station, Tuesday, Nov. 8, 2011, in Corpus Christi, Texas. Eric Gay

Pentagon Seeks a List of Ethical Principles for Using AI in War

An advisory board is drafting guidelines that may help shape worldwide norms for military artificial intelligence — and woo Silicon Valley to defense work.

U.S. defense officials have asked the Defense Innovation Board for a set of ethical principles in the use of artificial intelligence in warfare. The principles are intended to guide a military whose interest in AI is accelerating — witness the new Joint Artificial Intelligence Center — and to reassure potential partners in Silicon Valley about how their AI products will be used.

Today, the primary document laying out what the military can and can’t do with AI is a 2012 doctrine that says a human being must have veto power over any action an autonomous system might take in combat. It’s brief, just four pages, and doesn’t touch on any of the uses of AI for decision support, predictive analytics, etc. where players like Google, Microsoft, Amazon, and others are making fast strides in commercial environments.

“AI scientists have expressed concern about how DoD intends to use artificial intelligence. While the DoD has a policy on the role of autonomy in weapons, it currently lacks a broader policy on how it will use artificial intelligence across the broad range of military missions,” said Paul Scharre, the author of Army of None: Autonomous Weapons and the Future of War.

Related: Here’s What the White House’s AICommittee Will Focus On

Related: The Pentagon Is Building an AI Product Factory

Related: Here’s How the Russian Military Is Organizing to Develop AI

Josh Marcuse, executive director of the Defense Innovation Board, said crafting these principles will help the department “safely and responsibly” employ new technologies. “I think it's important when dealing with a field that’s emergent to think through all the ramifications,” he said.

The Board, a group of Silicon Valley corporate and thought leaders chaired by former Google and Alphabet chairman Eric Schmidt, will make the list public at its June meeting. Defense Department leaders will take them under consideration.

Marcuse believes that the Pentagon can be a leader not just in employing AI but in establishing guidelines for safe use — just as the military pioneered safety standards for aviation. “The Department of Defense should lead in this area as we have with other technologies in the past. I want to make sure the department is not just leading in developing AI for military purposes but also in developing ethics to use AI in military purposes,” he says.

The effort, in part, is a response to what happened with the military’s Project Maven, the Pentagon’s flagship AI project with Google as its partner. The effort applied artificial intelligence to the vast store of video and and image footage that the Defense Department gathers to guide airstrikes. Defense officials emphasized repeatedly that the AI was intended only to cut down the workload of human analysts. But they also acknowledged that the ultimate goal was to help the military do what it does better, which sometimes means finding and killing humans. An employee revolt ensued at Google. Employees resigned en masse and the company said that they wouldn’t renew the contract.

Scharre, who leads the Technology and National Security Program at the Center for a New American Security, said, "One of the challenges for things like Project Maven, which uses AI technology to process drone video feeds, is that some scientists expressed concern about where the technology may be heading. A public set of AI principles will help clarify DoD's intentions regarding artificial intelligence.”

The Maven episode represents a rare role reversal for a contractor and the Pentagon, with the Defense Department being more open—or at least consistent—in their messaging than the contractor they were paying.  Defense officials hope that having a list of principles will allow companies that want to work with the Pentagon do so without having give mixed messages to public and their employees.

We have to have questions about how these tools are used appropriately, how data is used, how data is labeled, how to ensure the appropriate degrees of autonomy.
Josh Marcuse, executive director of the Defense Innovation Board

Marcuse says a published list of ethical guidelines for the Department to follow will allay some of the suspicions many in Silicon Valley have about the military’s use of AI. “If we show leadership, responsibility, show that we’re circumspect and cautious where we need to be, and rigorous in our testing and making smart tradeoff decisions, I think that will address a lot of the concerns that partners and potential partners have raised,” he said.

As social media companies and large tech players like Amazon have developed and used new AI offerings, and offered them for sale to law enforcement agencies, the military, or just to maximize their own profits, ethicists, academics, and observers have raised serious questions about how big corporate players are developing what many consider to be the most important technology of the 21st century.

A common concern is data bias, or the occurrence of algorithmic decisions informed by large—but not necessarily accurate—data sets. Consider a 2015 controversy that erupted when it was revealed that Google’s search algorithm was mislabeling black people as gorillas. The private sector hasn’t worked out the answers to these questions even as it’s rushed to commercialize AI. The U.S. military, says Marcuse, doesn’t have the luxury of deploying new tech solutions and rushing them into the field without a good understanding of how they work. A massive data-labeling error for the military isn’t just a public relations disaster, it’s a potential killer, especially combined with highly autonomous robotic weapons. The concerns that ethicists and academics have raised are good ones, he says.

“We have to have questions about how these tools are used appropriately, how data is used, how data is labeled, how to ensure the appropriate degrees of autonomy,” he says.

The form that the recommendations take won’t be a strict list of commandments so much a range of recommendations informed by scenario-based methodology: what’s the most appropriate use of AI in one battle situation versus another? Those scenarios are informed by interviews with actual commanders, said Marcuse. “We actually ask commanders what their questions are...The scenarios are designed to test those questions.”

Of course, Defense Department leaders will decide on their own, and behind closed doors, how to put the principles into practice. Marcuse says it’s important for them to be able to do that away from the public eye. But, he says, the demand signal for the principles list came from within the department — in particular, CIO Dana Deasy. And It had the “full and enthusiastic” support of former Defense Secretary James Mattis, said Marcuse.

And they aren’t intended for a single user or audience within the military. Once they’re published, the hope is that combatant commanders, service chiefs, all sorts of different players will find them useful in drafting future strategies, concepts of operation, training guides, etc. “If you are the marine corps or army, and you are trying to come up with training manuals and conops, there is a lot of planning, thinking and training. We think the principles would be informing those things,” he says

The process kicks off on Jan. 23 with a public listening session at the Harvard Belfer Center, an effort to court exactly the sort of academics and ethicists that have expressed the most skepticism of the Defense Department’s AI ambitions.

“It is critical that the forthcoming [Defense Innovation Board] recommendation to DoD is consistent with an ethics-first approach and upholds existing legal norms around warfare and human rights, while continuing to carry out the Department's enduring mission to keep the peace and deter war. DIB members will collect and review all comments for consideration as they draft these AI principles in the coming months,” according to an announcement.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.