Hacker-themed Guernica, Paris Hotel, DEF CON, Las Vegas, Nevada. Photo used under Creative Commons 2.0 license, and may be distributed under that same license. This photo has been cropped.

Hacker-themed Guernica, Paris Hotel, DEF CON, Las Vegas, Nevada. Photo used under Creative Commons 2.0 license, and may be distributed under that same license. This photo has been cropped. Cory Doctorow

Hackers to Military: Replace Us With Robots? Ha!

Next year’s Cyber Grand Challenge event will pit humans against machines in a grand hacking war. DEF CON’s war gamers like their chances.

LAS VEGAS, Nev.— Every year, thousands of information-security specialists, computer scientists, and a few mohawked geeks who proudly wear the moniker of hacker gather here for a very particular digital war game:, the DEF CON capture- the-flag, or CTF, competition. To win, you have to find weaknesses in other teams’ defenses, steal their data flags, and protect your own.

But next year, it won’t just be humans squaring off. In addition to the regular DEF CON CTF event, the 2016 meeting will pit seven teams’ robotic hackers against each other in an AI capture-the-flag contest. Then humans will take on the robots.

The robot-vs.-robot battle is part of the Defense Advanced Research Projects Agency’s Cyber Grand Challenge series of competitions. (DARPA is not involved with the robots-vs.-humans competition, although some teams may participate in both, agency spokesman Jared Adams said.)

The arrival of an AI system that can outflank humans in breaching security and protecting data in a dynamic game environment would be a force multiplier for defensive cyber security and even offensive cyber warfare. But will war in a machine environment necessarily favor the machines? Not according to many of the hackers at this year’s DEF CON. Everyone who talked to Defense One about next year’s competition were confident that it would be years before a robot team would beat human hackers at their own game. 

Cyber Grand Challenge program manager Michael Walker laid out why it’s a better test for artificial intelligence than many other game scenarios, like chess or checkers. “You have to do binary reverse engineering the entire time,” he said, referring to the practice of dissecting and reconstructing program files. “The only way to figure out how the software works is to reverse…and do it as fast as you can while your opponents are trying to the same over you,” Walker said. “To even explore the state space, I have to be able to synthesize logic.” 

Robot hackers also have to be able to exhibit some very humanistic behaviors — skepticism, creativity, and even the ability to bluff — gray areas that get machines into trouble in games that aren’t perfectly straight forward. It’s one reason why computers that can dominate at chess get into trouble when the game requires what might be called instinct, like poker. “If machines can’t win at go, can’t win at poker, do they have a chance at all? That’s exactly what we’re talking about,” Walker said. (For more on the Cyber Grand Challenge, check out his Reddit AMA session from last year, or his appearance on “60 Minutes”.)

But if one of the seven robot teams wins, will it  signal the end of the era of human hacking in the same way that the self-driving cars foretell the end of human driving? Well, not quite. The Cyber Grand Challenge won’t be the free- for-all that is the regular CTF. It will take place within DARPA’s DECREE operating system, released as open source last year.  DECREE has seven system call types, or syscalls, ways a user can talk to the operating system’s input/ output manager. In the context of information security, syscalls are tools you can use for attack. Because the DARPA CTF will be limited to seven syscalls, it will be a rather more tame version of the regular DEF CON CTF, in which teams working in an X86 environment might use 200 syscalls.

This all means is that the contest will be more of a boxing match and less of a street brawl. 

So do the hackers think a robot is going to beat them? “Absolutely not,” said one, who declined to be named but is a self-described hacker who was providing technical support to the DEF CON CTF this year. “There are classes of challenges that will always be outside of the capabilities  of machines,” he said. “CGC is primarily focused on memory corruption vulnerabilities. That doesn’t include classes of bugs that are logic errors which are ridiculously difficult to detect autonomously. Like, how do you tell if something is intentional behavior, a back door, or a programming mistake?”

Ryan Grandgenett, an information assurance researcher at the University of Nebraska, agreed that humans would probably beat out machines for the foreseeable future. “I know that Google has made some pretty big advancements in chatbots that look like humans, but I don’t know about something this complex,” he said. 

Added Cmdr. Michael Bilzor, an instructor at the United States Naval Academy, “Finding exploits is so much an art form right now. Particularly because the large space of operating systems.”

Not everyone was quite so pessimistic about the machine teams’ chances. One observer, who asked to be identified only as someone who had worked in a security operations center for a large university, said that he was impressed by the DARPA talk, and estimated that a machine would beat a human at seven to ten years from now. “If capture-the-flag is a number of flags in a time limit, a computer is going to have an advantage,” he said. 

And Bilzor said the terms of the fight mean that it’s no real contest at all. After all, in an actual battle setting, no hacker would limit the types of strikes or holds (syscalls) that they could use. “The only way to get the automated systems to play is to constrain the problem, which they’ve done.” he said. “If you’re talking about full spectrum vulnerability identification and exploit generation on any architecture, using any operation base and any syscall set? You’re probably talking at least a decade, in my opinion,” he said. 

All trash talk aside, the DEF CON attendees were broadly appreciative of the DARPA effort and all the new open-source tools, like DECREE, that the agency has released for it. Overall, it’s already been a PR win for the agency, unlike the recent Robotics Grand Challenge event, which produced, primarily, laugh reels of robots falling down.  

The hackers just don’t think you can automate exploit fencing in a way that will threaten their livelihoods any time soon. Hear that, robots? The gauntlet has been thrown. 

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.