A research associate professor at the Naval Postgraduate School demonstrates the virtual sand table for urban warfare operations training rehearsals at Monterey, Calif., July 22, 2009.

A research associate professor at the Naval Postgraduate School demonstrates the virtual sand table for urban warfare operations training rehearsals at Monterey, Calif., July 22, 2009. U.S. Navy photo by Mass Communication Specialist 3rd Class John Fischer

Can the Military Make a Prediction Machine?

The planet is awash in open, free data. Can military-funded research turn it into a crystal ball?

What could the military do if it could better understand the massive amounts of data that humanity creates, an estimated 2.5 quintillion bytes every day? Could it predict aspects of the future?

If Pentagon funds can help create—even partially—a machine capable of understanding cause and effect, or causality, and do so on the scale of thousands of signals, data points, and possible conclusions, then, perhaps, big data will reach its real potential: a predictive tool that allows leaders to properly position soldiers, police forces, and humanitarian relief long before the action starts.

Among the military programs probing this new realm is Big Mechanism, run by the Defense Advanced Research Projects Agency, or DARPA. It seeks to turn machine-collected (or machine-generated) data into real insights into complex systems, and do so automatically.


Join Patrick Tucker online for a video discussion with DARPA'S Paul Cohen and IARPA's Jason Matheny at 11 a.m. EDT on Monday, April 13. Find out more below, or sign up here for the Defense One viewcast.


Some, such as Wired’s Chris Anderson, have suggested that access to huge amounts of data, which makes correlational analysis easier, has made old-fashioned, theory-based science obsolete. But in a recent conversation with Defense One, DARPA program manager Paul Cohen said he was looking more to mechanize the human capacity for causation, rather than innovate around it. “We’re a very much aiming toward a new science, but we’re very much interested in causal relationships,” he said. “What we’re finding is that mathematical modeling of systems is very hard to maintain.”

The supply of data, it turns out, is growing too quickly for the human race to use it effectively to solve big problems. The expanding reach and power of computational intelligence is both cause and, at least potentially, cure.

“Having big data about complicated economic, biological, neural and climate systems isn't the same as understanding the dense webs of causes and effects—what we call the big mechanisms—in these systems,” Cohen said last year. “Unfortunately, what we know about big mechanisms is contained in enormous, fragmentary and sometimes contradictory literatures and databases, so no single human can understand a really complicated system in its entirety. Computers must help us.”

These big systems can be as large as the entire world or as small as cancer cells, an initial area of focus for the program.

Machine intelligence can collect and process data on a scale unimaginable to regular humans. But processing data is very different from making sense of it, and from making predictions. If we could get computer systems to predict in the way that humans do, but with the data and processing power only available to massively interconnected systems, could we open up areas of the future to new inference? Cohen has suggested that the answer is yes.

“The beautiful thing about causal models is that they make predictions, so we can return to our big data and see whether we’re [retrospectively] right,” Cohen said. “And we can propose new experiments, suggest interventions and advance our knowledge more rapidly.” 

The ways in which humans interact with government, with one another, with medical facilities, transit systems and brands, etc. can predict events of national security significance. They can indicate, for instance, if a deadly disease outbreak is taking hold in a small rural community or if civil unrest is on the rise.

One example of that is the Open Source Indicators Program, launched in 2011 by from the Intelligence Advanced Research Programs agency. Led by program manager Jason Matheny, Open Source Indicators funds projects to predict events of national security relevance by monitoring tens of thousands of blogs, RSS feeds, news reports, social network chatter from sites like Twitter and Facebook, and other open sources.

Very early on, program participants began to generate some surprising results. In 2012, Virginia Tech computer scientist Naren Ramakrishnan, working solely with signals culled from the open Internet, effectively predicted both Mexico’s Yo Soy 132 protest movement, sometimes called the Mexican Arab Spring, and the “Friendship Bridge” protests that riled parts of Brazil and Paraguay.

Around the same time, Georgetown University data scientist Kalev Leetaru used a database of millions of open-source indicators to correctly (but retroactively) predict the spot in Abbottabad, Pakistan, where Osama Bin Laden was found.

But for every instance where big data correctly predicted a big national-security event, critics can point to a big miss. Last year, for example, such indicators failed to predict the Ebola outbreak.

The military and national security communities have only begun to explore the potential of big data to solve these kinds of enormously complex problems. But before open-source signal hunting can reach its full potential, people like Cohen and Matheny need to answer some serious questions.

Among them, how to balance privacy concerns with national security objectives? Open-source intel is, by definition, freely available on the Internet. But when most people give their data away, they don’t imagine military technologists trying to extrapolate predictions from that data. As more people become concerned about how their data is used, especially by government actors, they’re changing the data that they make and release.

Last summer, Lt. Gen. Michael Flynn, then the director of the Defense Intelligence Agency, discussed how the military had “completely revamped” the way it collects intelligence around open-source data. He said that, as the military’s reliance on such data grows, online behavior is changing and adapting. When asked if he was concerned by that, he answered, “Yes.”

Another question: how do we get machine intelligence to discover causation and not just correlation? How do you teach a big-data machine to provide a fully true answer, not just an output? Answer: we may need to re-design, on a fundamental level, the way we communicate with machines.

Perhaps one of the greatest paradoxes of the modern age is that our method for interacting with machine intelligence remains relatively crude: typing, and more and more with our thumbs. That limits our collaboration with machines to specific tasks with strict parameters.

When given a chance to collaborate with a machine or with a human on a project of any complexity, we’ll press zero to talk to the human almost every time. That’s a problem in a national-security context: service members are being asked to interact with an ever-larger number of systems in life-and-death situations.

Another one of Cohen’s DARPA programs, announced in February, seeks to change that. The Communicating with Computers effort seeks to “bridge the language barrier” between humans and machines, Cohen remarked in a press release.

"Human-machine communication falls short of the human-human standard, where speakers and listeners consider such contextual aspects as what has been said already, the purposes of the communication, the best ways to express ideas, who they are speaking with, prevailing social conventions and the availability of other modes of expression such as gestures. And so computers that might otherwise contribute more significantly to solving problems in a range of areas, including national security, remain in relatively simplistic roles such as crunching large datasets and providing driving directions."


A Little More About Our April 13 Viewcast

Before the military can use big data to understand complex systems and, possibly, the future, it must first help humans better communicate with the machine intelligences that are collecting or creating that data. That’s how the Communicating with Computers program, Big Mechanism, and Open Source Indicators all work together.

To explore these and other topics, Defense One will hold a special, exclusive viewcast on Monday, April 13, with Cohen and Methany, two technology leaders very much on the forefront of determining how the national-security community uses big data in the future. We hope you'll join us. 

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.