CIA Headquarters

CIA Headquarters AP

Meet the Man Reinventing CIA for the Big Data Era

An exclusive interview with CIA's new director of digital innovation about his agency’s biggest change in decades.

The CIA will stand up its new Directorate for Digital Innovation on Thursday. It's the first directorate the agency has added since 1963 and the biggest change to America’s key spy service since before the moon landing. The new office will look beyond the spycraft of today to the very big question of how to turn the vast amounts of data that the agency collects into useful insight for analysts, agents, the agency, and the nation. The goal is to turn chatter and daily digital exhaust into a window into the future.

Defense One visited CIA headquarters in Langley, Va., for an exclusive interview with CIA Deputy Director Andrew Hallman, picked by CIA director John Brennan to lead the new office.

To encounter Hallman is to meet a man who is, in appearance and mannerism, the living embodiment of a more stately era of spying. He’s poised, articulate, thin, looks at home in a suit with suspenders and his favorite show is “Mad Men.” (He seems both more adult and more a Boy Scout than Don Draper.) Though he has the bearing of a Cold War operative, Hallman is also deeply in touch with the promise, potential, and hype surrounding big data and the interconnected era.

“The days of attending a cocktail party and writing up your notes are over,” Hallman told us. “A digital world challenges the way we work in a clandestine world. We have to come up with new ways to operate in a much more-connected environment and still be clandestine,” he said. Read that to mean: it’s hard to maintain a cover (or “legend” as the CIA calls them) in an era where everyone has a television studio in his or her pocket. The digital era presents a sea change potential for intelligence collection. That’s true for adversaries as well.

Hallman sees the new directorate solving three big problems for CIA. The first is helping agents and operatives hone their hacking and sleuthing skills. “The way we help people use digital and cyber techniques, this will raise it to a new level,” he said. Part of that is developing intelligence about those places where the United States can’t or won’t put boots on the ground, or at least, as many boots as the U.S. would like. “Yes, you can develop standoff access, depending on the target,” he said. “ISIL social media is a rich source of info for us ... They’re the most aggressive of actors in using social media, one of the most prolific. It’s elevated the importance of it as a source.”

The second problem: improving the CIA’s data management, or what Hallman calls “governance.” That sounds like fixing the filing system but Hallman says that better data governance will help CIA answer questions about precisely what it’s doing and what it isn’t doing. ”As taxpayers frame what we do, our ability to govern our data, our ability to respond to the public and to Congress is key. This directorate will help us balance those interests,” he said, referring to the balance between security and privacy. “We can be more assuring to Congress and the public in terms of how to govern that data.”

The third and greatest challenge Hallman is looking to solve through the directorate, taking the vast volumes of digital intelligence that CIA receives from around the world and transforming it into a moving, credible picture of the future. Intelligence, in this context, becomes almost a super power.

“We have the ability to do more sense-making to provide for analysts a real ability to forecast,” said Hallman, quickly adding that forecasting, arriving at a better understanding of multiple probabilities, is fundamentally different from prediction. “We are gaining the ability to anticipate the conditions of change to determine if they are anomalies or areas to focus on. This directorate will bring focus to that anticipatory intelligence capability,” he says. “It’s often in the aggregate that we get an impression.” That’s particularly true of social media, which provides of course not only intelligence for specific operations--targeting this or that ISIS leader in Syria--but also taking the temperature of an entire population. It’s sentiment analysis on a massive, population-wide scale. “If you have time, you can develop insight into patterns,” with social media data, he says.

What does that mean? In describing how the directorate will provide anticipatory intelligence, Hallman highlighted a 2011 program sponsored by the Intelligence Advanced Research Projects Agency, or IARPA, called Open Source Indicators, or OSI. The goal was to “develop methods for continuous, automated analysis of publicly available data in order to anticipate and/or detect significant societal events, such as political crises, humanitarian crises, mass violence, riots, mass migrations, disease outbreaks, economic instability, resource shortages, and responses to natural disaster,” according to the IARPA Web site. In other words, the agency wanted to take massive amounts of publically available data from news reports, social networks and every other available open information source and paint a picture of some event that might occur of national security significance, like a revolution or an assassination, perhaps.

The program achieved some notable success. In 2012, a team of Virginia Tech researchers affiliated with OSI effectively predicted two events on the basis of open source data from social networks and elsewhere. One was that Mexico’s election of Enrique Peña Nieto to the presidency would be met with protests. Sure enough, at 11:15 p.m. on July 1, the student group Yo Soy 132 began marching in protest against election results declaring Nieto the winner. The marches set off weeks of arrests and police actions. The Virginia Tech team also correctly forecast a series of protests around the impeachment of Paraguay’s President Fernando Lugo. Sure enough, on June 29, 2012, 2,500 people effectively closed the bridge linking to Paraguay Ciudad del Este to the Brazilian town of Foz de Iguaçu. The team’s methods likely would have predicted other flare-ups had they not made the specific decision to limit themselves to Central and South America.

When asked if that was the sort of thing Hallman wanted to be able to do everywhere, he answered “yes.” He called OSI, “an important feature in our resources and development…a cutting edge program.” To have real relevance, the directorate will need to be able to predict change through sentiment or other means before it’s obvious to everyone else, even the participants. Lead time is the difference between magic and a bad press conference.

Better anticipatory analyses in the hands of agents and analysts will require a lot of new software to allow agents to directly tap into the agency’s largest data sets and pull out insights directly relevant to them and their current operations. The directorate will create new application program interfaces, or APIs, toward that end and will also pioneer entirely new mathematical approaches to data analysis and extrapolation. Hallman already has plans to hire a chief data officer to spearhead the development of advanced analysis techniques, to “manage the data science cadre and provide standards for development APIs that allow various applications to talk to each other,” he said. (It’s safe to say it will be the sexiest job in data science outside of Facebook.)

The result, he hopes, will be agents and analysts developing a slightly more prophetic understanding of the operation in front of them, the mission at hand, and doing so across a variety of devices. No it’s not a crystal ball, but it could help in decision-making in a way that enables to agency to focus on the right suspects better and faster.

Everything about that--writing software, creating new APIs, and big data analysis--represents an evolution beyond the role that the CIA traditionally has played, the collection of human intelligence on foreign subjects for national security. It elevates a new role that involves the collection of signals intelligence. But signals intelligence collection and analysis has long been the job NSA. Hallman says that there will “always be a distinction” between the two agencies. But today’s human targets produce a lot of digital data, and as targets become more digital, the two missions are merging somewhat, becoming more integrated.

“We can use human-enabled targeting of signals intelligence in a way that’s more operationally relevant. That means a greater ability to provide a human-enabled way to help the NSA target, to be very selective in how we apply our exquisite forms of collection,” he said.

The notion of a United States intelligence agency, one that’s caught a lot of recent criticism from both lawmakers and from the public, taking on big data quickly become a controversial one. When asked if he anticipated public suspicion toward the new directorate, Hallman answered: “There might be. But the directorate will have a strong public profile...” It will have to, since it will have to work with more corporate partners than the agency does traditionally, which forces a certain level of transparency.

It goes back to the governance challenge. Narrower intelligence collection and analysis will--at least in theory--result in less need to use large amounts data from more people, a key consideration in the post-meta data era. When intelligence collection becomes more focused, fewer people get caught up in the data dragnet. That’s a net privacy win, says Hallman. “It’s a benefit,” he says. “The public debate about signals intelligence is reaffirming. It reaffirms our need to apply our intelligence collection to where it has the most value.”

Of course, big data has been the beneficiary of a lot of hype and hasn’t met with everyone’s expectations. It’s proven effective for predicting consumer behavior and elections, bad for predicting the outcome of sporting events and the stock market. In other words, in domains where two sides have a lot of options for out maneuvering each other, where the game has more variables, it fails. Will it be useful for the CIA?

Perhaps, for Hallman the metrics for success start with changing the way the agency uses technology and data. “We will be successful when we know that digital and cyber is central to all we do. There’s a fast and rapid increase in technological ability,” in the hands of both friends and adversaries. “We need to be able to keep pace.”

Keep pace, and hopefully increase it.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.