Islamic State fighters appear in a November 2015 video.

Islamic State fighters appear in a November 2015 video. ISIL video via Reuters

How to Stop the Next Viral Jihadi Video

A new software tool could help social-media companies shut down the distribution of violent ISIS recruiting videos.

The Islamic State recruits supporters and fellow travelers from around the world largely by spreading photos and videos of its violent exploits online. What if social-media companies could automatically detect and delete such imagery?

The Counter Extremism Project, working with Dartmouth University computer scientist Hany Farid and funding from Microsoft, have developed a new method for doing just that. They hope to provide the software to help companies like Twitter, Facebook, and Google stop extremist groups from distributing such material on social media.

Based on a concept called robust hashing, the idea was invented in 2008 by Farid, who was trying to help stop the flow of child pornography. But telling a machine to recognize a picture of child abuse “is not possible,” he told reporters on a conference call on Friday. “We have not gotten to the stage where we can reason about fairly high-level things having to do with content. And so they were stuck.”  

Consider that Google and Stanford needed a neural network of some 16,000 processors looking at more than 10 million YouTube videos before the system could recognize a cat.

“Deep learning, and other related modern-day learning algorithms are not capable of quickly and reliably distinguishing between appropriate and inappropriate content,” Farid told Defense One in an email. “The speed and error rate for these approaches is simply prohibitive for dealing with Internet-scale image uploads” —  billions of new images per day.

So rather than teach a network of machines to understand the concept of abuse, Farid sought to teach software to recognize images that were spreading across the Internet after someone had tagged them as inappropriate. This way, the computer must simply recognize the image, not the idea that the image is depicting, and stop it from proliferating further.

Farid describes robust hashing as a method of extracting a “distinct signature” from an image. The signature is “an abstract numerical representation that is both distinct and stable to modifications to the underlying medium,” meaning that it remains useful even if the image is modified a bit. He further described the signature as “a string of numbers that embody that actual, underlying content,” like a photo’s DNA.

Farid donated the software to the National Center for Missing & Exploited Children, which licenses it for free to social media companies to police their sites for pictures of child pornography. It has an error rate of about one in five billion, he says.

About eight months ago, Farid teamed up with the Counter Extremism Project to apply robust hashing to a different challenge: recognizing jihadist content. They trained a system on the Project’s ever-expanding propaganda-images database, and set it to work.

“The technology will be running on any company that chooses to participate in this process. Associated with the technology is a database of hashes of known content. That hash table can be distributed easily through any number of different means and so every time there is new content, the hash table gets expanded, that hash table gets distributed to the companies that are participating and then the process is fully automatic at that point,” he explained.  

While social media companies will be able to license the software at no cost, Farid doesn’t intend to give away the intellectual property underlying it. The Counter Extremism Project is proposing a new center, called the National Office for Reporting Extremism, or NORex, to host the database and handle licensing and logistics for using the new technology. NORex would be modeled, in part, off the National Center for Missing and Exploited Children.

Social media companies have been stepping up efforts to combat the spread of violent jihadist imagery online. In February, Twitter announced that they had suspended more than 120,000 accounts used to spread ISIS content.

Counter Extremism Project CEO Ambassador Mark Wallace said that he had had “collegial discussions” with social media companies about the proposed NORex center.

One tech company representative told Defense One that many in the industry were skeptical of Wallace and the proposed plan. The representative said that on April 29th, a Facebook executive organized a conference call with Wallace and several key players in the social media space. The Facebook executive described a framework for an industry-funded center like NCMEC, but for extremist content. The tech companies were not sold. "The skepticism from participants focused primarily on the effectiveness of such a group. Someone raised privacy concerns, but the conversation was more about whether it would be able to accomplish what [the Facebook executive] described. Child pornography is very different from terrorist content. The NCMEC database is known, illegal, child sexual exploitation images and the definition of extremist content varies by country," the representative told Defense One.

That suggests that CEP still has some work to to in reassuring industry that the process of tagging content going into the database as "extremist" will be fair, since the definition of extremism is not uniform. CEP would be play an important role in deciding whether or not tagged content was actually extremist in nature, or simply controversial. Wallace said that he anticipated future debate and disagreement about some content and its tagging. In the meantime, some of the film and audio and footage that CEP already had in its possession met an obvious and common sense definition of extremism.  "I think we can all agree that there is going to be a set of images, video and audio that should be removed expeditiously," he said.

The effort has also won some important backing from the White House. "We welcome the launch of initiatives such as the Counter Extremism Project's National Office for Reporting Extremism (NORex) that enable companies to address terrorist activity on their platforms and better respond to the threat posed by terrorists' activities online. The innovative private sector that created so many technologies our society enjoys today can also help create tools to limit terrorists from abusing these technologies in ways their creators never intended," Assistant to the President for Homeland Security and Counterterrorism Lisa Monaco said on Sunday.

Updated Sunday, June 19th, 11:22 PM.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.