ARL wants anticipatory AI to adapt to soldiers' needs

Scientists at the Army Research Lab are studying how brain activity could be harnessed to optimize a performance of a human-machine team.

By feeding artificial intelligence systems information about a soldier's intent, advanced systems "could dynamically respond and adapt to assist the soldier in completing the task," said Jean Vettel, a senior neuroscientist at ARL's Combat Capabilities Development Command and co-author of a recent paper on the topic.

On the battlefield, soldiers perform many tasks at once -- surveying and navigating the landscape, listening to communications, assessing threats -- each of which is carried out in different regions of the brain. Researchers want to be able to analyze brain data in the moment so they can tell what tasks soldiers are performing.  Learning how areas of the brain work together to accomplish a task will help scientists build AI systems that can anticipate how a task should be accomplished and where they can assist.  

The researchers ran MRIs on 30 people to map the tissue pathways that relay information among nine different large-scale cognitive systems in the brain, such as those associated with attention, motor skills, visual and auditory systems.  The maps were converted into computational models, which allowed the researchers to see patterns among the regions in the brain and run simulations to show what would happen when specific areas were stimulated. A mathematical framework they developed allowed them to measure how brain activity became synchronized across various cognitive systems during specific  simulations.

Although the research has broad applications, Vettel said the primary purpose "is to understand how an individual's brain activity can be used to create novel strategies for a teaming environment, both for teams with soldiers as well as teams with autonomy."

In 2015, Vettel described some of her work using brain waves to train computers to detect threats. An experienced soldier wearing headgear lined with electroencephalography sensors would be shown images – some threating, some benign – and the sensors would be able to detect heighted anxiety, fear and tension to the threatening pictures.  Using brain waves to label threatening pictures would be a quick way to amass enough labeled training data to teach a machine learning algorithm to identify a threatening image. On the battlefield, brain waves detected by helmet sensors indicating heighted anxiety could be transmitted to other members of the squad to alert them of possible danger.

The foundational principles of brain coordination this new research describes could potentially be extended outside the brain, as well, creating dynamic teaming assignments in the future, Kanika Bansal, the paper's lead author, said.

The research was a collaboration between ARL, the University of Buffalo, Columbia University, the University of Pennsylvania, Carnegie Mellon University and the University of California, Santa Barbara.