Too often, accidents still generate controversy, with various parties blaming either the flight deck’s design or the crew’s performance. The French judicial case over the fatal crash of Air France Flight 447 in 2009 is still open, and the latest report transmitted to the judges added fuel to the fire. Meanwhile, airframers still have a hard time convincing pilots that adding automation will benefit safety.

But renewed scientific research applied to the cockpit may help move the industry past that tired, overrated debate. Rapid progress in neuroscience-based ergonomics is opening the door to an enhanced human-machine interface in the cockpit, with improvements in factoring in of the weaknesses and strengths of a pilot’s brain.

A Toulouse research laboratory, part of the ISAE-SupAero engineering school, has been making strides in what Director Frederic Dehais calls neuroergonomics, or neuroscience applied to human factors. With his 20-strong team, he is moving forward “from understanding to predicting.”

One of his focuses has been alarm deafness. In several accidents, pilots did not react to an aural alarm, which could be heard on the cockpit voice recording. Thanks to electroencephalography, the activity of the auditory cortex can be measured, and Dehais found the precursor of alarm deafness: a visible change on the electroencephalogram.

Combining real-time measurement of the brain’s activity with machine learning, a computer can detect and even predict the cognitive state leading to alarm deafness. Therefore, when an aircraft system wants to send an aural warning, a central computer could choose an alternative, such as a visual alarm, Dehais suggests.

The pilot would not have to wear a full, uncomfortable electroencephalography “helmet.” Two or three electrodes behind the ears could suffice. A challenge, however, is to make the technology work in an actual cockpit environment—an electroencephalograph “hears” every electromagnetic signal. Its signal-to-noise ratio is poor, Dehais notes.

To describe the circumstances that create alarm deafness, he took pilots through a simulated “Red Bull” pylon race in a magnetic resonance imaging (MRI) system. The researcher then sent aural warnings. “Pilots miss the alarms when they fly through the gates,” says Dehais. An “attentional filter” in the prefrontal cortex automatically lets relevant information in. In that instance, this is visual not auditory information.

Dehais and his team also have discovered that attentional skills are linked to working memory. The latter is the capability, for example, to immediately remember several numbers and use them. The greater the working memory, the better the attentional skills. Working memory tests are easy to devise and could be used as a criterion to select pilots.

In Bordeaux, a startup company called Akiani is advancing knowledge of the pilot’s cognitive state. The cognitive state differs depending on the task’s difficulty. “We look at cardiac arrhythmia, sweat, pupil dilation and some electroencephalography waves,” says co-founder Sami Lini. The brain’s workload can thus be described. “We go beyond ‘heavy or light workload,’ we determine whether the pilot is ahead or behind.” Akiani’s algorithms, however, are only 75-80% reliable, and the company’s researchers are working to improve that rate, along with the algorithm’s processing speed.

What about informing the pilot about his own cognitive state? “We asked ourselves the question three or four years ago; for some profiles, that piece of information proved deleterious,” Lini answers. Maybe it discourages the pilot, or adds to the workload in an already tricky situation, he suggests.

Electroencephalography can help measure engagement. Two different brain areas are considered connected when their electric waves have the same beat. The more a person is engaged, the more areas are found to be connected. And two people working together efficiently will have brain areas connected in a similar way, says Dehais. In a crew, monitoring such activity could help find different levels of engagement.

In turn, detecting insufficient engagement could trigger a “wake-up” procedure. The crewmember could be stimulated with a new task.

Another way to improve crew coordination could be to emphasize the importance of visual patterns. In a test conducted with France’s air accident investigation office, the BEA, Dehais and his team had crews performing a simulated go-around during which an additional requirement came from the control tower—a new heading. Due to a lack of cross-checking, 50% of the pilots overshot. In those crews, 90% of the pilots monitoring did not see the deviation.

An eye-tracking device helped the researchers understand that the pilots were not looking where they should have been. One, for example, was focusing on the airspeed to avoid flying faster than VFE (the maximum velocity with flaps extended).

One reason for an inadequate visual pattern is the brain’s need for fresh information. “Neurons being sensitive to gradients, the brain needs novelty or incongruity,” Dehais explains. The bottom line of the test was the lack of a defined visual pattern in the go-around procedure.

Akiani used eye-tracking to help the French Air Force with the Airbus A400M transport’s entry into service in tactical missions. “We watched the way crews appropriate information and measured the dispersion of sources; we therefore made recommendations on the time spent head-down and head-up,” says Lini

Eye-tracking has also helped Dehais find an eye-movement signature that indicates the pilot is looking at some information, but not understanding it. “One could highlight those items, on the display, that will help him or her understand,” Dehais suggests. “The pilot is an expert; he sometimes has weaknesses: Let’s put him back into the loop.” Lini concurs: “We have to find a strategy to assist the pilot without him feeling belittled in his job.”

An experiment Akiani conducted showed pilots could be kept engaged in low-workload flight phases, for a very convincing result. The pilots were given “what if?” scenarios. They prepared and rehearsed. When the hypothetical scenario materialized in a real situation, the brain’s workload, measured with pupil dilation, was lower than for a crew caught unprepared. “Hence better decisions,” Lini concludes.

An extreme level of disengagement can be reached when the brain is overloaded. Such a situation can be measured with near-infrared spectrography, which “sees” the concentration of cerebral oxygenation. In case of overload, some specific areas of the prefrontal cortex stop working, Dehais explains. The pilot can make a mistake when facing an obvious choice, such as shutting down the wrong engine.

To deal with overload, it is critical that pilots should be heavily trained for reflex actions, says Dehais.

He hopes to make the most of a new tool. In addition to experiments in a laboratory and a fixed-base flight simulator, Dehais and his team have been using a Socata TB20 light aircraft in flight. It is being replaced with a heavier Vulcanair P68 Observer 2. The latter aircraft is expected to be more representative of a commercial one, thanks to its two engines, modern avionics and sophisticated autopilot. With its 8-hr. endurance, researchers plan to test pilot fatigue extensively.

The P68 was delivered in December, and will receive eye-tracking cameras. A tablet will be used as an experimental primary flight display, able to adapt to the pilot’s cognitive state and issue new kinds of warnings. Zodiac Data Systems will supply a data acquisition, recording and real-time transmission system, similar to those Airbus uses in flight-testing.

The aircraft is planned to be fully equipped by year-end. In return, Zodiac may use Dehais’ work—to integrate eye-tracking data in a future flight data recorder, for instance.

Long term, too, low-cost portable devices could be used as an alternative to MRI, providing a deeper knowledge of the brain’s activity during flight.