In September 2012, Rockwell Collins gathered 18 airline, business aviation and owner-operator pilots in its Cedar Rapids hometown for some straight talk on the automation problems with today's integrated cockpits. What they learned could help the avionics company create next-generation flight decks that are safer and more mission-efficient.

“Mode surprises are a real problem today—they happen nearly daily,” says Geoff Shapiro, senior systems engineer in Rockwell Collins's advanced technology center. Shapiro had assembled the group to gather input on an advanced technology project to simplify the autopilot or “autoflight” modes that accompany flight management systems. A legacy airliner or business jet can have nearly 40 distinct autoflight modes for lateral, vertical or airspeed maneuvers.

“Pilots are busting altitudes. They have unintended stalls because the auto-throttle trips off and they don't know why,” Shapiro says. “The aircraft has a mind of its own and it's not really talking about why it's doing what it's doing.”

Mode confusion is one of many high-priority items on the to-do list of avionics makers like Rockwell Collins, Honeywell, Thales, Garmin and others that are striving to build cockpits of the future that will balance existing and emerging technologies such as fly-by-wire controls, broadband connectivity and increasingly complex automation.

From weather to traffic to pilot and aircraft performance and health, those flight decks will have unprecedented amounts of data that the avionics must transform into clear and concise information pilots or automation systems can use to take action—or perhaps not.

Pilots for their part will become more tightly coupled to their aircraft via a growing number of man-machine interfaces, known in human factors circles as “modalities.” Included are interactions with increasingly complex automation systems, larger displays complete with touchscreen capability, voice-recognition systems and visual systems that will soon be capable of “seeing” the real world through practically any weather. Farther out could be brain or heart activity monitors that pilots might use for taking actions.

“The next frontier is the connectivity of everything,” says Carl Esposito, vice president of marketing and product management at Honeywell Aerospace. “Most of the aviation systems we've designed to date have been relatively self-contained with little communication to the outside world.” He says the change is being driven by what we've come to expect in the consumer electronics market, where anyone with a smartphone and a connectivity plan can pull data from multiple sources and use applications to aggregate that data into useful information.

The connected aircraft will also require better man-machine relations to help pilots make optimal and safe decisions.

“Ten years ago, we introduced highly interactive flight decks that brought a high degree of capability, but we're still learning how to properly exploit it and the relationship with the human,” says Matt Carrico, senior engineering manager of advanced concepts for Rockwell Collins. “The next 10 years will be optimizing that interaction to balance the capability we provide versus the complexity of using the automation. We need to be more effective at using automation to help the crew accomplish the mission in a way that decreases training, increases awareness, reduces workload and increases safety.”

Key to reducing the workload will be engineering simpler interfaces to the equipment, an improvement that will go into future products, but could also be helpful in legacy cockpits to cut down on “mode confusion,” situations where the pilots become stumped by the seemingly capricious actions of the automation system.

Shapiro brings airline and corporate pilots into his laboratory to look at potential product improvements, one of which is a flight deck simplification. He says one pilot reflected that “the busier he gets, the less he uses” the automation. “That's a big failure for automation” says Shapiro. “Automation should offload you, so you can focus on the piloting tasks.”

How did automation fall out of synch with pilot capabilities?

Shapiro says it is because capabilities historically were introduced with federated boxes “and engineers making what they thought were good decisions. . . . But if they don't understand the pilot's point of reference, they miss the target,” he says. “We're going to reengage the pilots—building a completely pilot-driven interface versus an engineering-driven interface.”

Carrico notes that the industry has been wrestling with simplifying autoflight modes for more than two decades, but there is significant inertia preventing a step change. “Any radical changes to the flight management and autopilot human-machine interface will need to address the mixed fleet operations and cross-training issues caused by having a mix of 'old' and 'new” operational procedures in the fleet,” he says. “There will need to be a compelling cost/benefit story for the airlines to adopt.

The Rockwell Collins's advanced technology team is researching a “revolutionary flight deck” that includes mode simplifications along with increased situational awareness and FAA Next-Generation Air Transportation System (NextGen) features like 4-D time elements and self-separation.

“We're looking at multiple interface technologies that let your intentions get into the system faster,” says Shapiro. “You can't take five minutes to do something that should only take three minutes to do when you're getting close to the airport.”

Honeywell is investigating the new modalities as well. “As you get more different ways to interact with the systems and the aircraft, we want to make sure we do that up-front work of how pilots interact with systems, and what is the best way to present information at different times,” says Esposito.

The presentation, as it turns out, might need to be customized to the pilot's upbringing. “What are potentially regional differences in how people grow up, or how they understand certain visual or audio cues?” he says. “If you look at human factors, it's not only ergonomics, but its ethnocentricity or ethnic cultural pieces. We have done some things we can't talk about yet, but it involves customizing some products for different areas around the world, because of how different people react under stressful situations.”

Esposito says Honeywell by necessity is growing its ranks of human factors experts. “We always have had our pilot experts, but now we have such a confluence of data on the aircraft, and the ability to show so much data, that it's more critically important to show that data logically to pilots—so they can understand and absorb it quickly.”

Bob Witwer, vice president of advanced technology at Honeywell, says the “big challenge” in having a “totally connected” aircraft is translating data to actionable information. “Ten years and beyond, will have to develop more expert systems to sift this data out and provide actionable information to pilots,” he says, adding that there will likely be new FAA certification approaches needed. “I think we're going to start to approach and get into, over time, situations where full determinism of every single operation will start to become geometrically more complex,” he says. “We'll see more automation of the certification process.”

Honeywell, Saab Sensis and other companies are working with NASA to define and develop automated certification criteria, which would be needed to bound the performance and safety of so-called indeterminate systems, those with an infinite number of output states depending on the input and system state.

For the new technologies it is developing, Honeywell is placing more emphasis than in the past on the impacts they have on the mission and pilot.

“For the cockpit of the future, we try to understand what's the environment the crew will be in as opposed to 'let's put cool technology in to give new capability,'” says Witwer. “There's going to be so much data available to that aircraft, we have to make sure the cockpit systems we build will make it easy and intuitive for the pilot to constantly know what's going on inside and outside the aircraft. The pilot also must have a clear and intuitive understanding of whether there is action he wants or needs to take, and the cockpit systems must make it crystal clear to him what to do. That's incredibly hard to do well.”

Witwer says a question Honeywell is asked “a lot” is whether glass and touch screens will proliferate across the entire cockpit. “We had a customer that told us that for their next airplane they wanted to have even more glass,” he says. “We asked, 'Are you sure? Is more really better?'” He says modalities like touch screens or voice recognition cannot be “gimmicky.” “We've got to make the modality match the mission. We have to do mission analysis. That includes nominal situations, off–nominal and abnormal situations,” he says. “We also have to provision for backups. What if that modality isn't available? Will you have appropriate backups to keep the pilot's workload low?”

He says when Honeywell's human-centered systems “tech guys” look at additions to the cockpit, there are four criteria that must be met: Does it give the pilot what he needs, only what he needs and only when he needs it? And does it give him the information in a way that is intuitive, unambiguous and easy to understand? If not, it's clutter.”

Rockwell Collins's Carrico says a properly designed, highly automated next-generation cockpit will not necessarily require fewer pilots. “We're agnostic on the topic,” he says. “We're focused on workload reduction and the right role of automation versus what you let the pilot do. Whether that results in a plane that could be safely flown by one pilot is not an objective, but it could fall out from the result.”

He suggests there may be social and human problems with a one-pilot flight deck. “What do you do under failure conditions, or unusual operating conditions? Can a single human do that? Who backs up that human?” He says decision-making is best done with two people “overseeing” each other.

Tap the icon in the digital edition of AW&ST to see a video of next-generation cockpit technologies being developed by Honeywell and Rockwell Collins,or go to AviationWeek.com/video