Flight simulators are training marvels, but whatever lessons they impart at an aircraft's performance periphery are best left unlearned because they're likely to be wrong.
The voice and digital recordings from Flight 447 revealed that the flight crew incorrectly diagnosed their aircraft's true condition and were unable to correct the loss of control (LOC) as the plummeted into the Atlantic Ocean. Learning that veteran airline pilots had allowed a transport aircraft to degrade past the edges of its flight envelope left many aviation professionals troubled and perplexed.
One of the important questions that resulted is whether upset recovery training can and should be conducted in simulators and, if so, would it be effective? Many have grown so accustomed to the ever-increasing fidelity of full-flight simulators (FFSes) that it is easy to assume they can accurately replicate any motion possible for an aircraft.
That misapprehension was given voice one day last September when I presented “Loss of Control: Investigating and Preventing the Loss of Control Accident” at the International Society of Air Safety Investigators annual symposium. My remarks done, a member of the audience asked, “Can't we just use simulators for upset recovery training?” Many of the accident investigators, engineers, training vendors and academics gathered there nodded their agreement.
But such a conclusion doesn't account for some important limitations with simulators that haven't been understood by airline training departments and others within the industry, and which have led to deadly unintended consequences.
Notably, the session chairman was Robert MacIntosh, chief advisor of International Safety Affairs for the, who had been the Safety Board's investigator in charge of the fatal accident involving an Airborne Express DC-8 on Dec. 22, 1996, near Narrows, Va. That accident ended a post-maintenance check flight when the pilot flying (PF) applied inappropriate back pressure on the control column during an attempted clean stall recovery. The NTSB cited the differences in the aircraft's handling characteristics near the stall versus the handling characteristics programmed into the simulator as contributing to the crash.
The NTSB was standing on good science when it expressed strong concern about the potential consequences of handling differences between the actual aircraft and the data programmed into an FFS.
Unfortunately, it seems many of the important lessons that should have been learned from that accident were overlooked during the design and implementation of first-generation advanced maneuver programs. Simulator fidelity was again one of the main concerns during the NTSB hearing onFlight 587, an Airbus A300 that crashed shortly after takeoff from New York's John F. on Nov. 21, 2001. The jumbo went down after the first officer over-controlled rudder inputs in response to wake turbulence and in the doing snapped the composite vertical fin off the aircraft. American's Advanced Maneuvering Program had been teaching pilots during simulator sessions to use pronounced rudder inputs when encountering similar upsets.
At the time, Larry Rockcliff, an Airbus captain and co-chair of the Upset Recovery Industry Team, testified, “the actual fidelity — the actual information that goes into providing the simulation, the actual copy of the aircraft — is in a relatively narrow band as compared to what an aggressive upset could actually cause upon a pilot.”
He said his group discovered that even with “some fairly simple maneuvers” the simulation was not representative of what actually occurs in the real world. “For example,” he continued, “in one simulator, recovery from a full stall condition could be achieved by holding the control column back and using power to fly out of it, which is absolutely incorrect.”
The NTSB final report on the accident noted that the simulators did not adequately portray the actual large buildup in sideslip angle and side loads that would accompany such rudder inputs in an actual airplane.
It is keenly important that those proposing the use of simulators for upset recovery training programs understand FFS limitations. Essentially at the core of these advanced training systems are computers, and as such the quality of their output is dependent upon the quality of the data input. Much of those data are derived from wind-tunnel and flight tests, and when operating within the performance envelope that results, FFSes fly much like the actual aircraft. However, when the sim is flown near the envelope's edges, the fidelity begins to fade.
Certification criteria for a simulator's physical, motion, visual and cognitive, or engagement, fidelity do not extend to the “edge of the envelope” maneuvers.
At the Airbus-sponsored 16th Flight Safety Conference in Brussels on March 15-18, 2010, David Owens, the planemaker's senior director of Flight Crew Development, pointed out there is no assurance that a simulator's motion replicates that of the actual airplane once it is outside of the established envelope. The mathematical equations at the edges of the flight performance envelope, particularly when high angles of attack (AOA) and yawing are involved, are difficult to analyze and solve. Noted Capt. Owens, “Whenever extrapolations from real measurements are used, the simulator's aerodynamic model may become questionable.”
senior research engineers Thomas Jordan, John Foster, Roger Bailey and Christine Belcastro echo this warning. In a paper presented at the American Institute of Aeronautics and Astronautics' Aerodynamic Measurement Technology and Ground Testing Conference, held in June 2006 in San Francisco, they wrote, “Accurate modeling in regions of the flight envelope characterized by high wind angles, high angular rates and separated flow, is a formidable challenge”
Often when evaluating flight path data from some of the more prominent LOC accidents, investigators have found that the pilots were contending with an extreme AOA and/or high sideslip angle. Typically, the only time a pilot will ever be faced with flying a transport aircraft at such extreme angles is during simulator training. So, how representative is a simulator to an actual aircraft's motion when at such extreme angles?
To answer this question,and engineers spent 1,600 “occupancy-hours” in three wind tunnels collecting aerodynamic data on two scale models of a twin-engine airliner similar to NASA's -200. The models were positioned for angles of attack up to 90 deg. and sideslip angles out to 45 deg. In addition to static runs, where the model is rigidly mounted, the engineers ran forced oscillation and rotary balance testing with the model in rolling, pitching and yawing motions to gather rate damping terms.
The results were notable. For instance, the wind-tunnel-derived coefficient of pitching moment (a measure of the aerodynamics forcing the airplane's nose up and down) was drastically different from the existing simulation at relatively high angles of attack. The same was true for elevator power, a measure of how much control the pilot can exert with the elevator.
Data on performance near the envelope's edges must be derived because test pilots do not deliberately lose control of an aircraft just to get data for a simulator. Even when a temporary excursion from controlled flight does occur during testing, that isolated incident does not provide much information because of the complicated equations that govern dynamic maneuvers involving non-linear aerodynamics and inertia effects.
Further, flight test data are drawn from quasi-static maneuvers, during which a pilot makes a measured control input and then goes quickly back to neutral and holds there, as the aircraft's resulting motions in terms of the amplitudes and frequency are recorded and measured. By contrast, a “dynamic” maneuver, such as occurs in regular flying, would involve the pilot making that initial measured input and then continuing to make control adjustments in response to the aircraft's response.
In his article, “Airplane Upset Recovery, a Test Pilot's Point of View,” published in the Airbus Technical Digest: FAST in May 1999, William Wainwright, the company's chief test pilot, states, “Firm conclusions about aircraft behavior can only be drawn from the parts of the flight envelope that are based on hard data. This means being not far from the center of the flight envelope. It does not cover the edges of the envelope.
“There are other limitations of the flight test data,” he continued. “For example, when flight testing explores an aircraft's stall behavior, the aircraft is normally flown with minimal sideslip in order to minimize any lateral-directional abnormalities on the aircraft's handling and to make the data analysis 'cleaner.' So the emphasis during these tests is the motion about the longitudinal axis. The correlation of the data between the aircraft stalling tests and the simulator is quite good up to a point for the stalling speeds and pitching behavior (basically up to the point of stall warning), but fidelity is not ensured for any inputs that might bring about an asymmetric stalling of the wings.”
Since it is too dangerous to expose a flight test crew to inflight upsets, a team ofresearchers is testing a UAV with a 7-ft. wingspan that is a 5.5% scale model of a low-wing large transport with wing-mounted engines. The aircraft incorporates dimensional, weight, inertial, actuation and control system scaling so that it can realistically simulate the flight characteristics of the full-scale aircraft. By flying out-of-control scenarios, the team expects to be able to validate the aerodynamic models for those events.
The flight envelope expansion and the need to gather large amounts of data at high rates presented unique challenges. The model will be operating outside of the benign flight envelope, sometimes to AOA exceeding 40 deg. and simultaneously to sideslip angles approaching 30 deg. — not a place living pilots want to explore. An additional advantage is the significantly reduced cost compared to flying a full-scale vehicle. The UAV runway at the NASA Flight Facility on Wallops Island, Va., is being used to conduct the experiment.
While this exploration should prove enlightening, NASA Langley engineers long ago learned that seemingly insignificant changes to an aircraft can have pronounced and unpredicted effects, particularly at high AOA. For example, merely changing the fillet along the wing-root-fuselage junction on a Grumman AA-1 Yankee turned the stalls and spins from recoverable to completely unrecoverable in the actual aircraft, whereas the subscale models failed to predict any such change.
An unintended consequence of the upset recovery training revealed by the American Airlines Flight 587 investigation is that pilots have been improperly trained to counter LOC scenarios in those regions outside of suitable fidelity. Capt. Wainwright found, “Training managers were all in the habit of demonstrating the handling characteristics beyond the stall, often telling their trainees that the rudder is far more effective than aileron and induces less drag and has no vices! In short, they were developing handling techniques from simulators that were outside their guaranteed domain.”
Capt. Owens further warns, “When experiencing an actual LOC situation, the pilot's response based upon full-flight simulation training may impact their ability to safely recover from the event while flying an aircraft.”
As a result, many pilots have improper expectations regarding how the aircraft will handle near the edges of its performance envelope, and may have been taught incorrect recovery techniques. Furthermore, a number of business jet LOC accidents occurred when flight crews applied “standard” recovery techniques that had adverse handling characteristics in the specific type of aircraft. Capt. Wainwright makes the point that, “There is no such thing as a standard upset” and expressed a “reluctance to endorse simplified procedures for recoveries from an upset.” The “Airplane Upset Recovery Training Aid Revision 2,” a document produced for theby an industry/government working group in 2008, clearly warns that techniques and procedures that work with one type of aircraft may be entirely inappropriate for another.
So, even if the NASA engineers further refine the aerodynamic data for an aircraft near the edges of a flight envelope, it will accurately replicate the sensations of an LOC event only in a few very specialized simulators (see “A European Priority” sidebar).
That's because a controlled movement depends on the pilot's sensory, perceptual and motor systems, whose input is relayed to the brain where it is organized and integrated. The kinesthetic modality is an important source of information and movement.
Since simulators are essentially fixed to the floor and the travel distance and duration of their actuators are limited, they can't accurately replicate important vestibular sensations and g-forces. An FFS employs a technique called “acceleration onset queuing” to represent initial accelerations well. But in order to prevent the motion system from reaching its limits of travel, the platform returns to a neutral position at a rate below the pilot's ability to sense the motion. These motion systems are not capable of representing long duration accelerations accurately.
Many colleagues report their six-month recurrent training contains scenarios in which they fly the simulator to and beyond the edges of the machine's performance and handling envelope. This includes full upsets.
Capt. Owens acknowledges this noting that even some aviation authorities “still promote the use of simulator training beyond the area of validity.” Airbus does not recommend the use of FFSes for dynamic upset demonstration and recovery, and remains concerned about the excessive risk of negative training.
Flight simulators “are 'virtual aircraft' and they should not be used to develop techniques at the edges of the flight envelope,” Cautions Capt. Wainwright. Rather, he advises, “Concentrate everyone's attention on taking action early enough to prevent the occurrence of loss of control.” That advice is echoed by other studies.
Despite the limitations cited, an FFS can serve to prevent LOC by helping pilots recognize the onset of an upset, and respond appropriately to check its development. In the doing, however, the validity of the simulator model must be respected.
Pilot training methods still have a long way to go to address the LOC problem situation in a measurable and meaningful way. Simulator training can help, but those programs must adhere to valid flight envelopes. BCA