NTSB seeks lessons learned in 787 failures
A November 2006 fire that destroyed a 10,000-sq.-ft. Securaplane Technologies building in Tucson, Ariz., sent and its team back to the drawing board to reevaluate whether its chosen battery, chemistry and safety systems for the 787 were up to the job.
Securaplane, the battery-charging system contractor, had been testing a first-generation GS-Yuasa lithium-ion battery, which failed, causing the fire.
More than six years and several more battery incidents and fixes later, Boeing has constructed an increasingly comprehensive and complex mechanical and software safety net around the battery to protect the aircraft from any lurking “unknown unknowns.” What has remained the same is Boeing's commitment to the size, weight and energy-density advantages of lithium-ion batteries with cobalt-coated cathodes.
Thequestioned Boeing and the about that troubled path to certification during an investigative hearing April 23-24, part of its continuing probe into the failed-battery incident on a 787 in January.
The review was not meant to find the root cause of the failure, which continues to elude investigators, but to understand why the FAA-approved special certification conditions for the battery did not prevent the failure. Also on the agenda was whether FAA's delegation of authority to Boeing prevented the regulator from being close enough to testing the special conditions.
The FAA and Boeing rigorously defended both the certification and delegation of responsibility, saying the special conditions were “robust enough” and that delegation is the only way the agency can keep pace with technology. “One thing we can't do is wait until we know everything before we field a new technology,” says Steve Boyd, manager of the FAA's airplane and flight-crew interface branch.
Given its limited manpower, the FAA largely depends on authorized representatives (AR) to develop test plans and to witness tests. ARs work for companies which, like Boeing, have been granted organization designation authorization (ODA) by the agency. Doug Lane, the director of regulatory administration for, says there are 950 people in the program, about two-thirds of whom were working full-time as ARs during the regulations compliance phase of the 787 certification program. The FAA, in contrast, had the equivalent of about 40 full-time engineers on the 787 program, with an average of 30 years of experience in the group.
Lane says Boeing ARs approved qualification test procedures, performed conformity inspections, witnessed tests, and approved data and official test documents that were sent to the FAA engineers.
Ali Bahrami, manager of the FAA Transport Airplane Directorate, says it is most important for the agency to be involved in the first five years of a certification program—when the design “is relatively fluid”—and then again late in the program, when certification test flights begin. Communications lines with Boeing were kept open via the ODAs and monthly meetings to “discuss changes.” Regarding the FAA's decision to grant Boeing, late in the program, the authorization to make changes to the certification test plan, Bahrami says it was because changes at that point were more of an “administrative and timing” nature. “We have made it clear if there was some kind of anomaly, the ARs were required to report back to the FAA.”
The FAA's special conditions called for hazards like a runaway thermal condition to be “extremely remote,” leading to only once in 10 million flight hours. Boeing had experienced two events with the 50that had accumulated just 52,000 flight hours.
“We don't get every prediction right every time,” says Boyd. “But we try to build whole families of requirements that trap the safety issues, so even if something happens a little more frequently than expected, we have other parts of the regulations that provide the safety net to keep the airplane safe while we refine our processes and methods.”
When asked why the FAA did not incorporate a test regimen (DO-311) published by RTCA in 2008, the year after Boeing's special conditions were issued, Boyd said RTCA standards are not regulatory requirements.
“We looked at DO-311 and decided we had already incorporated what we needed based on special conditions and our test plan,” he said. “There are aspects in DO-311 that are explicitly more severe than regulatory standards and, in some cases, the special conditions.”
The government industry group met from 2006 to 2008 to develop minimum performance standards and suggested testing protocols for large lithium-ion batteries in aircraft use, as no guidelines existed for the new technology other than the FAA's special conditions.
Boeing systems engineer Jerry Hulm said some of the testing in DO-311 is “extreme” and “very harsh,” including a test that calls for overcharging every cell in the battery. “Those standards are there for a supplier who wants to go develop a battery so they can take [it] and sell to whomever,” he says. “We did an overcharge test and saw what it did, but that wasn't part of the certification. We had other protections.”
On the RTCA committee were representatives from Boeing,, Securaplane and GS-Yuasa. Thales builds the 787 power conversion system and subcontracts to Securaplane and GS-Yuasa. “If any one of those members saw anything in the standards that needed to be addressed from a safety standpoint, we would not have hesitated to address it,” says Hulm.
The risk assessment for the batteries, in part, came from GS-Yuasa's experience in building more than 14,000 lithium-ion cells for its industrial customers since 2001.
From the FAA's perspective, the nine special conditions were performance-based and flexible enough to cover whatever chemistry was used. Those conditions—meant to control lithium-ion-specific issues such as over-charging, over-discharging and cell flammability—cast a wide safety net with the flexibility that, so far, has covered the evolution of the battery through four design iterations.
Though there were more than 100 battery tests in the qualification/certification program, key to proving compliance with the special conditions were “abuse” tests—one of which called for a nail to be driven through a cell to force a short-circuit.
The testing, which GS-Yuasa says it performed at least 3-5 times for each generation of batteries, did not lead to thermal runaway in neighboring cells.
“There was no propagation to other cells,” says Boyd. “That was the basis of our conclusion that Boeing's analysis was reasonable.” In hindsight Boeing and the agency concede that certain battery tests to show compliance with FAA special conditions were too lenient.
“One particular test—nail penetration—resulted in a short that wasn't as energetic as we've seen in service,” says 787 chief project engineer, Mike Sinnett. “What we've since found out, in retrospect, is that we don't feel [the nail test] was conservative enough.”
En route to the new battery system being retrofitted on the grounded fleet, Boeing has made significant changes. After the Securaplane factory fire, attributed to the battery control system being disconnected, Boeing stayed with cobalt-based lithium-ion chemistry, but built-in an additional battery monitoring unit.
Three years later, while testing the electrical subsystem at Hamilton Sundstrand's Airplane Power Systems Integration Facility, one cell in a 787 battery experienced thermal runaway and vented. As a result, the third-generation battery included better sealing of the box, enhancements to the monitoring unit and a fuse-like “latch” function that requires a battery with too low a charge to be serviced by Boeing before being used again.