False predictive maintenance alerts can lead to unnecessary inspections, component removals and aircraft ground time, while missed faults can increase the risk of operational disruption, according to industry specialists. This is forcing airlines and MRO providers to take a more measured approach to deciding when a predictive alert is strong enough to trigger maintenance action.
“There is no single accuracy threshold that automatically triggers maintenance action,” says Alice Belcher, director of predictive technology engineering at Delta TechOps.
According to Belcher, Delta TechOps extensively tests and back-tests predictive algorithms against historical data before they are deployed. Analytics that demonstrate consistent value are then reviewed by support services teams, including former line technicians, who assess whether the recommendations are actionable.
“The analytic itself does not make maintenance decisions,” Belcher says. Instead, engineers investigate alerts, validate the findings and decide whether further action is required, including whether an issue should be escalated to an OEM.
The approach is deliberately cautious, with Belcher describing the process as “go slow to go fast” as the confidence builds in predictive analytics before wider deployment.
Lufthansa Technik (LHT) is similarly avoiding a one-size-fits-all approach to prediction accuracy. Its predictive maintenance teams assess models using two key measures: precision, which indicates how often a predicted fault is confirmed, and recall, which measures how often a genuine fault is missed.
Peter Golibrzuch, senior engineer for predictive maintenance with LHT’s Aviatar team, says the balance between the two depends on the system being monitored and the consequences of failure.
Factors include whether a suspected fault can be easily confirmed on-wing, the potential cost of a failure and parts availability. Models can therefore be configured either to minimize false alerts or to capture as many potential faults as possible, even if that generates more false positives.
“There is no universal accuracy threshold that applies to every predictive maintenance use case,” says Mia Witzig, product lead for predictive health analytics at LHT.
For systems where failure could lead to an AOG, delay or cancellation, Witzig says a higher level of reliability is required. LHT typically sets a higher performance threshold for systems where these failures could directly lead to such incidents, and it continues refining the underlying models until they consistently achieve the desired level of reliability.
The consequences of getting that balance wrong are being felt directly in maintenance operations. Miguel Sosa, vice president and head of innovation at Trax, says false or low-confidence alerts can trigger unnecessary inspections and component removals, consume parts and tie up aircraft on the ground. But the longer-term concern is the impact on confidence among maintenance personnel.
“If teams repeatedly investigate alerts that do not lead to real findings, they can become skeptical of the predictive maintenance system as a whole,” Sosa notes.
Trax therefore advocates keeping lower-confidence alerts with engineering teams, where they can be assessed alongside other operational and technical information before reaching frontline maintenance personnel.
For now, experienced engineers still make the final call, with the required level of confidence depending on the potential impact of getting the prediction wrong.
Related Content
