Aerospace Daily & Defense Report

Watch: Robotic Copilot Demonstrated By Aurora Flight Sciences

RSS

Aurora Flight Sciences is flying a Cessna Caravan fitted with a robotic copilot as it completes work under Phase 2 of a Darpa program to demonstrate automation that could reduce the crew required to fly existing aircraft.

Under Darpa’s Alias (Aircrew Labor In-cockpit Automation System) program, Aurora has also demonstrated its technical approach on a Diamond DA42 piston twin and is installing the system in a Bell UH-1 helicopter.

Aurora and competing Alias contractor Sikorsky have submitted bids for Phase 3 of the program, which will mature technology developed and demonstrated under Phases 1 and 2, says Dan PattDarpa program manager.

“Alias is focused on cockpit automation to enable operation of current aircraft with fewer crew, and not on a new aircraft designed for reduced-crew operation,” he says.

Citing shortages of trained military pilots, Patt says “Making each human operator more effective [through robust automation] would be a really large payoff for the Defense Department.”

Alias is not about eliminating the human, but managing all the basic procedures on an aircraft so the pilot is not in the cockpit to flip switches but for their longer-term strategic thinking, he says.

“Pilots can use their time more productively,” Patt says. On an intelligence, surveillance and reconnaissance mission, Alias would “allow the human to think about the context of the mission and the information being collected versus managing the stick and throttle.”

Aurora’s approach involves a perception system in the cockpit that uses multiple cameras to read the gauges and monitor the switches, and feeds the data into real-time computer model of the aircraft’s state. This then allows the cockpit-mounted system to fly the aircraft via actuators that move the control column and rudder pedals and the multi-axis robot arm that moves the throttle.

The system has been installed in place of the right seat in the otherwise-unmodified Caravan for flights that comprise the graduate exercise of Phase 2 of the Alias program. The system is controlled by the pilot in the left seat using a tablet mounted on the glareshield.

Aurora flight tested an earlier version of its Alias in its Centaur optionally piloted version of the DA42. Installing the system in an aircraft from another manufacturer, the Caravan, demonstrated the Alias goal of making the system portable between existing types.

The system has also been installed in both a DA42 and a Caravan being used on the ground as hardware-in-the-loop simulators (Hilsim) for ALIAS. The UH-1 is being modified into a third Hilsim to demonstrate extensibility to another type from another manufacturer.

For Phase 3 of the program, Aurora has proposed maturing the camera-based perception system along with speech recognition technology that will allow the pilot to interact with the automation in the same way they would with a human copilot.

Aurora sees the near-term potential for the perception system to reduce pilot workload and increase safety by assisting the pilot with checklists, particularly in emergencies, and by monitoring analog gauges or digital displays and alerting the pilot to anomalies.

In a demo witnessed by Aviation Week at Aurora’s Manassas, Virginia, headquarters on Oct. 17, the perception system monitored first a chip detection light illuminating then oil pressure falling, and Alias alerted the pilot to a possible engine failure via the cockpit-mounted tablet.

Alias then guided the pilot though an engine-failure checklist in which emergency functions were assigned to either the pilot or the automation, with the perception system monitoring the gauges and switches to verify the correct actions had been taken in the right sequence.

“We can enhance safety today, without removing anybody from the cockpit,” says Aurora CEO John Langford. “And we can go into aircraft not set up to have electronic controls.”

Discuss this Video 56

on Oct 18, 2016

Where is going the pleasure of flying?

on Oct 18, 2016

Down the pan, of course.

on Oct 18, 2016

This adds yet another layer of complexity to already complex systems. There are three things guaranteed in life:-

1. Secure Internet is an oxymoron

2. If something can go wrong it will go wrong. Muphreys Law

3. The more complex a computer system is the less likely it is that the designers and operators will know about the possible failure modes.

This looks like an auto-pilot to look after an auto-pilot and begs a number of questions:-

In the event of a failure how does the real pilot decide which of the systems has failed?

One of the big problems faced by the crew of the current generation of fly-by-wire aeroplanes is figuring out "What The Hell Is Happening" when there is a failure of the automatic systems. Air France 447 anyone. Will there be time in the event of a catastrophic failure of the system for the pilot to figure out what is happening, switch off the two systems and the auto-throttles, and recover the aeroplane to a safe flight condition?

What happens when the two systems decide to fight each other?

The aim of an aeroplane designer should be to reduce complexity not increase it. This system sounds good in theory but I fear could cause some interesting smoking holes in the ground.

on Oct 18, 2016

2. If something can go wrong it will go wrong. Muphreys Law

As evidenced right hereby the fact that it is 'Murphy's Law'

on Oct 19, 2016

Well spotted Roger, Muphry's Law appeared in an article on Murphy's law in a Royal Air Force flight safety publication in the 1960's. It is the perfect illustration of Murphy's law.

on Oct 18, 2016

More thoughts!

The arm is mechanical. What happens if it JAMS in an inconvenient position?

It seems to have the same capability as my 10 year old Grandson when I fly him in my Beagle Pup 150!

What is it doing that an existing auto-pilot cannot do?

on Oct 18, 2016

Current autopilots are not capable of interpreting multiple warning indicators/failure situations and then work alongside the pilot to suggest potential solutions, go through checklist procedures, and potentially performing some of those steps as the pilot sees fit.

on Oct 18, 2016

After reading this article, then going to the several DARPA websites on the topic - please someone tell me what this device/system does that is not otherwise done by autopilots and systems monitoring schemes already in place. How does Alias "know" and "perform" what it does? Absent that information, it seems like it has taken a remotely piloted vehicle pilot out of a ground station, turned it into an electro-mechanical robot, and installed it at a human pilots position. What does Alias do vs. enhanced autopilots and improved systems sensing and response to airplane malfunction?
Hopefully a follow-on article will provide why, quote:
"Aurora’s approach involves a perception system in the cockpit that uses multiple cameras to read the gauges and monitor the switches, and feeds the data into real-time computer model of the aircraft’s state. This then allows the cockpit-mounted system to fly the aircraft via actuators that move the control column and rudder pedals and the multi-axis robot arm that moves the throttle."
is somehow different and better than spending NASA $$ and time on improving autopilots and automated systems monitoring and operation.

on Oct 18, 2016

a) Is the world's labour market that short of qualified pilots?
b) Should I be right assuming that the device interacts with external systems (re weather information, navigation and gods know what else), how safe would it be against cyber attacks?
c) The poor chap looks trapped in his port side seat by the contraption.
d) Yet another device which can go kaput.

on Oct 18, 2016

In regards to question A the answer is no. But if you look at the records, historically, the vast majority of airline accidents and fatalities have been a result of human error. Either in their cause or in failure to recognize and rectify.

Also, the airlines being for profit businesses are always looking for ways to increase those profits. And one of the most sure fire ways to increase profits is to decrease the cost of performing your business. The fully loaded costs for a fleet of pilots (e.g. pay, benefits, sick time, pension, recurrent training, certification and facilities to provide that training, etc...) is no small expenditure.

on Oct 18, 2016

makes sense in deed; good luck for next steps into a great future of aviation.

on Oct 18, 2016

Humans can be as messed up as they are but I prefer a mess up stuff than a robot! The future aviation sucks really, It would be far safer and less expensive to to have X-plane or FSX and do virtual reality!!
For god sake what the hell is going on here, am I stupid or too old but real world even if it is ugly is preferable than this reality disconnected virtual world!

on Oct 18, 2016

Quite simple.
Solves the shortage of co-pilots for the military.
The bot will function fully as a copilot.
Cheaper than adding autonomy to older airframes still in use.

on Oct 18, 2016

To "Solves the shortage of co-pilots for the military" it is simple educate better youngsters, make them practice sports that is easy to solve this issue!

on Oct 18, 2016

What???

on Oct 18, 2016

The automatic pilot from the classic movie/documentary "Airplane" appeared less threatening. Just design an inflatable pilot with a smile around the mechanism and nobody will have a problem with it.

In all seriousness, this seems like an awkward, expensive and roundabout way of designing what is ultimately an autopilot.

on Oct 18, 2016

That's true, but I just don't want to be sitting in the left seat when it develops a leak.

on Oct 18, 2016

Myke, you obviously did not watch the movie. That situation occurred and was, shall we say - mitigated to everyone's mutual satisfaction.

on Oct 18, 2016

Mene, Mene, Tekel, Parsin. The handwriting on the wall.

Mene, or Mina, sixty shekels. As usual it starts with money. Tekel, aka shekel. More money. But Daniel read the nouns and interpreted them as verbs. He read the handwriting on the wall as "you have been weighed and found wanting."

All of the professional drivers are on notice they will soon be out of jobs; and the Golem of automation is coming for professional pilots as well. The handwriting is on the wall.

There are plans for Artificial Intelligence to replace most office workers, lawyers, doctors . . .

They are all going to join the ranks of unemployed factory workers, drivers, pilots.

There are dreams of the profits to be made by eliminating payroll expenses.

There are no plans being made for what to do with the unemployed and dispossessed.

This is a democracy.

Who knows how the unemployed and dispossessed will turn, will vote.

Worse, no one cares.

Wait! Was it not at Belshazzar's feast those words were written on the wall? Mene, Mene, Tekel, Parsin. "You have been weighed and found wanting" Daniel told the king.

on Oct 18, 2016

A more detailed story on ALIAS will appear in AW&ST due course, but let's see if I can answer some of the questions about Aurora's system:

1 - it is the autopilot. The Caravan testbed has a Garmin autopilot, but it is not engaged when ALIAS is operating. The UH-1 has not autopilot, but the same ALIAS system works the same way in old the helicopter as in the Caravan or DA42

2 - Autopilots are relatively simple. They are not aware of the state of the aircraft. They execute dialed-in commands and have limited authority so they cannot do anything bad to the aircraft in the process. They are usually the first thing disconnected in an emergency, ALIAS is designed to stay engage and help the pilot

3 - it is designed to do everything the copilot does: monitoring, cross-checking, challenge-and-response, etc. It is programmed with every procedure in the pilot's flight manual and has 100% recall. It monitors every instrument simultaneously and continuously

4 - The actuator and arms are there because DARPA's goal is for ALIAS to be portable between any type of aircraft, without modifying the host aircraft. Its is designed to work in aircraft that do not have electronic flight controls (ie fly-by-wire) that the system could otherwise interface with. Virtual reality would require fly-by-wire.

5 - after programming with the procedures from the flight manual, the system in its current form does not interact with anything outside the aircraft. All inputs come from monitoring the aircraft gauges and switches, sensors on the control surfaces and the pilot's tablet

6 - The actuation system can be quickly disconnected and the robot arm is designed to fail safe and to disengage if it conflicts with the pilot. That said, the system can also quickly engage to stabilize the aircraft while the pilot recovers from disorientation

7 - although the robot arm is the focus of the video, it is not the focus of Aurora's program going forward. They will focus on the perception system and the assistance it can provide the pilot. The pilot group Aurora is working with see value in the perception system, but are not ready to accept the robotic actiuation

on Oct 18, 2016

It seems the future of flying is in robotics. 2) the name of the program is similar to the secret program of the hypersonic airplane that they were working on in Area 51 that was reported by everybody into that kind of stuff. This would be the perfect cover.

on Oct 18, 2016

DARPA program manager Dan Patt says ALIAS is actually a "joke" acronym - an alias for the program - because he dislikes all the acronyms DARPA comes up with for its projects.

on Oct 18, 2016

Some further thoughts. There was some discussion during Monday's briefing at Aurora about how ALIAS could help in an emergency like the Qantas Flight 32 A380 engine failure, where the crew was overwhelmed with cascading failure messages.

Because the system is programmed with the flight manual and can monitor the displays, Aurora argues the automation system - which is external to the aircraft's caution/warning system - would help the crew sort through the conflicting messages and more quickly bring the situation under control.

AF447 was not discussed, but the points were made that ALIAS contains a dynamic flight model of the aircraft and is always aware of the aircraft's state; it can monitor all of the instruments to detect conflicting data and, if necessary, synthesize missing data from other sources; it knows how to fly the aircraft and it can challenge the pilot if they are not following the correct procedure, as a copilot should

I am not a pilot, and so not overly burdened by ego, but I think there is potential in the ALIAS concept that should be discussed and not dismissed.

That is particularly true near term, in operations that are already single-pilot and do not have the safety blanket of a copilot, such as personal flying and package feeder services.in aircraft like the Caravan

This is akin to the relatively simple driver assistance features that are already in my car and which have already saved me from a couple of embarrassing dings and taught me to be a bit more disciplined driver when it comes of keeping lanes and distances from other cars.

I welcome it

on Oct 18, 2016

"I am not a pilot, and so not overly burdened by ego....". OMG.... really?!?!?
Anyone who would make a broad brush statement like that should be fired from AW&ST!
I AM a pilot and DEEPLY RESENT your sentiment!

Since you appear to have difficulties understand spacing and distance between the ditches with your car, I'd just as soon you stay clear of airplanes also.

on Oct 18, 2016

"I am not a pilot, and so not overly burdened by ego." What a strange attitude on the "staff" of an organization concerned with aviation. I wonder how much "aviation" there would be to write about if there were none of those pesky pilots? LOL!

While I know some rather ego-centric pilots, I also know of people like that who are NOT pilots. I suspect one of those may even be a "writer"! 8+)

on Oct 18, 2016

As a pilot and a car driver I'll never abandoned my freedom for a computer!!! Better purchase an old bird or car for my freedom. People are free to do what ever they want!!!

on Oct 18, 2016

From another perspective though, namely passengers, they would also like the freedom to live and not loose that freedom due to pilot error. For example every single person on board AF447.

on Oct 18, 2016

I prefer to die as a freeman than enslaved to the technology!

on Oct 18, 2016

Easily said from the comfort of your home - probably a different response though when sitting in seat 25A.

on Oct 19, 2016

when you go from a glider or a Cessna to a commercial bird it is heaven apart of farting (good to have the prop hiding the noise) and smelly people next to you!!

on Oct 18, 2016

This reminds me of the demotivational poster from despair.com I kept in my cubicle as a Global Hawk instructor and as a 777 instructor; "Motivation: If a pretty poster and a cute saying are all it takes to motivate you, you probably have a very easy job. The kind robots will be doing soon."
I tell my fellow airline pilots, "We won't and our kids probably won't but at some time in their lifetime, our grandkids will probably fly in an unmanned airliner.
It will start with cargo flights and transition from there."
Fortunately, Despair.com has a new poster now; "Adaptation: The bad news is robots can do your job now. The good news is we're hiring robot repair technicians.The worse news is we're working on robot-fixing robots and we do not anticipate any further good news."

on Oct 18, 2016

Interesting, but silly. More "I hate hiring employees" nonsense.

on Oct 18, 2016

In the early days of aviation the designer of the auto-pilot was a pilot, Lawrence Sperry, and actually did all the test flying of what was called "the Airplane Stabiliser". Today the designers of auto-pilots are University graduates in computing and many of them have no idea what it is like to be on the flight deck when it has "All Turned To Worms". This looks like a good candidate for "The worst idea of 2016" award.

on Oct 18, 2016

That is an issue. Today, advances in avionics almost always involve complex algorithms and computer scientists, and there are examples of assumptions made in shirt-sleeve software programming environments leading to accidents in the real world. I think the answer is to heavily involve pilots in defining, developing and testing systems.

on Oct 19, 2016

Graham, have you ever tried to talk to computer people about what it is like in the cockpit when it is all "Turning To Worms". Most of them have no concept of what it is like, the G loading, fear, confusion et al. My company built the worlds first performance computers for use on the flight deck in the 1990's. We tried computer companies and conventional programmers, it failed! We then taught pilots to program, it worked because they understood what went on on the flight deck.

on Oct 18, 2016

So we will eventually have unmanned airliners, pilot-less fighters (Japan), driverless cars, drone package delivery, virtual staff meetings (skype), computer diagnosed ailments, fast food delivered by computer, etc. What will humans be doing other than sitting on their sofas getting fatter and fatter. This is supposed to be progress?

on Oct 18, 2016

The questions that need to be addressed relate to safety first, then productivity. Can we be safer if a machine does something a human now does, or if we use a machine to help the operator avoid mistakes?

On productivity, is it correct to insist that a human continues to watch gauges and flip switches as they always have when the job also requires their mental capabilities, but the physical tasks consume time and training?

The issue of robots taking over jobs is huge and daunting. Some see disaster. Some see the unlocking of human potential. I don't have any correct answers, I just think we need to be sure we are asking the right questions.

on Oct 18, 2016

The statements made here so far are missing one salient point. The co-pilot’s job, aside from the obvious is to “learn” through hands on experience and tutelage how to fly at the highest level of competence, so at some point they graduate to the left set, when one’s proficiency can be confirmed. What happens when the captains retire and there are no experience co-pilots to fill the left seat?? Are robots going to take over both seats? Maybe, but while the auto industry is pushing the concept of “driver-less cars”, that’s a far cry from convincing the passengers flying at 35,000 ft. and mach.8 that there is not a human at the controls!!
I’m guessing it might happen at some point, but not in our lifetime!

on Oct 18, 2016

There are implications in the ALIAS concept for how you train pilots, what you focus their training on and how much training you give them. But they are likely far down the road, because first we have to build confidence in such a system as an advisory capability, before we can make any fundamental changes.

The car industry will face similar issues - driver assist is okay as a concept and so are driverless cars, but how do toy manage the transition from people who know how to drive, but chose to let a machine take control from time to time to those who have no concept of how a car works.

on Oct 18, 2016

I would need too much time monitoring what this mechanical co-pilot is doing for it to be worthwhile. A good automatic pilot is much better. This is a waste of money.

on Oct 18, 2016

How to move from shortage of pilots to surplus in a heartbeat ...

on Oct 18, 2016

Before I forget, I should apologize to any pilot offended by my "ego" gibe. Humor is a double-edged sword.

on Oct 19, 2016

Why should you apologize? A good sick joke is always enjoyable!!!
Enough with people having a broom stick in their spine!!

on Oct 18, 2016

In the Lockheed C141 to 'impress the flight nurses" we would 'hook up" the oxygen hose with a glove attached to the yoke - and show how the "arm" flew the jet. (They did not notice the autopilot on and we were turning the bank knob). Gee - I wonder!

on Oct 18, 2016

"After reading this article, then going to the several DARPA websites on the topic - please someone tell me what this device/system does that is not otherwise done by autopilots and systems monitoring schemes already in place."

It is integrating the capabilities to replace first the copilot and then the pilot. It is a step towards the autonomous aircraft as opposed to the remotely piloted one.

on Oct 18, 2016

"I wonder how much "aviation" there would be to write about if there were none of those pesky pilots?"

Pesky indeed. They act like they are important and want to be paid.

Think business not flying.

Years ago this was a publication concerned with every aspect of aviation. Now it might be better named Aviation Business Week.

on Oct 18, 2016

" Lawrence Sperry, and actually did all the test flying of what was called "the Airplane Stabiliser".

Inspired, perhaps after he and a lady friend crashed in an airplane and were rescued naked. The first recorded flying . . .

on Oct 19, 2016

Ah yes, but could a lady have as much fun with a naked robotic arm. Lawrence Sperry was actually showing the lady how effective his autopilot was at the time. Autopilot is in this context not an Euphemism.

on Oct 19, 2016

Lawrence Sperry was an innovative man. Not only did he initiate the "Mile High Club" with Mrs. Waldo Polk. He was also the first Base Jumper testing a parachute of his off the roof of the Garden City Hotel, on Long Island.

on Oct 18, 2016

"Maybe, but while the auto industry is pushing the concept of “driver-less cars”, that’s a far cry from convincing the passengers flying at 35,000 ft. and mach.8 that there is not a human at the controls!!"

After a few years of ads, articles and TV programs, and news, pimping propaganda on how much safer "self flying airplanes" are. Seasoned by a few specials, magazine articles, newspaper stories and blogs, about pilot error, folks will welcome the innovation.

Propaganda works, if it didn't there would be a world of ad men out there, squeegee in hand, on street corners.

Please or Register to post comments.

Connected Aerospace Coverage Sponsor
Advertise with Us
 

Sign up to Aviation Week Newsletter

Daily analysis on technology advances impacting the global aviation, aerospace & defense industries.

Penton Corporate

Sponsored Introduction Continue on to (or wait seconds) ×