AN OVER-DEPENDENCE ON AUTOMATION WILL KILL US ALL

An Emirates flight nearly crashed due to automation bias — here is how we fix the issue

By Brandon Bordenkircher

Sorry for reeling you in with the hyperbolic title, but we need to have a serious discussion about automation. In December of 2021 an Emirates flight from Dubai took off past the end of the runway, damaging the plane. As it struggled to climb it nearly hit the rooftops of buildings. Luckily all 300+ passengers survived, but it makes one thing clear — we need better automation safety precautions.

The botched takeoff was due to pilots failing to check the Master Control Panel altitude setting that was set to 0 ft by the previous pilots, causing the jet’s internal systems to not indicate takeoff rotation and climb. It happened due to something called “automation bias.”

The Double Edge Blade of Automation

Automation in the cockpit has helped make airline travel safer by a factor of two in every decade for 50 years by preventing unsafe maneuvers by pilots and reducing pilot cognitive load and cognitive fatigue.

It’s ironic that automation helped make airline travel safer, because it has also paved the way for automation related issues such as:

1. Automation Bias: automation is trusted more than the pilot’s own judgment
2. Automation Surprise: an unpredictable issue occurs when the pilot is “out-of-the-loop”
3. Absorption: pilot over focuses on a task leading to other issues being excluded
4. Fixation: pilot focuses on one solution despite other solutions being apparent
5. Preoccupation: pilot is distracted because the plane is flying smoothly
6. Underload: workload is low and it becomes difficult to pay attention

Pilots are spending less and less time practicing hands-on flying and more time learning new automated systems, resulting in pilots less comfortable overriding automated systems.

It’s not just a problem for pilots.

Surgery Robots and Autonomous Vehicles:

Automation in surgery may lead to less blood loss, shorter hospital stays and faster recoveries, but a study revealed robotic surgery practices limit the amount of hands-on surgical practice trainees receive, leaving new surgeons unequipped to perform surgery without the aid of artificial intelligence.

Autonomous vehicles have the potential to make the road safer, as 94% of all serious motor vehicle crashes are due to human error; however, several autonomous-vehicle crashes have demonstrated that automation bias is not just phenomena found in aviation .

Solutions:

The positive aspects it brings to our world far out way the negatives. We just need to make sure safeguards are in place such as:

1. More flight simulator training
2. Abnormal-situation training simulations
3. Mandated manual flying training
4. More automated system training and encouragement to override automation
5. Enforcement and regulation
6. Simplifying cockpit design
7. Monitoring pilots’ attention via artificial intelligence

The genie is out of the bottle and automation is here to stay. It is time we start to not only redefine the role of the pilot, but redefine the role of humans in many fields (surgery, etc).

For a more in depth look at automation in the cockpit, check out my article in the Spring 2021 edition of the Aviation Law & Policy Journal titled, “The Unintended Consequences of Automation and Artificial Intelligence: Are Pilots Losing Their Edge?”