AI-supported Software Engineering - and some lessons the aviation industry can teach us

What can aviation teach us about working with AI tools in software engineering? A lot, it turns out.

This talk draws parallels between the evolution of cockpit automation and the rise of AI coding assistants. Aviation achieved its remarkable safety record not by removing humans or removing automation, but by learning how humans and machines work best together — and, crucially, by learning what goes wrong when that partnership breaks down.

Through case studies including US Airways 1549 ("Miracle on the Hudson"), Air France 447, and Asiana 214, the talk explores the automation paradox: the more reliable our tools become, the less prepared we are when they fail. The same risks apply to software engineers who over-rely on AI - hallucinated libraries, atrophied skills, automation bias, and the slow erosion of the "code smell" intuition that only comes from practice.

The talk closes with three practical principles for navigating AI-assisted development: staying in the loop, maintaining your fundamentals, and knowing when to take manual control.

Developers who ignore AI will be left behind. But the real winners will be those who learn to fly with this new kind of copilot.

Transcript

1. AI-supported Software Engineering and some lessons the aviation industry can teach us Kai Koenig 20 October 2025
2. Content warnings: Mentions of aviation accidents, incl. human casualties Photos of aircrafts and humans in distress No graphic details/illustrations/photos
3. 208 seconds That's all the time Captain Chesley Sullenberger had.
10. We’re “flying” with a new kind of copilot 󰥔 + 🤖
11. Act 1 When partnerships work
14. The Evolution of the cockpit From manual flying to 90% automation
17. Result: Safest form of transportation in human history Not by removing humans. Not by removing automation. By learning to work together.
18. Promises of AI Partnership
19. The Partnership Pattern Pilot Sets course Monitors systems Ready in intervene Software Engineer Defines rules Reviews AI suggestions Catches edge cases
20. Act 2 When partnerships break down
21. Air France 447 The Automation Paradox
22. ❄ Ice blocks sensors ⚠ Autopilot disconnects 😱 Pilots startled, out of practice 🔔 Stall warning sounds 75 times ❌ They don't recognize it
23. “The more reliable automation becomes, the less prepared humans are when it fails.”
24. import pandas as pd import numpy as np import financial_analytics as fa data = pd.read_csv('financial_data.csv') result = fa.calculate_metrics(data) ✅ Looks perfect: proper imports, clean formatting ❌ Problem: 'financial_analytics' doesn't exist
25. Asiana 214 When Skills Atrophy
29. ✈ Pilot with 10,000 flight hours ❌ Never flew visual approach without automation in the 777 📋 Company policy: Use maximum automation ⚠ When automation confused: Couldn't manually fly 😱 Korean cockpit culture: Don’t question superiors
30. Act 3 The Human Factors
31. Mind Traps Automation Bias: "The AI said so" = "The computer said so" Confirmation Bias: AI reinforces your existing approach Attentional Tunneling: Focus on alerts, miss bigger picture
32. Flying by the Seat of Your Pants ✈ Experienced pilots feel when something's wrong 󰞵 Senior engineers sense code smells ⚠ If you never practice without AI, you lose this intuition “Physical piloting skills develop through flying aircraft with little automation”
34. What Aviation Does Aviation Training Starts with hand-flying Required sim sessions Regular proficiency checks Required manual hours Checklists for everything Tech Equivalent? 😞
38. Checklist Mindset ☐ Can I solve this without AI? ☐ Do I understand what this code does? ☐ Have I questioned this suggestion? ☐ Am I maintaining my skills? ☑ Is this critical and needs manual oversight?
39. Act 4 Becoming Beer Pilots
40. Engineering’s Path Forward ❌ AI won't fully replace developers (yet) ⚠ Developers without AI will for sure be left behind ✨ The winners: Those who learn to fly with this new copilot
41. Three Principles of dealing with AI Stay in The Loop Understand what AI does Have awareness Never trust it blindly Maintain Skills Code manually regularly Practice fundamentals Develop and keep the “feel” sharp When to Take Control Oversight over critical changes Security requires verification Edge case judgement
42. Remember the 208 seconds Sullenberger couldn’t have saved US 1549 without understanding automation. Automation could not have saved it alone. “For 42 years, I've been making deposits in the bank of experience...and on January 15, the balance was sufficient.”
43. Thank you!