AI Changes Everything - AWS Community Day NL 2026

An incident call starts with an AI assistant explaining the changed code. Useful. But the room still goes quiet because nobody knows why this flow behaves differently, or which other promises depend on it. AI makes code, tests and summaries cheaper. The harder work is preserving the shared understanding behind the system. This session connects Naur's theory building to daily AI-assisted development and leaves developers with practical habits for keeping the why alive.

Transcript

1. AI Changes Everything. Except what makes great systems. Nico Krijnen · Architect
2. A reasonable request Most payments finish within two seconds Failures take a minute Customers are complaining to Support Product wants this in the next release Fictional reconstruction inspired by real personal observations
3. A look at “our” service API Gateway Step Functions payment workflow Lambda → payment provider Lambda checks payment status confirm order OR report payment failed
4. A likely cluprit API Gateway Step Functions payment workflow Lambda → payment provider "WaitBeforeNextStep": { "Type": "Wait", "Seconds": 47, "Next": "CheckPaymentStatus" } timeout Wait 47 seconds Lambda checks payment status confirm order OR report payment failed ✅ The test expects 47 expect(wait.Seconds).toBe(47);
5. Where does the 47 come from?
6. Where does the 47 come from? The commit says only: “Add delay after a payment incident.”
7. The real work is building a theory Peter Naur The N in Backus–Naur Form Programmers build a mental theory of how the system works, why it was built that way and how it fits the world. It is that mental theory which allows safe modification and quick incident reaction. Code is merely a by-product. Paraphrased from Peter Naur, “Programming as Theory Building” (1985)
8. We know the code. Do we understand the system? 01 Can we explain what the system does? Yes 02 Can we explain why it is right for the real world? No 03 Can we change it with confidence? No
9. Two lines change. All 28 tests pass. Would you deploy it?
10. INC-241 Paid orders are being marked as failed. Payment-failure-rate alarm · 14:12
12. OUR SYSTEM 14:06:03 14:06:08–09 Our request times out NOT_FOUND; order marked failed 14:07:02 Customer retries 14:12:00 Failure-rate alarm fires
13. OUR SYSTEM 14:06:03 CUSTOMER CONSEQUENCE Our request times out NOT_FOUND; order marked failed Customer sees “payment failed” 14:07:02 Customer retries Customer pays again 14:12:00 Failure-rate alarm fires Team starts incident response 14:06:08–09
14. PROVIDER TRANSACTION FEED OUR SYSTEM 14:06:03 14:06:08–09 14:07:02 Our request times out NOT_FOUND; order marked failed 14:06:38 Original payment appears as accepted. Customer retries ASSUMPTIONS PROVEN WRONG 14:12:00 Failure-rate alarm fires our request timed out ≠ provider rejected the payment status says NOT_FOUND ≠ payment will not complete
15. Days later An engineer at the provider checked their code: Accepted payments can remain invisible to status checks for up to 45 seconds. 45-second uncertainty window + 2-second operational margin = 47 seconds
16. Theory restored. A better fix. What happened What went wrong Better change A timeout leaves the payment outcome unknown. Declaring failure to customer can invite a second real payment. Block retries until safe. Make the implicit explicit: express UNKNOWN and SAFE_TO_RETRY instead of hiding behavior in a delay.
17. The real work is building a theory Your mental theory is what allows • confident changes • predicting what will happen • understand new situation
18. Root cause: AI changed 47 seconds to 5 Contributing condition: theory behind 47 seconds outlived the understanding of the team.
19. AI made both the edit and the investigation faster. The deployment decision was still ours. AI does not take responsibility. That lies with us.
20. Build understanding, not just better artifacts 01 Build a mental theory together Pair on consequential changes. Explain the assumptions, likely failures and consequences to each other. 02 Leave evidence that helps rebuild the theory Record the observation, customer consequence, rejected alternatives and what would change the decision.
21. A theory is not handed over in a document. People need to build it together. Theory Building Techniques
22. The correct fix required a correct mental theory We were not shortening a wait. We were deciding when to tell a customer their payment had failed. Which “simple” request in your backlog depends on an explanation your team has lost? · Nico Krijnen Theory Building Techniques