Understanding What We Have Wrought: Systemic Risks as told by a System Engineer

The slides accompanying Patrick McKenzie's talk, as delivered to the Bank of England. He later recorded a derivative of the talk for Complex Systems.

Transcript

1. Understanding What We Have Wrought Patrick McKenzie — Bank of England October 1st, 2025
3. “Hidden” SPOFs
10. Five Whys • Why did several banks share a software monoculture? • Why was this configuration change not caught in testing? • Why was this configuration change so hard to roll back and/or mitigate? • Why did this configuration change have such a large “blast radius?” • Why did this configuration change substantially disrupt “important business services”?
11. A Recipe For Software Monoculture
13. Defense in Depth: Automated Testing Crowdstrike Root Cause Analysis
14. Control Plane Issues Are Frequently Sev-0
15. 4,500 Banks Couldn’t Have Correlated Failures, Right?
16. This was a Near Miss… Including For You • Banking system in U.S. largely recovered to normal by extended hours on Friday. • Most affected service, teller transactions, are societally critical but can be deferred a short while. Competent, immediate efforts to divert transactions to electronic channels. • We had not yet had Crowdstrike completely deployed within the banks – Saved by our sloth and incompetence! – Consider an alternate universe in which all US-based counterparties go “dark for a day” • You may experience a technical crisis when weather is not normal, either incidental to the fact of market stress or in a complex casual relationship with that stress.
17. Potential Policy Responses • Blameless postmortems – Ask fintechs if you can read these! – Particularly the near misses! • If an engineer can cause an outage on accident then what could their laptop do on purpose? – Red team exercises
18. Stablecoins
27. “I think the answer is that it’s messy, but the funds are real. We see legitimate inflows into Tether from many sources—large ones—that result in market makers selling, creating, and sending billions of dollars to Tether’s bank accounts. They do this to mint the tokens and maintain relationships with Tether and its banks. Everything checks out, albeit in a messy way.” — Sam Bankman-Fried to Bloomberg (Aug 8th 2021)
28. “They have the money they say they have … I’ve seen a whole lot and the firm has seen whole a lot and they have the money. And so there has always been a lot of talk ‘Do they have it or not?’ and I’m here with you guys and I’m telling you we’ve seen it and they have it.” — Howard Lutnick to Bloomberg TV, Jan 16th, 2024
29. “Cantor Fitzgerald is not conducting continuous diligence on Tether’s financial statements, but I believe my statements were accurate when made.” — Howard Lutnick to U.S. Senate, Jan 25th , 2025
32. Patrick McKenzie on blog, October 28th, 2019
33. AI and future of trading
34. Kaplan et al., 2020
39. I Spent August and September Falling in Love
40. AI Risks in Trading Space • ”Every hedge fund will train their own model” not the outcome to bet on. – Concentration risk among 3 large lab providers, of which one could be (in any given six month window) effectively sole source to the UK trading community. • Time to detection and resolution increasingly dependent on whether Claude Code / OpenAI Codex / etc is up or not. – Knight Capital: 20 minute critical window – CrowdStrike: 90 minute time to resolution • Recursive dependencies put larger chunks of economy on narrower shoulders
41. Thank You patrick@kalzumeus.com Happy to chat if I can ever be useful, particularly informally. https://www.bitsaboutmoney.com Bits about Money is freely available and relevant to your interests.