“Seeing Like a Programmer”—Resiliency, Limits, and Moral Hazards in Software Engineering (LambdaConf 2024)

Engineering is building systems that both reduce mistakes and safely absorb the mistakes that do occur. In software engineering specifically, we should apply test-driven development (TDD) and adopt domain-driven design (DDD), use types systems to make illegal states unrepresentable, model what kinds of effects are legal in languages like Unison, and formally model and even prove how parts of our systems work: all tools for more effective “theory-building”, to borrow Peter Naur’s term.

But we also have two serious problems: (1) the limits of translating our thoughts into code; and (2) the limits of translating any human activity into software systems. Many “schemes to improve the human condition” have failed, as James C. Scott observes. We should learn from those failures.

We should not give up on software—or on software engineering! But great engineers know not only when and how to apply their tools, but also the limits of their tools and their whole trade.

Transcript

1. Chris Krycho – LambdaConf 2024 Resiliency, Limits, & Moral Hazards in Software Engineering Seeing Like a Programmer
2. A little bit about me Hello! • 15 years in software engineering • Aircraft software • Energy industry safety • Restaurant ordering • Deve l oper too l s and web frameworks at LinkedIn
3. How do we make good software?
4. Engineering Modernity and intellectual modernism Humility
5. What is software?
6. “[It] is important to have an appropriate understanding of what programming is. If our understanding is inappropriate we will misunderstand the di ffi culties that arise in the activity and our attempts to overcome them will give rise to con fl icts and frustrations. —Peter Naur, “Programming as Theory Building”, 1985
7. Software is artifact and system.
8. Software as artifact
9. “The score is potential ener gy . It’s the potential for music to happen, but it’s not the music. —Eímear Noone, composer and conductor
10. Software as system
11. Software is artifact and system.
12. Good software artifacts
13. What do we know? Good software artifacts • Domain-driven design • “Make I l l ega l States Unrepresentab l e”
14. “Make Illegal States Unrepresentable” Good software artifacts struct RequestData<Value> { is_loading: bool, is_loaded: bool, is_error: bool, value: Option<Value>, error: Option<String>, } let data = RequestData { is_loading: true, is_loaded: false, is_error: true, value: Some("oh no"), error: None, };
15. “Make Illegal States Unrepresentable” Good software artifacts enum RequestData<Value> { Loading, Loaded(Value), Error(String), } let data = RequestData :: Loaded("Phew");
16. What do we know? Good software artifacts • Domain-driven design • “Make I l l ega l States Unrepresentab l e” • Proofs and forma l veri fi cation • Forma l mode l ing • Testing: regression tests, test-driven design (TDD), property-based testing, performance testing
17. Good software systems
18. —Donella Meadows, Thinking in Systems “Systems happen all at once. They are connected not just in one direction, but in many directions simultaneously.
19. —Donella Meadows, Thinking in Systems “Resilience is not the same thing as being static or constant over time. Resilient systems can be very dynamic. Short-term oscillations, or periodic outbreaks, or long cycles of succession, climax, and collapse may in fact be the normal condition, which resilience acts to restore! And, conversely, systems that are constant over time can be unresilient.
20. What do we know? Good software systems • Let it crash (Er l ang) • Hand l e crashiness we l l
21. —Keunwoo Lee, “On Rebooting” “Your beautiful crash-only error handling strate gy has nudged your system into a new equilibrium where the backend is continually receiving too much load. A local defect has been ampli fi ed into a global system outage. Even if you remove the crashing defect, the fl ood of retrying startup queries may persist as a metastable failure mode of your system.
22. What do we know? Good software systems • Let it crash (Er l ang) • Hand l e crashiness we l l • Observabi l ity (visibi l ity): tracing and monitoring
23. Programming as Theory-Building
24. What state transitions are legal here? Programming as Theory-Building enum RequestData<Value> { Loading, Loaded(Value), Error(String), }
25. Why this representation and not another?
26. —Peter Naur, “Programming as Theory-Building”, 1985 “…the theory built by the programmers has primacy over such other products as program texts, user documentation, and additional documentation such as speci fi cations… in at least three essential areas:
27. —Peter Naur, “Programming as Theory-Building”, 1985 1. The programmer… can explain how the solution relates to the a ff airs of the world that it helps to handle.… 2. The programmer… can explain why each part of the program is what it is, in other words is able to support the actual program text with a justi fi cation of some sort.… 3. The programmer…is able to respond constructively to any demand for a modi fi cation of the program so as to support the a ff airs of the world in a new manner.
28. —Peter Naur, “Programming as Theory-Building”, 1985 “In several major cases it turned out that the solutions suggested by group B [(not the original authors)] were found by group A [(the original authors)] to make no use of the facilities that were not only inherent in the structure of the existing [program] but were discussed at length in its documentation.
29. “Legacy code” is code for which we do not have a mental model— a theory.
30. The model—the theory— lives in our minds.
31. Decisions
32. Heuristics and aphorisms Decisions “Prefer duplication to the wrong abstraction. —Sandi Metz —approximately everyone “Don’t repeat yourself.
33. —Peter Naur, “Programming as Theory-Building”, 1985 “ …if the exercise of intelligence depended on following rules there would have to be rules about how to follow rules, and about how to follow the rules about following rules, etc. in an in fi nite regress, which is absurd.
34. —James C. Scott, Seeing Like a State, p. 313 “…a wide array of practical skills and acquired intelligence in responding to a constantly changing natural and human environment. Mētis
35. Legibility
36. …for our own systems! Legibility enum RequestData<Value> { Loading, Loaded(Value), Error(String), }
37. Two paths Legibility 1. Embrace the fact that l egibi l ity is l imited. 2. Reshape the wor l d to be more l egib l e.
38. High modernism
39. —James C. Scott, Seeing Like a State, p. 4 “…a strong… version of the self-con fi dence about… the mastery of nature (including human nature), and, above all, the rational design of social order…
40. —Donella Meadows, Thinking in Systems “People who are raised in the industrial world and who get enthused about systems thinking are likely to make a terrible mistake.… to assume that here, in systems analysis, in interconnection and complication… is the key to prediction and control.… because the mind-set of the industrial world assumes that there is a key to prediction and control.
41. —James C. Scott, Seeing like a State, p. 262 “The necessarily simple abstractions of large bureaucratic institutions… can never adequately represent the actual complexity of natural or social processes. The categories that they employ are too coarse, too static, and too stylized to do justice to the world that they purport to describe.
42. This is lossy compression. Software and modernism enum RequestData<Value> { Loading, Loaded(Value), Error(String), }
43. —John Salvatier “Reality has a surprising amount of detail.
44. Humility
45. —Levi Wall “Disruptive logic often requires an oversimpli fi cation of a perceived problem..
46. Is this software good?
47. Ambition
48. —Peter Naur, “Programming as Theory-Building” “In including fl exibility in a program we build into the program certain operational facilities that are not immediately demanded, but which are likely to turn out to be useful. Thus a fl exible program is able to handle certain classes of changes of external circumstances without being modi fi ed.… However, fl exibility can in general only be achieved at a substantial cost.
49. To make good software. Robust and resi l ient. Respect the surprising amount of detai l in rea l ity. Reinforce mētis. Enab l e and empower and e l evates peop l e. Ambition
50. Rigor Perseverance Imagination
51. I appreciate your attention. Thank you! • Fo l l ow me: chriskrycho.com • Emai l me: he l l o@chriskrycho.com • Hire me! • front-end technica l strategy • TS adoption & conversion • Rust workshops • hard prob l em ana l ysis • bui l ding “ratchets” to improve software qua l ity 51