Beyond "it depends" - How much architectural work do we need?

This slide deck discusses the controversial topic when to do how much architectural work in software development projects - not only, but especially in agile contexts.

It starts with the observation that _the_ right answer does not exist. Instead it should be aligned with the amount of uncertainty due to the task at hand and other uncertainty drivers. Based on this level of uncertainty other types of software development approaches are needed where the architectural work is embedded in.

After a short discussion, that "agile" is not necessarily agile, a series of software development approaches are listed, aligned with the degree of uncertainty they support best and how to embed the architectural work into these approaches.

This is just a very simple framework to sketch the ideas and based on individual needs, different approaches and embeddings can be used of course. Still, the key message is: There is no "one size fits all", neither for software development approaches nor for the embedding of architectural work.

As always, the voice track is missing. Nevertheless, I hope it contains some useful ideas for you.

Transcript

1. Beyond “it depends” How much architectural work do we need? Uwe Friedrichsen – codecentric – 2006-2022
2. Uwe Friedrichsen Works @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://ufried.com/
3. When to do what and how much
4. There is no “one-size-fits-all”
5. We came from “waterfall all-the-things”
6. Waterfall all-the-things • We didn’t like it • It didn’t work well in quite some contexts • We complained • Too limited, too uniform for our heterogeneous demands • “We need something that adapts better to our needs”
7. We went “Agile”
8. Well, quite often it feels more like this ...
9. Still, fine … but …
10. … now we are “Agile all-the-things”
11. Agile all-the-things • We love it • It doesn’t work well in quite some contexts • We do not complain • Instead, we try to squeeze everything into an agile corset • “We’re agile. Period!”
12. We just created the next “one-size-fits-all”
14. Understanding uncertainty
15. Uncertainty breaks the linear relation between effort and value
16. Eh, WTF?
17. Internally controlled Externally controlled Effort spent Units produced / Output Certainty-driven value prediction model (a.k.a. industrial value prediction model) Value created Units sold / Outcome Linear relation due to certainty We are looking for this Due to the linear relation it is sufficient to plan effort in order to predict value
18. Internally controlled Externally controlled Effort spent Units produced / Output The effect of uncertainty Value created Units sold / Outcome We are still looking for this Influencing factors outside our control We cannot predict it reliably upfront We can only observe it after the fact No longer suitable to predict value Breaks linear relation Uncertainty ?
19. Internally controlled Externally controlled Effort spent Units produced / Output The effect of uncertainty Under uncertainty we cannot predict upfront which kind of performance we are going to produce Support performance Idle performance No value Value-adding performance Creates value Value-reducing performance Destroys value
20. Uncertainty means you cannot predict the value that will result from your effort
21. Under uncertainty you do not maximize value by optimizing efficiency of efforts (a.k.a. cost efficiency), but by detecting and cutting idle and value-reducing performances as soon as possible
22. Acting under uncertainty 1. Create a hypothesis regarding the effect of an effort 2. Do smallest action suitable to measure an effect 3. Measure effect and evaluate hypothesis a. Further develop hypothesis if expectations met b. Drop hypothesis otherwise (optionally pivot) 4. Repeat
23. Sources of uncertainty in IT • Post-industrial markets • Complexity of context and tasks • Digital transformation (a.k.a. digitization, digitalization) • Disruptive technologies and more
24. Going post-agile or killing the one-size-fits-all dogma
25. The suitability of an approach depends on its context
26. Uncertainty Certainty • Extensive effort planning • Maximize efficiency • Detailed controlling • Hypotheses validation • Small steps • Quick feedback cycles Value maximization approaches
27. And acting under uncertainty is “Agile”, isn’t it?
28. Well, not really …
29. Types of uncertainty • Due to homework not done • Accidental, complicated, avoidable • Lack of diligence • Due to missing internal feedback • Semi-accidental, complex, harder to avoid • Rooted in company culture deficiencies • Due to required external feedback • Essential, complex, not avoidable • Cannot be solved internally
30. Uncertainty and “Agile” • Due to homework not done • Accidental, complicated, avoidable • Lack of diligence • Due to missing internal feedback • Semi-accidental, complex, harder to avoid • Rooted in company culture deficiencies • Due to missing external feedback • Essential, complex, not avoidable • Cannot be solved internally This is what Agile originally tried to address This is what Enterprise Agile * usually addresses This is what Enterprise Agile * often causes * Enterprise Agile: Typical “Agile” implementation in most companies
31. Agile as implemented in most companies does not address essential uncertainty
32. Uncertainty Certainty Software development approaches Focus on cost (Waterfall, V-model, SAFe, …) Enterprise “Agile” (SCRUM, LeSS, …) Focus on flow (DevOps, Lean, …) Focus on feedback (Developer anarchy, …) Focus on innovation (No software development)
33. Uncertainty Certainty Invent potential new business offerings Grow a successfully launched new business offering Explore a potential new business offering Extend/improve an established business offering Create a new digital interaction point for an established business offering Implement new legal regulation Replace system that technologically reached EOL Typical tasks with varying uncertainty
34. Uncertainty Certainty Task examples with varying uncertainty “Create a better world for our children” (vision) Evolution of new Living 360 offering (insurance) Fruits4U (startup) Release of new Warp wireless plans (telecommunications) Subcontractor self-service app OTID (logistics) Implement the new customer documentation law (banking) Replacement of AS/400 based ERP AMS/400 (retailer)
35. Uncertainty Certainty Software development approaches Cost “Agile” Flow Feedback Innovation “Create a better world for our children” (vision) Evolution of new Living 360 offering (insurance) Fruits4U (startup) Release of new Warp wireless plans (telecommunications) Subcontractor self-service app OTID (logistics) Implement the new customer documentation law (banking) Replacement of AS/400 based ERP AMS/400 (retailer)
36. Uncertainty Certainty Software development approaches Focus on cost (Waterfall, V-model, SAFe, …) Enterprise “Agile” (SCRUM, LeSS, …) Focus on flow (DevOps, Lean, …) Focus on feedback (Developer anarchy, …) Focus on innovation (No software development) Is this all we need to take into account?
37. We need to take the lifecycle of a system into account
38. Initial empty state S0 C1 S1 First change Resulting state C2 S2 Next change Next state SI Initial release (V1.0) SN CN Final state (EOL) Last change System lifecycle (simplified)
39. Most architectural work considers this part only … … but 80%+ of the costs and efforts accrue after the initial system release S0 C1 S1 C2 S2 SI SN CN
40. How can the architecture deteriorate over the lifecycle of a system?
41. Architecture deterioration drivers • Changes in business domain • Involuntary inadmissible implementation • Focus on implementation efforts only • Accumulated “technical debt” (quick-fixes, workarounds, ...) and more
42. We need to reassess our architecture over the lifecycle of a system regularly
43. Uncertainty Certainty Approach summary Project start Project ongoing Maintenance Cost “Agile” Flow Feedback Innovation BDUF Big Design UpFront BDUF Complete design Duration: ~weeks --- Reassessment Frequency: ~2y Duration: ~days Decent DUF & occasional updates DDUF Most of design Duration: ~days Missing details added if needed Frequency: > month Duration: ~hours Reassessment Frequency: ~2y Duration: ~days JDUF Quick pass (“Just enough”) Duration: < day Just enough DUF & continuous evolution „Whatever works“ (Minimal agreement) Discuss, complement and reassess continuously along the way Frequency: ~weekly Duration: < hour Just enough agreement to get things done fast (90%+ of all written code will be deleted anyway) --- Approach not useful for systems in maintenance “Too early for IT” ---
44. Additional influencing factors • Time and budget constraints • Availability of people (who and when) • Available skills, knowledge and experience • Risk of resulting system (e.g., can harm human lives) • …
45. Wrap-up
46. Wrap-up • Align architectural work with uncertainty • Should fit the chosen software development approach • Take the lifecycle into account • Most architectural efforts only target initial development • Reassess regularly to avoid architecture deterioration • There is no “one-size-fits-all”
47. Uwe Friedrichsen Works @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://ufried.com/