Essential architectural thinking

Based on what I see there are tons of advice, opinions and recommendations how to do architectural work. Unfortunately, many of them contradict each other. I think one of the core reasons for this unfortunate situation is that we forgot to ask "Why": Why are we doing architectural work in the first place? What are the goals of architectural work (besides having it done because "it is important")?

Thus, I start this slide deck with the "Why" question and try to answer this question. Based on that answer, I move on to the "What", i.e., what work we need to do in order to satisfy the "Why". Following the "What"-question, I come up with a 4E-model, describing the four pillars of architectural work as I understand it. I complement the 4 pillars with some tools and ideas that I found helpful in my career so far.

Finally, I move on to the "How", or to be more precise to "When to do what and how much of it". There is a widespread tendency to follow only one approach, being more waterfall-like in the past, being more "Agile" today. And these approaches are usually defended with religious zeal.

Unlike the aforementioned tendencies I recommend a different approach: Based on the uncertainty of the project, I prefer different approaches that meet the given situation best.

Therefore, I first explain the effects of uncertainty and how the typical "Agile" implementations fit in there. Then, I derive a range of software development approaches based on the given uncertainty. Finally, I map the architectural work activities on the different development approaches.

Of course, I do not claim that the ideas shared in this slide deck are the ultimate truth (or even close). Instead, you should take it as a, well, "idea", a point of view that you can use, augment or reject as you like.

However, I hope the slides are of some use for you, even if - as always - the voice track is missing.

Transcript

1. Essential architectural thinking Why, what, when, how and how much? Uwe Friedrichsen – codecentric AG – 2006-2019
2. Uwe Friedrichsen CTO @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://medium.com/@ufried
3. Why we need architectural work
4. Architectural work without asking “why” …
5. BDUF architecture Big Design Upfront Stack-overflow architecture closely related to Conference-driven architecture Google-driven architecture “Agile” architecture a.k.a. “Emergent” architecture often in conjunction with “Death to all architects!” Developer-centric architecture “Strategic” architecture often found in conjunction with the “Powerpoint architect” “Architect” – step on the company career ladder working as fire accelerant … and many more anti-patterns
6. … results in a mess
7. And how do we try to address the problem?
8. We tend to discuss what architecture is instead of asking us why we need it or how to work on architecture
9. … which results in even worse archetypes
10. Hype-driven architecture
11. Tunnel-vision architecture
12. Blast-from-the-past architecture
13. One-size-fits-all architecture
14. Accidental architecture
16. We need to ask why to focus our work Without asking why the value of our work will be accidental
17. Let’s (re-)establish focus
18. Why do we need architecture? • We need architecture to address • an optimization problem over time • with changing constraints
19. Eh, WTF?
20. Architecture attempts to satisfy quality attributes
21. see, e.g., ISO 25010, ISO 9126 or SEI for comprehensive collections of quality attributes Maintainability Changeability Availability Performance Scalability Functionality Security … Understandability Accessibility Reliability Usability Portability Operability
22. Quality attributes can be segmented in two general classes 1. Development (incl. build & test) related attributes 2. Runtime (usage and operations) related attributes
23. Development related attributes • Describe the desired behavior of a system from a development perspective • Influence how efficiently the system can be modified • Target “cost of change” • Effectiveness can usually only be measured in hindsight • “Arcane guesswork” – subject of many (heated) debates à Optimization problem over time
24. Runtime related attributes • Describe the desired behavior of a system from a usage and operations perspective • Target “correctness” • Constrain the solution options • Support evaluation of solution options • May change over time à Changing constraints
25. Correctness of behavior Runtime (usage and operations) related attributes Cost of change Development related attributes Maintainability Changeability Availability Performance Scalability Functionality Security … Understandability Accessibility Reliability Usability Portability Operability
26. Note that cost of change is not sufficient as an optimization goal as it neglects all other relevant types of costs. Thus, the optimization goal for architectural work needs to target the cumulative costs instead. (unfortunately, usually not covered by quality attributes)
27. Why do we need architecture? • With architecture we attempt • to minimize the cumulative costs of a system over its lifetime • while ensuring the correctness of behavior at runtime • On a side note: What is architecture? • Architecture is whatever is needed to implement the required quality attributes (i.e., achieve the why)
28. What the essential activities are
29. 1ST LAW OF ARCHITECTURAL WORK EVERY DECISION HAS ITS PRICE NO DECISION IS FOR FREE (NO DECISION ONLY HAS UPSIDES. EVERY DECISION ALSO HAS DOWNSIDES)
30. 2ND LAW OF ARCHITECTURAL WORK A DECISION CAN ONLY BE EVALUATED WITH RESPECT TO ITS CONTEXT (DECISIONS ARE NOT INVARIABLY “GOOD” OR “BAD”, BUT ONLY IN A GIVEN CONTEXT)
31. The 4E model of architectural work
32. The 4E model of architectural work Explore
33. Explore Find solution options
34. Explore • About designing solution options • Designing structures and behavior • Combining frameworks, tools, technologies, … • Important activity • Always done, either explicitly or implicitly • Architectural work often reduced to Explore activities • Unclear if and how solution delivers to the Why
35. The 4E model of architectural work Explore Execute
36. Execute Help stakeholders to make the best possible decisions
37. Execute • About making decisions • Often not made by architect, but by other stakeholders • Communication, collaboration, convincing, … • Very important activity • Need to collaborate with all stakeholder groups • Learn their needs, points of view, language, … • Do not forget dev’s and ops as important stakeholder groups
38. The 4E model of architectural work Explore Evaluate Execute
39. Evaluate Identify the trade-offs
40. Evaluate • About trade-offs • Comparing solution options • How well NFRs are satisfied • Essential activity • Delivers to Why and 1st law of architectural work • Needed for Execute • Keep different perspectives in mind
41. The 4E model of architectural work Examine Explore Evaluate Execute
42. Examine Understand the problem
43. Examine • About understanding the problem and context • Examine problem space holistically • Functionality, NFRs, time, costs, risks, constraints, … • The basis of all other activities • Provides focus and context • Delivers to Why and 2nd law of architectural work • Needed for Explore and Evaluate
44. Examine Explore Evaluate Execute
45. Examine (1/3) • Understand the degree of uncertainty • What is the business model context? • How good does the customer understand the problem? • How often do they change their mind? • Have you done something similar before? • Do you have a specification/backlog? How specific is it? • Do you already have an IT solution for the problem? • Are success criteria defined? How do you measure them? • …
46. Examine (2/3) • Understand the business problem • Hypothesis collection and confidence check • Event storming • Actors and use cases • Dissect requirements document • Analyze existing IT solution • Interview key users • …
47. Examine (3/3) • Understand architectural context • Context view (incl. all actors, external systems and interfaces) • Quality tree with prioritized quality attributes • Collect key figures • Gather known constraints (financial, organizational, legal, …) • Assess external system interfaces • … • One or more workshops with required stakeholders
48. Explore (1/3) • The creative part of the work • Lots of material available • Yet, little advice regarding the actual task • Usually just lots of basics and (unfortunately) opinions • Designing good options is hard • Requires broad knowledge and experience • Few recommendations here that worked well for me …
49. Explore (2/3) • Explore variability axes to identify options, e.g. • Cloud vs. on-premise • Make vs. buy (OSS vs. managed service) • Architectural styles, communication styles, integration styles • … • About creating a high-level design • Understand required functionality and the business domain • The magic lies in the behavior, not the static structure! • Prefer architectural styles over blueprints
50. Explore (3/3) • About creating a set of options (max. 3 options) • Use constraints and key figures to reduce options • Take your environment into account • Document rationale (why you selected an option) • Use spikes and prototypes as exploration tools • Have evolvability in mind • But balance flexibility and understandability
51. Evaluate (1/2) • Understand scenario based analysis • E.g., ATAM (and how to strip it down) • Consider evaluation matrix for summary overview • Use SWOT as an augmenting risk evaluation tool • Helps to balance short-term and long-term consequences • Take the financial consequences of choices into account • Consider all dimensions: development, testing, deployment, operations, maintenance, licenses, infrastructure, usage, administration, revenue (e.g., windows of opportunity), … • Do not forget the long-term financial consequences!
52. Evaluate (2/2) • Take additional constraints into account • E.g., market situation, required deadlines, available skills • Evaluate with focus on all stakeholders • Especially do not forget consequences for operations • Have a “cheat sheet” for popular options • General usage scenarios, costs, risks and trade-offs • Use spikes and prototypes to leverage risk • Automated evaluation as an outlook * * https://www.thoughtworks.com/insights/blog/fitness-function-driven-development
53. Execute (1/2) • Know how to create a recommendation • Be familiar with decision templates and how to tweak them • Learn to present and discuss a recommendation • Try to understand biases and concerns upfront • Adjust language and arguments to your stakeholder group • Communicate all perspectives – value, risks and trade-offs • “Give 'em all the info and then let 'em decide”
54. Execute (2/2) • Accept that it (usually) is neither your risk nor your money • Thus accept the decisions of the risk takers and sponsors • If people decide against your recommendations • Stay supportive and sympathetic • Be authentic • Use time as an ally • Using code as design documentation • Great for developers, shitty for everyone else …
55. When to do what and how much
56. Understanding uncertainty
57. Uncertainty breaks the linear relation between effort and value
58. Eh, WTF?
59. Internally controlled Externally controlled Effort spent Units produced / Output Certainty-driven value prediction model (a.k.a. industrial value prediction model) Value created Units sold / Outcome Linear relation due to certainty We are looking for this Due to the linear relation it is sufficient to plan effort in order to predict value
60. Internally controlled Externally controlled Effort spent Units produced / Output The effect of uncertainty Value created Units sold / Outcome We are still looking for this Influencing factors outside our control We cannot predict it reliably upfront We can only observe it after the fact No longer suitable to predict value Breaks linear relation Uncertainty ?
61. Internally controlled Externally controlled Effort spent Units produced / Output The effect of uncertainty Under uncertainty we cannot predict upfront which kind of performance we are going to produce Support performance Idle performance No value Value-adding performance Creates value Value-reducing performance Destroys value
62. Uncertainty means that you cannot predict the value that will result from your effort
63. Acting under uncertainty 1. Create a hypothesis regarding the effect of an effort 2. Do smallest action suitable to measure an effect 3. Measure effect and evaluate hypothesis a. Further develop hypothesis if expectations met b. Drop hypothesis otherwise (optionally pivot) 4. Repeat
64. Under uncertainty you do not maximize value by optimizing efficiency of efforts (a.k.a. cost efficiency), but by detecting and cutting idle and value-reducing performances as soon as possible
65. Uncertainty Certainty • Extensive effort planning • Maximize efficiency • Detailed controlling • Hypotheses validation • Small steps • Quick feedback cycles Value maximization approaches
66. Uncertainty drivers in IT • Complexity of context and tasks • Post-industrial (& startup) markets • Digital transformation (a.k.a. digitization, digitalization) • Disruptive technologies • …
67. Types of uncertainty • Due to homework not done • Accidental, complicated, avoidable • Lack of diligence • Due to missing internal feedback • Semi-accidental, complex, harder to avoid • Rooted in company culture deficiencies • Due to missing external feedback • Essential, complex, not avoidable • Cannot be solved internally
68. Uncertainty and “Agile” • Due to homework not done • Accidental, complicated, avoidable • Lack of diligence • Due to missing internal feedback • Semi-accidental, complex, harder to avoid • Rooted in company culture deficiencies • Due to missing external feedback • Essential, complex, not avoidable • Cannot be solved internally This is what Agile originally tried to address This is what Enterprise Agile * usually addresses This is what Enterprise Agile * often causes * Enterprise Agile: Typical “Agile” implementation in most companies
69. Agile as implemented in most companies does not address essential uncertainty
70. Uncertainty Certainty Invent potential new business offerings Grow a successfully launched new business offering Explore a potential new business offering Extend/improve an established business offering Create a new digital interaction point for an established business offering Implement new legal regulation Replace system that technologically reached EOL Typical tasks with varying uncertainty
71. Uncertainty Certainty Task examples with varying uncertainty “Create a better world for our children” (vision) Evolution of new Living 360 offering (insurance) Fruits4U (startup) Release of new Warp wireless plans (telecommunications) Subcontractor self-service app OTID (logistics) Implement the new customer documentation law (banking) Replacement of AS/400 based ERP AMS/400 (retailer)
72. Uncertainty Certainty Software development approaches Focus on cost (Waterfall, V-model, SAFe, …) Enterprise “Agile” (SCRUM, LeSS, …) Focus on flow (DevOps, Lean, …) Focus on feedback (Developer anarchy, …) Focus on innovation (No software development)
73. Uncertainty Certainty Software development approaches Cost “Agile” Flow Feedback Innovation BDUF Big Design UpFront All 4 activities completely upfront Rare reassessments Decent DUF & occasional updates All 4 activities largely upfront, missing details added along the way Rare reassessments Just enough DUF & continuous evolution Quick pass through all 4 activities upfront Then discuss, complement and re-evaluate continuously along the way „Whatever works“ (Minimal agreement) Just enough agreement to get things done fast (90%+ of all written code will be deleted anyway) Note: Approach not suitable for systems in maintenance “Too early for IT” ---
74. Uncertainty Certainty Software development approaches Cost “Agile” Flow Feedback Innovation “Create a better world for our children” (vision) Evolution of new Living 360 offering (insurance) Fruits4U (startup) Release of new Warp wireless plans (telecommunications) Subcontractor self-service app OTID (logistics) Implement the new customer documentation law (banking) Replacement of AS/400 based ERP AMS/400 (retailer)
75. Additional influencing factors • Time and budget constraints • Availability of people (who and when) • Available skills and knowledge • Risk of resulting system (e.g., can harm human lives) • …
76. Wrap-up
77. Wrap-up • Why architectural work • To minimize cumulative costs without violating correctness • How to do architectural work • 4E model: Examine, Explore, Evaluate, Execute • When to do architectural work and how much • Align your activities with uncertainty • There is no one-size-fits-all • Do what is needed, not what is fashionable
78. Uwe Friedrichsen CTO @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://medium.com/@ufried