Essential architectural thinking

Based on what I see there are tons of advice, opinions and recommendations how to do architectural work. Unfortunately, many of them contradict each other. I think one of the core reasons for this unfortunate situation is that we forgot to ask "Why": Why are we doing architectural work in the first place? What are the goals of architectural work (besides having it done because "it is important")?

Thus, I start this slide deck with the "Why" question and try to answer this question. Based on that answer, I move on to the "What", i.e., what work we need to do in order to satisfy the "Why". Following the "What"-question, I come up with a 4E-model, describing the four pillars of architectural work as I understand it. I complement the 4 pillars with some tools and ideas that I found helpful in my career so far.

Finally, I move on to the "How", or to be more precise to "When to do what and how much of it". There is a widespread tendency to follow only one approach, being more waterfall-like in the past, being more "Agile" today. And these approaches are usually defended with religious zeal.

Unlike the aforementioned tendencies I recommend a different approach: Based on the uncertainty of the project, I prefer different approaches that meet the given situation best.

Therefore, I first explain the effects of uncertainty and how the typical "Agile" implementations fit in there. Then, I derive a range of software development approaches based on the given uncertainty. Finally, I map the architectural work activities on the different development approaches.

Of course, I do not claim that the ideas shared in this slide deck are the ultimate truth (or even close). Instead, you should take it as a, well, "idea", a point of view that you can use, augment or reject as you like.

However, I hope the slides are of some use for you, even if - as always - the voice track is missing.

Transcript

1. Essential architectural thinking
   Why, what, when, how and how much?
   Uwe Friedrichsen – codecentric AG – 2006-2019
   

   View Slide

2. Uwe Friedrichsen
   CTO @ codecentric
   https://twitter.com/ufried
   https://www.speakerdeck.com/ufried
   https://medium.com/@ufried
   

   View Slide

3. Why we need architectural work
   

   View Slide

4. Architectural work without asking “why” …
   

   View Slide

5. BDUF architecture
   Big Design Upfront
   Stack-overflow architecture
   closely related to
   Conference-driven architecture
   Google-driven architecture
   “Agile” architecture
   a.k.a.
   “Emergent” architecture
   often in conjunction with
   “Death to all architects!”
   Developer-centric architecture
   “Strategic” architecture
   often found in conjunction with
   the “Powerpoint architect”
   “Architect” – step
   on the company
   career ladder
   working as fire accelerant
   …
   and many more anti-patterns
   

   View Slide

6. … results in a mess
   

   View Slide

7. And how do we try to address the problem?
   

   View Slide

8. We tend to discuss what architecture is
   instead of asking us why we need it
   or how to work on architecture
   

   View Slide

9. … which results in even worse archetypes
   

   View Slide

10. Hype-driven architecture
    

    View Slide

11. Tunnel-vision architecture
    

    View Slide

12. Blast-from-the-past architecture
    

    View Slide

13. One-size-fits-all architecture
    

    View Slide

14. Accidental architecture
    

    View Slide

15. View Slide

16. We need to ask why to focus our work
    Without asking why the value of our work will be accidental
    

    View Slide

17. Let’s (re-)establish focus
    

    View Slide

18. Why do we need architecture?
    • We need architecture to address
    • an optimization problem over time
    • with changing constraints
    

    View Slide

19. Eh, WTF?
    

    View Slide

20. Architecture attempts to satisfy quality attributes
    

    View Slide

21. see, e.g., ISO 25010, ISO 9126 or SEI for comprehensive collections of quality attributes
    Maintainability
    Changeability
    Availability
    Performance
    Scalability
    Functionality
    Security
    …
    Understandability
    Accessibility
    Reliability
    Usability
    Portability
    Operability
    

    View Slide

22. Quality attributes can be segmented in two general classes
    1. Development (incl. build & test) related attributes
    2. Runtime (usage and operations) related attributes
    

    View Slide

23. Development related attributes
    • Describe the desired behavior of a system
    from a development perspective
    • Influence how efficiently the system can be modified
    • Target “cost of change”
    • Effectiveness can usually only be measured in hindsight
    • “Arcane guesswork” – subject of many (heated) debates
    à Optimization problem over time
    

    View Slide

24. Runtime related attributes
    • Describe the desired behavior of a system
    from a usage and operations perspective
    • Target “correctness”
    • Constrain the solution options
    • Support evaluation of solution options
    • May change over time
    à Changing constraints
    

    View Slide

25. Correctness of behavior
    Runtime (usage and operations) related attributes
    Cost of change
    Development related attributes
    Maintainability
    Changeability
    Availability
    Performance
    Scalability
    Functionality
    Security
    …
    Understandability
    Accessibility
    Reliability
    Usability
    Portability
    Operability
    

    View Slide

26. Note that cost of change is not sufficient as an optimization goal
    as it neglects all other relevant types of costs.
    Thus, the optimization goal for architectural work
    needs to target the cumulative costs instead.
    (unfortunately, usually not covered by quality attributes)
    

    View Slide

27. Why do we need architecture?
    • With architecture we attempt
    • to minimize the cumulative costs of a system over its lifetime
    • while ensuring the correctness of behavior at runtime
    • On a side note: What is architecture?
    • Architecture is whatever is needed to implement
    the required quality attributes (i.e., achieve the why)
    

    View Slide

28. What the essential activities are
    

    View Slide

29. 1ST LAW OF ARCHITECTURAL WORK
    EVERY DECISION HAS ITS PRICE
    NO DECISION IS FOR FREE
    (NO DECISION ONLY HAS UPSIDES. EVERY DECISION ALSO HAS DOWNSIDES)
    

    View Slide

30. 2ND LAW OF ARCHITECTURAL WORK
    A DECISION CAN ONLY BE EVALUATED
    WITH RESPECT TO ITS CONTEXT
    (DECISIONS ARE NOT INVARIABLY “GOOD” OR “BAD”, BUT ONLY IN A GIVEN CONTEXT)
    

    View Slide

31. The 4E model of architectural work
    

    View Slide

32. The 4E model of architectural work
    Explore
    

    View Slide

33. Explore
    Find solution options
    

    View Slide

34. Explore
    • About designing solution options
    • Designing structures and behavior
    • Combining frameworks, tools, technologies, …
    • Important activity
    • Always done, either explicitly or implicitly
    • Architectural work often reduced to Explore activities
    • Unclear if and how solution delivers to the Why
    

    View Slide

35. The 4E model of architectural work
    Explore
    Execute
    

    View Slide

36. Execute
    Help stakeholders to make the best possible decisions
    

    View Slide

37. Execute
    • About making decisions
    • Often not made by architect, but by other stakeholders
    • Communication, collaboration, convincing, …
    • Very important activity
    • Need to collaborate with all stakeholder groups
    • Learn their needs, points of view, language, …
    • Do not forget dev’s and ops as important stakeholder groups
    

    View Slide

38. The 4E model of architectural work
    Explore
    Evaluate
    Execute
    

    View Slide

39. Evaluate
    Identify the trade-offs
    

    View Slide

40. Evaluate
    • About trade-offs
    • Comparing solution options
    • How well NFRs are satisfied
    • Essential activity
    • Delivers to Why and 1st law of architectural work
    • Needed for Execute
    • Keep different perspectives in mind
    

    View Slide

41. The 4E model of architectural work
    Examine Explore
    Evaluate
    Execute
    

    View Slide

42. Examine
    Understand the problem
    

    View Slide

43. Examine
    • About understanding the problem and context
    • Examine problem space holistically
    • Functionality, NFRs, time, costs, risks, constraints, …
    • The basis of all other activities
    • Provides focus and context
    • Delivers to Why and 2nd law of architectural work
    • Needed for Explore and Evaluate
    

    View Slide

44. Examine Explore
    Evaluate
    Execute
    

    View Slide

45. Examine (1/3)
    • Understand the degree of uncertainty
    • What is the business model context?
    • How good does the customer understand the problem?
    • How often do they change their mind?
    • Have you done something similar before?
    • Do you have a specification/backlog? How specific is it?
    • Do you already have an IT solution for the problem?
    • Are success criteria defined? How do you measure them?
    • …
    

    View Slide

46. Examine (2/3)
    • Understand the business problem
    • Hypothesis collection and confidence check
    • Event storming
    • Actors and use cases
    • Dissect requirements document
    • Analyze existing IT solution
    • Interview key users
    • …
    

    View Slide

47. Examine (3/3)
    • Understand architectural context
    • Context view (incl. all actors, external systems and interfaces)
    • Quality tree with prioritized quality attributes
    • Collect key figures
    • Gather known constraints (financial, organizational, legal, …)
    • Assess external system interfaces
    • …
    • One or more workshops with required stakeholders
    

    View Slide

48. Explore (1/3)
    • The creative part of the work
    • Lots of material available
    • Yet, little advice regarding the actual task
    • Usually just lots of basics and (unfortunately) opinions
    • Designing good options is hard
    • Requires broad knowledge and experience
    • Few recommendations here that worked well for me …
    

    View Slide

49. Explore (2/3)
    • Explore variability axes to identify options, e.g.
    • Cloud vs. on-premise
    • Make vs. buy (OSS vs. managed service)
    • Architectural styles, communication styles, integration styles
    • …
    • About creating a high-level design
    • Understand required functionality and the business domain
    • The magic lies in the behavior, not the static structure!
    • Prefer architectural styles over blueprints
    

    View Slide

50. Explore (3/3)
    • About creating a set of options (max. 3 options)
    • Use constraints and key figures to reduce options
    • Take your environment into account
    • Document rationale (why you selected an option)
    • Use spikes and prototypes as exploration tools
    • Have evolvability in mind
    • But balance flexibility and understandability
    

    View Slide

51. Evaluate (1/2)
    • Understand scenario based analysis
    • E.g., ATAM (and how to strip it down)
    • Consider evaluation matrix for summary overview
    • Use SWOT as an augmenting risk evaluation tool
    • Helps to balance short-term and long-term consequences
    • Take the financial consequences of choices into account
    • Consider all dimensions: development, testing, deployment,
    operations, maintenance, licenses, infrastructure, usage,
    administration, revenue (e.g., windows of opportunity), …
    • Do not forget the long-term financial consequences!
    

    View Slide

52. Evaluate (2/2)
    • Take additional constraints into account
    • E.g., market situation, required deadlines, available skills
    • Evaluate with focus on all stakeholders
    • Especially do not forget consequences for operations
    • Have a “cheat sheet” for popular options
    • General usage scenarios, costs, risks and trade-offs
    • Use spikes and prototypes to leverage risk
    • Automated evaluation as an outlook *
    * https://www.thoughtworks.com/insights/blog/fitness-function-driven-development
    

    View Slide

53. Execute (1/2)
    • Know how to create a recommendation
    • Be familiar with decision templates and how to tweak them
    • Learn to present and discuss a recommendation
    • Try to understand biases and concerns upfront
    • Adjust language and arguments to your stakeholder group
    • Communicate all perspectives – value, risks and trade-offs
    • “Give 'em all the info and then let 'em decide”
    

    View Slide

54. Execute (2/2)
    • Accept that it (usually) is neither your risk nor your money
    • Thus accept the decisions of the risk takers and sponsors
    • If people decide against your recommendations
    • Stay supportive and sympathetic
    • Be authentic
    • Use time as an ally
    • Using code as design documentation
    • Great for developers, shitty for everyone else …
    

    View Slide

55. When to do what and how much
    

    View Slide

56. Understanding uncertainty
    

    View Slide

57. Uncertainty breaks the linear relation
    between effort and value
    

    View Slide

58. Eh, WTF?
    

    View Slide

59. Internally controlled Externally controlled
    Effort spent
    Units produced / Output
    Certainty-driven value prediction model
    (a.k.a. industrial value prediction model)
    Value created
    Units sold / Outcome
    Linear relation due to certainty
    We are looking for this
    Due to the linear relation it
    is sufficient to plan effort
    in order to predict value
    

    View Slide

60. Internally controlled Externally controlled
    Effort spent
    Units produced / Output
    The effect of uncertainty
    Value created
    Units sold / Outcome
    We are still looking for this
    Influencing factors outside our control
    We cannot predict it reliably upfront
    We can only observe it after the fact
    No longer suitable
    to predict value
    Breaks linear relation
    Uncertainty ?
    

    View Slide

61. Internally controlled Externally controlled
    Effort spent
    Units produced / Output
    The effect of uncertainty
    Under uncertainty we cannot predict
    upfront which kind of performance
    we are going to produce
    Support
    performance
    Idle performance No value
    Value-adding
    performance
    Creates value
    Value-reducing
    performance
    Destroys value
    

    View Slide

62. Uncertainty means that you cannot predict
    the value that will result from your effort
    

    View Slide

63. Acting under uncertainty
    1. Create a hypothesis regarding the effect of an effort
    2. Do smallest action suitable to measure an effect
    3. Measure effect and evaluate hypothesis
    a. Further develop hypothesis if expectations met
    b. Drop hypothesis otherwise (optionally pivot)
    4. Repeat
    

    View Slide

64. Under uncertainty
    you do not maximize value
    by optimizing efficiency of efforts
    (a.k.a. cost efficiency),
    but by detecting and cutting
    idle and value-reducing performances
    as soon as possible
    

    View Slide

65. Uncertainty
    Certainty
    • Extensive effort planning
    • Maximize efficiency
    • Detailed controlling
    • Hypotheses validation
    • Small steps
    • Quick feedback cycles
    Value maximization approaches
    

    View Slide

66. Uncertainty drivers in IT
    • Complexity of context and tasks
    • Post-industrial (& startup) markets
    • Digital transformation (a.k.a. digitization, digitalization)
    • Disruptive technologies
    • …
    

    View Slide

67. Types of uncertainty
    • Due to homework not done
    • Accidental, complicated, avoidable
    • Lack of diligence
    • Due to missing internal feedback
    • Semi-accidental, complex, harder to avoid
    • Rooted in company culture deficiencies
    • Due to missing external feedback
    • Essential, complex, not avoidable
    • Cannot be solved internally
    

    View Slide

68. Uncertainty and “Agile”
    • Due to homework not done
    • Accidental, complicated, avoidable
    • Lack of diligence
    • Due to missing internal feedback
    • Semi-accidental, complex, harder to avoid
    • Rooted in company culture deficiencies
    • Due to missing external feedback
    • Essential, complex, not avoidable
    • Cannot be solved internally
    This is what Agile
    originally tried
    to address
    This is what
    Enterprise Agile *
    usually addresses
    This is what
    Enterprise Agile *
    often causes
    * Enterprise Agile: Typical “Agile” implementation in most companies
    

    View Slide

69. Agile as implemented in most companies
    does not address essential uncertainty
    

    View Slide

70. Uncertainty
    Certainty
    Invent potential new
    business offerings
    Grow a successfully
    launched new
    business offering
    Explore a
    potential new
    business offering
    Extend/improve
    an established
    business offering
    Create a new digital
    interaction point for an
    established business
    offering
    Implement
    new legal
    regulation
    Replace system
    that technologically
    reached EOL
    Typical tasks with varying uncertainty
    

    View Slide

71. Uncertainty
    Certainty
    Task examples with varying uncertainty
    “Create a better world
    for our children” (vision)
    Evolution of new
    Living 360 offering
    (insurance)
    Fruits4U (startup)
    Release of new Warp
    wireless plans
    (telecommunications)
    Subcontractor
    self-service app
    OTID (logistics)
    Implement the
    new customer
    documentation
    law (banking)
    Replacement of
    AS/400 based ERP
    AMS/400 (retailer)
    

    View Slide

72. Uncertainty
    Certainty
    Software development approaches
    Focus on cost
    (Waterfall, V-model, SAFe, …)
    Enterprise “Agile”
    (SCRUM, LeSS, …)
    Focus on flow
    (DevOps, Lean, …)
    Focus on feedback
    (Developer anarchy, …)
    Focus on innovation
    (No software development)
    

    View Slide

73. Uncertainty
    Certainty
    Software development approaches
    Cost “Agile” Flow Feedback Innovation
    BDUF
    Big Design
    UpFront
    All 4 activities
    completely
    upfront
    Rare
    reassessments
    Decent DUF &
    occasional
    updates
    All 4 activities
    largely upfront,
    missing details
    added along the
    way
    Rare
    reassessments
    Just enough DUF &
    continuous evolution
    Quick pass through all
    4 activities upfront
    Then discuss, complement
    and re-evaluate continuously
    along the way
    „Whatever works“
    (Minimal
    agreement)
    Just enough
    agreement to get
    things done fast
    (90%+ of all written
    code will be deleted
    anyway)
    Note: Approach not
    suitable for systems
    in maintenance
    “Too early for IT”
    ---
    

    View Slide

74. Uncertainty
    Certainty
    Software development approaches
    Cost “Agile” Flow Feedback Innovation
    “Create a better world
    for our children” (vision)
    Evolution of new
    Living 360 offering
    (insurance)
    Fruits4U (startup)
    Release of new Warp
    wireless plans
    (telecommunications)
    Subcontractor
    self-service app
    OTID (logistics)
    Implement the
    new customer
    documentation
    law (banking)
    Replacement of
    AS/400 based ERP
    AMS/400 (retailer)
    

    View Slide

75. Additional influencing factors
    • Time and budget constraints
    • Availability of people (who and when)
    • Available skills and knowledge
    • Risk of resulting system (e.g., can harm human lives)
    • …
    

    View Slide

76. Wrap-up
    

    View Slide

77. Wrap-up
    • Why architectural work
    • To minimize cumulative costs without violating correctness
    • How to do architectural work
    • 4E model: Examine, Explore, Evaluate, Execute
    • When to do architectural work and how much
    • Align your activities with uncertainty
    • There is no one-size-fits-all
    • Do what is needed, not what is fashionable
    

    View Slide

78. Uwe Friedrichsen
    CTO @ codecentric
    https://twitter.com/ufried
    https://www.speakerdeck.com/ufried
    https://medium.com/@ufried
    

    View Slide