Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Squaring the circle - Mastering the next level of architectural design

Squaring the circle - Mastering the next level of architectural design

In this presentation, I discuss how architectural work changed in the last 10-15 years due to new and changed drivers of various sorts.

I start with architectural work as it was typically done in the early 2000s. Then I briefly touch how architectural work mostly stood still through the agile confusion due to some widespread misunderstandings regarding architecture and architectural work. After that, I discuss 12 drivers of the last 10-15 years that changed how we need to address architecture and architectural work. Finally, I provide a few recommendations how we could address the new challenges and hopefully turn them into opportunities.

As this a really big topic, the presentation only scratches the surface - creating awareness regarding the changed challenges in architectural work at best. Additionally, the voice track is lacking. Still, I hope it provides some food for thought.

Uwe Friedrichsen

June 02, 2022
Tweet

More Decks by Uwe Friedrichsen

Other Decks in Technology

Transcript

  1. Squaring the circle
    Mastering the next level of architectural design
    Uwe Friedrichsen – codecentric AG – 2006-2022

    View full-size slide

  2. Uwe Friedrichsen
    Works @ codecentric
    https://twitter.com/ufried
    https://www.speakerdeck.com/ufried
    https://ufried.com/

    View full-size slide

  3. The “good ol’ times”
    Software architectural work until 2008

    View full-size slide

  4. Typical setting (1/2)
    • Many internally used systems
    • Relatively few external systems
    • Mostly monolithic systems
    • Rich frontend apps or server-side rendered web pages
    • On-premises data centers

    View full-size slide

  5. Typical setting (2/2)
    • Perimeter-based security
    • Pre-definable load patterns
    • “IT is a cost center” thinking
    • Effects of distribution handled by infrastructure

    View full-size slide

  6. Typical mindset
    • “Single process, single thread” thinking
    • Availability demands handled by infrastructure
    • Security demands handled by infrastructure
    • Project-scope thinking
    • Integration as 2nd class citizen

    View full-size slide

  7. Architectural work (1/3)
    • Design solution that satisfies project needs
    • Project cost efficiency as dominant design driver
    • Focus on system structure
    • Layered architecture was predominant

    View full-size slide

  8. Architectural work (2/3)
    • Availability, security, scaling, … left to Ops
    • Standard, often mandatory solutions offered by Ops
    • Often respective NFRs determined by solutions offered
    • Performance as 2nd class citizen
    • Usability and architecture completely unrelated
    • Integration as 2nd class citizen
    • Either solved ad hoc or via centralized integration solution

    View full-size slide

  9. Architectural work (3/3)
    • Core area of tension
    • Solution maintainability and changeability
    • Project cost efficiency
    • Speed of change usually constrained by project lead times
    • Typically measured in months or years

    View full-size slide

  10. Architectural work summary
    • Dominated by project constraints
    • Focus on system structure
    • Required architectural work quite well understood
    • At least in theory
    • Still not easy in practice
    • Many solutions from that time as witnesses

    View full-size slide

  11. Focus on structure
    Core area of tension
    Changeability
    Maintainability
    Flexibility
    Project efficiency
    Opposing goals
    Amplifying goals
    Usability
    Detached topic
    Performance
    Minor design
    consideration
    Availability
    Scalability
    Reliability
    Confidentiality Integrity
    Security
    “Ops problem”
    Dominant solution driver
    Portability
    Interoperability
    Minor design
    considerations

    View full-size slide

  12. Architectural work typically meant
    „Create building block design and decide technology” *
    * within the predefined constraints

    View full-size slide

  13. The age of confusion
    Software architectural work between 2008 and 2015

    View full-size slide

  14. The age of confusion
    • The IT world went “Agile”
    • Developers were “The new kingmakers” *
    • Countermovement to the architecture-centric movement
    of the 1990s
    * https://www.oreilly.com/library/view/the-new-kingmakers/9781449368036/

    View full-size slide

  15. “We do not need architects. The dev team is the architect.”
    “We do not need architectural work. Architecture emerges.”

    View full-size slide

  16. IT changed a lot in those days

    View full-size slide

  17. Architectural work mostly stood still

    View full-size slide

  18. Focus on structure
    Core area of tension
    Changeability
    Maintainability
    Flexibility
    Project efficiency
    Opposing goals
    Amplifying goals
    Usability
    Detached topic
    Performance
    Minor design
    consideration
    Availability
    Scalability
    Reliability
    Confidentiality Integrity
    Security
    “Ops problem”
    Dominant solution driver
    Portability
    Interoperability
    Minor design
    considerations
    We were still here

    View full-size slide

  19. Or sometimes even worse
    “From 0 to legacy in just 6 months”

    View full-size slide

  20. The present
    Software architectural work since 2015

    View full-size slide

  21. Novel drivers (1/3)
    • Cloud revolution *
    • Completely new solution options
    • Pay per use
    • Elasticity
    • Moving data around has a price tag
    • Availability and reliability on non-HA hardware
    * https://www.ufried.com/blog/public_cloud_revolution_1/

    View full-size slide

  22. Changeability
    Maintainability
    Usability
    Performance
    Availability
    Scalability
    Reliability
    Confidentiality Integrity
    Security
    Portability
    Interoperability
    Flexibility
    Cloud revolution
    Project efficiency
    Opposing goals
    Amplifying goals
    Suitability
    New solution options
    Pay per use
    Interoperability
    Data flow
    Economic efficiency
    Data flow
    Elasticity
    Elasticity
    Scalability
    Resilience
    Availability
    Reliability
    Scale out
    on non-HA
    hardware
    Project efficiency

    View full-size slide

  23. Novel drivers (1/3)
    • Cloud revolution
    • Big data & ML/AI
    • Huge amounts of data to store, move and process
    • Data flow becoming 1st class citizen of system design
    • Polyglot persistence
    • MLaaS/AIaaS

    View full-size slide

  24. Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    Big data & ML/AI
    Project efficiency
    Suitability
    Data flow
    Resilience
    Project efficiency
    Economic efficiency
    Move, store &
    process data
    Mlaas/AIaas
    Economic efficiency
    Availability
    Scalability
    Reliability
    Interoperability
    Performance
    Process &
    move data
    Polyglot persistence
    Suitability
    Processing options
    Interoperability
    Move data
    Scalability
    Move, store &
    process data
    Store data
    Reliability
    Availability

    View full-size slide

  25. Novel drivers (1/3)
    • Cloud revolution
    • Big data & ML/AI
    • Mobile devices
    • Completely new interaction patterns
    • Various novel device classes
    • Less reliable and often limited connectivity
    • Unpredictable load patterns

    View full-size slide

  26. Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    Mobile devices
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Reliability
    Interoperability
    Performance
    Limited battery
    Scalability
    Unpredictable
    usage patterns
    Suitability
    New interaction patterns
    Usability
    Increased user expectations
    Adaptability
    Novel device classes
    Portability
    Novel attack vectors
    Security
    Confidentiality Integrity
    Availability
    Dependability
    Less reliable
    connectivity
    Versatility
    Manifold subclasses
    Interoperability
    Less reliable
    connectivity
    Changeability
    Increased user expectations

    View full-size slide

  27. Novel drivers (1/3)
    • Cloud revolution
    • Big data & ML/AI
    • Mobile devices
    • IoT & Edge
    • Most of the effects of mobile devices “to the square”
    • Solutions enter safety-critical domains

    View full-size slide

  28. Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    IoT & Edge
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Reliability
    Interoperability
    Performance
    Limited battery
    Suitability
    New interaction patterns
    Usability
    Novel device classes
    Portability
    Novel attack vectors Security
    Confidentiality Integrity
    Availability
    Dependability
    Less reliable
    connectivity
    Versatility
    Manifold subclasses
    Interoperability
    Less reliable
    connectivity
    Adaptability
    Safety Safety-critical
    applications

    View full-size slide

  29. Novel drivers (2/3)
    • Distributed systems
    • Highly connected, distributed system landscape
    • Service-based applications, …
    • Intricate new failure types need to be handled
    • Effects of distribution penetrate to the application level
    • Near-realtime response expectations intensify effects

    View full-size slide

  30. Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    Distributed systems
    Project efficiency
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Performance
    Latency
    Confidentiality Integrity
    Adaptability
    Safety
    Reliability
    Remote
    communication
    Security
    Safety
    Availability
    Dependability
    Distributed
    failure modes
    Reliability
    Resilience
    Interoperability
    Suitability
    Communication flow
    Usability
    Suitability
    Communication flow
    Interoperability
    Observability
    Distributed
    failure modes

    View full-size slide

  31. Novel drivers (2/3)
    • Distributed systems
    • Near-realtime
    • Batch updates turn into online updates
    • Complex web of continuously communicating systems
    • Integration becomes first-class citizen
    • Near-realtime insights feedback loop
    • Intensifies effects of system distribution

    View full-size slide

  32. Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    Near-realtime
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Performance
    Latency
    Confidentiality Integrity
    Adaptability
    Safety
    Reliability
    Availability
    Dependability
    Distribution
    failure modes
    Reliability
    Resilience
    Interoperability
    Integration as
    1st class citizen
    Observability

    View full-size slide

  33. Novel drivers (2/3)
    • Distributed systems
    • Near-realtime
    • 24x7
    • Maintenance windows are gone
    • Overnight batch processing windows are gone
    • Upgrade/repair systems in-flight

    View full-size slide

  34. Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    24x7
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality Integrity
    Adaptability
    Safety
    Reliability
    Availability
    Dependability
    In-flight system
    upgrade/repair
    Reliability
    Observability
    In-flight system
    upgrade/repair
    Observability
    Batch windows gone
    Performance

    View full-size slide

  35. Novel drivers (2/3)
    • Distributed systems
    • Near-realtime
    • 24x7
    • Perimeter-less security
    • Trusted zones are gone: “The Internet is everywhere”
    • Remote work, BYOD, mobile devices, etc. intensify challenges
    • “Shift left security” mandatory

    View full-size slide

  36. Observability
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Perimeter-less security
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    Observability
    Trusted zones gone
    Integrity
    Trusted zones gone
    Security
    Confidentiality

    View full-size slide

  37. Novel drivers (3/3)
    • Digital transformation
    • “IT and business are the same side of the same coin;
    the other side are the market and your customers”
    • IT being integral part of all business offerings
    • IT being integral part of most customer interactions
    • IT enabling new, previously unthinkable business models
    • IT becoming indispensable part of business and private lives

    View full-size slide

  38. Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Digital transformation
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    New business models Adaptability
    System lifecycle
    Products,
    not projects
    Economic efficiency
    Availability
    Dependability
    Integral part of
    business offerings
    Reliability
    Resilience
    Scalability
    Usability
    Suitability
    Integral part of
    customer interactions
    Integrity
    Security
    Confidentiality
    Safety
    Indispensable
    part of our lives
    Interoperability
    API-driven
    revenue streams
    Changeability
    New business models
    Flexibility
    Maintainability
    Human needs
    Socio-technical systems
    Affecting
    humans

    View full-size slide

  39. Novel drivers (3/3)
    • Digital transformation
    • Post-industrial markets
    • Supply exceeds demand by far à customer-driven markets
    • Fast adoption to changing demands becomes vital
    • Lead and cycle times become essential
    • Continuous feedback on all levels needed
    • More complex multidisciplinary collaboration needed

    View full-size slide

  40. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Post-industrial markets
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    Fast adoption Adaptability
    Human needs
    Socio-technical systems
    Multidisciplinary
    collaboration
    Changeability
    Fast adoption
    Flexibility
    Maintainability
    Observability
    Observability Continuous feedback
    Usability
    Suitability
    Customer-driven market
    System lifecycle

    View full-size slide

  41. Novel drivers (3/3)
    • Digital transformation
    • Post-industrial markets
    • Exploding complexity
    • Probably the 2nd biggest challenge of the upcoming years
    • IT drowns in complexity and every day it becomes worse
    • Due to shortsighted thinking
    • Due to important decisions not made

    View full-size slide

  42. Gregor’s Law: Excessive complexity is nature’s punishment
    for organizations that are unable to make decisions.
    -- Gregor Hohpe
    Source: https://architectelevator.com/architecture/it-complexity/

    View full-size slide

  43. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Exploding complexity
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    System lifecycle
    Simplicity
    Tackling complexity
    System empathy
    System viability
    and collaboration

    View full-size slide

  44. Novel drivers (3/3)
    • Digital transformation
    • Post-industrial markets
    • Exploding complexity
    • Ecological sustainability
    • Probably the biggest challenge of the upcoming years
    • Making sustainability 1st class citizen of system design
    • Counterforce to many other forces

    View full-size slide

  45. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Ecological sustainability
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    System lifecycle
    Fostering sustainability
    Sustainability
    Simplicity
    System empathy

    View full-size slide

  46. Finally, let us add the amplifying and opposing forces

    View full-size slide

  47. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Opposing goals
    Amplifying goals
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    System lifecycle
    Simplicity
    System empathy
    Sustainability
    Project efficiency
    … and many more

    View full-size slide

  48. All quality goals need to be balanced
    according to the given needs *
    * needs of different stakeholder groups, short-term and long-term needs, amplifying vs. opposing goals, …

    View full-size slide

  49. All influencing forces and the derived quality goals
    vastly influence the resulting solution architecture

    View full-size slide

  50. Observations
    What has changed in the last decade?

    View full-size slide

  51. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Opposing goals
    Amplifying goals
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    System lifecycle
    Simplicity
    System empathy
    Sustainability
    Project efficiency
    Focus on structure
    not sufficient anymore
    Data flow
    Communication flow
    Limited view on projects
    not sufficient anymore
    Economic perspective
    over lifecycle as 1st
    class citizen
    Prevalent complexity of
    IT system landscapes
    requires new design
    approaches
    Exploding number
    of device classes
    Integration as
    1st class citizen
    Thinking in
    feedback loops
    Holistic view required.
    Focus on software alone
    not sufficient anymore
    Novel drivers of future system
    design. Counterforces to many
    other design forces
    Enterprise software
    affects safety-critical
    systems more and more
    Security as 1st class
    citizen in design
    (“Shift left security”)
    Huge new
    challenges due
    to highly
    interconnected
    distributed
    system
    landscapes
    (“microservices”)
    Unpredictable
    load patterns
    Massively increased user
    expectations influence
    system design
    Technology evolution provides
    new options and changes what
    a feasible solution is

    View full-size slide

  52. Lots of new forces. Lots of new tasks and challenges.
    Partially completely novel forces and resulting tasks.

    View full-size slide

  53. Reinforces long know, yet still unsolved problems

    View full-size slide

  54. Unsolved problems
    • Getting functional design right (“modularization”)
    • Pondering behavior and data flow, not just structure
    • Systemic thinking instead of local optimizations
    • Balancing long-term and short-term effects
    • Understanding distributed systems
    • …

    View full-size slide

  55. This is a completely
    different game ..
    … than that

    View full-size slide

  56. Useful side effect
    It is acceptable again to talk about architecture *
    * and use words like “architect” or “architectural work” … ;)

    View full-size slide

  57. Moving on
    Addressing the new challenges

    View full-size slide

  58. The collaboration perspective

    View full-size slide

  59. “Emergent architecture” is dead

    View full-size slide

  60. Tackling the challenges of today
    requires hard work and careful decision making

    View full-size slide

  61. Does that mean the “all-knowing” architect is back?

    View full-size slide

  62. No!
    But you need the skills and someone who drives the architectural work that needs to be done.

    View full-size slide

  63. Even the smartest architect imaginable will not be able
    to address all these challenges alone

    View full-size slide

  64. In the end, architectural work always is a team effort

    View full-size slide

  65. Does that mean the BDUF * is back?
    * Big Design UpFront

    View full-size slide

  66. No!
    But you need to have an idea where the goal is before you start running.

    View full-size slide

  67. Implementation without any
    upfront architectural work
    Implementation with some
    upfront architectural work

    View full-size slide

  68. Sometimes, a new or previously undetected requirement
    may take us by surprise

    View full-size slide

  69. But basically, your architecture will evolve over time

    View full-size slide

  70. Never forget Gall’s Law

    View full-size slide

  71. “A complex system designed from scratch never works
    and cannot be made to work. You have to start over,
    beginning with a working simple system.”
    -- John Gall
    Source: John Gall, "Systemantics", 1975

    View full-size slide

  72. The skills perspective

    View full-size slide

  73. Learn about architectural work
    • Understand the problem from all perspectives
    • Find alternative solution options
    • Evaluate the trade-offs
    • Help stakeholders make best decisions possible

    View full-size slide

  74. Improve your RE skills
    • Need to understand demands and consequences
    • Impeded by widespread division-of-labor approach
    • Help find requirements that do not harm the design
    • Often, alternatives with same outcome are possible
    • If not, you need to change the design of course
    • Uncover the actual “needs” behind the “wants”
    • Helps to avoid lots of accidental complexity

    View full-size slide

  75. Learn better functional design
    • Especially master the foundations of good design
    • “Loose coupling, high cohesion” & “separation of concerns”
    • “Information hiding” especially with respect to APIs
    • Domain-driven design may be a useful starting point
    • Good design skills are more relevant than ever
    • Affect understandability, changeability and extensibility
    • Affect usability, stability and acceptance at API level
    • Affect robustness, availability and scalability at runtime

    View full-size slide

  76. “I would advise students to pay more attention to the
    fundamental ideas rather than the latest technology.
    The technology will be out-of-date before they graduate.
    Fundamental ideas never get out of date.”
    -- David L. Parnas (http://www.sigsoft.org/SEN/parnas.html)

    View full-size slide

  77. Embrace distributed systems
    “(Almost) every system is a distributed system” -- Chas Emerick
    “Everything fails, all the time” -- Werner Vogels
    • Understand the non-determinism of distributed systems
    • “Memory, guesses and apologies” & promise theory
    • Understand that infrastructure cannot guarantee robustness

    View full-size slide

  78. Help people understand software
    • Not comparable to physical goods
    • Needs to be changed after initial release to keep its value
    • Software development is design, not shop floor assembly
    • Extreme malleability of software can turn into a curse
    • Invisibility makes it very hard to grasp “software”

    View full-size slide

  79. Understand the new challenges
    • Simplicity – tackling ever-growing complexity
    • System empathy – accepting heterogeneity
    • Dependability – making sure we all can rely on IT
    • Resilience – being prepared for the unexpected
    • Safety – dealing with the growing blast radius
    • Security – indispensable for dependable system
    • Sustainability – reducing the ecological footprint of IT

    View full-size slide

  80. Wrap-up
    • Architectural challenges have changed a lot
    • Complexity went up significantly
    • New counterforces have emerged
    • Decision making has become a lot more demanding
    • Updated perception of architectural work needed
    • New collaboration modes required
    • Different skills have become vital

    View full-size slide

  81. "Ultimately, design is a tool to enhance our humanity”
    -- Ilse Crawford

    View full-size slide

  82. Uwe Friedrichsen
    Works @ codecentric
    https://twitter.com/ufried
    https://www.speakerdeck.com/ufried
    https://ufried.com/

    View full-size slide