Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Where do we go from here? – Mastering the changed needs of architectural work

Where do we go from here? – Mastering the changed needs of architectural work

This presentation discusses how the needs and demands regarding architectural work massively changed over the course of the past 10-15 years and how we can respond to it.

It first sketches architectural work in the past based on the typical settings back then. It then briefly covers why the agile movement (as valuable as it was and still is from many other perspectives) impeded properly adapting architectural work to a changing environment.

After that, the presentation outlines a number of changes in technology, system properties and overarching trends that massively influence architecture and architectural work, resulting in a new image what we need to take into account when doing architectural work (spoiler alert: the new image is very different from the original image and a lot more complex).

The presentation finally offers a few recommendations in terms of guiding principles how to address architecture and architectural work today to live up to the changed needs and demands.

As always, the voice track is missing. Still, I hope the slides offer a few useful insights and ideas to ponder.

Side note: This presentation is the successor of one of my prior presentations called "Squaring the circle". While the first part of the presentation only slightly changed, I completely rewrote the recommendations part (of course, there were also a lot of updates regarding the overall voice track but you won't see that on the slides).

Uwe Friedrichsen

February 01, 2024
Tweet

More Decks by Uwe Friedrichsen

Other Decks in Technology

Transcript

  1. Where do we go from here?
    Mastering the changed needs of architectural work
    Uwe Friedrichsen – codecentric AG – 2006-2024

    View full-size slide

  2. Uwe Friedrichsen
    Works @ codecentric
    https://twitter.com/ufried
    https://www.speakerdeck.com/ufried
    https://ufried.com/

    View full-size slide

  3. The “good ol’ times”
    Software architectural work until 2008

    View full-size slide

  4. Typical setting (1/2)
    • Many internally used systems
    • Relatively few external systems
    • Mostly monolithic systems
    • Rich frontend apps or server-side rendered web pages
    • On-premises data centers

    View full-size slide

  5. Typical setting (2/2)
    • Perimeter-based security
    • Pre-definable load patterns
    • “IT is a cost center” thinking
    • Effects of distribution handled by infrastructure

    View full-size slide

  6. Architectural work (1/3)
    • Design solution that satisfies project needs
    • Project cost efficiency as dominant design driver
    • Focus on system structure
    • Layered architecture was predominant

    View full-size slide

  7. Architectural work (2/3)
    • Availability, security, scaling, … left to Ops
    • Standard, often mandatory solutions offered by Ops
    • Often respective NFRs determined by solutions offered
    • Performance as 2nd class citizen
    • Usability and architecture completely unrelated
    • Integration as 2nd class citizen
    • Either solved ad hoc or via centralized integration solution

    View full-size slide

  8. Architectural work (3/3)
    • Speed of change usually constrained by project lead times
    • Typically measured in months or years
    • Core area of tension
    • Solution maintainability and changeability
    • Project cost efficiency

    View full-size slide

  9. Focus on structure
    Core area of tension
    Changeability
    Maintainability
    Flexibility
    Project efficiency
    Opposing goals
    Amplifying goals
    Usability
    Detached topic
    Performance
    Minor design
    consideration
    Availability
    Scalability
    Reliability
    Confidentiality Integrity
    Security
    “Ops problem”
    Dominant solution driver
    Portability
    Interoperability
    Minor design
    considerations

    View full-size slide

  10. Architectural work typically meant
    „Create building block design and decide technology” *
    * within the predefined constraints

    View full-size slide

  11. The age of confusion
    Software architectural work between 2008 and 2015

    View full-size slide

  12. The age of confusion
    • The IT world went “Agile”
    • Developers were “The new kingmakers” *
    • Countermovement to the architecture-centric movement
    of the 1990s
    * https://www.oreilly.com/library/view/the-new-kingmakers/9781449368036/

    View full-size slide

  13. “We do not need architects. The dev team is the architect.”
    “We do not need architectural work. Architecture emerges.”

    View full-size slide

  14. IT changed a lot in those days

    View full-size slide

  15. Architectural work mostly stood still

    View full-size slide

  16. Focus on structure
    Core area of tension
    Changeability
    Maintainability
    Flexibility
    Project efficiency
    Opposing goals
    Amplifying goals
    Usability
    Detached topic
    Performance
    Minor design
    consideration
    Availability
    Scalability
    Reliability
    Confidentiality Integrity
    Security
    “Ops problem”
    Dominant solution driver
    Portability
    Interoperability
    Minor design
    considerations
    We were still here

    View full-size slide

  17. Or sometimes even worse
    “From 0 to legacy in just 6 months”

    View full-size slide

  18. The present
    Software architectural work since 2015

    View full-size slide

  19. Novel drivers (1/3)
    • Cloud revolution *
    • Completely new solution options
    • Pay per use
    • Elasticity
    • Moving data around has a price tag
    • Availability and reliability on non-HA hardware
    * https://www.ufried.com/blog/public_cloud_revolution_1/

    View full-size slide

  20. Changeability
    Maintainability
    Usability
    Performance
    Availability
    Scalability
    Reliability
    Confidentiality Integrity
    Security
    Portability
    Interoperability
    Flexibility
    Cloud revolution
    Project efficiency
    Opposing goals
    Amplifying goals
    Suitability
    New solution options
    Pay per use
    Interoperability
    Data flow
    Economic efficiency
    Data flow
    Elasticity
    Elasticity
    Scalability
    Resilience
    Availability
    Reliability
    Scale out
    on non-HA
    hardware
    Project efficiency

    View full-size slide

  21. Novel drivers (1/3)
    • Cloud revolution
    • Big data & ML/AI
    • Huge amounts of data to store, move and process
    • Data flow becoming 1st class citizen of system design
    • Polyglot persistence
    • MLaaS/AIaaS

    View full-size slide

  22. Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    Big data & ML/AI
    Project efficiency
    Suitability
    Data flow
    Resilience
    Project efficiency
    Economic efficiency
    Move, store &
    process data
    Mlaas/AIaas
    Economic efficiency
    Availability
    Scalability
    Reliability
    Interoperability
    Performance
    Process &
    move data
    Polyglot persistence
    Suitability
    Processing options
    Interoperability
    Move data
    Scalability
    Move, store &
    process data
    Store data
    Reliability
    Availability

    View full-size slide

  23. Novel drivers (1/3)
    • Cloud revolution
    • Big data & ML/AI
    • Mobile devices
    • Completely new interaction patterns
    • Various novel device classes
    • Less reliable and often limited connectivity
    • Unpredictable load patterns

    View full-size slide

  24. Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    Mobile devices
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Reliability
    Interoperability
    Performance
    Limited battery
    Scalability
    Unpredictable
    usage patterns
    Suitability
    New interaction patterns
    Usability
    Increased user expectations
    Adaptability
    Novel device classes
    Portability
    Novel attack vectors
    Security
    Confidentiality Integrity
    Availability
    Dependability
    Less reliable
    connectivity
    Versatility
    Manifold subclasses
    Interoperability
    Less reliable
    connectivity
    Changeability
    Increased user expectations

    View full-size slide

  25. Novel drivers (1/3)
    • Cloud revolution
    • Big data & ML/AI
    • Mobile devices
    • IoT & Edge
    • Most of the effects of mobile devices “to the square”
    • Solutions enter safety-critical domains

    View full-size slide

  26. Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    IoT & Edge
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Reliability
    Interoperability
    Performance
    Limited battery
    Suitability
    New interaction patterns
    Usability
    Novel device classes
    Portability
    Novel attack vectors Security
    Confidentiality Integrity
    Availability
    Dependability
    Less reliable
    connectivity
    Versatility
    Manifold subclasses
    Interoperability
    Less reliable
    connectivity
    Adaptability
    Safety Safety-critical
    applications

    View full-size slide

  27. Novel drivers (2/3)
    • Distributed systems
    • Highly connected, distributed system landscape
    • Service-based applications, …
    • Intricate new failure types need to be handled
    • Effects of distribution penetrate to the application level
    • Near-realtime response expectations intensify effects

    View full-size slide

  28. Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    Distributed systems
    Project efficiency
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Performance
    Latency
    Confidentiality Integrity
    Adaptability
    Safety
    Reliability
    Remote
    communication
    Security
    Safety
    Availability
    Dependability
    Distributed
    failure modes
    Reliability
    Resilience
    Interoperability
    Suitability
    Communication flow
    Usability
    Suitability
    Communication flow
    Interoperability
    Observability
    Distributed
    failure modes

    View full-size slide

  29. Novel drivers (2/3)
    • Distributed systems
    • Near-realtime
    • Batch updates turn into online updates
    • Complex web of continuously communicating systems
    • Integration becomes first-class citizen
    • Near-realtime insights feedback loop
    • Intensifies effects of system distribution

    View full-size slide

  30. Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    Near-realtime
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Performance
    Latency
    Confidentiality Integrity
    Adaptability
    Safety
    Reliability
    Availability
    Dependability
    Distribution
    failure modes
    Reliability
    Resilience
    Interoperability
    Integration as
    1st class citizen
    Observability

    View full-size slide

  31. Novel drivers (2/3)
    • Distributed systems
    • Near-realtime
    • 24x7
    • Maintenance windows are gone
    • Overnight batch processing windows are gone
    • Upgrade/repair systems in-flight

    View full-size slide

  32. Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality Integrity
    Security
    Portability
    Flexibility
    24x7
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality Integrity
    Adaptability
    Safety
    Reliability
    Availability
    Dependability
    In-flight system
    upgrade/repair
    Reliability
    Observability
    In-flight system
    upgrade/repair
    Observability
    Batch windows gone
    Performance

    View full-size slide

  33. Novel drivers (2/3)
    • Distributed systems
    • Near-realtime
    • 24x7
    • Perimeter-less security
    • Trusted zones are gone: “The Internet is everywhere”
    • Remote work, BYOD, mobile devices, etc. intensify challenges
    • “Shift left security” mandatory

    View full-size slide

  34. Observability
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Perimeter-less security
    Project efficiency
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    Observability
    Trusted zones gone
    Integrity
    Trusted zones gone
    Security
    Confidentiality

    View full-size slide

  35. Novel drivers (3/3)
    • Digital transformation
    • “IT and business are the same side of the same coin;
    the other side are the market and your customers”
    • IT being integral part of all business offerings
    • IT being integral part of most customer interactions
    • IT enabling new, previously unthinkable business models
    • IT becoming indispensable part of business and private lives

    View full-size slide

  36. Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Digital transformation
    Suitability
    Resilience
    Project efficiency
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    New business models Adaptability
    System lifecycle
    Products,
    not projects
    Economic efficiency
    Availability
    Dependability
    Integral part of
    business offerings
    Reliability
    Resilience
    Scalability
    Usability
    Suitability
    Integral part of
    customer interactions
    Integrity
    Security
    Confidentiality
    Safety
    Indispensable
    part of our lives
    Interoperability
    API-driven
    revenue streams
    Changeability
    New business models
    Flexibility
    Maintainability
    Human needs
    Socio-technical systems
    Affecting
    humans

    View full-size slide

  37. Novel drivers (3/3)
    • Digital transformation
    • Post-industrial markets
    • Supply exceeds demand by far à customer-driven markets
    • Fast adoption to changing demands becomes vital
    • Lead and cycle times become essential
    • Continuous feedback at all levels needed
    • More complex multidisciplinary collaboration needed

    View full-size slide

  38. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Post-industrial markets
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    Fast adoption Adaptability
    Human needs
    Socio-technical systems
    Multidisciplinary
    collaboration
    Changeability
    Fast adoption
    Flexibility
    Maintainability
    Observability
    Observability Continuous feedback
    Usability
    Suitability
    Customer-driven market
    System lifecycle

    View full-size slide

  39. Novel drivers (3/3)
    • Digital transformation
    • Post-industrial markets
    • Exploding complexity
    • Probably the 2nd biggest challenge of the upcoming years
    • IT drowns in complexity and every day it becomes worse
    • Due to shortsighted thinking
    • Due to important decisions not made

    View full-size slide

  40. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Exploding complexity
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    System lifecycle
    Simplicity
    Tackling complexity
    System empathy
    System viability
    and collaboration

    View full-size slide

  41. Novel drivers (3/3)
    • Digital transformation
    • Post-industrial markets
    • Exploding complexity
    • Ecological sustainability
    • Probably the biggest challenge of the upcoming years
    • Making sustainability 1st class citizen of system design
    • Counterforce to many other forces

    View full-size slide

  42. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Ecological sustainability
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    System lifecycle
    Fostering sustainability
    Sustainability
    Simplicity
    System empathy

    View full-size slide

  43. Finally, let us add the amplifying and opposing forces

    View full-size slide

  44. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Opposing goals
    Amplifying goals
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    System lifecycle
    Simplicity
    System empathy
    Sustainability
    Project efficiency
    … and many more

    View full-size slide

  45. All quality goals need to be balanced
    according to the given needs *
    * needs of different stakeholder groups, short-term and long-term needs, amplifying vs. opposing goals, …

    View full-size slide

  46. All influencing forces and the derived quality goals
    vastly influence the resulting solution architecture

    View full-size slide

  47. Observations
    What has changed in the last decade?

    View full-size slide

  48. Human needs
    Socio-technical systems
    Integrity
    Integrity
    Dependability
    Versatility
    Changeability
    Maintainability
    Usability
    Performance
    Confidentiality
    Security
    Portability
    Flexibility
    Opposing goals
    Amplifying goals
    Suitability
    Resilience
    Economic efficiency
    Availability
    Scalability
    Interoperability
    Confidentiality
    Adaptability
    Safety
    Reliability
    Observability
    System lifecycle
    Simplicity
    System empathy
    Sustainability
    Project efficiency
    Focus on structure
    not sufficient anymore
    Data flow
    Communication flow
    Limited view on projects
    not sufficient anymore
    Economic perspective
    over lifecycle as 1st
    class citizen
    Prevalent complexity of
    IT system landscapes
    requires new design
    approaches
    Exploding number
    of device classes
    Integration as
    1st class citizen
    Thinking in
    feedback loops
    Holistic view required.
    Focus on software alone
    not sufficient anymore
    Novel drivers of future system
    design. Counterforces to many
    other design forces
    Enterprise software
    affects safety-critical
    systems more and more
    Security as 1st class
    citizen in design
    (“Shift left security”)
    Huge new
    challenges due
    to highly
    interconnected
    distributed
    system
    landscapes
    (“microservices”)
    Unpredictable
    load patterns
    Massively increased user
    expectations influence
    system design
    Technology evolution provides
    new options and changes what
    a feasible solution is

    View full-size slide

  49. This is a completely
    different game ..
    … than that

    View full-size slide

  50. Lots of new forces. Lots of new tasks and challenges.
    Partially completely novel forces and resulting tasks.

    View full-size slide

  51. Moving on
    Addressing the new challenges

    View full-size slide

  52. A few suggestions …

    View full-size slide

  53. Architecture is complex
    • The dev team as architect is dead
    • The omniscient architect is dead, too
    • Architectural work has become a team sport …
    • … but needs a conductor

    View full-size slide

  54. Architecture unfolds
    • Emergent architecture is dead
    • BDUF is dead, too
    • Start with JDUF and then evolve along the way …
    • … but make deliberate architectural decisions

    View full-size slide

  55. “A complex system designed from scratch never works
    and cannot be made to work. You have to start over,
    beginning with a working simple system.”
    -- John Gall
    Source: John Gall, "Systemantics", 1975

    View full-size slide

  56. Architectural work is systemic
    • Decisions have complex impact patterns
    • Think global, act local
    • Foster feedback at all levels
    • Minimize overall complexity growth

    View full-size slide

  57. Do you really think hundreds of local optimizations
    completely ignoring their context
    will result in a global optimum?
    Or do you rather think quite the opposite is going to happen?

    View full-size slide

  58. Architecture is socio-technical
    • Mind the impacted socio-technical systems
    • Effectiveness of software design and development
    • Ease of operations
    • Effectiveness of problem detection and analysis
    • Efficiency of problem fixing or mitigation
    • Support of usage patterns, processes and organization
    • ...

    View full-size slide

  59. "Ultimately, design is a tool to enhance our humanity”
    -- Ilse Crawford

    View full-size slide

  60. Architectural work is challenging
    • Level up architectural work
    • Level up requirements engineering and communication
    • Master the foundations of good design
    • Help people understand software
    • Embrace distributed systems
    • Understand dependability, sustainability and simplicity

    View full-size slide

  61. “There is nothing a mere scientist can say that will stand against
    the flood of a hundred million dollars.
    But there is one quality that cannot be purchased in this way -
    and that is reliability. The price of reliability is the pursuit of the
    utmost simplicity.
    It is a price which the very rich find most hard to pay.”
    -- Sir C. A. R. Hoare
    Source: “The emperor's old clothes“, Communications of the ACM, Vol. 24, No. 2, February 1981, pp. 75-83

    View full-size slide

  62. Wrap-up
    • Architectural challenges have changed a lot
    • Complexity went up significantly
    • New counterforces have emerged
    • Decision making has become a lot more demanding
    • Updated perception of architectural work required
    • Forget the traditional perception of architects and their work
    • New collaboration modes needed
    • Different skills have become vital

    View full-size slide

  63. Uwe Friedrichsen
    Works @ codecentric
    https://twitter.com/ufried
    https://www.speakerdeck.com/ufried
    https://ufried.com/

    View full-size slide