$30 off During Our Annual Pro Sale. View Details »

Squaring the circle - Mastering the next level of architectural design

Squaring the circle - Mastering the next level of architectural design

In this presentation, I discuss how architectural work changed in the last 10-15 years due to new and changed drivers of various sorts.

I start with architectural work as it was typically done in the early 2000s. Then I briefly touch how architectural work mostly stood still through the agile confusion due to some widespread misunderstandings regarding architecture and architectural work. After that, I discuss 12 drivers of the last 10-15 years that changed how we need to address architecture and architectural work. Finally, I provide a few recommendations how we could address the new challenges and hopefully turn them into opportunities.

As this a really big topic, the presentation only scratches the surface - creating awareness regarding the changed challenges in architectural work at best. Additionally, the voice track is lacking. Still, I hope it provides some food for thought.

Uwe Friedrichsen

June 02, 2022
Tweet

More Decks by Uwe Friedrichsen

Other Decks in Technology

Transcript

  1. Squaring the circle Mastering the next level of architectural design

    Uwe Friedrichsen – codecentric AG – 2006-2022
  2. Uwe Friedrichsen Works @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://ufried.com/

  3. The “good ol’ times” Software architectural work until 2008

  4. Typical setting (1/2) • Many internally used systems • Relatively

    few external systems • Mostly monolithic systems • Rich frontend apps or server-side rendered web pages • On-premises data centers
  5. Typical setting (2/2) • Perimeter-based security • Pre-definable load patterns

    • “IT is a cost center” thinking • Effects of distribution handled by infrastructure
  6. Typical mindset • “Single process, single thread” thinking • Availability

    demands handled by infrastructure • Security demands handled by infrastructure • Project-scope thinking • Integration as 2nd class citizen
  7. Architectural work (1/3) • Design solution that satisfies project needs

    • Project cost efficiency as dominant design driver • Focus on system structure • Layered architecture was predominant
  8. Architectural work (2/3) • Availability, security, scaling, … left to

    Ops • Standard, often mandatory solutions offered by Ops • Often respective NFRs determined by solutions offered • Performance as 2nd class citizen • Usability and architecture completely unrelated • Integration as 2nd class citizen • Either solved ad hoc or via centralized integration solution
  9. Architectural work (3/3) • Core area of tension • Solution

    maintainability and changeability • Project cost efficiency • Speed of change usually constrained by project lead times • Typically measured in months or years
  10. Architectural work summary • Dominated by project constraints • Focus

    on system structure • Required architectural work quite well understood • At least in theory • Still not easy in practice • Many solutions from that time as witnesses
  11. Focus on structure Core area of tension Changeability Maintainability Flexibility

    Project efficiency Opposing goals Amplifying goals Usability Detached topic Performance Minor design consideration Availability Scalability Reliability Confidentiality Integrity Security “Ops problem” Dominant solution driver Portability Interoperability Minor design considerations
  12. Architectural work typically meant „Create building block design and decide

    technology” * * within the predefined constraints
  13. The age of confusion Software architectural work between 2008 and

    2015
  14. The age of confusion • The IT world went “Agile”

    • Developers were “The new kingmakers” * • Countermovement to the architecture-centric movement of the 1990s * https://www.oreilly.com/library/view/the-new-kingmakers/9781449368036/
  15. “We do not need architects. The dev team is the

    architect.” “We do not need architectural work. Architecture emerges.”
  16. IT changed a lot in those days

  17. Architectural work mostly stood still

  18. Focus on structure Core area of tension Changeability Maintainability Flexibility

    Project efficiency Opposing goals Amplifying goals Usability Detached topic Performance Minor design consideration Availability Scalability Reliability Confidentiality Integrity Security “Ops problem” Dominant solution driver Portability Interoperability Minor design considerations We were still here
  19. Or sometimes even worse “From 0 to legacy in just

    6 months”
  20. The present Software architectural work since 2015

  21. Novel drivers (1/3) • Cloud revolution * • Completely new

    solution options • Pay per use • Elasticity • Moving data around has a price tag • Availability and reliability on non-HA hardware * https://www.ufried.com/blog/public_cloud_revolution_1/
  22. Changeability Maintainability Usability Performance Availability Scalability Reliability Confidentiality Integrity Security

    Portability Interoperability Flexibility Cloud revolution Project efficiency Opposing goals Amplifying goals Suitability New solution options Pay per use Interoperability Data flow Economic efficiency Data flow Elasticity Elasticity Scalability Resilience Availability Reliability Scale out on non-HA hardware Project efficiency
  23. Novel drivers (1/3) • Cloud revolution • Big data &

    ML/AI • Huge amounts of data to store, move and process • Data flow becoming 1st class citizen of system design • Polyglot persistence • MLaaS/AIaaS
  24. Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability Flexibility Big

    data & ML/AI Project efficiency Suitability Data flow Resilience Project efficiency Economic efficiency Move, store & process data Mlaas/AIaas Economic efficiency Availability Scalability Reliability Interoperability Performance Process & move data Polyglot persistence Suitability Processing options Interoperability Move data Scalability Move, store & process data Store data Reliability Availability
  25. Novel drivers (1/3) • Cloud revolution • Big data &

    ML/AI • Mobile devices • Completely new interaction patterns • Various novel device classes • Less reliable and often limited connectivity • Unpredictable load patterns
  26. Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability Flexibility Mobile

    devices Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Reliability Interoperability Performance Limited battery Scalability Unpredictable usage patterns Suitability New interaction patterns Usability Increased user expectations Adaptability Novel device classes Portability Novel attack vectors Security Confidentiality Integrity Availability Dependability Less reliable connectivity Versatility Manifold subclasses Interoperability Less reliable connectivity Changeability Increased user expectations
  27. Novel drivers (1/3) • Cloud revolution • Big data &

    ML/AI • Mobile devices • IoT & Edge • Most of the effects of mobile devices “to the square” • Solutions enter safety-critical domains
  28. Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability

    Flexibility IoT & Edge Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Reliability Interoperability Performance Limited battery Suitability New interaction patterns Usability Novel device classes Portability Novel attack vectors Security Confidentiality Integrity Availability Dependability Less reliable connectivity Versatility Manifold subclasses Interoperability Less reliable connectivity Adaptability Safety Safety-critical applications
  29. Novel drivers (2/3) • Distributed systems • Highly connected, distributed

    system landscape • Service-based applications, … • Intricate new failure types need to be handled • Effects of distribution penetrate to the application level • Near-realtime response expectations intensify effects
  30. Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability

    Flexibility Distributed systems Project efficiency Resilience Project efficiency Economic efficiency Availability Scalability Performance Latency Confidentiality Integrity Adaptability Safety Reliability Remote communication Security Safety Availability Dependability Distributed failure modes Reliability Resilience Interoperability Suitability Communication flow Usability Suitability Communication flow Interoperability Observability Distributed failure modes
  31. Novel drivers (2/3) • Distributed systems • Near-realtime • Batch

    updates turn into online updates • Complex web of continuously communicating systems • Integration becomes first-class citizen • Near-realtime insights feedback loop • Intensifies effects of system distribution
  32. Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability

    Flexibility Near-realtime Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Interoperability Performance Latency Confidentiality Integrity Adaptability Safety Reliability Availability Dependability Distribution failure modes Reliability Resilience Interoperability Integration as 1st class citizen Observability
  33. Novel drivers (2/3) • Distributed systems • Near-realtime • 24x7

    • Maintenance windows are gone • Overnight batch processing windows are gone • Upgrade/repair systems in-flight
  34. Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability

    Flexibility 24x7 Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Interoperability Confidentiality Integrity Adaptability Safety Reliability Availability Dependability In-flight system upgrade/repair Reliability Observability In-flight system upgrade/repair Observability Batch windows gone Performance
  35. Novel drivers (2/3) • Distributed systems • Near-realtime • 24x7

    • Perimeter-less security • Trusted zones are gone: “The Internet is everywhere” • Remote work, BYOD, mobile devices, etc. intensify challenges • “Shift left security” mandatory
  36. Observability Integrity Integrity Dependability Versatility Changeability Maintainability Usability Performance Confidentiality

    Security Portability Flexibility Perimeter-less security Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability Observability Trusted zones gone Integrity Trusted zones gone Security Confidentiality
  37. Novel drivers (3/3) • Digital transformation • “IT and business

    are the same side of the same coin; the other side are the market and your customers” • IT being integral part of all business offerings • IT being integral part of most customer interactions • IT enabling new, previously unthinkable business models • IT becoming indispensable part of business and private lives
  38. Integrity Integrity Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Security

    Portability Flexibility Digital transformation Suitability Resilience Project efficiency Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability New business models Adaptability System lifecycle Products, not projects Economic efficiency Availability Dependability Integral part of business offerings Reliability Resilience Scalability Usability Suitability Integral part of customer interactions Integrity Security Confidentiality Safety Indispensable part of our lives Interoperability API-driven revenue streams Changeability New business models Flexibility Maintainability Human needs Socio-technical systems Affecting humans
  39. Novel drivers (3/3) • Digital transformation • Post-industrial markets •

    Supply exceeds demand by far à customer-driven markets • Fast adoption to changing demands becomes vital • Lead and cycle times become essential • Continuous feedback on all levels needed • More complex multidisciplinary collaboration needed
  40. Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability

    Usability Performance Confidentiality Security Portability Flexibility Post-industrial markets Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability Fast adoption Adaptability Human needs Socio-technical systems Multidisciplinary collaboration Changeability Fast adoption Flexibility Maintainability Observability Observability Continuous feedback Usability Suitability Customer-driven market System lifecycle
  41. Novel drivers (3/3) • Digital transformation • Post-industrial markets •

    Exploding complexity • Probably the 2nd biggest challenge of the upcoming years • IT drowns in complexity and every day it becomes worse • Due to shortsighted thinking • Due to important decisions not made
  42. Gregor’s Law: Excessive complexity is nature’s punishment for organizations that

    are unable to make decisions. -- Gregor Hohpe Source: https://architectelevator.com/architecture/it-complexity/
  43. Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability

    Usability Performance Confidentiality Security Portability Flexibility Exploding complexity Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability System lifecycle Simplicity Tackling complexity System empathy System viability and collaboration
  44. Novel drivers (3/3) • Digital transformation • Post-industrial markets •

    Exploding complexity • Ecological sustainability • Probably the biggest challenge of the upcoming years • Making sustainability 1st class citizen of system design • Counterforce to many other forces
  45. Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability

    Usability Performance Confidentiality Security Portability Flexibility Ecological sustainability Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability System lifecycle Fostering sustainability Sustainability Simplicity System empathy
  46. Finally, let us add the amplifying and opposing forces

  47. Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability

    Usability Performance Confidentiality Security Portability Flexibility Opposing goals Amplifying goals Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability System lifecycle Simplicity System empathy Sustainability Project efficiency … and many more
  48. All quality goals need to be balanced according to the

    given needs * * needs of different stakeholder groups, short-term and long-term needs, amplifying vs. opposing goals, …
  49. All influencing forces and the derived quality goals vastly influence

    the resulting solution architecture
  50. Observations What has changed in the last decade?

  51. Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability

    Usability Performance Confidentiality Security Portability Flexibility Opposing goals Amplifying goals Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability System lifecycle Simplicity System empathy Sustainability Project efficiency Focus on structure not sufficient anymore Data flow Communication flow Limited view on projects not sufficient anymore Economic perspective over lifecycle as 1st class citizen Prevalent complexity of IT system landscapes requires new design approaches Exploding number of device classes Integration as 1st class citizen Thinking in feedback loops Holistic view required. Focus on software alone not sufficient anymore Novel drivers of future system design. Counterforces to many other design forces Enterprise software affects safety-critical systems more and more Security as 1st class citizen in design (“Shift left security”) Huge new challenges due to highly interconnected distributed system landscapes (“microservices”) Unpredictable load patterns Massively increased user expectations influence system design Technology evolution provides new options and changes what a feasible solution is
  52. Lots of new forces. Lots of new tasks and challenges.

    Partially completely novel forces and resulting tasks.
  53. Reinforces long know, yet still unsolved problems

  54. Unsolved problems • Getting functional design right (“modularization”) • Pondering

    behavior and data flow, not just structure • Systemic thinking instead of local optimizations • Balancing long-term and short-term effects • Understanding distributed systems • …
  55. This is a completely different game .. … than that

  56. Useful side effect It is acceptable again to talk about

    architecture * * and use words like “architect” or “architectural work” … ;)
  57. Moving on Addressing the new challenges

  58. The collaboration perspective

  59. “Emergent architecture” is dead

  60. Tackling the challenges of today requires hard work and careful

    decision making
  61. Does that mean the “all-knowing” architect is back?

  62. No! But you need the skills and someone who drives

    the architectural work that needs to be done.
  63. Even the smartest architect imaginable will not be able to

    address all these challenges alone
  64. In the end, architectural work always is a team effort

  65. Does that mean the BDUF * is back? * Big

    Design UpFront
  66. No! But you need to have an idea where the

    goal is before you start running.
  67. Implementation without any upfront architectural work Implementation with some upfront

    architectural work
  68. None
  69. None
  70. Sometimes, a new or previously undetected requirement may take us

    by surprise
  71. None
  72. But basically, your architecture will evolve over time

  73. None
  74. None
  75. None
  76. Never forget Gall’s Law

  77. “A complex system designed from scratch never works and cannot

    be made to work. You have to start over, beginning with a working simple system.” -- John Gall Source: John Gall, "Systemantics", 1975
  78. The skills perspective

  79. Learn about architectural work • Understand the problem from all

    perspectives • Find alternative solution options • Evaluate the trade-offs • Help stakeholders make best decisions possible
  80. Improve your RE skills • Need to understand demands and

    consequences • Impeded by widespread division-of-labor approach • Help find requirements that do not harm the design • Often, alternatives with same outcome are possible • If not, you need to change the design of course • Uncover the actual “needs” behind the “wants” • Helps to avoid lots of accidental complexity
  81. Learn better functional design • Especially master the foundations of

    good design • “Loose coupling, high cohesion” & “separation of concerns” • “Information hiding” especially with respect to APIs • Domain-driven design may be a useful starting point • Good design skills are more relevant than ever • Affect understandability, changeability and extensibility • Affect usability, stability and acceptance at API level • Affect robustness, availability and scalability at runtime
  82. “I would advise students to pay more attention to the

    fundamental ideas rather than the latest technology. The technology will be out-of-date before they graduate. Fundamental ideas never get out of date.” -- David L. Parnas (http://www.sigsoft.org/SEN/parnas.html)
  83. Embrace distributed systems “(Almost) every system is a distributed system”

    -- Chas Emerick “Everything fails, all the time” -- Werner Vogels • Understand the non-determinism of distributed systems • “Memory, guesses and apologies” & promise theory • Understand that infrastructure cannot guarantee robustness
  84. Help people understand software • Not comparable to physical goods

    • Needs to be changed after initial release to keep its value • Software development is design, not shop floor assembly • Extreme malleability of software can turn into a curse • Invisibility makes it very hard to grasp “software”
  85. Understand the new challenges • Simplicity – tackling ever-growing complexity

    • System empathy – accepting heterogeneity • Dependability – making sure we all can rely on IT • Resilience – being prepared for the unexpected • Safety – dealing with the growing blast radius • Security – indispensable for dependable system • Sustainability – reducing the ecological footprint of IT
  86. Wrap-up

  87. Wrap-up • Architectural challenges have changed a lot • Complexity

    went up significantly • New counterforces have emerged • Decision making has become a lot more demanding • Updated perception of architectural work needed • New collaboration modes required • Different skills have become vital
  88. "Ultimately, design is a tool to enhance our humanity” --

    Ilse Crawford
  89. Uwe Friedrichsen Works @ codecentric https://twitter.com/ufried https://www.speakerdeck.com/ufried https://ufried.com/