$30 off During Our Annual Pro Sale. View Details »

Modern Enterprise Architecture: Architecting for Outcomes

Modern Enterprise Architecture: Architecting for Outcomes

A presentation given at Agile Meets Architecture 2023 on a modern approach to Enterprise Architecture following the paradigm shift in software development driven by, amongst other things, the Agile Manifesto, DevOps, cloud, and the flip from project-to-product.

Simon Rohrer

October 08, 2023
Tweet

Other Decks in Technology

Transcript

  1. Modern Enterprise Architecture:
    Architecting for Outcomes
    Simon Rohrer

    View Slide

  2. My journey
    Developer
    Enterprise architect
    Agile transformation lead
    Head of tech arch & WoW
    Agile adoption lead
    Ways of working lead

    View Slide

  3. Modern Enterprise
    Architecture
    Teams-of-
    Teams-of-
    Teams: > 150
    Employees
    Interact with
    Your
    Customers
    Digitally
    Change is a
    Constant
    Heterogeneous
    Environment:
    Old/New, Big/Small,
    Slow/Fast,
    Monolith/Distributed
    Vendors/Internal
    Systems of
    Systems of
    Systems

    View Slide

  4. A
    B
    C
    D
    E
    Aligning Value, People
    and Technology
    Architecting for
    Outcomes: Better Value,
    Sooner, Safer, Happier
    Governing Continuously:
    Conversational and
    Automated
    Practicing DevOps at
    Enterprise Scale
    Curating the Evolution of
    Enterprise Architecture
    and Organisation
    • Conway’s law ++ and reverse Conway manoeuvre

    • Removing complexity and increasing autonomy

    • Enterprise architecture as politics and diplomacy
    • Outcomes of quality,
    fl
    ow, safety to balance pure “value”

    • Happier customers, colleagues, citizens & climate

    • Continuous improvement - be the best at being better
    • A stream of continuous architecture decisions

    • Moving decisions down into the right level of the org

    • Automating common top-level system policies
    • Learning by observing: intention & re
    fl
    ection

    • Humility in architecture: all decisions are contingent
    • Fitness functions for socio-technical architecture at scale

    • Evolving fast and slow
    Modern Enterprise Architecture

    View Slide

  5. Growth vs Fixed
    de
    fi
    nitions

    View Slide

  6. Uncovering better ways of
    developing software by doing it and
    helping others do it
    Early and continuous delivery

    of valuable software
    Continuous attention to technical
    excellence and good design
    “Agile”
    Processes & tools
    Sprints, standups, user stories, story
    points
    Scrum
    Fixed mindset Growth mindset
    Frameworks & certi
    fi
    cations &
    coaches
    Individuals and interactions
    over processes and tools

    View Slide

  7. You build it, you run it (/ you deploy
    it / you test it / you design it / etc)
    Culture, Automation, Lean, Metrics,
    Sharing
    The 3 ways: Systems Thinking, Amplify
    Feedback Loops, Culture of Continuous
    Experimentation & Learning
    Business = Dev, Customer = Ops
    “DevOps”
    A DevOps department / team /
    engineer
    Kubernetes, cloud, automating release
    pipelines
    CI/CD
    Fixed mindset Growth mindset
    One more hando
    ff
    in tech:

    Dev -> DevOps -> Ops/SRE

    View Slide

  8. Where are we?
    How did we get here?

    View Slide

  9. 1990s
    1995 1999
    1994 1999
    1994

    View Slide

  10. Software ate the world

    View Slide

  11. A paradigm shift happened in software development
    Plan
    • Requirements
    • High level
    design
    months
    Analysis &
    design team
    Code
    • Detailed
    design
    • Development
    months
    Development
    team
    Test
    months
    • Test plan
    • Test execution
    Test team
    Release
    • Change
    request
    • Deployment
    weeks
    Release team
    Operate
    • Incidents
    • Problem fixes
    forever
    Operations
    team
    Development/
    operation
    team
    Days or hours

    View Slide

  12. Not everyone noticed

    View Slide

  13. Complexity in the existing (“legacy”) system
    landscape started to dominate the pace of change
    Business
    service
    Technical
    service
    Database

    View Slide

  14. 1. Autonomy

    2. Explicit boundaries

    3. Partitioning of functionality

    4. Dependencies de
    fi
    ned by
    functionality

    5. Asynchronicity [by default]

    6. Partitioning of data

    7. No cross-fortress transactions

    8. Single point security

    9. Inside trust

    10. Keep it simple
    Patterns were documented to manage complexity
    but not everyone used them. They still don’t.
    Autonomous
    business
    capability
    2008

    View Slide

  15. Our problems are similar but different to those in
    the 1990s
    The enterprise software
    landscape is so complex we need
    to use project management
    techniques to coordinate multiple
    teams for simple features
    Civil engineering processes are
    the best way to deliver any
    software change
    1990s 2020s
    Plan
    • Requirements
    • High level
    design
    months
    Analysis &
    design team
    Code
    • Detailed
    design
    • Development
    months
    Development
    team
    Test
    months
    • Test plan
    • Test execution
    Test team
    Release
    • Change
    request
    • Deployment
    weeks
    Release team
    Operate
    • Incidents
    • Problem fixes
    forever
    Operations
    team

    View Slide

  16. An organization that lacks a viable program in
    enterprise architecture will pay a severe cost in IT
    complexity.

    Complexity is the enemy.

    Enterprise Architecture is the solution.

    The only solution.

    Roger Sessions, 2008
    https://web.archive.org/web/20211021220626/https://simplearchitectures.blogspot.com/2008/11/job-of-enterprise-architect.html

    View Slide

  17. Help the organisation view itself as a
    complex sociotechnical system(-of-
    systems-of-systems) & reduce that
    complexity over time based on
    outcomes and learning

    Enterprise architecture
    Tell the organization what to do
    Fixed mindset Growth mindset

    View Slide

  18. A Aligning Value,
    People and
    Technology

    View Slide

  19. Organisation
    architecture
    System
    architecture
    Drives
    Conway’s law (very roughly)

    View Slide

  20. If the architecture of the system and the
    architecture of the organization are at odds, the
    architecture of the organization wins.
    Ruth Malan, 2008
    https://web.archive.org/web/20221205155201/https://ruthmalan.com/traceinthesand/conwayslaw.htm

    View Slide

  21. BUT!

    View Slide

  22. Where all hell really breaks loose is when
    management decides to reorganise things.

    … if you try to change the organisation, the
    software won’t let it happen.
    Allan Kelly, 2018
    https://www.youtube.com/watch?v=Cu0AU8vw3xw

    View Slide

  23. Organisation
    architecture
    System
    architecture
    Drives
    System
    architecture
    Organisation
    architecture
    Constrains

    View Slide

  24. ???

    View Slide

  25. We shape our
    buildings;

    thereafter they shape
    us
    Winston Churchill, 1943

    View Slide

  26. We shape our architecture and then the
    architecture shapes us
    Gene Kim, 2022
    https://web.archive.org/web/20231015100627/https://twitter.com/lbmkrishna/status/1547447927567953920?s=20

    View Slide

  27. Organisation
    architecture
    System
    architecture
    Constrains
    Constrains

    View Slide

  28. System architects (who we call architects) and
    business/organization architects (who we call
    managers) should not work as if one has no
    impact on the other.
    Ruth Malan, 2008
    https://web.archive.org/web/20221205155201/https://ruthmalan.com/traceinthesand/conwayslaw.htm

    View Slide

  29. Organization architects
    (“managers”)
    System architects
    (“architects”)
    Organisation
    architecture
    System
    architecture
    Constrains
    Constrains

    View Slide

  30. View Slide

  31. Business
    architecture
    (“value
    streams”)
    Organisation
    architecture
    Drives
    Organization architects
    (“managers”)

    View Slide

  32. Organisation
    architecture
    System
    architecture
    Constrains
    Constrains
    Business
    architecture
    (“value
    streams”)
    Constrains
    Constrains
    Constrains
    Constrains
    Organization architects
    (“managers”)
    System architects
    (“architects”)

    View Slide

  33. Value
    Organization Technology & processes
    People, teams,
    teams-of-teams,
    departments
    Business services &
    business processes,
    IT services & IT
    components
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Work
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents
    Aligning all 3
    for sustainable
    fl
    ow

    View Slide

  34. Level 0: an
    ideal 2-
    pizza
    stream-
    aligned
    team
    Value
    2 pizza team
    - 20 people
    Software that fits
    inside the team’s head
    Full stack, full
    lifecycle, full burrito,
    T-shaped people -
    YBIYRI
    Self-organizing
    DDD
    Eventually consistent
    Independently deployable & testable
    “Monolith vs Microservice is missing the point”
    Emergent design & evolutionary* architecture
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents
    Value
    2 pizza team
    - 20 people
    Software that fits
    inside the team’s head
    Full stack, full
    lifecycle, full burrito,
    T-shaped people -
    YBIYRI
    Self-organizing
    DDD
    Eventually consistent
    Independently deployable & testable
    “Monolith vs Microservice is missing the point”
    Emergent design & evolutionary* architecture
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents
    Value
    2 pizza team
    - 20 people
    Software that fits
    inside the team’s head
    Full stack, full
    lifecycle, full burrito,
    T-shaped people -
    YBIYRI
    Self-organizing
    DDD
    Eventually consistent
    Independently deployable & testable
    “Monolith vs Microservice is missing the point”
    Emergent design & evolutionary* architecture
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents
    Value
    2 pizza team
    - 20 people
    Software that fits
    inside the team’s head
    Full stack, full
    lifecycle, full burrito,
    T-shaped people -
    YBIYRI
    Self-organizing
    DDD
    Eventually consistent
    Independently deployable & testable
    “Monolith vs Microservice is missing the point”
    Emergent design & evolutionary* architecture
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents

    View Slide

  35. View Slide

  36. View Slide

  37. Value
    pizza team
    20 people
    Software that fits
    inside the team’s head
    Full stack, full
    ecycle, full burrito,
    -shaped people -
    YBIYRI
    Self-organizing
    DDD
    Eventually consistent
    Independently deployable & testable
    “Monolith vs Microservice is missing the point”
    Emergent design & evolutionary* architecture
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents

    View Slide

  38. Level 1: an
    ideal team-
    of-teams
    Value
    Team-of-teams
    Dunbar number - <150
    System-of-systems & domain
    that fits inside the team’s head
    “Tribe”
    Nested value
    Self-organizing?
    Nested DDD
    Shared data model?
    Eventually consistent
    Each component independently deployable & testable
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents
    Value
    Team-of-teams
    Dunbar number - <150
    System-of-systems & domain
    that fits inside the team’s head
    “Tribe”
    Nested value
    Self-organizing?
    Nested DDD
    Shared data model?
    Eventually consistent
    Each component independently deployable & testable
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents
    Value
    Team-of-teams
    Dunbar number - <150
    System-of-systems & domain
    that fits inside the team’s head
    “Tribe”
    Nested value
    Self-organizing?
    Nested DDD
    Shared data model?
    Eventually consistent
    Each component independently deployable & testable
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents
    Value
    Team-of-teams
    Dunbar number - <150
    System-of-systems & domain
    that fits inside the team’s head
    “Tribe”
    Nested value
    Self-organizing?
    Nested DDD
    Shared data model?
    Eventually consistent
    Each component independently deployable & testable
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents

    View Slide

  39. Level 2: an
    ideal
    business line
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Value
    Team-of-team-of-teams
    1000–2000
    System-of-system-of-systems
    Does this fit in anyone’s head?
    Department /
    business line
    Nested value
    Self-organizing???
    Further nested DDD
    Minimally shared data model - just keys?
    Eventually consistent between subdomains
    Key paths and failure modes should be known
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Value
    Team-of-team-of-teams
    1000–2000
    System-of-system-of-systems
    Does this fit in anyone’s head?
    Department /
    business line
    Nested value
    Self-organizing???
    Further nested DDD
    Minimally shared data model - just keys?
    Eventually consistent between subdomains
    Key paths and failure modes should be known
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Value
    Team-of-team-of-teams
    1000–2000
    System-of-system-of-systems
    Does this fit in anyone’s head?
    Department /
    business line
    Nested value
    Self-organizing???
    Further nested DDD
    Minimally shared data model - just keys?
    Eventually consistent between subdomains
    Key paths and failure modes should be known
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Teams-of-
    Teams-of-
    Teams: > 150
    Employees
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Value
    Team-of-team-of-teams
    1000–2000
    System-of-system-of-systems
    Does this fit in anyone’s head?
    Department /
    business line
    Nested value
    Self-organizing???
    Further nested DDD
    Minimally shared data model - just keys?
    Eventually consistent between subdomains
    Key paths and failure modes should be known
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Systems of
    Systems of
    Systems

    View Slide

  40. View Slide

  41. The best architectures, requirements, and designs
    emerge from self-organizing teams.
    agilemanifesto.org/principles.html, 2001

    View Slide

  42. Value
    zza team
    20 people
    Software that fits
    inside the team’s head
    stack, full
    le, full burrito,
    ped people -
    YBIYRI
    -organizing
    DDD
    Eventually consistent
    Independently deployable & testable
    “Monolith vs Microservice is missing the point”
    Emergent design & evolutionary* architecture
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents
    Value
    Team-of-teams
    Dunbar number - <150
    System-of-systems & domain
    that fits inside the team’s head
    “Tribe”
    Nested value
    Self-organizing?
    Nested DDD
    Shared data model?
    Eventually consistent
    Each component independently deployable & testable
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Outcomes,
    hypotheses, bets
    User stories, tasks,
    changes, problems,
    incidents
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Value
    Team-of-team-of-teams
    1000–2000
    System-of-sys
    Does this fit in
    Department /
    business line
    Nested value
    Self-organizing???
    Further n
    Minimally shared da
    Eventually consistent
    Key paths and failure m
    OKRs looking forward (Dev)
    KPIs & OLIs right now &
    looking back (Ops)
    Teams-of-
    Teams-of-
    Teams: > 150
    Employees
    Self-organizing teams?

    View Slide

  43. B
    Architecting for
    Outcomes:
    Better Value,
    Sooner, Safer,
    Happier

    View Slide

  44. Standardisation
    Consistency
    Predictive planning
    Cost reduction
    Antipattern: traditional EA outcomes
    These are
    not the
    outcomes
    you’re
    looking for

    View Slide

  45. Cloud
    Kubernetes
    Microservices
    Antipattern: “newer” EA outcomes
    These are
    not the
    outcomes
    you’re
    looking for

    View Slide

  46. Customers
    Colleagues
    Citizens
    Climate
    Security
    Privacy
    Minimum
    viable
    compliance
    Quality
    Flow
    Pattern: align to business agility

    View Slide

  47. We shape our architecture and then the
    architecture shapes us
    Gene Kim
    As the systems we build become larger, the
    coordination increases…
    …it can grow so so large that all our time and
    energy is spent coordinating - it is at the expense
    of our value creating activities
    https://web.archive.org/web/20231015100627/https://twitter.com/lbmkrishna/status/1547447927567953920?s=20

    View Slide

  48. “Hello world” with tightly coupled complex
    architecture
    ¥$€
    SOLUTION
    ARCHITECTS
    TESTING
    TEAM
    UI
    API LAYER
    GREETINGS
    SERVICE
    PLANET
    SERVICE
    “Hello
    world!”
    “!”
    /api
    Hello
    World
    T_HW
    “Hello
    wrld!”

    View Slide

  49. View Slide

  50. Release trains are an iterative and incremental
    lifecycle, but they are not agile.
    https://www.jrothman.com/articles/2011/01/not-ready-for-agile-start-your-journey-with-release-trains/
    Johanna Rothman, 2011

    View Slide

  51. Autonomous team with their own valuable business capability
    Refactored Restructured organization
    and architecture
    ¥$€
    EARTH GREETER
    MICROFRONTEND
    EARTH GREETER
    MICROSERVICE DB
    EARTH GREETER
    MICROSERVICE
    “Hello
    world!”
    ]
    “Hello
    world!”

    View Slide

  52. View Slide

  53. C Governing
    Continuously:
    Conversational
    and Automated

    View Slide

  54. Creating and reducing complexity may sound like
    perfect opposites. But in fact fundamental
    asymmetries exist between the two.
    https://hbr.org/2020/01/taming-complexity

    View Slide

  55. Architecture governance to
    the rescue?

    View Slide

  56. Antipattern:

    Architecture Review Boards

    View Slide

  57. Initiate Plan Execute Close
    Analysis Design Build/Test Release
    Architecture
    board
    Review
    Architecture
    board
    Review
    How we used to govern architecture
    A project: waterfall / wagile / water-scrum-fall / “hybrid”

    View Slide

  58. How does this work with modern delivery?
    Initiate
    Plan
    Execute
    Close
    Analysis
    Test/design/build
    Release Release Release Release
    Release Release Release
    Architecture
    board
    Review
    An agile or lean project

    View Slide

  59. How does this work with modern delivery?
    Architecture
    board
    A product

    View Slide

  60. Pattern:
    Black box architecture
    governance

    View Slide

  61. Govern realised architectures, not (just) designs on paper
    “I think I need an new service”
    “I think my service needs to
    access data or functionality from
    another service”
    EA approve new services, new
    namespaces in Kubernetes (etc.)
    EA approve:
    • Service-to-service IAM
    • Firewalls
    “I want an exception to
    automated governance - e.g.
    synchronous request/response”
    EA approve:
    • Exceptions
    • Tech debt
    Enterprise Architecture as GitOps

    View Slide

  62. Pattern:
    Continuous conversational
    “governance”

    View Slide

  63. Continuous conversational governance
    Initiate
    Plan
    Execute
    Close
    Analysis
    Test / Design / Build
    Release Release Release Release
    Release Release Release
    Architecture
    C.o.P.
    Continuous conversation
    Agile/lean/DevOps/Continuous Delivery driven project

    View Slide

  64. Continuous conversational governance
    A single product
    Architecture
    C.o.P.
    Continuous conversation

    View Slide

  65. Continuous conversational governance across value streams
    Value stream B Value stream C
    Value stream A
    Value stream E Value stream F
    Value stream D
    Value stream M
    Value stream N
    Business unit
    Architecture
    C.o.P.
    Architecture
    centre of
    excellence

    View Slide

  66. D Practicing
    DevOps at
    Enterprise Scale

    View Slide

  67. Continuous conversational governance
    Architecture
    C.o.P.
    Architecture
    centre of
    excellence
    Enterprise
    scope
    Longer term
    timeframe
    Service
    management
    C.o.E.

    View Slide

  68. Intention Reflection
    Working code
    deployed as
    running system
    Model
    storming
    Emergent design
    Discoverability &
    observability
    tooling
    Test-driven
    design
    Infrastructure-as-
    code
    Architecture
    decision records
    Continuous
    refactoring
    Whiteboard
    sessions
    Continuous conversational governance (after Ruth Malan)

    View Slide

  69. View Slide

  70. E
    Curating the
    Evolution of
    Enterprise
    Architecture and
    Organisation

    View Slide

  71. Continuous attention to technical excellence
    and good design enhances agility.

    View Slide

  72. Evolutionary architecture fitness functions at
    enterprise level (“macro architecture”)
    •Number of 2-pizza teams required to implement a typical business feature
    - sociotechnical coupling & cohesion
    •Asynchronous connections ÷ synchronous connections

    View Slide

  73. Theory #1
    :
    gradual evolution
    Time
    Evolution

    View Slide

  74. Theory #2
    :
    punctuated equilibria
    Time
    Evolution

    View Slide

  75. Reconciled: punctuated gradualism
    Continuous attention to
    complexity debt pay down
    and continuous
    improvement.
    Continuous funding.
    A step change: a business case explicitly to
    rewrite (or buy, or outsource, or write something
    new).
    Standalone funding.
    Time
    Evolution

    View Slide

  76. Strangler
    fi
    g pattern

    View Slide

  77. View Slide

  78. A
    B
    C
    D
    E
    Aligning Value, People
    and Technology
    Architecting for
    Outcomes: Better Value,
    Sooner, Safer, Happier
    Governing Continuously:
    Conversational and
    Automated
    Practicing DevOps at
    Enterprise Scale
    Curating the Evolution of
    Enterprise Architecture
    and Organisation
    • Conway’s law ++ and reverse Conway manoeuvre

    • Removing complexity and increasing autonomy

    • Enterprise architecture as politics and diplomacy
    • Outcomes of quality,
    fl
    ow, safety to balance pure “value”

    • Happier customers, colleagues, citizens & climate

    • Continuous improvement - be the best at being better
    • A stream of continuous architecture decisions

    • Moving decisions down into the right level of the org

    • Automating common top-level system policies
    • Learning by observing: intention & re
    fl
    ection

    • Humility in architecture: all decisions are contingent
    • Fitness functions for socio-technical architecture at scale

    • Evolving fast and slow
    Modern Enterprise Architecture

    View Slide

  79. Thank you!
    @sirohrer

    View Slide