Domain-Driven Architecture at Scale - DDD EU 2022

My talk at DDD Europe 2022, Amsterdam.

Transcript

1. Domain-​ Driven Architecture at Scale
2. Making my humble contribution to sociotechnical architecture and learning organization About me 01 02 03 04 Xin Yao Chief Software Architect @settling_mud Loves gardening, yoga, sci-​ fi, good food, hanging out with friends and family Lives in Copenhagen, Denmark xy@danskebank.dk
3. Today's path collaborative modelling value coherence strategic correspondence recommoning (domain-​ driven platforming) conditions & consequences (and boundary spanning roles) Domain-​ Driven Architecture at Scale
4. Today's path collaborative modelling value coherence strategic correspondence recommoning (of services & platforms) conditions & consequences (and boundary spanning roles) Domain-​ Driven Architecture at Scale
5. End 2021 numbers Danske Bank A Nordic Bank
6. We were Spotified in 2020 Source: Henrik Kniberg & Anders Ivarsson Scaling agile
7. We reorganized ourselves to promote the desired domain structure
8. system designs & architecture communication structures in the organization (team structure) Conway's Law
9. target domain boundaries team structure (Conway's Maneuver) desired system architecture
10. Complex domains Legacy systems Cross-​ disciplinary view Integrated perspectives Reshuffled teams Tackling complexity with collaborative discovery common language compounding complexity DDD evangelists as facilitators
11. Domain story telling Event storming Context mapping Domain modelling Bounded contexts Hexagonal architecture Bounded context canvas Example mapping Wardley mapping Impact mapping User story mapping Team Topologies Northstar Framework Event modelling Domain driven data Service blueprints Collaborative discovery & design Lean value tree Specification by example Core domain chart Domain message flow Cynefin Behavior driven development
13. When we can understand the domain, we can influence it
14. Premature convergence Big design up front At the same time? domain team Domain discovery as disjointed, localized, timeboxed effort DDD workshops DDD workshops DDD workshops domain team domain team Impact things linearly Friction points
15. Coordination complexity drains teams' cognitive capacity Friction points A change initiative involves multiple teams, domains & systems domain domain domain domain Decision decentralized [autonomy] information not decentralized Dependencies and handoffs Optimized parts, not relations
16. Involved in many levels of design "burnout" Trying to have the whole System in my head Architect Pain Facilitate to integrate efforts from many teams
17. Where are we? collaborative modelling value coherence strategic correspondence conditions & consequences (and boundary spanning roles) "Domain Driven Architecture at Scale" Beyond structural coherence recommoning (domain-​ driven platforming)
18. Governance processes for alignment
19. A system is not the sum of its parts Alignment does not necessarily bring coherence Images by Ursus Wehrly. Inspiration from Sonja Blignaut "https://www.morebeyond.co.za/making-​ coherence-​ coherent/"
20. Large Scale Structure ... is a set of high-​level principles, rules, or both that establishes a pattern of design for an entire system. A language that allows the system to be discussed and understood in broad strokes. -Eric Evans "Systemic understanding in broad strokes"
21. ... greatly clarifies the system without forcing unnatural constraints on model development. Because an ill-​fitting structure is worse than none, it is best not to shoot for comprehensiveness, but rather to find a minimal set that solves the problems that have emerged. Less is more. -​Eric Evans Large Scale Structure "models at scale must be minimal"
22. Common purpose serves as an invisible leader.. A common purpose, born of the desires and the activities of the group. -​Mary Parker Follett
23. Lead with the WHY, not the WAY - John Cutler
24. If you can't locate yourself in the narrative you feel disadvantaged. You also don't feel empowered to change the narrative. Without coherence, people will not understand their place in the system. - Jabe Bloom
25. Purpose as the connective tissue squad Tribe A squad squad squad Tribe B squad A new initiative OKR OKR OKR OKR OKR OKR OKR A shared and deeper understanding of the "why" How does my contribution matter in the big picture, over time? Find my place in the story
26. Linking strategy to everyday work We experimented with "the North Star Framework" input input input input North Star (The purpose) The value "bets" The work "bets" Opportunities Interventions leverage point solution agnostic, but opinionated actual work experiment epics stories 1-3 years 1-3 quarters 1-3 months 1-3 sprints
27. input input input input North Star The value "bets" The work "bets" Opportunities Speak to the customers goals and concerns xxx xxx Interventions Design the mobile/web UX to ask for customer consent Withdraw consent Author consent text leverage point solution agnostic, but opinionated actual work experiment epics stories 1-3 years 1-3 quarters 1-3 months 1-3 sprints Fast credit decision using account transaction history with minimal manual input [Trust/Breadth] How many customers trust us with their data? Facilitate a trust-​ building and hassle free consent experience [Data] How can we build a usable & compliant PSD2 infra? Source, store and serve PSD2 data compliantly [Accuracy & compliance] How can we calculate an accurate budget? Make accurate & invidualised budget using transaction data [Engagement & Transparency] How can we engage customer to validate budget? Engage customers in a "conversation" to validate prefilled budget items & enable a holistic overview of customer's finances [Efficiency/speed] How fast can a customer succeed with her goal? Increase credit decision automation by using transaction data to cut down on manual rules as well as customer input questions Expand consent Collaboratively kickoff complex initiatives An field example: the PSD2 initiative
28. when [a car buyer] is able to [get a credit decision really fast after he has given PSD2 consent] when Customer is able to see and validate budget When the customer has to answer fewer questions he/she is able to get credit faster and with less hassle Happy when I see that the information I filled in once still is there. (Kids, Housing type, etc.) Less annoyed due to fewer questions, more help with debt information when a customer experience a successful journey, they are likely to return when [a car buyer] is presented with prefilled information and required to answer only a few questions Happy that I get help in accesing debt information provided to public authorities (by enabling eTax combined with PSD2) when a customer is able to drive away with a new car [PSD2 design sprint] empathy mapping - chart the path through a customer's emotional journey Reframe compliance requirements in the customer context Human-​ centered design
29. Model fragments Language drifts Friction points Difficult to integrate new information Are we still solving the right problem? Temporal anomoly?
30. Successful problem solving requires finding the right solution to the right problem. We fail more often because we solve the wrong problem than because we get the wrong solution to the right problem. -​Russel Ackoff, "Redesiging the Future" Solving the right problem?
31. Where are we? collaborative modelling value coherence strategic correspondence conditions & consequences (and boundary spanning roles) "Domain-​ Driven Architecture at Scale" recommoning (domain-​ driven platforming) "understand to influence" "find my place in the story" "alternate realities & possible futures"
32. Coherence vs. correspondence Truth is what coheres with the rest of our knowledge. "Theories of truth (truth=a useful model)" Credit: Jabe Bloom Coherence Correspondence Truth is what corresponds to facts. Some facts are outside of our knowledge.
33. Future Cone reinterpretated by Jabe Bloom Help us toward Coherence Help us move toward Correspondence "How do we allow more possibilities to become plausible or probable"
34. A field example "Navigating toward correspondence"
35. Personal customer A fast & affordable financing solution Understand my current economy Automated current budget User User need Capabilty notation
36. Personal customer An affordable financing solution Understand my current economy UX to provide customer data and consents View & edit budget Automatic budget calculation Automated current budget "User" User need Capabiilty notation
37. Personal customer A fast & affordable financing solution Understand my current economy View & edit budget UX to provide customer data and consents Automatic budget calculation Statistical estimation methods Team 2 1 1 Initiative 1 Budget API for housing journey 2020 Team 2 Team 1 Automated current budget
38. Personal customer A fast & affordable financing solution Understand my current economy Regulators Enforce compliant credit practice Individualised budget calculation View & edit budget Team 1 UX to provide customer data and consents Automatic budget calculation Statistical estimation methods Team 2 1 1 Several banks got orders from the Danish Financial Supervisory Authority budget must be individualised Initiative 1 Budget API for housing journey 2020 Automated current budget risk of losing license to operate
39. Personal customer A fast & affordable financing solution Understand my current economy Regulators Enforce compliant credit practice Individualised budget calculation UX to provide customer data and consents Automatic budget calculation Statistical estimation methods 1 1 Initiative 1 Budget API for housing journey Initiative 2 "Buy" - quick compliance 2020 2021 Buy - quick compliance Team 1 Team 2 Team 2 View & edit budget Team 3 2 Automated current budget
40. Why didn't initiative 2 kill initiative 1?
41. Personal customer A fast & affordable financing solution Understand my current economy Regulators Enforce compliant credit practice Individualised budget calculation UX to provide customer data and consents Automatic budget calculation Statistical estimation methods Team 2 1 1 t Initiative 1 Budget API for housing journey Initiative 2 "Buy" - quick compliance 2020 2021 Buy - quick compliance Vendor has monopoly now - very expensive Questionable vendor solution quality View & edit budget 2 Team 1 Team 2 Team 3 Automated current budget No funding to use vendor solution in all financing journeys
42. Personal customer A fast & affordable financing solution Understand my current economy Regulators Enforce compliant credit practice Individualised budget calculation UX to provide customer data and consents Automatic budget calculation Statistical estimation methods Team 2 1 1 Buy - quick compliance Estimate using account transaction data View & edit budget 2 t Initiative 1 Budget API for housing journey Initiative 2 "Buy" - quick compliance 2020 2021 Initiative 3 "Build" individualised budget in legacy systems 2022 Team 4 Team 3 Team 2 3 Team 1 Automated current budget
43. Personal customer A fast & affordable financing solution Understand my current economy Regulators Enforce compliant credit practice Individualised budget calculation UX to provide customer data and consents Automatic budget calculation Statistical estimation methods Team 2 1 1 Buy - quick compliance Estimate using account transaction data View & edit budget 2 t Initiative 1 Budget API for housing journey Initiative 2 "Buy" - quick compliance 2020 2021 Initiative 3 2022 Team 4 Team 3 Team 2 3 Team 1 compliance more technical debt + Automated current budget "Build" individualised budget in legacy systems
44. Personal customer A fast & affordable financing solution Understand my current economy Regulators Enforce compliant credit practice Individualised budget calculation UX to provide customer data and consents Automatic budget calculation Statistical estimation methods Team 2 1 4 Estimate using account transaction data View & edit budget 2 Initiative 1 Budget API for housing journey Initiative 2 "Buy" - quick compliance 2020 2021 Initiative 3 2022 Team 4 Team 3 Team 2 3 Team 1 4 2022 Initiative 4 "Build" individualised budget in new Budget API 4: migrate logic to with new architecture Automated current budget "Build" individualised budget in legacy systems
45. Personal customer A fast & affordable financing solution Understand my current economy Regulators Enforce compliant credit practice Individualised budget calculation UX to provide customer data and consents Automatic budget calculation Statistical estimation methods Team 2 1 4 Estimate using account transaction data View & edit budget 2 Initiative 1 Budget API for housing journey Initiative 2 "Buy" - quick compliance 2020 2021 Initiative 3 2022 Team 4 Team 3 Team 2 3 Team 1 4 2022 Initiative 4 4: Connect with initiative 1 with new architecture Complex negotiation Automated current budget "Build" individualised budget in new Budget API "Build" individualised budget in legacy systems
46. Personal customer An affordable financing solution Understand my current economy Regulators Enforce compliant credit practice Individualised budget calculation UX to provide customer data and consents Automatic budget calculation Statistical estimation methods Team 2 1 4 Estimate using account transaction data View & edit budget 2 Initiative 1 Budget API for housing journey Initiative 2 "Buy" - quick compliance 2020 2021 Initiative 3 2022 Team 4 Team 3 Team 2 3 Team 1 4 2022 Initiative 4 5 ??? Initiative 5 ? "Build" a full-​ stack budget product "Build" - budget domain as internal product Higher order capability Our credit and risk expertise Full-​ stack budget solution Our credibility Automated current budget "Build" - budget domain as external product "Build" individualised budget in new Budget API "Build" individualised budget in legacy systems lower dependency & negotiation complexity
47. Personal customer A fast & affordable financing solution Understand my current economy Regulators Enforce compliant credit practice Individualised budget calculation UX to provide customer data and consents Automatic budget calculation Statistical estimation methods Team 2 1 4 Estimate using account transaction data View & edit budget 2 Initiative 1 Budget API for housing journey Initiative 2 "Buy" - quick compliance 2020 2021 Initiative 3 2022 Team 4 Team 3 Team 2 3 Team 1 4 2022 Initiative 4 5 ??? Initiative 5? "Build" - budget domain as internal product "Build" - budget domain as external product Can we compete on speed and cost? Someone else thinking similar thoughts? Do we have the skills to pull it off? Automated current budget "Build" individualised budget in new Budget API "Build" individualised budget in legacy systems "Build" a full-​ stack budget product
48. Being able to ask better questions can improve our odds of deciding and acting with correspondence to an emergent reality.
49. Friction points Revenge of Conway's Law reinforced by the agile mandate of team autonomy Team autonomy gets in the way of good design Let's reshape architecture to fit the team boundaries, so we can work independently of other teams. Shipping org charts
50. Efforts pulling apart, not together Local optimization, just at the initiative level The problem is systemic Consumer finance journey credit card, consumer loan... "automation" competing agendas good will hunting? an initiative an initiative Current Budget (DK) Housing journey buying & owning a home Asset finance journey car, boat, auto-​ camper...
51. Look at the next larger context Source: Ruth Malan (adapted to context) initiatives individual contributors teams enterprise scope of complexity decisions across boundaries systems relationships
52. Where are we? collaborative modelling value coherence strategic correspondence conditions & consequences (and boundary spanning roles) "Domain-​ Driven Architecture at Scale" recommoning (domain-​ driven platforming)
53. The Three Economies "More than a single economic logic at play in large enterprises"
54. Increase variety (variety=intended difference in outcome) Examples: Customer experience Customer facing processes Products Digital convenience Value drivers Efficiency drivers
55. Increase variety (variety=intended difference in outcome) Examples: Customer experience Customer facing processes Products Digital convenience Value drivers Efficiency drivers Decrease variation (variation: unintended difference in output) Examples: Consumable Primitives Compute Storage Network Database
56. Increase variety (variety=intended difference in outcome) Examples: Customer experience Customer facing processes Products Digital convenience Value drivers efficiency drivers Decrease variation (variation: unintended difference in output) Examples: consumable primitives Compute Storage Network Database Standardize to differentiate (Help others differentiate by making myself less different) Examples: resources gain value when shared Well-​ formed functions (APIs) & data Repeatable business processes Standardized configurations of primitives Other ways of "commoning" resources to accelerate differentiation value & efficiency drivers
57. Increase variety (variety=intended difference in outcome) Value drivers efficiency drivers Decrease variation (variation: unintended difference in output) value & efficiency drivers Recommoning (Re-​platforming) Examples: Customer experience Customer facing processes Products Digital convenience Examples: consumable primitives Compute Storage Network Database Examples: resources gain value when shared Well-​ formed functions (APIs) & data Repeatable business processes Standardized configurations of primitives Other ways of "commoning" resources to accelerate differentiation Standardize to differentiate (Help others differentiate by making myself less different)
58. Recommoning: Platform redefined ... as an opportunity to transform parts of the value chain exhibiting recurring patterns to shared services Domain-​ driven platforms
59. Housing journey Domain-​ driven development environments Consumer finance journey Asset finance journey Common credit processes Isolate complexity Reduce cognitive load for differentiation teams Ensure consistency & compliance Enable configurability & adaptability Bridge differentiation and scale economies for resilience Infrastructure domains Exists to enable differentiation Exists to increase variety Exists to decrease variation Domain-​ driven data platforms ("Data products") Domain-​ Driven Platforms Security domains
60. Systems thinking is a core skill in recommoning differentiation squad platform squad Another differentiation squad not in current initiative ... how can the platform design improve the optionality for more differentiation squads? Current Initiative optimize in "lines" platform squad differentiation squad platform team optimize in "networks" differentiation squad differentiation squad differentiation squad
61. What is better shared - the common ground? How can we simplify for the common good? How can we make a supple design less shakable by changes in the environment? What should be configurable? Where is the best sociotechnical boundary? Recommoning needs domain-​ driven architecture Consumer finance journey credit card, consumer loan... "automation" an initiative an initiative Current Budget (DK) Housing journey buying & owning a home Asset finance journey car, boat, auto-​ camper... How can we craft deliberate dependencies? to tackle the sociotechnical complexity at scale
62. Where are we? collaborative modelling value coherence strategic correspondence conditions & consequences (and boundary spanning roles) "Domain-​ Driven Architecture at Scale" recommoning (domain-​ driven platforming)
63. Architecture at scale is to create good conditions blueprint compre-​ hensive structures Negotiate common ground Architect by artefacts ("views") collaborative discovery & design adaptive capacity to integrate information (OODA) be better at systems thinking Think in Conditions-​ consequences Think in Means-​ ends Upfront design decisions Hard to reverse
64. Boundary-​ driven architecture for requisite coherence Source: Jabe Bloom Inside Inside Inside In-​ between "Look at what's in between, the inside will adapt"
65. Minimalistic architecture runway Architecture Runway Conceptual contour has has Example A common language weakly structured - don't go for finality sticky notes architecture Boundary objects
66. My personal paradox Super analytical Super outcome driven An architect archetype?
67. Staying with the struggle A team comes to me --> the first immediate impulse is to solve the problem for them A team comes to me --> how can I help them to find the best solution? Facilitate a space for conversation and negotiation to continue. Stay together in the struggle. Absorb feedback signals. "From advocacy to inquiry" A team comes to me --> take a step back. My job is not solving the problem for them. It's not even helping them find the answer here and now.
68. A new frame for being an architect Solve the right problem the right way Cultivate the conditions for good consequences Maker Multiplier Inspired by: Patrick Kua
69. Enabling Interact with complexity My own role DDD Evangelist Team architect & "cluster" architect Listen to the wisdom of the system Socialize knowledge and practice Build relation, trust, safety Speak the common language linear non-​linear Less stressed about outcome Expose my own thinking effectively (advocacy) .. and make that thinking open to the influence of others (inquiry) Facilitate conversation & negotiation
71. What we've covered today collaborative modelling value coherence strategic correspondence recommoning (domain driven platforming) conditions & consequences (and boundary spanning roles) "Domain-​ Driven Architecture at Scale" "understand to influence" "find my place in the story" "standardize to differentiate" "alternate realities & possible futures" "listen to the wisdom of the system"