Sociotechnical design for software and human systems

Talk at Explore DDD 2024, Denver.

As a community, we’ve come a long way modeling software-centric systems. But what about the runtime dynamics of human systems in our working environment, especially when such systems are full of fixes that don’t work? As software professionals, can we just tell jokes about SAFe, about being able to copy the Spotify Model, but not paste it? Or should we give the usual shrug – “it’s the system’s fault”?

Xin has been a design and architecture practitioner for more than a decade. Drawing on her own developmental journey, Xin makes a case for the rising relevance of sociotechnical design in a post-modern world, where aging companies struggle with aging software, while adding new software and complexity to their IT portfolio. With good attractor effect, practices like Domain-Driven Design(DDD), Systems Thinking and Constraints Framework are evolving into a multi-dimensional, sociotechnical design toolbox.

From a systems standpoint, we are part of the same mess. If we can become better at seeing and naming the elephants in the room, not as random events, but as systemic patterns described through a consistent systems language, then the current messy reality is no longer our enemy but our ally. It can become a generative force for us to identify leverage and sustain influence in large organizational spaces.

Join Xin to reflect together on, how DDD and adjacent practices can be leveraged as powerful enabling constraints to help us see the system, share the system, and build the system in a reality of vast sociotechnical complexity and constant change.

Transcript

1. Xin Yao, Explore DDD '24 Sociotechnical design for software and human systems
2. DDD & architecture consultancy DDD output + + Sociotechnical design + Systems leadership + Who is Xin? Undercover change agent through DDD and architecture Xin Yao @settling_mud xin.yao.dk@gmail.com @xinyao@mathstodon.xyz /in/xinxin/ delay + delay Independent consultant
3. Navigating sociotechnical complexity Today's path Software-​ centric systems design 1. Work design (Social systems) 2.
4. I imagined a neat and tidy software career
5. Variables Algorithms
6. Software is logical, and "compiles" Right or wrong answer a mathematically provable range of optimized solutions or at least Deterministic solution
7. The universe is a giant clock. The laws of nature act like gears and springs, causing everything to move in a predictable and orderly way. -- Issac Newton (Not exactly, but kinda the exact words)
8. My first real-​ life software project Fixed price software delivery in a consultant team Thorough requirement spec from client Client analysts answering our queries, mostly by email Software architecture blueprint upfront Waterfall - Project lead had regular checkin with client Great team - we delivered on time! Domain: insurance workflow
9. We applied design patterns
10. Single Responsibility Principle Open/Closed Principle Liskov Substitution Principle Interface Segregation Principle Dependency Inversion Principle Image Credit: Robert Ecker We applied SOLID principles
11. But, our software never made it to the users
12. The client's analysts misunderstood (and oversimplified) core user needs. We made the wrong interpretation of the client analysts' email response to our questions. The client's business sponsor didn't align with the parent company about starting the project. No one told us a mandatory frontend framework had to be used. Rewrite considered too costly. Project X dropped, shelved, and forgotten. We (the consultant team) got paid well. We got praised for on-​ time delivery. Lots of tech learning and experiences, so why are we not feeling happy? So what's the rub
13. Design cannot happen in a closed system My existential conundrum Long way from neat and tidy to useful software <> "giant clock"
14. Software's value is always evaluated in context Context Software Software systems are open systems
15. Software's context is full of decisions that can go wrong Picks the wrong direction Builds things that don't matter Builds wrong things Incidents & outages Builds things the wrong way Credit: Jabe Bloom (adapted to context) Which area(s) do you see higher risks here in your organization, where well-​ meaning decisions do not end up having the desired impact? Leadership (team) Product (team) Development (team) Operations (team) Architecture (team)
16. Leadership (team) Product (team) Development (team) Operations (team) Architecture (team) Picks the wrong direction Builds things that don't matter Builds wrong things Incidents & outages Builds things wrong Decisions are interdependent The whole story How does each part understand the whole story? Architect Developer
17. Software's context is full of human relations Credit: Henrik Kniberg (Adapted) Maker User Decision makers handoffs/intermediaries Time (Delay in feedback)
18. Socio- technical Sociotechnical complexity Complexity in "Parts" (e.g. tasks, code, Jira tickets) Credit: University of Leeds (Sociotechnical Center) Complexity in "Relations" (e.g. interfaces, meetings, decisions, practices) Infrastructure People Technology Culture Processes /Procedures Goals / Metrics
19. What many people fail to realize is that the relationship is the delivery system of anything we wish to accomplish. -- Peter Block
20. Domain-​ Driven Design (DDD) as a practice Visually model the whole-​ part, part-​ whole relations
21. Who work and create value together? Who else do we need to align understanding with? DDD Three core practices in DDD Credit: Paul Rayner (adapted and extended) What software are we building? Why are we building it? How do we build and connect software - for long-​term changeability? Strategic Design Tactical design Visual Collaborative Modeling (models) (models)
22. Ever wondered why event storming works magic, almost every time?
23. See the forest and the trees
24. See the forest and the trees But not at the same time And not like this
25. Complex Systems scale by decomposition to the lowest consistent level of granularity, and recombination, not by aggregation or imitation. -- Dave Snowden Knowledge is created, not shared Granular discovery Distributed cognition Dis-​ intermediation of data Principles of managing complexity (Credit: Dave Snowden)
26. Linking strategy to everyday work Good experience with "the North Star Framework" input input input input North Star (The purpose) The value "bets" The work "bets" Opportunities Interventions My ticket leverage point solution agnostic, but opinionated actual work experiment epics stories 1-3 years 1-3 quarters 1-3 months 1-3 sprints Model of strategy Model of strategy deployment
27. squad Tribe A squad squad squad Tribe B squad A new initiative OKR OKR OKR OKR OKR OKR OKR A shared and deeper understanding of the "WHY" How does my contribution matter in the big picture, over time? Find my place in the story
28. Joy in work comes from understanding why your work is important. Not from work, but from the knowledge of who's going to use it. -- W. Edward Deming
29. Lead with the WHY, not the WAY Architect for value coherence Credit: John Cutler
30. A DDD workshop series Strategic Design (models) Tactical design (models) Visual Collaborative Modeling the WHY the WAY
31. We explored sociotechnical design with software as the centerpiece Context Software System boundary Notation
32. Software's context is a System of Systems Me A company My team Another team Environment System boundary Notation
33. Navigating sociotechnical complexity Today's path Software-​ centric systems design 1. Work design (Social Systems) 2.
34. Is complexity a bad thing? Should we simplify complexity? Software industry matures Every business is a software business Aging companies with aging software Compounding sociotechnical complexity
35. Ashby's Law of Requisite Variety If a system is to be stable the number of states of its control mechanism must be greater than or equal to the number of the states being controlled. --​Ross Ashby You can choose how much environmental complexity you want to consume based on how much of of that environment you want to occupy. -- Jabe Bloom Requisite Complexity
36. Complexity is not bad, it just needs to be designed for.
37. + Software-​ centric Sociotechnical complexity Work-​ centric Sociotechnical complexity Software design Work design
38. Software-​ centric Sociotechnical complexity Software design DDD Product thinking Strategy COMO** DevOps Software Architecture We've come a long way with DDD and friends* *Friend: the adjacent possible **COMO: Collaborative Modeling Systems Thinking
39. Work-​ centric Sociotechnical complexity Work design What about
40. Work-​ centric Sociotechnical complexity Work design Sociotechnical mirroring Agile process & tooling Inverse Conway Spotify model Learned helplessness Skepticism Unknowability A big divide Pessimism Non-​linearity Lack of agency
41. Human systems are too complex to manage Let's have the next joke about SAFe, anyone?
42. Software-​ centric Sociotechnical complexity Work-​ centric Sociotechnical complexity Software design Work design We've come a long way with DDD and friends* We are struggling with this
43. The gold is in the struggle
44. "My child has grown so much" It's the same person that has grown Credit: Humberto Maturana It's a system It's complex Something hopeful in complex systems
45. What is complex is also systemic Stable identity Boundary Order norms, rules, social contracts purpose, goal, outcome structure, culture ("preferential relations") disposition discoverable, workable, architectable In flux, changing Emergent uncertain unpredictable uncontrollable causality unknowable Slowness Entangled system properties Inter-​ dependent
46. Explanation Assumption More general theory Most general theory Theory of reality Concept about nature of the world Rests on Rests on Traces back to How we make sense of the world Credit: Russell Ackoff
47. Constraints Framework Unifying Systems Thinking & Complexity Theory Explanation Assumption More general theory Even more general theory Theory of reality Concept about nature of the world Rests on Rests on Traces back to Reality is complex, but systemic. Systems Thinking Complexity Theory Sociotechnical architectablilty Frameworks, Canvases, Tools, Processes, Practice Uncontrollable, Unpredictable, Unprovable Relatable Communicable Can be influenced Constraints Framework
48. 1. Enabling constraints coalesce coherence Bottom Up Enabling constraints 1 Bottom-​ up, Generative, Opening up options But not through cause-​ and-​ effect Induce conditional probability of initially separate entities/events/processes becoming interdependent. Enabled interdependencies coalesce as coherent dynamics entering the higher-​ level Context, the interdependent Whole. Example: Catalyst (like participatory discovery, visualizing, languaging, modeling, reflective conversation), Feedback loops. Credit: Jabe Bloom, Alicia Juarrero Enabling Constraints Whole
49. 2.Environmental feedback stabilizes coherence Bottom Up Enabling Constraints 1 The system outputs products/services to the environment. Value creation hypothesis is tested in the environment. Design is evaluated in use. Environmental feedback brings about Constraint Closure, stabilizes coherent dynamics, and expands the system's phase space (interrelatedness, complexity, optionality). Example: Startups become scaleups, Disrupted Industries. 2 Feedback Credit: Jabe Bloom, Alicia Juarrero Environmental Feedback ... or disrupts coherence if feedback is negative Environment Phase Space Whole
50. Bottom up enabled coherences manifest as interactional/relational types and patterns. These types are multiply realized in a systemwide archtiecture that preserves interdependencies within coherent dynamics. Thus, a second-​ order Constitutive Constraint Regime arises concomitantly and organically with the bottom up dynamics. Constitutive constraints embed into the higher-​ level Context, acting top down as Governing Constraints, maintaining order, restricting behavior and strengthening system identity. A system is thus constituted and preserved through preferential relations and experiential coherence. Example: An organization's contextual framework (norms, rules, social contracts, practices, culture) constrains and organizes events and processes within coherence boundaries. Constitutive/Governing Constraints Feedback Whole Bottom Up Enabling Constraints 1 2 3 3.Constitutive & governing constraints persist coherence Credit: Jabe Bloom, Alicia Juarrero, Humberto Maturana Constitutive / Governing Constraints Top Down Environment Constraints constitute the system through preferential relations
51. Credit: Jim Benson, Alicia Juarrero Catalytic Enabling Constraint: Visualizing work - See the Whole System Tactical info Strategic Info Cultural Info what is going on? what needs attention now? Where are we going? Where is my place in the story? How is my team a team? What does good look like? What is meaningful for me? (Agency) Example: 7 Elements of Visual Management Purposeful models Software-​ centric design Discoveries Decisions Shared understanding Software design State Triggers Visualized work Direction Narrative Culture Professionalism Identity Emergencies Action Improvement Opportunity Discovery need Shared work Complex work Current WIP Who is working Who is collaborating What is ready for release What is stuck How is our psychological flow Strategy Plans & Backlogs Upcoming work options Onboarding Upskilling Team identity Interaction & meetings Human repair Social contracts Improvement Correction Learning Quality Who am I? How is work meaningful? How is work developmental? Agency Spheres of influence Affordance
52. Visual Pull Planning as Enabling Constraint Credit: Milestone Systems, Systems Architecture Team Team Daily Stand-​ up Prompts what is going on? what needs attention now? Where are we going? Where is my place in the story? How is my team a team? What does good look like? What is meaningful for me? (Agency) Example: 7 Elements of Visual Management Purposeful models Software-​ centric design Discoveries Decisions Shared understanding
53. VS step VS variation / option Pain (Problem Challenge Issue) Improvement Opportunity Collaboration Opportunity Notation VS step VS step VS step VS step VS step VS step VS variation / option VS variation / option VS variation / option Pain (Problem Challenge Issue) Pain (Problem Challenge Issue) Pain (Problem Challenge Issue) Improvement Opportunity Improvement Opportunity Improvement Opportunity Pain (Problem Challenge Issue) Pain (Problem Challenge Issue) Improvement Opportunity Improvement Opportunity Pain (Problem Challenge Issue) Improvement Opportunity Collaboration Opportunity Collaboration Opportunity Collaboration Opportunity Collaboration Opportunity Constraint Closure: virtuous probability cycle Development Value Stream Mapping Deep, reflective conversations Constitutive / Governing Constraint enabling constraint t Constraint Closure Social Contracts Reflective Conversation VS step VS variation / option Pain (Problem Challenge Issue) Improvement Opportunity Collaboration Opportunity Pull planning PDSA* *PDSA: Plan, Do, Study, Act Accept good interrupts more causal, repeatable more probabilistic emergent Languaging & Emotioning Credit: Jim Benson, Humberto Maturana
54. Enabling and stabilizing scaffolds Boundary-​ driven sociotechnical architecture Credit: Jabe Bloom Inside Inside Inside In-​ between "Look at what's in between, the inside will adapt" Danger Define APIs between software and teams - sociotech mirroring as architeture blueprint Instead Boundary social practices (materials, skills, meaning) Instead Value coherence, interface negotiation, SLOs, visual modeling enabling constraint Constitutive constraint Governing Constraint
55. Many can copy the Spotify Model. None seems to be able to paste it. -- Anonymous Constraints are context-​ dependent
56. Software-​ centric Sociotechnical complexity Work-​ centric Sociotechnical complexity Software design Work design DDD Product thinking Strategy COMO DevOps Architecture We've come a long way with DDD and friends* DDD Reflective Conversation Visual modeling Sociotehnical architecture Systems Thinking Complexity Theory Constraint modeling Languaging DDD & adjacent practices can be effective enabling and constitutive constraints Sense making Emotioning
57. The gold is in the struggle Design for stability Sociotechnical complexity can be designed for stability slowness Identity Boundary Order Design for variability change emergence agency affordance
58. Enabling coherent heterogeneity in mereological, sociotechnical design Affordance Agency What is afforded from the context to allow the system to balance stability with variability Do I/we feel free to choose actions and relations that cohere with the Whole? Agency of purpose, goals, and process Agency of individuals, teams, divisions, orgs Social practice (material, skill, meaning), Leadership cover for emergent ideas and variability, Psychological safety, experimentation Credit: Herbert Spencer, Dave Snowden, Alicia Juarrero, Jabe Bloom Whole-​ to-​ Part Part-​ to-​ Whole ... and enable the coherence of a new Whole
59. Navigating complexity is a developmental journey
60. DDD consultancy/ facilitation DDD output + + Sociotechnical design + Systems leadership + delay + delay (Re)Walk the lines How can I contribute to designing enabling & constitutive constraints to influence the conditional probability of more healthy and humane systems dynamics, and more self-​ aware, improvable ways of working? Cause & Effect Conditional probability
61. Social systems design targets a possibility space, not a fixed outcome.
62. The future is not the same place we are going to, but one we are creating. The paths are not to be found, but made, and the activity of making them changes both the maker and the destinations. -- John Schaar We can only create the path by walking it
63. Enlightenment is when a wave realizes that it is the ocean. -- Thich Nhat Nanh
64. Thank you xin.yao.dk@gmail.com @xinyao@mathstodon.xyz /in/xinxin/ @settling_mud
65. Sociotechnical design for software and human systems Sociotechnical architecture modernization lead Xin Yao