Scaling a legacy big ball of mud product through collaborative software design @TSL

Numerous products kick off with a basic, initial model intended to demonstrate market viability. Often, this model must expand rapidly to include additional features, a necessity that can significantly complicate efforts to scale. This quick expansion can turn the initial model into an unwieldy "big ball of mud," rendering it less effective for addressing core business challenges. Development teams are thus faced with a dilemma: they need to refactor this cumbersome model without halting the addition of new features, which are essential for attracting more customers, scaling efficiently, and achieving profitability. The central question is, how can we navigate out of this complexity while also fostering product growth?

In my talk, I will share the journey of assisting a software development team in dividing into two more focused groups by breaking their "big ball of mud" into several distinct bounded contexts, all within the original monolithic framework. I will present various heuristics, practices, tools, and strategies used to simplify the existing model. This approach facilitated the continuous addition of new features and the collection of user feedback. A significant challenge was the team's initial lack of skills in software design and architecture. I will show how the team, along with their stakeholders, embraced a collaborative software design approach, enabling them to independently design, develop, and run software. This journey led to the development of a strategy to simplify the model, establish an idealised bounded context design, and software engineering becoming domain experts themselves. This approach, in turn, collaboratively guided and drove the product roadmap forward.

Transcript

1. @kenny_baas @kenny_baas@mastodon.social Scaling a legacy big ball of mud product through collaborative software design Photo by National Cancer Institute
2. @kenny_baas @kenny_baas@mastodon.social Written with MVC over 1.5-2 year time from scratch Based on previous models from the job application website Uses both the website, custom ERP and SaaS ERP Potential reusable capabilities for other products to automate manual processes Looks promising and strategy is to also go live in another country Scaled up from 3 to 13 engineers, hybrid working Keep delivering new features Photo by Osman Rana on Unsplash Status of the product
3. What do I do @kenny_baas @kenny_baas@mastodon.social Independent software consultant, tech lead, and software architect, catalysing organisations and teams towards designing and building sustainable and resilient software architectures. Team Topologies valued practitioner, enabling organisation to organizing business and technology teams for fast flow in combination with Domain-​ Driven Design. Enabling teams to independently undertake software design, while providing expertise to modernize their software architecture. This includes untangling and modernizing legacy software systems, as well as improving the overall product.
4. @kenny_baas @kenny_baas@mastodon.social Agenda How Domain-​ Driven Design enabled scaling Why understanding the underlying needs of users was crucial Embedding software architecture in the team
5. @kenny_baas @kenny_baas@mastodon.social It is designed to make programming web applications easier by making assumptions about what every developer needs to get started. It allows you to write less code while accomplishing more than many other languages and frameworks. Rails emphasizes the use of convention over configuration (CoC) and follows the principle of Don't Repeat Yourself (DRY).
6. @kenny_baas @kenny_baas@mastodon.social The Model is usually the boundary, where new features are being added...Ending up with a Big Ball of Mud !! This also counts for most implementations using an ORM framework
7. Team A Team B Application Database Somewhere around the Millenium @kenny_baas @kenny_baas@mastodon.social
8. @kenny_baas @kenny_baas@mastodon.social Domain models often start based on a naïve, over simplified entity-​relationship model based on reality. Good when you start, but when dealing with more complicated and complex problems it will become a big ball of mud
9. @kenny_baas @kenny_baas@mastodon.social Platonic Domain model thinking assumes idealized, abstract representations of reality which may not always capture the intricacies of a domain. This can be misleading in software design especially in core or complicated supporting domains where the nuances matter. A naïve model, influenced by this thinking, can confine you to an oversimplified view, potentially leading to incorrect, suboptimal and unadaptable software solutions. Inspired by Barry M O'Reilly - Plato is the devil Anti-​Pattern: Platonic Domain Model Thinking
10. @kenny_baas @kenny_baas@mastodon.social Users and stakeholders Code Stream-​Aligned Team Tests Domain Model Other people possibly involved in designing and building software Managers Architects User researcher .....
11. Photo by pure julia on Unsplash For each specific problem, a different map For each specific problem, a different model @kenny_baas @kenny_baas@mastodon.social
12. @kenny_baas @kenny_baas@mastodon.social Job template Jobs Jobs Applications Candidate applications Application Job drafting Job Fulfilment Candidate applications
13. @kenny_baas @kenny_baas@mastodon.social Job template Jobs Jobs Applications Candidate applications Application ...... Domain-​Driven Design allows you to break down your domain into smaller models, enabling a single team to take ownership, even within a monolithic architecture. This empowers the team to make changes more independently, as it reduces the impact of those changes on other models. Job drafting Job Fulfilment Candidate applications
14. @kenny_baas @kenny_baas@mastodon.social Photo by oxana v on Unsplash Start extracting bounded context guided by the roadmap What bounded context do we touch that is on the backlog? What is something on the backlog what is hard to do with the current model? Keep the shop open
15. DDD is a discipline rooted in the belief that creating good software systems for problems in the complex domain cannot be done without a deep understanding of the business problems you are trying to solve in the domain. Summary @kenny_baas @kenny_baas@mastodon.social
16. @kenny_baas @kenny_baas@mastodon.social Why understanding the underlying needs of users was crucial Photo by Kelly Sikkema on Unsplash
17. @kenny_baas @kenny_baas@mastodon.social Onboarding a student Signup for an account Screen a candidate Upload compliancy information and documents Plan a kickstart Check student compliancy Enable on platform Student 18-25 years Work single day job when money is needed Recruiter Find the best job for a student to work on apply for a job user (external) needs user (internal) Capabilities Account service Kickstart service Screening service Compliancy service Candidate service
18. @kenny_baas @kenny_baas@mastodon.social Anti-​Pattern: Splitting a big ball of mud based on objects or process Refactoring a monolithic system by dividing it based on abstract or "platonic" object concepts can lead to a fragmented architecture, an approach that's an acknowledged anti-​ pattern. This method, focused on idealized object categories rather than concrete functionalities, often results in ambiguous module responsibilities, impractical implementations, increased maintenance overhead, and reduced adaptability to new requirements. Inspired by Barry M O'Reilly - Plato is the devil
19. What problems are we trying to solve? Indi Young @kenny_baas @kenny_baas@mastodon.social indiyoung.com X Explanations Problem Space The problem space is a person's domain. It's where they are addressing something they want to make progress on, or put off, or just think about. The problem space is separate from your solutions, although the person may reach out for your tool along the…
20. Purpose & Problems Value stream: All activities a team performs from understanding the purpose & problem to building that solution without being blocked Solution Feedback Stream-​aligned team: Long-​ term design + build + run for software-​ enriched services in a small, stable team of around ~5-9 people Stream-​aligned teams @kenny_baas @kenny_baas@mastodon.social 💡Stop managing dependencies start unblocking flow 💡You build it, you design and run it!
21. is a visualisation technique to analyse complex and conflict-​ laden decision-​ making processes with all relevant stakeholders and decision-​ makers to create a shared understanding. @kenny_baas @kenny_baas@mastodon.social Collaborative modelling
22. @kenny_baas @kenny_baas@mastodon.social https://www.eventstorming.com/ First time we started with only the team, on-​site. Afterward we move it to a miro board, so we could continue online. EventStorming became the default modelling language to design software for the engineers. Once the engineers had a good understanding of EventStorming, we started doing session with stakeholders 1. 2. 3. 4.
23. @kenny_baas @kenny_baas@mastodon.social ........ ........ ........ ........ ........ ........ ........ Start with 1 virtual board, and let the team self organise and enable structuring by splitting the miro board when needed. Giving each bounded context it's own board * Note, the team themselves decided on using miro
24. @kenny_baas @kenny_baas@mastodon.social Prepare templates before doing collaborative modelling with certain key stakeholder
25. @kenny_baas @kenny_baas@mastodon.social Distilling Bounded Context Onboarding a student Signup for an account Screen a candidate Upload compliancy information and documents Plan a kickstart Check student compliancy Enable on platform Student 18-25 years Work single day job when money is needed Recruiter Find the best job for a student to work on apply for a job user (external) needs user (internal) Capabilities Candidate screening Candidate compliancy Candidate management Account management Split according to the: Language ambiguity Departments User needs And run through the design with scenarios 1. 2. 3.
26. @kenny_baas @kenny_baas@mastodon.social Photo by Trnava University on Unsplash Start applying Domain-​Driven Design on a complicated, medium risk problem. That way the team can experiment with the new patterns safely, and get used too the paradigm shift without to much risk and organisation exposure
27. @kenny_baas @kenny_baas@mastodon.social We are so caught up in the solution space that we miss perspectives that we forget to understand that we miss opportunities   ​  ​  ​  ​  ​  ​— Indi Young CC BY-​ SA: INDI YOUNG Summary
28. @kenny_baas @kenny_baas@mastodon.social The goal is to enable software teams to collaborate with their stakeholders, to understand their needs, and to allow this understanding to shape their software architecture. This is the essence of what we describe as teams engaging in collaborative software design! Embedding software architecture in the team 💡Stop designing software for teams to build, start enabling teams to design it themselves
29. @kenny_baas @kenny_baas@mastodon.social A paradigm shift Command Bounded Context Entity Domain Event Event Handler Command Bounded Context Entity Domain Event Controllers (MVC or Rest)
30. @kenny_baas @kenny_baas@mastodon.social Photo by Nima Mot on Unsplash "Pair programming is not just about writing code together; it's about weaving two minds into a singular thread of creativity and problem-​solving."
31. @kenny_baas @kenny_baas@mastodon.social Controllers (MVC or Rest) Rest request (command) Rest request (query) Company management Command Services Aggregate Company Registered WorkLoc ationUp dated Legacy model Rest request (query) Event Handler Do something else Database Entity table Column Column Column Updates Views Entity Reads request (command) request (query) Bubble context Controllers (MVC or Rest) Rest request (command) Rest request (query) Company management Command Services Aggregate Company Registered WorkLoc ationUp dated Legacy model Rest request (query) Event Handler Database Aggregate table Column Column Column Views Entity Reads Read entity Table Updates Column Column Column Event Handler Do something else request (command) request (query) Autonomous Bubble context
32. @kenny_baas @kenny_baas@mastodon.social https://bytesizearchitecturesessions.com/ Andrea Magnorsky Bytesize architecture session
33. Sustainable design decisions are decisions that consider the sociotechnical impact of the alternatives. After designing and analysing the alternatives, you pick the one that gives you the best sociotechnical trade-​ off with the information that is available. You create a feedback loop into your design process to improve it along the way. Over time, you will find yourself making decisions in a similar way. This is when your decisions become sustainable. Complex Environment Sociotechnical System Structure (Organization) Adapted from "Assessing the impact of new technology on complex sociotechnical systems", South African Journal of Industrial Engineering August 2016 Vol 27(2), pp 15-29, R. Oosthuizen & L. Pretorius https://www.researchgate.net/publication/306242078_Assessing_the_impact_of_new_technology_on_complex_sociotechnical_systems Licensed under https://creativecommons.org/licenses/by-​ sa/4.0/ People (Cognitive & Social) Physical System (Hardware, Software, Facilities) Task (Work) @kenny_baas @kenny_baas@mastodon.social Summary
34. @kenny_baas @kenny_baas@mastodon.social kenny@weave-​ it.org - weave-​ it.org Get the book here Where every voice shapes the software Thank you!