Data Ownership in Legacy Systems - Building a Foundation for Data_Mesh

Data analysis and reporting have traditionally operated using tailor-made data models after the operational data has been extracted and transformed. Behind it we find that the application development uses a data model which is much less suited for data analysis and reporting. One of the main reasons for this mismatch is the approach denoted as functional decomposition, where the system is designed based on functions as described in requirements, use-cases, product backlogs etc. Moreover, this approach is the one that most application developers take as a default choice.

What if there is another way to do the application design that explicitly allows the data to play a much more prominent role in system decomposition? What if this alternative approach can bridge the gap between application development and data analysis and reporting?

Join and explore an alternative to complex system design that emphasizes data ownership with foundation in business capabilities, which presents a very different opportunity for rich data analysis and reporting. The talk is based on real-world cases from the domains for healthcare and insurance.

Transcript

1. Data Ownership in Legacy Systems Building a Foundation for Data Mesh Software Craftspeople India Community, Online 2023-11-22 Mufrid Krilic, Domain-Driven Design Coach CoWork, Norway
2. About myself…. • Developer, architect, agile and technical coach • Healthcare, Telecom, Insurance • “Domain-Driven Design enthusiast” Crafting Your Digital Confidence
3. CoWork - something to inspire you • Trust-Based Leadership in Practice Tight-Loose-Tight - TLT Crafting Your Digital Confidence
4. Some light reading
5. Takeaways after the session • Different perspectives between application development and data analysis • Understanding needs for business insights with Domain Storytelling • Establishing data ownership with Domain-Driven Design
6. On Different Perspectives Application Development and Data Analysis
7. The problem we are solving • Being data-driven is a mandatory in modern business • However …. • Data management at scale is highly complex area with socio- technical implications
8. Different perspectives • Application Development • Data Analysis • Operational data • Transaction based and state-aware • Analytical data • Aggregated business events to provide future insight or retrospect
9. Organizational Legacy • Common to find a team for data analysis and reporting • Separate from application development • Based on a real needs for different data model for analytical data • Data model for operational data is usually not adequate
11. Four Principles 1. Domain-oriented data ownership 2. Data as a product 3. Self-served data platform 4. Federated Computational Governance
12. Established Domain Boundaries • Prerequisite for Data Mesh! • «domain-oriented decentralized data ownership and architecture»
13. Domain-oriented? • Data ownership closely aligned with the data source • Multiple autonomous yet integrated models
14. How to Reduce the Gap? • Reducing the gap between the operational and analytical data addresses complexity with socio- technical approach • Combined with decentralization to address scaling • Can we take a different approach? • Modelling of application data
15. Domain-Driven Design Principles of Strategic DDD
17. Strategic DDD • Focusing on Core domain • Problem and solution space • Subdomains and Bounded Contexts • Linguistic boundaries • Ubiquitous Language • The two pillars of DDD
18. Core Domain • The thing that distinguishes you from the competitors • “Not every part of the system will be well-designed” • Generic subdomain • Supporting subdomain
19. Problem Space and Solution Space Problem space • Domain analysis to discover inherent Subdomains Solution space • Domain modeling to discover corresponding Bounded Contexts • Hard to find 1:1 mapping though
20. Linguistic Boundaries Ubiquitous Language • The same language everywhere • Conversations • Documentation • Code
21. The Two Pillars of DDD 1. Ubiquitous Language 2. Bounded Context Bounded contexts in your application are defined by linguistic boundaries
22. Cross-Functional Collaboration • Build software in a way that domain experts would have built it themselves • Discovery process where everyone, including domain experts, learn and understand more about the domain
23. Strategic DDD • Focusing on Core domain • Problem and solution space • Subdomains and Bounded Contexts • Linguistic boundaries • Ubiquitous Language • The two pillars of DDD
24. It’s Story Time Domain of Healthcare – Group Treatment
25. ▪ For patients with identical diagnoses sometimes the treatment has the best effect when done in a group of patients ▪ Examples from different subdomain in healthcare: • Conversation therapy groups in psychiatry • Training sessions post-injury Domain: Group Treatment
26. ▪ Setting up a group • Planning ▪ Conducting group appointments • Check-in process Group Treatment Use Cases
28. Group Treatment Planning Somatic Department Rehabilitation after lifestyle affected diagnosis
35. Group Treatment Check-In Somatic Department Rehabilitation after lifestyle affected diagnosis
41. Group Treatment Planning Psychiatric Department Adult Psychiatry
47. Group Treatment Check-In Psychiatric Department Adult Psychiatry
54. Data Ownership and Modeling On different approaches to application development
55. Data Ownership • A business term is defined as a set of data properties
56. Functional Decomposition • Boundaries in the system follow the function that a user needs to do hers/his job • Classic approach • Subdomain = Function
57. Group Treatment in Healthcare
58. Domain Model
59. Data Ownership in Domain Model – Functional Decomposition • One single “Group” for different functions • Physician and specialists • Patients • Appointments • Location/Venue • Patient attendance, including no-show • Patients exempted from payment • Invoice – preferred method • Diagnosis-based patient fee • Reference to § in law for compulsory interventions in psychiatry
60. Role-based decomposition • Boundaries in the system based on which roles perform different functions • Leads to more task-oriented model • Subdomain = Role based function
63. Data Ownership in Domain Model – role-based decomposition “Group” planning context • Specialists • Patients • Appointments • Location • Reference to § in law for psychiatry “Group” check-in context • Patient attendance incl. no-show • Cancelled appointments • Patients exempted from payment • Invoice – preferred method • Diagnosis-based patient fee
64. Time- and role-based decomposition • Boundaries in the system based on which roles perform different function at different times • Supports context-oriented systems • Subdomain = Role-based function in a user context
67. Data Ownership in Domain Model – time- and role-based decomposition “Group” planning context • Specialists • Patients • Appointments • Location • Reference to § in law for psychiatry “Group” check-in context • Patients exempted from payment • Invoice preferred method • Diagnosis-based patient fee “Group” billing context • Patient attendance, incl. no-show • Cancelled appointments
68. Aligned with the Business? The first principle of Data Mesh “Domain-oriented decentralized data ownership and architecture”
69. The Business and the Legacy System • The more complex the legacy system…. • ….and the longer the system is in production • ….the more likely that the domain language will be affected …. by the language of the legacy system!
70. Distilling the Domain with Pure Domain Stories • Capturing the very essence of the business processes Questions to ask: • How would you do your work without the software system? • What are you trying to achieve? • Why are you doing this?
71. Group Planning Somatic Department
72. Group Planning Psychiatric Department
73. Departments •Accident and Emergency Department •Anaesthesia and Surgical Services •Cancer Treatment and Medical Physics •Children and Youth Clinic •Clinical Nutrition •Communication •Department of Occupational Therapy •Dermatology •Emergency Clinic •Emergency Department Short Stay Unit •Finance •Haukeland hotel •Heart Disease •Human Resources •Internal Medicine •International Collaboration •Laboratory Medicine and Pathology •Maternity Ward •Medical Biochemistry and Pharmacology MBF •Medical Genetics •Neurology •Neurosurgery •Occupational Medicine •Occupational Outpatient Clinic •Ophthalmology •Oral Surgery •Orthopedic Clinic •Physiotherapy •Psychiatry •Radiology department •Recruitment and Temporary Staffing Office •Regional Centre for Asthma, Allergy and Other Hypersensitivity illnesses in Western Norway •Research and Development •Rheumatology •Secretariat for hospital management •Surgical Clinic •The Cancer Center for Education and rehabilitation- CCER •The Norwegian Arthritis Registry - NorArthritis •The Norwegian Porphyria Centre NAPOS •Thoracic Medicine •Treatment abroad •Tuberculosis clinic •Women's Clinic
74. Decompositions by business capabilities • Boundaries in the system follow the capabilities that the business offers its’ customers • Subdomain = Business Capability • Subdomain ≠ Function • Foundation for the product architecture
77. Data Ownership – Decomposition by Business Capabilities “Group” – psychiatry capability • Psychologist • Psychiatrist • Diagnosis • Reference to § in law • Patients • Appointments • Location for appointment outside hospital premises “Group” – medical clinical services • Physician • Diagnosis • Patients • Appointments
78. More business capabilities in healthcare • Neurosurgery • Birth clinic • Cardiology • Etc.
79. Business capabilities in other domains Insurance - Centered on products or capabilities • House • Car • Life Instead of functions • Such as policy, coverage
80. Product architecture • There are some product related decisions to be made. • Which users are we tailoring our products for? • What is the cost of customizing the product for diverse user groups? • What is the cost of developing separate products for separate user groups?
81. Conclusion and Takeaways
82. About Functional Decomposition • Customer/end-user needs are hidden behind functions! • No incentives to decompose • Pull towards canonical domain model • One of the main reasons behind the gap between operational data model and needs for data-insight aligned model
83. Data-Driven • Consider modeling around Business Capabilities to unlock the real Business Needs • Reducing the gap between operational and analytical data needed for business insights
84. Language as a tool • The same domain term in different subdomains is defined with different or possibly overlapping set of data properties • Use domain language to find appropriate level for decomposition
85. Takeaways ▪ Respect and align different needs for application development and data analysis – Data Mesh ▪ Different approaches to modeling in application development to get closer to the real business needs – Avoid functional decomposition ▪ Establish data ownership with Domain-Driven Design Ella Fitzegerald – «They Cant’t Take That Away From Me»
87. Images • https://unsplash.com/@x_vinicius • https://unsplash.com/@mpho_mojapelo • https://unsplash.com/@fazurrehman • https://unsplash.com/@freegraphictoday • https://unsplash.com/@dancristianp • https://unsplash.com/@patrickperkins • https://unsplash.com/@nadineshaabana • https://unsplash.com/@chuttersnap • https://unsplash.com/@ceebeesnap • https://unsplash.com/@renemolenkamp • https://unsplash.com/@rosiekerr • https://unsplash.com/@janilson123 • https://unsplash.com/@profwicks • Bing AI