$30 off During Our Annual Pro Sale. View details »

Domain Driven Design & Data Mesh

Domain Driven Design & Data Mesh

Data mesh is a decentralized sociotechnical approach for data management. Focusing on the technology, we tend to neglect the socio aspect of moving to new decentralized approaches. If we want to invest in data mesh architecture work, we must also invest in the people to make our data mesh more sustainable. How do we make people feel responsible for the data they expose and use? How do we make data management meet business objectives instead of a bureaucratic burden.

Decentralized approaches, e.g. like microservices, have been around for a while now. And we notice similar patterns of companies moving to a microservices architecture happening again when moving towards data mesh. Join this webinar to learn from these experiences and prevent obstacles for the future.

Kenny Baas-Schwegler

March 15, 2022
Tweet

More Decks by Kenny Baas-Schwegler

Other Decks in Technology

Transcript

  1. Domain Driven Design & Data Mesh Webinar 15/03/2022 Kenny Baas-Schwegler

    Peter Kromhout Niels Zeilemaker
  2. Photo by Şahin Sezer Dinçer on Unsplash

  3. “Most organizations don’t have their Data Management in order, and

    have found themselves in a data mess because of it”
  4. Businesses miss out on opportunities

  5. It is a waste of resources

  6. You break the law 746 225 90 60 60 50

    35 27,8 22 20,4 17 12,3 10,4 9 8,5 8,15 7 6 5 4,5 0 100 200 300 400 500 600 700 800 Amazon WhatsApp Google Ireland Facebook Google LLC Google H&M TIM British Airways Marriott Wind Vodafone Italia Notebooksbilliger Austrian Post Eni Vodafone Spain Google Caixabank BBVA bank Fastweb In millions € 20 biggest GDPR fines till date (900 in total)
  7. Agenda • Why do we do what we do? •

    From centralised to decentralised, we've been here before.. • What do the data mesh principles have to do with data management? • How do we get to a domain-oriented ownership? • The engineering way of solving this problem • Why a Data Mesh is a decentralized sociotechnical approach
  8. Why do we do what we do? * * =

    Data- engineers, analysts, scientists, architects, leaders ..
  9. How do we try to achieve this goal of Data

    Driven Organizations? Volume, Velocity & Variety Tooling & technology Growth in number of data professionals
  10. “The percentage of firms identifying themselves as being data-driven has

    declined in each of the past 3 years — from 37.1% in 2017 to 32.4% in 2018 to 31.0% this year” (HBR - Companies Are Failing in Their Efforts to Become Data-Driven - 2019) “Today, most firms use data in some fashion for decision making, but many can’t claim to be fully “data-driven.” In fact, only 24% said they had created a data-driven organization.” (NVP - Big Data and AI Executive Survey - 2021)
  11. From centralised to decentralised approach to data

  12. From centralised to decentralised, we've been here before..

  13. None
  14. How do we split a Monolithic application? https://martinfowler.com/articles/microservices.html

  15. Big ball of Mud When we have no correct boundaries

    of functionality, meaning a software systems lack an explicit architecture around these functionalities. Changing one functionality impacts the other functionalities.
  16. Leads to

  17. From 2003 - Get a shared understanding of the problems

    our domain experts have.
  18. None
  19. None
  20. Instead of one canonical language, create multiple bounded languages

  21. A model is a simplified representation of a thing or

    phenomenon that intentionally emphasizes certain aspects while ignoring others. Abstraction with a specific use in mind. — Rebecca Wirfs-Brock
  22. Stakeholders Team developing software Others involved designing the software

  23. What do the data mesh principles have to do with

    data management?
  24. One of the opportunities with Data Mesh is to address

    the Data Management problem organizations face (O’Reilly – Data Mesh – Zhamak Dehgani - 2022) Data Mesh principles Very much needed & Huge risk for chaos, inefficiency and misalignment Engineering way of solving the data management issue
  25. “Data mesh, at its core, is founded in decentralization and

    distribution of data responsibility to people who are closest to the data” “Data product ownership: long-term ownership of responsibilities to create, model, maintain, evolve, and share data as a product to meet the needs of data users.” (O’Reilly – Data Mesh – Zhamak Dehgani - 2022) (O’Reilly – Data Mesh – Zhamak Dehgani - 2022) Data Owner Data Users
  26. (O’Reilly – Data Mesh – Zhamak Dehgani - 2022) One

    long-standing challenge of existing analytical data architectures is the high friction and cost of using data: discovering, understanding, trusting, exploring, and ultimately consuming quality data. Data becomes a 1st class citizen instead of an after thought Data product characteristics: (O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
  27. How do we get to a domain-oriented ownership?

  28. Domain • A sphere of knowledge, influence, or activity. The

    subject area to which the user applies a program is the domain of the software. • an area of interest or an area over which a person has control • It is all about grouping concepts. Photo by Randy Fath on Unsplash
  29. Photo by Michal Balog on Unsplash Photo by Raul Gonzalez

    Escobar on Unsplash
  30. None
  31. None
  32. None
  33. None
  34. What influences our grouping of the domains? • Customers •

    Resources • People • Team cognitive load • Cohesion of change • Change rate • Culture • Business strategy and there value streams • Software architecture • Organisation structure • Knowledge & Practice • ….. • Aka Contextual!
  35. Collaborative Modelling

  36. None
  37. The engineering way of solving this problem

  38. A refreshed look at data governance is the missing and

    final piece to make data mesh work. (O’Reilly – Data Mesh – Zhamak Dehgani - 2022) … rely on automated mechanisms built by the platform and embedded into the distributed architecture ... (O’Reilly – Data Mesh – Zhamak Dehgani - 2022) Examples of governance that can be automated: But How?!
  39. Data product Catalogue Data product monitoring Experience plane Data Product

    plane Output port Output port Bounded context Metadata store Bounded context Metadata: - Descriptions - Classifications - SLA’s - Quality - Sample data - Endpoint - Owner - Usage - Access The 'Experience plane'
  40. Data product Catalogue Data product monitoring Metadata store Experience plane

    Data Product plane Output port Output port Bounded context Bounded context Bounded context Output port The 'Experience plane'
  41. Why a Data Mesh is a decentralized sociotechnical approach

  42. None
  43. None
  44. End2End Live Conference: https://www.youtube.com/watch?v=VxPVWjEVc6Q

  45. Wrap up Data Mesh is an investment in technology and

    in the people! Use collaborative modelling to design domain-oriented ownership Use an engineering approach to solve the governance problem Getting your data management in order is not limited to Data Mesh
  46. Contact us Peter Kromhout – pkromhout@xebia.com Kenny Baas-Schwegler – kbaas@xebia.com

    – twitter: @kenny_baas Niels Zeilemaker – nielszeilemaker@godatadriven.com