Domain Driven Design & Data Mesh

Data mesh is a decentralized sociotechnical approach for data management. Focusing on the technology, we tend to neglect the socio aspect of moving to new decentralized approaches. If we want to invest in data mesh architecture work, we must also invest in the people to make our data mesh more sustainable. How do we make people feel responsible for the data they expose and use? How do we make data management meet business objectives instead of a bureaucratic burden.

Decentralized approaches, e.g. like microservices, have been around for a while now. And we notice similar patterns of companies moving to a microservices architecture happening again when moving towards data mesh. Join this webinar to learn from these experiences and prevent obstacles for the future.

Transcript

1. Domain Driven
   Design & Data
   Mesh
   Webinar
   15/03/2022
   Kenny Baas-Schwegler
   Peter Kromhout
   Niels Zeilemaker
   

   View Slide

2. Photo by Şahin
   Sezer Dinçer on
   Unsplash
   

   View Slide

3. “Most organizations
   don’t have their
   Data Management
   in order, and have
   found themselves in
   a data mess
   because of it”
   

   View Slide

4. Businesses miss out on opportunities
   

   View Slide

5. It is a waste of resources
   

   View Slide

6. You break the law
   746
   225
   90
   60
   60
   50
   35
   27,8
   22
   20,4
   17
   12,3
   10,4
   9
   8,5
   8,15
   7
   6
   5
   4,5
   0 100 200 300 400 500 600 700 800
   Amazon
   WhatsApp
   Google Ireland
   Facebook
   Google LLC
   Google
   H&M
   TIM
   British Airways
   Marriott
   Wind
   Vodafone Italia
   Notebooksbilliger
   Austrian Post
   Eni
   Vodafone Spain
   Google
   Caixabank
   BBVA bank
   Fastweb
   In millions €
   20 biggest GDPR fines till date (900 in total)
   

   View Slide

7. Agenda
   • Why do we do what we do?
   • From centralised to decentralised, we've
   been here before..
   • What do the data mesh principles have
   to do with data management?
   • How do we get to a
   domain-oriented ownership?
   • The engineering way of solving this
   problem
   • Why a Data Mesh is a decentralized
   sociotechnical approach
   

   View Slide

8. Why do we do what we do?
   *
   * = Data- engineers, analysts, scientists, architects, leaders ..
   

   View Slide

9. How do we try to achieve this goal of Data Driven Organizations?
   Volume, Velocity & Variety
   Tooling &
   technology
   Growth in number of
   data professionals
   

   View Slide

10. “The percentage of firms identifying themselves as being data-driven
    has declined in each of the past 3 years — from 37.1% in 2017 to
    32.4% in 2018 to 31.0% this year”
    (HBR - Companies Are Failing in Their Efforts to Become Data-Driven - 2019)
    “Today, most firms use data in some fashion for decision making, but
    many can’t claim to be fully “data-driven.” In fact, only 24% said they
    had created a data-driven organization.”
    (NVP - Big Data and AI Executive Survey - 2021)
    

    View Slide

11. From
    centralised to
    decentralised
    approach
    to data
    

    View Slide

12. From centralised to decentralised,
    we've been here before..
    

    View Slide

13. View Slide

14. How do we split a
    Monolithic
    application?
    https://martinfowler.com/articles/microservices.html
    

    View Slide

15. Big ball of Mud
    When we have no correct boundaries of
    functionality, meaning a software systems lack an
    explicit architecture around these functionalities.
    Changing one functionality impacts the other
    functionalities.
    

    View Slide

16. Leads to
    

    View Slide

17. From 2003 - Get a shared understanding of the
    problems our domain experts have.
    

    View Slide

18. View Slide

19. View Slide

20. Instead of one canonical
    language,
    create multiple bounded
    languages
    

    View Slide

21. A model is a simplified
    representation of a thing
    or phenomenon that
    intentionally emphasizes
    certain aspects while
    ignoring others.
    Abstraction with a
    specific use in mind.
    — Rebecca Wirfs-Brock
    

    View Slide

22. Stakeholders
    Team
    developing
    software
    Others involved
    designing the
    software
    

    View Slide

23. What do the data mesh
    principles have to do with data
    management?
    

    View Slide

24. One of the opportunities with Data Mesh is to address the Data Management
    problem organizations face
    (O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
    Data Mesh principles
    Very much needed
    &
    Huge risk for chaos, inefficiency and
    misalignment
    Engineering way of solving the data
    management issue
    

    View Slide

25. “Data mesh, at its core, is founded in decentralization and
    distribution of data responsibility to people who are closest to the
    data”
    “Data product ownership: long-term ownership of
    responsibilities to create, model, maintain, evolve, and share data
    as a product to meet the needs of data users.”
    (O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
    (O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
    Data Owner
    Data Users
    

    View Slide

26. (O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
    One long-standing challenge of existing analytical data architectures is
    the high friction and cost of using data: discovering, understanding,
    trusting, exploring, and ultimately consuming quality data.
    Data becomes a 1st class citizen instead of an after thought
    Data product characteristics:
    (O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
    

    View Slide

27. How do we get to a
    domain-oriented ownership?
    

    View Slide

28. Domain
    • A sphere of knowledge, influence, or activity.
    The subject area to which the user applies a
    program is the domain of the software.
    • an area of interest or an area over which a
    person has control
    • It is all about grouping concepts.
    Photo by Randy Fath on Unsplash
    

    View Slide

29. Photo by Michal Balog
    on Unsplash
    Photo by Raul
    Gonzalez Escobar
    on Unsplash
    

    View Slide

30. View Slide

31. View Slide

32. View Slide

33. View Slide

34. What influences our
    grouping of the domains?
    • Customers
    • Resources
    • People
    • Team cognitive load
    • Cohesion of change
    • Change rate
    • Culture
    • Business strategy and there value streams
    • Software architecture
    • Organisation structure
    • Knowledge & Practice
    • …..
    • Aka Contextual!
    

    View Slide

35. Collaborative
    Modelling
    

    View Slide

36. View Slide

37. The engineering way of
    solving this problem
    

    View Slide

38. A refreshed look at data governance is the missing and final piece
    to make data mesh work.
    (O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
    … rely on automated mechanisms built by the platform and
    embedded into the distributed architecture ...
    (O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
    Examples of governance that can be automated:
    But How?!
    

    View Slide

39. Data product
    Catalogue
    Data product
    monitoring
    Experience plane
    Data Product plane
    Output
    port
    Output
    port
    Bounded
    context
    Metadata store
    Bounded
    context
    Metadata:
    - Descriptions
    - Classifications
    - SLA’s
    - Quality
    - Sample data
    - Endpoint
    - Owner
    - Usage
    - Access
    The 'Experience plane'
    

    View Slide

40. Data product
    Catalogue
    Data product
    monitoring
    Metadata store
    Experience plane
    Data Product plane
    Output
    port
    Output
    port
    Bounded
    context
    Bounded
    context
    Bounded
    context
    Output
    port
    The 'Experience plane'
    

    View Slide

41. Why a Data Mesh is a decentralized
    sociotechnical approach
    

    View Slide

42. View Slide

43. View Slide

44. End2End Live Conference: https://www.youtube.com/watch?v=VxPVWjEVc6Q
    

    View Slide

45. Wrap up
    Data Mesh is an investment in technology
    and in the people!
    Use collaborative modelling to design
    domain-oriented ownership
    Use an engineering approach to solve the
    governance problem
    Getting your data management in order is
    not limited to Data Mesh
    

    View Slide

46. Contact us
    Peter Kromhout – pkromhout@xebia.com
    Kenny Baas-Schwegler – kbaas@xebia.com – twitter: @kenny_baas
    Niels Zeilemaker – nielszeilemaker@godatadriven.com
    

    View Slide