Slide 1
Slide 1 text
Domain Driven
Design & Data
Mesh
Webinar
15/03/2022
Kenny Baas-Schwegler
Peter Kromhout
Niels Zeilemaker
Slide 2
Slide 2 text
Photo by Şahin
Sezer Dinçer on
Unsplash
Slide 3
Slide 3 text
“Most organizations
don’t have their
Data Management
in order, and have
found themselves in
a data mess
because of it”
Slide 4
Slide 4 text
Businesses miss out on opportunities
Slide 5
Slide 5 text
It is a waste of resources
Slide 6
Slide 6 text
You break the law
746
225
90
60
60
50
35
27,8
22
20,4
17
12,3
10,4
9
8,5
8,15
7
6
5
4,5
0 100 200 300 400 500 600 700 800
Amazon
WhatsApp
Google Ireland
Facebook
Google LLC
Google
H&M
TIM
British Airways
Marriott
Wind
Vodafone Italia
Notebooksbilliger
Austrian Post
Eni
Vodafone Spain
Google
Caixabank
BBVA bank
Fastweb
In millions €
20 biggest GDPR fines till date (900 in total)
Slide 7
Slide 7 text
Agenda
• Why do we do what we do?
• From centralised to decentralised, we've
been here before..
• What do the data mesh principles have
to do with data management?
• How do we get to a
domain-oriented ownership?
• The engineering way of solving this
problem
• Why a Data Mesh is a decentralized
sociotechnical approach
Slide 8
Slide 8 text
Why do we do what we do?
*
* = Data- engineers, analysts, scientists, architects, leaders ..
Slide 9
Slide 9 text
How do we try to achieve this goal of Data Driven Organizations?
Volume, Velocity & Variety
Tooling &
technology
Growth in number of
data professionals
Slide 10
Slide 10 text
“The percentage of firms identifying themselves as being data-driven
has declined in each of the past 3 years — from 37.1% in 2017 to
32.4% in 2018 to 31.0% this year”
(HBR - Companies Are Failing in Their Efforts to Become Data-Driven - 2019)
“Today, most firms use data in some fashion for decision making, but
many can’t claim to be fully “data-driven.” In fact, only 24% said they
had created a data-driven organization.”
(NVP - Big Data and AI Executive Survey - 2021)
Slide 11
Slide 11 text
From
centralised to
decentralised
approach
to data
Slide 12
Slide 12 text
From centralised to decentralised,
we've been here before..
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
How do we split a
Monolithic
application?
https://martinfowler.com/articles/microservices.html
Slide 15
Slide 15 text
Big ball of Mud
When we have no correct boundaries of
functionality, meaning a software systems lack an
explicit architecture around these functionalities.
Changing one functionality impacts the other
functionalities.
Slide 16
Slide 16 text
Leads to
Slide 17
Slide 17 text
From 2003 - Get a shared understanding of the
problems our domain experts have.
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
Instead of one canonical
language,
create multiple bounded
languages
Slide 21
Slide 21 text
A model is a simplified
representation of a thing
or phenomenon that
intentionally emphasizes
certain aspects while
ignoring others.
Abstraction with a
specific use in mind.
— Rebecca Wirfs-Brock
Slide 22
Slide 22 text
Stakeholders
Team
developing
software
Others involved
designing the
software
Slide 23
Slide 23 text
What do the data mesh
principles have to do with data
management?
Slide 24
Slide 24 text
One of the opportunities with Data Mesh is to address the Data Management
problem organizations face
(O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
Data Mesh principles
Very much needed
&
Huge risk for chaos, inefficiency and
misalignment
Engineering way of solving the data
management issue
Slide 25
Slide 25 text
“Data mesh, at its core, is founded in decentralization and
distribution of data responsibility to people who are closest to the
data”
“Data product ownership: long-term ownership of
responsibilities to create, model, maintain, evolve, and share data
as a product to meet the needs of data users.”
(O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
(O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
Data Owner
Data Users
Slide 26
Slide 26 text
(O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
One long-standing challenge of existing analytical data architectures is
the high friction and cost of using data: discovering, understanding,
trusting, exploring, and ultimately consuming quality data.
Data becomes a 1st class citizen instead of an after thought
Data product characteristics:
(O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
Slide 27
Slide 27 text
How do we get to a
domain-oriented ownership?
Slide 28
Slide 28 text
Domain
• A sphere of knowledge, influence, or activity.
The subject area to which the user applies a
program is the domain of the software.
• an area of interest or an area over which a
person has control
• It is all about grouping concepts.
Photo by Randy Fath on Unsplash
Slide 29
Slide 29 text
Photo by Michal Balog
on Unsplash
Photo by Raul
Gonzalez Escobar
on Unsplash
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
No content
Slide 34
Slide 34 text
What influences our
grouping of the domains?
• Customers
• Resources
• People
• Team cognitive load
• Cohesion of change
• Change rate
• Culture
• Business strategy and there value streams
• Software architecture
• Organisation structure
• Knowledge & Practice
• …..
• Aka Contextual!
Slide 35
Slide 35 text
Collaborative
Modelling
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
The engineering way of
solving this problem
Slide 38
Slide 38 text
A refreshed look at data governance is the missing and final piece
to make data mesh work.
(O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
… rely on automated mechanisms built by the platform and
embedded into the distributed architecture ...
(O’Reilly – Data Mesh – Zhamak Dehgani - 2022)
Examples of governance that can be automated:
But How?!
Slide 39
Slide 39 text
Data product
Catalogue
Data product
monitoring
Experience plane
Data Product plane
Output
port
Output
port
Bounded
context
Metadata store
Bounded
context
Metadata:
- Descriptions
- Classifications
- SLA’s
- Quality
- Sample data
- Endpoint
- Owner
- Usage
- Access
The 'Experience plane'
Slide 40
Slide 40 text
Data product
Catalogue
Data product
monitoring
Metadata store
Experience plane
Data Product plane
Output
port
Output
port
Bounded
context
Bounded
context
Bounded
context
Output
port
The 'Experience plane'
Slide 41
Slide 41 text
Why a Data Mesh is a decentralized
sociotechnical approach
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
End2End Live Conference: https://www.youtube.com/watch?v=VxPVWjEVc6Q
Slide 45
Slide 45 text
Wrap up
Data Mesh is an investment in technology
and in the people!
Use collaborative modelling to design
domain-oriented ownership
Use an engineering approach to solve the
governance problem
Getting your data management in order is
not limited to Data Mesh
Slide 46
Slide 46 text
Contact us
Peter Kromhout – [email protected]
Kenny Baas-Schwegler – [email protected] – twitter: @kenny_baas
Niels Zeilemaker – [email protected]