Unlock your data - A reference data mesh

Many companies struggle with the problem of effectively using their existing data. Teams do not have sovereignty over their data and often have to compromise with the DBA team. Data mesh is a concept for the data architecture of companies that aims to improve collaboration and data exchange between different business areas and teams. It is based on the concept of microservices and sees data as independent services developed, provided, and managed by the respective teams. The concept of data mesh includes four main principles:
Domain ownership
Data as a product
Self-service data platform
Federated Governance
The goal of data mesh is to overcome the traditional centralized data architecture and create a decentralized, more flexible, and agile data infrastructure.

Transcript

2. Your guiding light to innovation
3. 4 • Software engineering – TU Wien • Backend with Java and Kotlin • Frontend with Typescript and React • DevOps and Cloud platforms • Data mesh • https://www.linkedin.com/in/damjan-gjurovski- 362b3a206/ Damjan Gjurovski CTO Posedio GmbH
4. UNLOCK YOUR DATA: A REFERENCE DATA MESH 10 June 2024
5. WHAT’S IN IT FOR YOU? 1
6. 7 WHAT TO DO WITH ALL THAT DATA 1 2 3 4 • Generate data
7. 8 WHAT TO DO WITH ALL THAT DATA 1 2 3 4 • Generate data • ???
8. 9 WHAT TO DO WITH ALL THAT DATA 1 2 3 4 • Generate data • Make data driven decisions • Experiment • Outmaneuver the competition • ???
9. 10 WHAT TO DO WITH ALL THAT DATA 1 2 3 4 • Generate data • Make data driven decisions • Experiment • Outmaneuver the competition • Profit • ???
10. DATA MESH IS NOT A NEW CONCEPT – A LOOK BACK 2
11. 12 THE GOOD OLD DAYS
12. 13 SOFTWARE PROJECT MANAGEMENT • Difficult to get the solution *just* right – not enough expertise • Slow to react to changes • Time and budget estimations are nearly always wrong
13. 14 AGILE SOLUTIONS • Team decides on work - Responsibility given to the people closest to the problem • Outcome is presented to (internal) customers / stakeholders • Work is re-organized to fit a self- organizing team • Centralized control through feature prioritization to keep everything on track
14. 15 MONOLITHIC ARCHITECTURE AND DEV VS OPS • Difficult to get the solution *just* right – not enough expertise • Slow to react to changes • Time and budget estimations are nearly always wrong
15. 16 DEVOPS & MICRO- SERVICES • Team takes responsibility to develop and maintain software (you build it, you run it) • Microservices is a full (internal) product, with all required functionality (and maybe billing) • A runtime platform (like Kubernetes) allows teams to deploy easily • Centralized security and governance, API layer and service catalogue
16. 17 HOW DOES IT COMPARE Objective Agile methodology DevOps and Microservices Responsibility is given to the people closest to the problem Team decides on work - Responsibility given to the people closest to the problem Team takes responsibility to develop and maintain software (you build it, you run it) Artifact is presented as a product to (internal) customer Outcome is presented to (internal) customers / stakeholders Microservices is a full (internal) product, with all required functionality (and maybe billing) Self-service leading to independence Work is re-organized to fit a self-organizing team A runtime platform (like Kubernetes) allows teams to deploy easily Centralized governance Centralized control through feature prioritization to keep everything on track Centralized security and governance, API layer and service catalogue
17. 18 WHAT ABOUT DATA? Objective Agile methodology DevOps and Microservices Data Mesh Responsibility is given to the people closest to the problem Team decides on work - Responsibility given to the people closest to the problem Team takes responsibility to develop and maintain software (you build it, you run it) Domain driven ownership Artifact is presented as a product to (internal) customer Outcome is presented to (internal) customers / stakeholders Microservices is a full (internal) product, with all required functionality (and maybe billing) Data as a product Self-service leading to independence Work is re- organized to fit a self-organizing team A runtime platform (like Kubernetes) allows teams to deploy easily Self-serve data platform Centralized governance Centralized control through feature prioritization to keep everything on track Centralized security and governance, API layer and service catalogue Federated computational governance
18. A REFERENCE ARCHITECTURE 3
19. 20 SIMPLE ARCHITECTURE
20. 21 GCP ARCHITECTURE
21. THANK YOU! POSEDIO GMBH +43 660 8484500 onur.kasimlar@posedio.com damjan.gjurovski@posedio.com Millenium Park 4, 6980 Lustenau, Weyringergasse 1-3/DG, 1040 Wien www.posedio.com