Fulfilling Data Needs between Microservices

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Need an idea for a talk...

Slide 3

Slide 3 text

Something about how we do Microservices!

Slide 4

Slide 4 text

https://www.primevideotech.com/video- streaming/scaling- up- the- prime- video- audio- video- monitoring- service- and- reducing- costs- by- 90 Then this happened...

Slide 5

Slide 5 text

Now Microservices are basically dead and my talk is pointless

Slide 6

Slide 6 text

Fullfilling Data Needs between Microservices Bounded Contexts

Slide 7

Slide 7 text

What is a "microservice"? What are "data needs"?

Slide 8

Slide 8 text

What is a "microservice"? Autonomous Specialized Develop, Build, Deploy independently Data ownership Communicate through well- defined API's Knowledge isolation https://aws.amazon.com/microservices/

Slide 9

Slide 9 text

What are "data needs"? When a service needs Data it doesn't own To perform a task from another service

Slide 10

Slide 10 text

Task 1 Return data you don't own

Slide 11

Slide 11 text

Task 1 Return data you don't own Request

Slide 12

Slide 12 text

Concrete example Service Projects is requested to return the details of a specific project, together with the name of the customer

Slide 13

Slide 13 text

Concrete example Projects Customers Customer ID Name Address Project ID Title Customer 443f16e3- e398-4e57-88ea-9fa4972be6a0 443f16e3- e398-4e57-88ea-9fa4972b Bezos Company Parkway Drive 666, 90210 Bvrly Hills 17a4b5ac-38f0-4945- b3a5-5aed968ce0b0 Install Solar Panels

Slide 14

Slide 14 text

Solution Projects Customers API Gateway one of many solutions actually Routes the request Consolidates responses Protocol translation { "id" : "17a4b5ac-38f0-4945- b3a5-5aed968ce0b0", "title" : "Install Solar Panels", "customer" : { "type": "customer", "id" : "443f16e3- e398-4e57-88ea-9fa4972be6a0" } } { "id" : "443f16e3- e398-4e57-88ea-9fa4972be6a0", "Name" : "Bezos Company", "address" : "Parkway Drive 666, 90210 Bvrly Hills" } { "id" : "17a4b5ac-38f0-4945- b3a5-5aed968ce0b0", "title" : "Install Solar Panels", "customer" : { "id" : "443f16e3- e398-4e57-88ea-9fa4972be6a0", "Name" : "Bezos Company", "address" : "Parkway Drive 666, 90210 Bvrly Hills" } } API Gateway

Slide 15

Slide 15 text

Task 1 Return data you don't own Request Solution: introduce a separate component AKA not a microservice's task

Slide 16

Slide 16 text

Task 2 Use data for a business process

Slide 17

Slide 17 text

Concrete example A project's initial title is constructed from the customer name and an incrementing number

Slide 18

Slide 18 text

Projects Solution #1 Customers Database sharing Just read from their database!

Slide 19

Slide 19 text

Solution #1Database sharing Pro's Cons Why even do microservices?? Easy Fast High "implementation" coupling

Slide 20

Slide 20 text

What is a "microservice"? Autonomous Specialized Develop, Build, Deploy independently Data ownership Communicate through well- defined API's Knowledge isolation https://aws.amazon.com/microservices/

Slide 21

Slide 21 text

Should not even be considered as a solution

Slide 22

Slide 22 text

Solving the implementation coupling leads to...

Slide 23

Slide 23 text

Projects Solution #2API Calls Just use their API! Customers

Slide 24

Slide 24 text

Solution #2API Calls Pro's Cons Loose "implementation" coupling High "temporal" coupling aka "latency" But... seems acceptable Very microservice'y! API Contract

Slide 25

Slide 25 text

Fast- forward in time...

Slide 26

Slide 26 text

Projects Solution #2API Calls Customers Invoicing The Internet Currency conversion

Slide 27

Slide 27 text

Solution #2API Calls Pro's Cons Loose "implementation" coupling High "temporal" coupling But... seems acceptable Very microservice'y!

Slide 28

Slide 28 text

But what about caching?

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

Solution #2API Calls Sync calls Async calls Eventually needs caching UX Hell

Slide 31

Slide 31 text

How to do it well?

Slide 32

Slide 32 text

API Contracts Make your API a 1st Class Citizen it was implied in the definition of a Microservice JSON Schema

Slide 33

Slide 33 text

Solving the temporal coupling leads to...

Slide 34

Slide 34 text

Projects Solution #3Event- Carried State Transfer Customers Service Bus event event Just use events to transfer data!

Slide 35

Slide 35 text

Solution #3Event- Carried State Transfer Customers Service Bus T r a n s l a t e Domain Event Customer Activated Customer Updated Produce Domain events 1. 3. Send out integration event 2. Translate into 1 integration event with all entity data { "action" : "UPDATE", "id" : "17a4b5ac-38f0-4945- b3a5-5aed968ce0b0", "data" : { "first_name" : "Josh", "last_name" : "Holme", // ... } } Producing Events

Slide 36

Slide 36 text

Solution #3Event- Carried State Transfer Fat Event https://verraes.net/2019/05/patterns- for- decoupling- distsys- segregated- event- layers/ https://verraes.net/2019/05/patterns- for- decoupling- distsys- fat- event/ Segregated Event Layers Rich

Slide 37

Slide 37 text

Projects Solution #3Event- Carried State Transfer Service Bus { "action" : "UPDATE", "id" : "17a4b5ac-38f0-4945- b3a5-5aed968ce0b0", "data" : { "first_name" : "Josh", "last_name" : "Holme", // ... } } Customer Updated A C L Ingest integration event 1. 2. Pick interesting data 3. Store in a read model ID | NAME 17a4b5ac-38f0-4945- b3a5-5aed968ce0b0 | Josh Holme Consuming Events

Slide 38

Slide 38 text

Solution #3Event- Carried State Transfer Anti- Corruption Layer https://learn.microsoft.com/en- us/azure/architecture/patterns/cqrs https://www.youtube.com/watch? v=Dok2Ikcjaro Read Model

Slide 39

Slide 39 text

Solution #3Event- Carried State Transfer https://www.youtube.com/watch? v=IzBEbfSg0uY

Slide 40

Slide 40 text

Solution #3Event- Carried State Transfer Pro's Cons Loose "implementation" coupling Loose "temporal" coupling Data duplication Is this really a problem these days? Eventual Consistency

Slide 41

Slide 41 text

How to do it well?

Slide 42

Slide 42 text

Event Contracts Make your Events 1st Class Citizens

Slide 43

Slide 43 text

Solving the data duplication & eventual consistency leads to...

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

https://particular.net/webinars/2023- live- qa- with- udi

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

Projects Solution #4Service Composition Customers C C Just inject a component!

Slide 49

Slide 49 text

Solution #4Service Composition Component Piece of functionality, exposed as a reusable package

Slide 50

Slide 50 text

Solution #4Service Composition Component Created & maintained where it logically belongs Inside the knowledge boundary or "system"

Slide 51

Slide 51 text

How to do it well? experimental ! in PHP

Slide 52

Slide 52 text

Guard the boundaries C o m p o n e n t O t h e r c o d e exposed hidden Cannot depend on each other https://qossmic.github.io/deptrac/ same codebase

Slide 53

Slide 53 text

e.g. Github Actions Mirror to Read- Only repository C o m p o n e n t O t h e r c o d e C o m p o n e n t same repository separate repository mirror

Slide 54

Slide 54 text

Re- use s e r v i c e C o m p o n e n t composer require // composer.json { "repositories": [ { "type": "vcs", "no- api": true, "url": "" } } (the Read- only repository)

Slide 55

Slide 55 text

Decouple s e r v i c e Component Interfaces depends on & uses for typehinting depends on implements Dependency- inversion principle (separate repository)

Slide 56

Slide 56 text

Just like PSR!

Slide 57

Slide 57 text

Why?

Slide 58

Slide 58 text

To avoid cyclic dependencies Customers C L i b r a r y depends on depends on v1 depends on v2 conflict!

Slide 59

Slide 59 text

Some heuristics

Slide 60

Slide 60 text

Solution #4Service Composition Component Should only do reads and/or business logic No writes in its knowledge boundary

Slide 61

Slide 61 text

Solution #4Service Composition Component Should not do writes to your database Service no longer retains data ownership

Slide 62

Slide 62 text

Solution #4Service Composition Component Interface(s) are distributed separately To break cyclic dependencies

Slide 63

Slide 63 text

Solution #4Service Composition Component Caching is not its responsibility Use decorator pattern & cache locally

Slide 64

Slide 64 text

Solution #4Service Composition Component Ideally has a single purpose It's not a library!

Slide 65

Slide 65 text

Quick Recap

Slide 66

Slide 66 text

Task 1 Return data you don't own

Slide 67

Slide 67 text

Solution Projects Customers API Gateway API Gateway

Slide 68

Slide 68 text

Task 2 Use data for a business process

Slide 69

Slide 69 text

Projects Solution #1 Customers Database sharing seriously... don't!

Slide 70

Slide 70 text

Projects Solution #2API Calls Customers

Slide 71

Slide 71 text

Projects Solution #3Event- Carried State Transfer Customers Service Bus event event