Classic Data Architecture
> Centralized databases
> …or services that provide data
> Ensures consistency across systems
> …for data model
> ...and updates to data
> Reuse
Slide 6
Slide 6 text
Classic Data Architecture
Billing
Order
Process
CRM
Order
Slide 7
Slide 7 text
Who is using a
centralized database?
Slide 8
Slide 8 text
Who likes the
centralized database?
Slide 9
Slide 9 text
Microservices:
Definition
> No consistent definition
> Microservices are modules
> Independent deployment units
> E.g. processes, Docker container
> Microservice owned by one team
Slide 10
Slide 10 text
Microservices: Definition
Server /
Container
Server /
Container
Micro
Service
Micro
Service
Slide 11
Slide 11 text
Why Microservices?
> Develop a feature
> …bring it into production
> ...with no coordination
> Independent scaling
> Free choice of technology
> Robustness
> Security
Slide 12
Slide 12 text
Microservices aim for
decoupling
Slide 13
Slide 13 text
Microservices & Data
Server /
Container
Server /
Container
Micro
Service
Micro
Service
Order Schema
Slide 14
Slide 14 text
Microservices & Data
Server /
Container
Server /
Container
Micro
Service
Micro
Service
Order Schema
Slide 15
Slide 15 text
Microservices & Data
> Decoupling for data, too
> Separate data storage
Slide 16
Slide 16 text
Data Microservices
Server /
Container
Server /
Container
Micro
Service
Micro
Service
Order Data
Microservice
Slide 17
Slide 17 text
Data Microservices
Server /
Container
Micro
Service
Order Data
Microservice
Customer Data
Microservice
Remote calls
influence
performance
availability
No transaction
across customer
and order
Slide 18
Slide 18 text
Data Microservice
> Change two microservices if new feature
requires change to data schema
> But: data in one place
> No consistency issues
Slide 19
Slide 19 text
Data microservice
limits decoupling.
Slide 20
Slide 20 text
Encapsulation
> Information hiding
> Hide the internal data structure
> Provide access only through a well defined
interface
> Data and databases should not be exported
Slide 21
Slide 21 text
Violates Encapsulation
Billing
Order
Process
CRM
Order
Shared
data model
Logic
Slide 22
Slide 22 text
Violates Encapsulation
Server /
Container
Server /
Container
Micro
Service
Micro
Service
Order Data
Microservice
Shared
data model
Logic
Slide 23
Slide 23 text
Separate Databases
Server /
Container
Server /
Container
Micro
Service
Micro
Service
Order Order
Slide 24
Slide 24 text
Different Databases
Server /
Container
Server /
Container
Micro
Service
Micro
Service
neo4j Oracle
Slide 25
Slide 25 text
Different Databases
> “Polyglot persistence”
> Use the best tool for the job
> Technology freedom
– advantage of microservices
> …but extra effort
> Backup, disaster recovery etc.
> Not as easy as e.g. different frameworks
Slide 26
Slide 26 text
Separate Schema
Server /
Container
Server /
Container
Micro
Service
Micro
Service
Oracle
Schema Schema
Slide 27
Slide 27 text
Separate Schemas
> Less effort
> Decoupled data models
> ...but limited independent scaling and
robustness
Slide 28
Slide 28 text
Billing
Order
Process
CRM
Order
Order Order
Slide 29
Slide 29 text
Redundancy!!!
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
Domain-driven Design
Slide 33
Slide 33 text
Domain-driven Design
> 2004
> Still very relevant
> By Eric Evans
> Focus on part IV
> Free reference:
http://domainlanguage.com/
ddd/reference/
Slide 34
Slide 34 text
Order
Shipping address
Tracking #
Items
Item Categories
Priority shipping
Customs #
Account #
...
Credit card #
Order #
Slide 35
Slide 35 text
My Domain Model
is a mess!
Slide 36
Slide 36 text
Bounded Context
> Domain model is only valid for one context
> There is no universal data model!
> See all failed SOA attempts
Slide 37
Slide 37 text
Order
Shipping address
Tracking #
Items
Item Categories
Priority shipping
Customs #
Account #
...
Credit card #
Order #
Customs
Order
Recommen-
dations
Order
Tracking
Order
Shipping
address
Tracking #
Item
Categories
Priority
shipping
Customs #
Payment
Order
Account #
Credit card #
Slide 38
Slide 38 text
Billing
Order
Process
CRM
Order
Order Order
Slide 39
Slide 39 text
Bounded Context
> Microservice =
BOUNDED CONTEXTS
> Changes for new features are local
> …even if data models need to be changed
Slide 40
Slide 40 text
Billing
Order
Process
CRM
Order
Order Order
Slide 41
Slide 41 text
Redundancy?
Slide 42
Slide 42 text
Redundancy?
Not really
Slide 43
Slide 43 text
Bounded Context
Slide 44
Slide 44 text
What about
basic data of an
order?
Slide 45
Slide 45 text
Strategic Design
> How do BOUNDED CONTEXTS relate to each
other?
> Context can have relationships
> DDD defines several relationship patterns
Slide 46
Slide 46 text
Shared Kernel
> Subset of a model
> …that two teams share
> Eric Evans: Including code and database
> Microservices: Just sharing a model
Slide 47
Slide 47 text
Anti-corruption Layer
> Don’t let e.g. a legacy model influence a
new model
> Isolate model by additional layer
> No need to modify the old system
Slide 48
Slide 48 text
Context Relationships
> Team = Deployment Unit = BOUNDED CONTEXT
> Context Relationships define how BOUNDED
CONTEXT are used…
> ...and how much teams need to collaborate
Context Map
> Show the different BOUNDED CONTEXT
> …and the relation to each other
> BOUNDED CONTEXT might be microservices
> ...or communication links
Slide 52
Slide 52 text
Order Process
Registration
Basic
Customer
Data
Basic
Customer
Data
Customer
Order
Data
Delivery
Customer
Order
Data
Billing
Anticorruption
Layer
Mainframe
Customer
Data
Customer
Order
Data
Customer
Order
Data
Basic Customer
Data +
Customer Order
Data
= Shared Kernel
Slide 53
Slide 53 text
Billing
Order
Process
CRM
Shared
Kernel
Order
Additional
data
Additional
data
Additional
data
Order
Data
Slide 54
Slide 54 text
Centralized
Shared Kernel
> Ensures consistency
> ...but needs to be called for a lot of
operations
> Resilience / performance / transactions
> Have one master as the source of truth
Slide 55
Slide 55 text
Billing
Order
Process
CRM
Shared
Kernel
Order
Shared
Kernel
Order
Shared
Kernel
Order
Additional
data
Additional
data
Additional
data
Slide 56
Slide 56 text
Decentralized
Shared Kernel
> Might be inconsistent
> ...but all data for all requests is available in
the local database
> Better resilience…
> ...and performance
Slide 57
Slide 57 text
How to Replicate
Data?
Slide 58
Slide 58 text
Database Replication
> Built into the database
> Replicate schema across database instances
> But: Microservices have separated schemas
> Every Microservice might have different data
> …so database replication is not a good fit
Slide 59
Slide 59 text
Replication with
Events
Slide 60
Slide 60 text
Events
> Later addition to Domain-driven Design
> Events with a business meaning
> Decouple time:
Asynchronous
> Decouple logic:
System can handle event as it pleases
Slide 61
Slide 61 text
Billing
Order
Process
CRM
Shared
Kernel
Order
Shared
Kernel
Order
Shared
Kernel
Order
Additional
data
Additional
data
Additional
data
New Order Event
Slide 62
Slide 62 text
Events & Data Replication
> Events lead to data replication
> i.e. each system stores information it
received in an event
> Data stored in separate schema
> Very decoupled
> Hard to repair inconsistencies
Slide 63
Slide 63 text
More Fun With Events
Slide 64
Slide 64 text
Event Sourcing
> Internal Structure for Microservice with events
> Current state result of all events
> Calculate state on the fly?
Event Sourcing
> Event store and snapshot help to repair
inconsistencies
> Event-based architecture in microservices
Slide 67
Slide 67 text
CQRS
> Command – Query Responsibility Segregation
> Commands change data
> Query provide data
> Implement in separate modules
> …or even microservices
> ...with potentially different BOUNDED CONTEXTS
Slide 68
Slide 68 text
Commands vs Events
> Command: Change that data!
> Event: Something has happened
> Component decides if data should be
changed
Batch
> Get all data
> Provide API
> …to decouple schema
> Copy interesting data into local database
Slide 72
Slide 72 text
Billing
Order
Process
CRM
Shared
Kernel
Order
Shared
Kernel
Order
Shared
Kernel
Order
Additional
data
Additional
data
Additional
data
Batch
Batch API API
Slide 73
Slide 73 text
Batch & Data Replication
> Easy to repair inconsistencies
> Batch run at specific points
> i.e. updates take time
> Data not consistent across microservices
Slide 74
Slide 74 text
CAP: Challenge for
Replication
Slide 75
Slide 75 text
CAP Theorem
> Consistency
> All nodes see the same data
> Availability
> Node failures do not prevent survivors
from operating
> Partition Tolerance
> System continues to operate despite
arbitrary message loss
C
P A
Slide 76
Slide 76 text
CAP Theorem: P
> Network partitions do occur
> Even with highly available network
hardware
> Also: very slow response = partition
> Need to deal with P
Slide 77
Slide 77 text
CAP Theorem: C or A?
> Node cannot access other nodes
> Might have missed updates
> A, not C:
Answer with a potentially wrong answer
> C, not A:
Don’t answer – the answer might be wrong
Slide 78
Slide 78 text
Billing
Order
Process
CRM
Shared
Kernel
Order
Shared
Kernel
Order
Shared
Kernel
Order
Additional
data
Additional
data
Additional
data
New Order Event
inconsistent or unavailable
Slide 79
Slide 79 text
Conclusion
Slide 80
Slide 80 text
Classic:
Centralized
Database
Microservices:
private
database
decoupling
Data Microservices:
Consistent but
resilience / performance
/ transactions / decoupling?
Database per
Microservice:
Polyglot Persistence
Schema per
Microservice:
Simple infrastructure
Slide 81
Slide 81 text
Redundant Data or
Bounded Context?
Batch
Database
Replication
Events
Redundancy?
Context Map and
Context Relations
Replication
CAP
Event
Sourcing
CQRS
e.g. Shared
Kernel