Slide 1
Slide 1 text
Migrating to Microservice Databases:
From Relational Monolith to Distributed Data
Edson Yanaga
Director of Developer Experience
@yanaga
Slide 2
Slide 2 text
Join developers.redhat.com
2
Java Champion Microsoft MVP
Slide 3
Slide 3 text
http://developers.redhat.com/promotions/
migrating-to-microservice-databases
Slide 4
Slide 4 text
“Now, every company is a software company”
— Forbes
Slide 5
Slide 5 text
Join developers.redhat.com
5
Slide 6
Slide 6 text
DevOps &
Microservices
Slide 7
Slide 7 text
Feedback
Loop
Slide 8
Slide 8 text
Batch Size
Slide 9
Slide 9 text
Join developers.redhat.com
9
Maintenance
Window
Slide 10
Slide 10 text
Zero Downtime
Slide 11
Slide 11 text
Join developers.redhat.com
11
Blue Green
Deployments
Slide 12
Slide 12 text
Join developers.redhat.com
12
Deployment
Slide 13
Slide 13 text
Join developers.redhat.com
13
Deployment
Proxy
Slide 14
Slide 14 text
Join developers.redhat.com
14
Blue
Proxy
Green
Slide 15
Slide 15 text
Join developers.redhat.com
15
Blue
Proxy
Green
Slide 16
Slide 16 text
Join developers.redhat.com
16
Blue
Proxy
Green
Slide 17
Slide 17 text
Join developers.redhat.com
17
Blue
Proxy
Green
Slide 18
Slide 18 text
Code is easy,
state is hard
Slide 19
Slide 19 text
What about
my
relational
database?
Slide 20
Slide 20 text
Join developers.redhat.com
20
Slide 21
Slide 21 text
Zero Downtime
Migrations
Slide 22
Slide 22 text
Back and Forward
Compatibility
Slide 23
Slide 23 text
Baby Steps = Smallest
Possible Batch Size
Slide 24
Slide 24 text
Too many rows = Long Locks
Slide 25
Slide 25 text
Shard your
updates
Slide 26
Slide 26 text
Join developers.redhat.com
ALTER TABLE customers RENAME COLUMN wrong TO correct;
26
Slide 27
Slide 27 text
Join developers.redhat.com
ALTER TABLE customers ADD COLUMN correct VARCHAR(20);
UPDATE customers SET correct = wrong
WHERE id BETWEEN 1 AND 100;
UPDATE customers SET correct = wrong
WHERE id BETWEEN 101 AND 200;
ALTER TABLE customers DELETE COLUMN wrong;
27
Slide 28
Slide 28 text
Join developers.redhat.com
28
Scenarios
Add a Column
Rename a Column
Change Type/Format of a Column
Delete a Column
Slide 29
Slide 29 text
Join developers.redhat.com
29
Add a Column
1 ADD COLUMN
2 Code computes the read value and writes to new column
3 Update data using shards
4 Code reads and writes from the new column
Slide 30
Slide 30 text
Join developers.redhat.com
30
Rename a Column
1 ADD COLUMN
2 Code reads from the old column and writes to both
3 Copy data using small shards
4 Code reads from the new column and writes to both
5 Code reads and writes from the new column
6 Delete the old column (later)
Slide 31
Slide 31 text
Join developers.redhat.com
31
Change Type/Format of a Column
1 ADD COLUMN
2 Code reads from the old column and writes to both
3 Copy data using small shards
4 Code reads from the new column and writes to both
5 Code reads and writes from the new column
6 Delete the old column (later)
Slide 32
Slide 32 text
Join developers.redhat.com
32
Delete a Column
1 DON’T
2 Stop using the read value but keep writing to the column
3 Delete the column
Slide 33
Slide 33 text
What about referential integrity
constraints?
Slide 34
Slide 34 text
Drop them and recreate when
migration is done
Slide 35
Slide 35 text
Microservices
Characteristics
https://martinfowler.com/microservices/
Slide 36
Slide 36 text
• Componentization via Services
• Organized around Business Capabilities
• Products not Projects
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation
• Design for failure
• Evolutionary Design
Slide 37
Slide 37 text
Extracting your Microservice
database
Slide 38
Slide 38 text
Join developers.redhat.com
38
One database per Microservice
Slide 39
Slide 39 text
Join developers.redhat.com
39
But I have a monolithic database!
Slide 40
Slide 40 text
Join developers.redhat.com
40
Slide 41
Slide 41 text
Splitting is not easy,
but how do I integrate later?
Slide 42
Slide 42 text
Consistency Models
Slide 43
Slide 43 text
Strong Consistency
Eventual Consistency
Slide 44
Slide 44 text
CRUD & CQRS
Slide 45
Slide 45 text
Join developers.redhat.com
45
CRUD (Create, Read, Update, Delete)
Slide 46
Slide 46 text
Join developers.redhat.com
46
CQRS (Command Query Responsibility Segregation)
Slide 47
Slide 47 text
Join developers.redhat.com
47
CQRS with separate data stores
Slide 48
Slide 48 text
CQRS & Event Sourcing
Slide 49
Slide 49 text
Join developers.redhat.com
49
Scenarios
Shared Tables
Database View
Database Materialized View
Mirror Table using Trigger
Mirror Table using Transactional Code
Mirror Table using ETL
Mirror Table using Data Virtualization
Event Sourcing
Change Data Capture
Slide 50
Slide 50 text
Join developers.redhat.com
50
Shared Tables
Fastest Data Integration
Strong Consistency
Low cohesion and high coupling
Slide 51
Slide 51 text
Join developers.redhat.com
51
Database View
Easiest one to implement
Largest support from DBMS vendors
Possible performance issues
Strong Consistency
One database must be reachable by the other
Updatable depending on DBMS support
Slide 52
Slide 52 text
Join developers.redhat.com
52
Database Materialized View
Better performance
Strong or Eventual Consistency
One database must be reachable by the other
Updatable depending on DBMS support
Slide 53
Slide 53 text
Join developers.redhat.com
53
Database Trigger
Depends on DBMS Support
Strong Consistency
One database must be reachable by the other
Slide 54
Slide 54 text
Join developers.redhat.com
54
Transactional Code
Any code: usually Stored Procedures or Distributed
Transactions
Strong Consistency
Possible cohesion/coupling issues
Possible performance issues
Updatable depending on how it is implemented
Slide 55
Slide 55 text
Join developers.redhat.com
55
ETL Tools
Lots of available tools
Requires external trigger (usually time-based)
Can aggregate from multiple datasources
Eventual Consistency
Read only integration
Slide 56
Slide 56 text
Join developers.redhat.com
56
Data Virtualization
Real Time Access
Strong Consistency
Can aggregate from multiple datasources
Updatable depending on Data Virtualization Platform
Slide 57
Slide 57 text
Join developers.redhat.com
57
Slide 58
Slide 58 text
Join developers.redhat.com
58
Event Sourcing
State of data is a stream of events
Eases auditing
Eventual Consistency
Usually combined with a Message Bus
High scalability
Slide 59
Slide 59 text
Join developers.redhat.com
59
Change Data Capture
Read datasource is updated through a stream of events
Eventual Consistency
Usually combined with a Message Bus
High scalability
Slide 60
Slide 60 text
http://debezium.io
Slide 61
Slide 61 text
Join
developers.redhat.com
Feedback welcome!
@yanaga
Slide 62
Slide 62 text
plus.google.com/+RedHat
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHatNews
Thank you!