Microservices Evolution: Breaking your monolithic database

Microservices are the current "big thing" and most of the current technologies seems to be a fit for greenfield projects. Unfortunately the great majority of developers can't build something from scratch and have a big legacy database with multiple relationships. The code usually is the easy part. The question that every enterprise developer asks is: what about my database?

Come to this session to learn the required database refactorings, architecture and deployment strategies that will enable you to split your big monolithic database into Microservices. And downtime is not an option! We'll show how we need to encompass code, architecture, DevOps and infrastructure altogether to accomplish this awesome evolution of your application with your database.

Transcript

1. Microservices Evolution: Breaking Your Monolithic Database Edson Yanaga Director of Developer Experience @yanaga
2. Join developers.redhat.com 2 Java Champion Microsoft MVP
3. “Now, every company is a software company” — Forbes
4. Join developers.redhat.com 4
5. DevOps & Microservices
6. Feedback Loop
7. Join developers.redhat.com 7
8. Join developers.redhat.com 8
9. Batch Size
10. Join developers.redhat.com 10 Maintenance Window
11. Zero Downtime
12. Join developers.redhat.com 12 Blue Green Deployments
13. Join developers.redhat.com 13 Blue
14. Join developers.redhat.com 14 Blue Proxy
15. Join developers.redhat.com 15 Blue Proxy Green
16. Join developers.redhat.com 16 Blue Proxy Green
17. Join developers.redhat.com 17 Blue Proxy Green
18. Join developers.redhat.com 18 Blue Proxy Green
19. Code is easy, state is hard
20. What about my relational database?
21. Zero Downtime Migrations
22. Back and Forward Compatibility
23. Baby Steps = Smallest Possible Batch Size
24. Too many rows = Long Locks
25. Shard your updates
26. Join developers.redhat.com ALTER TABLE customers RENAME COLUMN wrong TO correct; 26
27. Join developers.redhat.com ALTER TABLE customers ADD COLUMN correct VARCHAR(20); UPDATE customers SET correct = wrong WHERE id < 100; UPDATE customers SET correct = wrong WHERE id < 200; <code update> <code update> <code update> (later…)ALTER TABLE customers DELETE COLUMN wrong; 27
28. Join developers.redhat.com 28 Scenarios Add a Column Rename a Column Change Type/Format of a Column Delete a Column
29. Join developers.redhat.com 29 Add a Column 1 ADD COLUMN 2 NULL/DEFAULT Value/Computed Value 3 Use Column
30. Join developers.redhat.com 30 Rename a Column 1 ADD COLUMN 2 Code reads from the old column and writes to both 3 Copy data using small shards 4 Code reads from the new column and writes to both 5 Code reads and writes from the new column 6 Delete the old column
31. Join developers.redhat.com 31 Change Type/Format of a Column 1 ADD COLUMN 2 Code reads from the old column and writes to both 3 Copy data using small shards 4 Code reads from the new column and writes to both 5 Code reads and writes from the new column 6 Delete the old column
32. Join developers.redhat.com 32 Delete a Column 1 DON’T 2 Stop using the read value but keep writing to the column 3 Delete the column
33. What about referential integrity constraints?
34. Drop them and recreate when migration is done
35. Extracting your Microservice database
36. Join developers.redhat.com 36 One database per Microservice
37. Join developers.redhat.com 37 But I have a monolithic database!
38. Join developers.redhat.com 38
39. CRUD & CQRS
40. Join developers.redhat.com 40 CRUD (Create, Retrieve, Update, Delete) Write Operations Read Operations
41. Join developers.redhat.com 41 CQRS (Command Query Responsibility Segregation) Write Operations Read Operations
42. Join developers.redhat.com 42 Scenarios View Materialized View Mirror Table using Trigger Mirror Table using Transactional Code Mirror Table using ETL tools Event Sourcing
43. Join developers.redhat.com 43 View Easiest one Largest support Possible performance issues Strong Consistency One database must be reachable by the other DBLink Updatable (?)
44. Join developers.redhat.com 44 Materialized View Better performance Strong or Eventual Consistency One database must be reachable by the other DBLink Updatable (?)
45. Join developers.redhat.com 45 Mirror Table using Trigger Depends on Database Support Strong Consistency One database must be reachable by the other DBLink
46. Join developers.redhat.com 46 Mirror Table using Transactional Code *Any* code Strong Consistency Stored Procedures or Distributed Transactions Cohesion and coupling issues Updatable (?)
47. Join developers.redhat.com 47 Mirror Table using ETL Lots of available tools Requires external trigger (usually time-based) Can aggregate from multiple datasources Read Only
48. Join developers.redhat.com 48 Event Sourcing State of data is a stream of events Eases auditing Eventual Consistency Distributable stream through a Message Bus
49. http://debezium.io
50. Join developers.redhat.com Feedback welcome! @yanaga
51. plus.google.com/+RedHat linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHatNews Thank you!