CQRS & Event-Sourcing in Microservice-Architectures

Domain-driven Design, Events, CQRS, and Event Sourcing are important foundations for a microservices architecture. These slides present an overview over these concepts and show benefits as well als challenges.

1. Eberhard Wolff Fellow INNOQ @ewolff http://ewolff.com
2. http://continuous-delivery-buch.de/ http://continuous-delivery-book.com/
3. http://microservices-buch.de/ http://microservices-book.com/
4. http://microservices-book.com/ primer.html http://microservices-buch.de/ ueberblick.html FREE!!!!
5. http://microservices-praxisbuch.de/rezepte.html FREE!!!!
6. http://microservices-praxisbuch.de/
7. What are Microservices?
8. What are Microservices? Modules
9. What are Microservices? Separately deployable Modules
10. What are Microservices? Separately deployable Modules =Container
11. What are Microservices? • Modules, so: • Using microservices is a decision per system • Microservice are just one way to do modules. • Old rules apply e.g. low coupling & high cohesion • (There is more: http://isa-principles.org )
12. What is a Great Microservice? • Feature implemented in one microservice. • Feature put into production with one deployment. • A separate part of the domain!
13. Most important part of doing microservice: Getting the split right!!!
14. Domain-driven Design’s Bounded Context provides a specialized domain model for some use cases.
15. Order Invoice Delivery Items Rebates Credit cards Taxes Parcel service Returns
17. Events • Allow communication between Bounded Contexts • Model an event in the business domain • Something happened (past) • Immutable • No delete • Instead: cancel event
18. Events • Easily extensible systems • Traditional benefits of events
19. Order Invoice Delivery Asynchronous Communication ???
20. What is in the Event?
21. New Order Item prices Billing address Item sizes Item weights Delivery address ??? Invoice Delivery ??? • Lots of data • Which Bounded Context uses what? • Changes can impact all modules • DDD: Published Language • THE order model? • Contradicts Bounded Context?
22. Delivery New Order Invoice New Order Item prices Billing address Item sizes Item weights Delivery address • Specialized “events” • Independent evolution • Easy to figure out which Bounded Context uses what • Extensibility need new events • DDD: Customer / Supplier ??? New Order ???
23. New Order Order ID • Just an ID • Provide additional specialized API for data for each consumer • Compromise between extensibility and specialized events
24. Events: Conclusion • What data should be included? • Extensibility is not so easy • Probably do not include all data • Prefer specialized events and just IDs
25. Events: Conclusion • Best option for communication between Bounded Contexts • …and “correct” from a domain point. • Implicit replication
26. Inconsistencies!!!
28. What Kind of Inconsistencies? • One system for invoices • One for delivery • One for orders • One source of truth for each information. • No “invoice might be there or not”
29. Inconsistencies • Either delivery without invoice • …or invoice without delivery. • Will eventually be solved. • Delays are not unheard of. • What is the alternative? • Centralized complex database? • Really??
30. Inconsistencies • Ask: What are the consequences if it takes 0.1s / 1s / 1min / 1h / 1d for the new data to appear? • “Invoice has to be there in <1min!!!!!”
31. Cancel Order 42 New Order 23 New Order 42 There is no order 42! Uuups Order 42 might be delivered and invoiced Here is a new order! …even though it was cancelled. Order of events matters!
33. Event Sourcing • Event queue provides events • Event handlers react to events • Event handler might create a snapshot • Event store stores all events
34. Event Queue Event Event Event Event Handler Event Handler Event Store Snapshot
35. Cancel Order 42 • Note problem • Middleware might notify about Out-of-order • Discard snapshot • …rollback to old snapshot • …or complete rebuild • Order events correctly • Replay New Order 23 New Order 42
36. Cancel Order 42 • Not trivial • Has an invoice / delivery been created for order 42? • ”New Order” event not enough • Need to read potentially very old events • …with old schema • …and missing information New Order 23 New Order 42
37. Other Benefits • Radical: State becomes just a snapshot. • Migrate database schemata easier • Replay • Fully History • Time-based queries • Auditing • Event processing
38. Event Queue Event Event Event Event Handler Event Handler Event Store Snapshot
39. Event Store • Must Store all events • …for replay etc • Some event queues store old events. • E.g. Kafka or Atom • Why implement locally?
40. Global Event Store • Event must include more than just the ID. • Microservices need to handle (very?) old events. • Truly global event store = Published Language • …with its challenges.
41. Event Sourcing is Pattern for State Management.
42. State Management • Responsibility of the microservices • Might be changed • Might be different for each microservice • Event Sourcing should be a local decision
43. Invoice • Order needs to be paid • Invoice is generated • Afterwards invoices are only read • Invoices must be immutable • Catalog has a lot more reads than writes • Scale reads independent from writes?
45. CQRS • Command – Query Responsibility Segregation • Commands change data • Query provide data • Implement in separate microservices
46. Commands vs Events • Command: Change that data! • Event: Something has happened • Component decides if data should be changed
47. CQRS • Command store just stores commands • Command handler change data • Query Handler read data • Different specialized data models • …or even Bounded Contexts possible • More independent modules
48. Command Queue Command Command Command Command Handler Query Handler Command Store Database Read Invoice Payment Read Replica? Create Invoice!
49. CQRS • Queries and commands separated • But: shared database model • Quite strong coupling • No independent evolution • Same technology
50. Command Queue Command Command Command Command Handler Query Handler Command Store Database Read Invoice Payment Read Replica? Create Invoice!
51. Command Queue Command Command Command Command Handler Query Handler Command Store Read Invoice Payment Snapshot Event Sourcing Create Invoice! Database?
52. CQRS & Event Sourcing • Full separation of commands and queries • Might choose different technologies • Separate deployment possible • …including data models • Truly separated models / Bounded Context
54. Conclusion • Most important: Getting the split right! • Bounded Context • …communicate via events • Inconsistencies can be dealt with • What is in the event? • Event Sourcing: A pattern for state management. • CQRS: Split commands and queries • Do not overuse Event Sourcing and CQRS
55. • EMail oop2018@ewolff.com to get: • Slides • + Microservices Primer • + Microservices Rezepte • + Sample Microservices Praxisbuch • + Sample Microservices Buch / Book • + Sample of Continuous Delivery Buch / Book • Powered by Amazon Lambda & Microservices