Building Event-Driven Microservices with Event Sourcing and CQRS

Most systems today store only the current state of their business entities. However, we can look at the current state through a different lens- as a derivative of previous behaviors, and store those behaviors as a sequence of events instead of the current state of the entity. The current state will be derived by replaying the events.

Event sourcing is a great way to implement event-driven applications, and it’s often combined with CQRS (Command Query Responsibility Segregation) which is a key part of an architecture based on event sourcing.

In this talk I will cover the principles of event sourcing pattern, how to model your data and implement business logic using this pattern and how to improve your system through the segregation of reads and writes with specialized read models.

You will also learn how those concepts fit in with event-driven micro services through a real life example- an invoices management app we built at Wix.

Transcript

1. Engineering Building Event-Driven Microservices with Event Sourcing and CQRS Lidan Hifi Team Lead, Wix @lidanh
2. Lidan Hifi Team Lead, Wix User Notifications Platform Israel Tel Aviv Be’er Sheva Lithuania Vilnius Ukraine Dnipropetrovsk Kiev
4. Accidental Complexity
7. Restaurants Blog Events Bookings Forum Photography Stores Hotels Video Music ShoutOut Invoices Engage 300+ Apps Contacts
8. Restaurants Blog Events Bookings Forum Photography Stores Hotels Video Music ShoutOut Invoices Engage 300+ Apps Contacts
9. Invoices
13. WEB SERVICE /invoices/:id Application Logic BUSINESS LOGIC DATABASES AND OTHER SERVICES DAO Domain Model DB Analytics Emails
14. WEB SERVICE /invoices/:id Application Logic BUSINESS LOGIC DATABASES AND OTHER SERVICES DAO Domain Model DB Analytics Emails
15. WEB SERVICE /invoices/:id Application Logic BUSINESS LOGIC DATABASES AND OTHER SERVICES DAO Domain Model DB Analytics Emails
16. WEB SERVICE /invoices/:id Application Logic BUSINESS LOGIC DATABASES AND OTHER SERVICES DAO Domain Model DB Analytics Emails
17. WEB SERVICE /invoices/:id Application Logic BUSINESS LOGIC DATABASES AND OTHER SERVICES DAO Domain Model DB Analytics Emails
18. We Lose Data By Design
19. Regulatory compliance
20. Regulatory compliance Scalability
21. Regulatory compliance Scalability Ability to add new features easily
23. Event Sourcing Express what the user wants in domain language
24. Create Invoice id: 809f142f-afba-4cad-93b5-d26b3ae7a021 customer: { name: “lidanh”, id: … } currency: USD line items: [ … ] COMMAND Named with an imperative verb Express the user’s intent POST /invoices/a33b7d1f More meaningful than HTTP methods CreateInvoice vs.
25. Create Invoice id: 809f142f-afba-4cad-93b5-d26b3ae7a021 customer: { name: “lidanh”, id: … } currency: USD line items: [ … ] COMMAND EVENTS 1 InvoiceCreated 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT 5 LineItemAdded: { … } 6 LineItemAdded: { … } Named with a past-participle verb Events are immutable Prefer fine-grained events Named with an imperative verb Express the user’s intent POST /invoices/a33b7d1f More meaningful than HTTP methods CreateInvoice vs. time
26. Create Invoice id: 809f142f-afba-4cad-93b5-d26b3ae7a021 customer: { name: “lidanh”, id: … } currency: USD line items: [ … ] COMMAND EVENTS 1 InvoiceCreated 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT 5 LineItemAdded: { … } 6 LineItemAdded: { … } DB INSERT INTO invoice_events time
27. Regenerating State 1 InvoiceCreated 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT 5 LineItemAdded: { … } DB AGGREGATE EVENTS
28. Regenerating State r { customer: lidanh, currency: USD, status: DRAFT, lineItems: [ {…} ] }
29. fold f(state, event) => state
30. Regulatory compliance Scalability Ability to add new features easily No data loss
31. READS WRITES >>
32. 10 % 90 % READS WRITES Easier to Scale Flexible View Models Normalized Form Validations
33. 10 % 90 % READS WRITES Easier to Scale Flexible View Models Normalized Form Validations SELECT A.SD1, B.ED1 FROM (SELECT SD1, ROW_NUMBER() OVER (ORDER BY SD1) AS RN1 FROM (SELECT T1.Start_Date AS SD1, T2.Start_Date AS SD2 FROM (SELECT * FROM Projects ORDER BY Start_Date) T1 LEFT JOIN (SELECT * FROM Projects ORDER BY Start_Date) T2 ON T1.Start_Date=(T2.Start_Date+1) ORDER BY T1.Start_Date) WHERE SD2 IS NULL) A INNER JOIN (SELECT ED1, ROW_NUMBER() OVER (ORDER BY ED1) AS RN2 FROM (SELECT T1.End_Date AS ED1, T2.Start_Date AS SD2 FROM (SELECT * FROM Projects ORDER BY Start_Date) T1 LEFT JOIN (SELECT * FROM Projects ORDER BY Start_Date) T2 ON T1.End_Date=(T2.Start_Date) ORDER BY T1.Start_Date) WHERE SD2 IS NULL) B ON A.RN1=B.RN2 ORDER BY (B.ED1-A.SD1), A.SD1;
34. 10 % 90 % READS WRITES Easier to Scale Flexible View Models Normalized Form Validations SELECT A.SD1, B.ED1 FROM (SELECT SD1, ROW_NUMBER() OVER (ORDER BY SD1) AS RN1 FROM (SELECT T1.Start_Date AS SD1, T2.Start_Date AS SD2 FROM (SELECT * FROM Projects ORDER BY Start_Date) T1 LEFT JOIN (SELECT * FROM Projects ORDER BY Start_Date) T2 ON T1.Start_Date=(T2.Start_Date+1) ORDER BY T1.Start_Date) WHERE SD2 IS NULL) A INNER JOIN (SELECT ED1, ROW_NUMBER() OVER (ORDER BY ED1) AS RN2 FROM (SELECT T1.End_Date AS ED1, T2.Start_Date AS SD2 FROM (SELECT * FROM Projects ORDER BY Start_Date) T1 LEFT JOIN (SELECT * FROM Projects ORDER BY Start_Date) T2 ON T1.End_Date=(T2.Start_Date) ORDER BY T1.Start_Date) WHERE SD2 IS NULL) B ON A.RN1=B.RN2 ORDER BY (B.ED1-A.SD1), A.SD1;
35. CRUD is Difficult to Scale
36. Let’s CQRSify /invoices/:id Application Logic BUSINESS LOGIC DATABASES AND OTHER SERVICES Domain Model DAO WEB SERVICE DB GET / POST
37. Let’s CQRSify /invoices/:id BUSINESS LOGIC DATABASES AND OTHER SERVICES Domain Model DAO WEB SERVICE /invoices/:id Command Logic DB Read Optimized Views Domain Model DAO Query Logic POST GET
38. Let’s CQRSify /invoices/:id BUSINESS LOGIC DATABASES AND OTHER SERVICES Domain Model DAO WEB SERVICE /invoices/:id Command Logic DB Read Optimized Views Domain Model DAO Query Logic Inconsistency Window POST GET
39. CQRS is an Optimization Do you really have scaling issues?
40. CONSIDER DOING CQRS Scaling your application is difficult Highly asymmetrical read/write ratio Complex domain logic
41. WEB SERVICE /execute Command Decoder COMMAND HANDLING DATABASES AND EVENT HANDLERS Events Store Analytics Emails Command Handler Event Bus /invoice/:id Read Optimized Views
42. WEB SERVICE /execute Command Decoder COMMAND HANDLING DATABASES AND EVENT HANDLERS Events Store Command Handler Event Bus { command: “SendInvoiceToCustomer", aggregateId: "809f142f-afba-4cad-93b5-d26b3ae7a021", payload: { subject:"New Invoice from Lidan Ltd.", message: "You received an invoice from Lidan Ltd." } }
43. WEB SERVICE /execute Command Decoder COMMAND HANDLING DATABASES AND EVENT HANDLERS Events Store Command Handler Event Bus
44. WEB SERVICE /execute Command Decoder COMMAND HANDLING DATABASES AND EVENT HANDLERS Events Store Command Handler Event Bus
45. WEB SERVICE /execute Command Decoder COMMAND HANDLING DATABASES AND EVENT HANDLERS Events Store Command Handler Event Bus
46. WEB SERVICE /execute Command Decoder COMMAND HANDLING DATABASES AND EVENT HANDLERS Events Store Command Handler Event Bus
47. WEB SERVICE /execute Command Decoder COMMAND HANDLING DATABASES AND EVENT HANDLERS Events Store Analytics Emails Command Handler Event Bus Read Optimized Views
48. WEB SERVICE /execute Command Decoder COMMAND HANDLING DATABASES AND EVENT HANDLERS Events Store Analytics Emails Command Handler Event Bus /invoice/:id Read Optimized Views
49. WEB SERVICE /execute Command Decoder COMMAND HANDLING DATABASES AND EVENT HANDLERS Events Store Analytics Emails Command Handler Event Bus /invoice/:id Read Optimized Views
50. Regulatory Compliance Scalability Ability to add new features easily No Data Loss
51. THINGS WE LIKE Audit Log “for free” Debugging Tool Historic State - Time Travel 1 Log, Multiple Schemas
53. Event Sourcing 
 is Not a Silver Bullet Beware of the added complexity
54. COMPLEXITY Development Operational
55. Handling Consistency Command Handler Event Bus Read Optimized Views Strong vs. Eventually
56. Handling Consistency Strong vs. Eventually On-the-fly views /invoices/:id In memory View Events Store
57. Handling Consistency Strong vs. Eventually On-the-fly views Synchronous Writes Command Handler Read Optimized View Read Optimized View Read Optimized View
58. Handling Consistency Strong vs. Eventually Events Store Read Optimized Views version? id: 1 version: 10 id: 1 version: 15 On-the-fly views Synchronous Writes Delta Updates
59. Handling Consistency Strong vs. Eventually Events Store Read Optimized Views version? id: 1 version: 10 id: 1 version: 15 update On-the-fly views Synchronous Writes Delta Updates
60. Your Business Domain Drives The Consistency Level
61. Handling Validations { command: “SendInvoiceToCustomer", aggregateId: "809f142f-afba-4cad-93b5-d26b3ae7a021", payload: { subject:"New Invoice from Lidan Ltd.", message: "You received an invoice from Lidan Ltd." } }
62. Handling Validations Single Command- simple input validation https://github.com/wix/accord val commandValidator = validator[CreateTax] { c => c.name is notEmpty c.rate is >=(0.0) }
63. Handling Validations Single Command- simple input validation https://github.com/wix/accord Aggregate Validation- can I apply this command? CreationCommand, DeletionCommand case class CreateLabel(...) extends LabelCommand with CreationCommand case class DeleteLabel(...) extends LabelCommand with DeletionCommand
64. Handling Validations Single Command- simple input validation https://github.com/wix/accord Aggregate Validation- can I apply this command? CreationCommand, DeletionCommand Complex Validation based on domain requirements Validate against the read store Emails per tenant id tenant id email 53d6211d-acfb-468c- bf7d-7245a630b64b lidan@wix.com 53d6211d-acfb-468c- bf7d-7245a630b64b ittai@wix.com 53d6211d-acfb-468c- bf7d-7245a630b64b ohad@wix.com
65. Your Business Domain Drives The Way You Validate the Data
66. Concurrent Commands How to Resolve Conflicts Update Invoice Delete Invoice ?
67. Concurrent Commands How to Resolve Conflicts Update Invoice Delete Invoice Pessimistic Locking An aggregate can only be accessed by one thread at a time EVENTS DB
68. Concurrent Commands How to Resolve Conflicts Optimistic Locking Expected Version < Actual Version ? Update Invoice version: 3 Delete Invoice version: 3 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD
69. Concurrent Commands How to Resolve Conflicts Optimistic Locking Expected Version < Actual Version ? Update Invoice version: 3 Delete Invoice version: 3 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT ConcurrentModificationException
70. Concurrent Commands How to Resolve Conflicts 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT Conflict Resolution
71. Concurrent Commands How to Resolve Conflicts 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT Conflict Resolution 4 LineItemAdded: { … }
72. Concurrent Commands How to Resolve Conflicts 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT Conflict Resolution 5 LineItemAdded: { … }
73. Concurrent Commands How to Resolve Conflicts 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT Conflict Resolution
74. Concurrent Commands How to Resolve Conflicts 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT Conflict Resolution 4 InvoiceStatusSet: DELETED
75. Concurrent Commands How to Resolve Conflicts 2 InvoiceCustomerSet: lidanh 3 InvoiceCurrencySet: USD 4 InvoiceStatusSet: DRAFT Conflict Resolution Concurrent Modification 4 InvoiceStatusSet: DELETED
76. Your Business Domain Drives The Locking Mechanism
77. Lessons Learned
78. Event sourcing is not a top-level architecture Small Steps
79. Beware of changing the event definitions in your code Code Changes
80. The CQRS’s read model is extremely flexible. Your DBA will love it. No Database Migration
81. REFERENCES Scaling Event Sourcing for Netflix Downloads https://youtu.be/rsSld8NycCU Greg Young Lectures
82. LOOKING FOR MORE?
83. LOOKING FOR MORE?
84. def load(tenantId: TenantId, aggregateId: AggregateId, revision: Option[AggregateVersion] = None): Option[SnapshotType] = { } val events = eventStreamReader.readStream( tenantId.id, aggregateId.id, revision.map(_ => VersionRange(to = revision))) events.lastOption.map { lastEvent => } val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) AggregateSnapshot(aggregate, lastEvent.aggregateVersion, events.head.timestamp, lastEvent.timestamp)
85. def load(tenantId: TenantId, aggregateId: AggregateId, revision: Option[AggregateVersion] = None): Option[SnapshotType] = { } val events = eventStreamReader.readStream( tenantId.id, aggregateId.id, revision.map(_ => VersionRange(to = revision))) events.lastOption.map { lastEvent => } val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) AggregateSnapshot(aggregate, lastEvent.aggregateVersion, events.head.timestamp, lastEvent.timestamp)
86. def load(tenantId: TenantId, aggregateId: AggregateId, revision: Option[AggregateVersion] = None): Option[SnapshotType] = { } val events = eventStreamReader.readStream( tenantId.id, aggregateId.id, revision.map(_ => VersionRange(to = revision))) events.lastOption.map { lastEvent => } val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) AggregateSnapshot(aggregate, lastEvent.aggregateVersion, events.head.timestamp, lastEvent.timestamp)
87. def load(tenantId: TenantId, aggregateId: AggregateId, revision: Option[AggregateVersion] = None): Option[SnapshotType] = { } val events = eventStreamReader.readStream( tenantId.id, aggregateId.id, revision.map(_ => VersionRange(to = revision))) events.lastOption.map { lastEvent => } val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) AggregateSnapshot(aggregate, lastEvent.aggregateVersion, events.head.timestamp, lastEvent.timestamp)
88. def load(tenantId: TenantId, aggregateId: AggregateId, revision: Option[AggregateVersion] = None): Option[SnapshotType] = { } val events = eventStreamReader.readStream( tenantId.id, aggregateId.id, revision.map(_ => VersionRange(to = revision))) events.lastOption.map { lastEvent => } val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) AggregateSnapshot(aggregate, lastEvent.aggregateVersion, events.head.timestamp, lastEvent.timestamp)
89. val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id))
90. val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) trait AggregateReducer[AggregateType] { } def reduce(aggregate: AggregateType, event: EventData): AggregateType def apply(events: Seq[EventData])(initialState: AggregateType) = { events.foldLeft(initialState)(reduce) }
91. val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) trait AggregateReducer[AggregateType] { } def reduce(aggregate: AggregateType, event: EventData): AggregateType def apply(events: Seq[EventData])(initialState: AggregateType) = { events.foldLeft(initialState)(reduce) }
92. val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) trait AggregateReducer[AggregateType] { } def reduce(aggregate: AggregateType, event: EventData): AggregateType def apply(events: Seq[EventData])(initialState: AggregateType) = { events.foldLeft(initialState)(reduce) }
93. val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) trait AggregateReducer[AggregateType] { } def reduce(aggregate: AggregateType, event: EventData): AggregateType def apply(events: Seq[EventData])(initialState: AggregateType) = { events.foldLeft(initialState)(reduce) }
94. val aggregate = reducer(events.map(e => e.data))(emptyState(aggregateId.id)) trait AggregateReducer[AggregateType] { } def reduce(aggregate: AggregateType, event: EventData): AggregateType def apply(events: Seq[EventData])(initialState: AggregateType) = { events.foldLeft(initialState)(reduce) }
95. fold f(state, event) => state
96. 1 Talk Started
97. 1 Talk Started 2 Event Sourcing Presented
98. 1 Talk Started 2 Event Sourcing Presented 3 CQRS Demonstrated
99. 1 Talk Started 2 Event Sourcing Presented 3 CQRS Demonstrated 4 Trade-offs Explained
100. 1 Talk Started 2 Event Sourcing Presented 3 CQRS Demonstrated 4 Trade-offs Explained Save it in your event store.
101. Engineering Take the Survey. Get the Slides. http://escqrs.online @lidanh