Designing Towards Event Sourcing Tugberk Ugurlu Senior Software Engineer Deliveroo 

Hello, I’m Tugberk! 

No content

Sign up and get £10! https://roo.it/tugberku-dgfd 

careers.deliveroo.co.uk We’re Growing! Unique challenges, amazing people and great food! 

So, Event Sourcing.... 

Audit Log 

Potential Changes to the Information Presentation 

Growing complexity making it hard to diagnose issues 

Event Sourcing 

1 BasketCreated ItemAdded ItemAdded ItemRemoved ItemQuantityIncreased ItemAdded BasketCompleted 2 BasketCreated ItemAdded ItemQuantityIncreased ItemAdded ItemAdded 3 BasketCreated ItemAdded ItemAdded BasketCompleted Event Streams 

Write Side 

Append Only Writes BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemAdded { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved { "itemId": "10", "time": 1559172286 } ItemQuantityIncreased { "itemId": "20", "increasedBy": 2, "time": 1559172325 } State { "id": 1, "items": [ { "id": "20", "quantity": 4 } ], "createdOn": 1559172146, "lastUpdatedOn": 1559172325, "version": 5 } 

Changing Projections BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemAdded { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved { "itemId": "10", "time": 1559172286 } ItemQuantityIncreased { "itemId": "20", "increasedBy": 2, "time": 1559172325 } BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemRemoved { "itemId": "10", "time": 1559172286 } BasketCompleted { "itemId": "20", "increasedBy": 2, "time": 1559172325 } State { "completedBasketWithRemovedItems": "50%" } State { "averageItemRemoveDurationInSeconds": 70 } State { "removedItems": [ { "id": "10", "count": 2 } ] } 

Consistency Checks BasketCreated { "id": "1", "time": 1559172146 } ItemAdded (v1) { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemAdded (v2) { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved (v3) { "itemId": "10", "time": 1559172286 } ItemQuantityIncreased (v3) { "itemId": "10", "increasedBy": 2, "time": 1559172325 } State { "id": 1, "items": [ { "id": "20", "quantity": 2 } ], "createdOn": 1559172146, "lastUpdatedOn": 1559172286, "version": 4 } 

1 BasketCreated ItemAdded ItemAdded ItemRemoved ItemQuantityIncreased ItemAdded BasketCompleted 2 BasketCreated ItemAdded ItemQuantityIncreased ItemAdded ItemAdded 3 BasketCreated ItemAdded ItemAdded BasketCompleted Naturally Partitioned 

Read Side 

Eventual Consistency ● Read side needs to embrace eventual consistency ● Unless you can reproject over the entire stream every time a read is requested ● Which is especially very inefficient where your projection needs to work across streams 

Consumer Driven Subscription ● Consumer dictates where the delivery needs to start from ● Unlike AMQP model, event messages can be requested over and over again 

Event Order Matters ● So that the state can be applied expectedly ● Unless read models pay a special attention to the sequence of the messages and try to BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved { "itemId": "10", "time": 1559172286 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemQuantityIncreased { "itemId": "20", "increasedBy": 2, "time": 1559172325 } 

Implementation 

Two Sides Low level How to coordinate the flow from request arriving to response going out the door, for both reads and writes. High level How the implementation fits into our end-to-end solution, where the system resources defined and being used. 

Low Level 

Aggregate ● Write model ● Represents the consistency boundary ● Stream of events are tied to an aggregate. Why? ○ Aggregate represents our consistency boundary and we can enforce the consistency within an aggregate even when applying multiple events to its state ● Not a strict relationship structure ○ Could result in very large aggregates w/o necessarily needing all the resources ● Repository handles aggregate construction by retrieving and applying all the events 

Aggregate: Construction https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L22-L37 

Aggregate: Construction https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L97-L118 

Commands ● Request to alter the state of a particular aggregate instance ● Command is different than events ○ commands can be rejected (e.g. due to invalidating business invariants) ○ whereas events represent a state change which has already happened ● Application of a command on an aggregate instance can cause one or multiple events to be emitted 

Command Handler ● Sits in between command sender and the aggregate ● Abstract away the write model from the outside world ● Performs invariant checks which an aggregate simply cannot do ○ E.g. in cases where the check needs to query an external data outside the boundaries of an aggregate instance 

Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L29-L41 

https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L19-L38 

Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L43-L46 

Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L43-L49 

Queries ● Construct a read model which satisfy the access pattern ● Listening on events on one aggregate or multiple, across streams ● Doesn’t have to project the data on the fly, can cache and serve data faster in that way ● Projection can also happen offline 

Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L75-L111 

Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L128-L170 

Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L39-L58 

High Level 

Hexagonal Architecture (a.k.a. Ports and Adaptors) ● Architectural approach to encapsulate changes ● Prevents infrastructure code to leak into the domain ● Consists of 3 layers: ○ Domain ○ Application ○ Framework ● Hexagonal Architecture != Layered Architecture http://tpierrain.blogspot.com/2016/04/hexagonal-layers.html 

https://fideloper.com/hexagonal-architecture 

Hexagonal Architecture and Even Sourcing: Test If you change your data storage system and this results in making a change in the domain layer, you failed! More accurately, your abstractions failed you! 

More Resources ● CQRS FAQ: https://cqrs.nu/Faq ● Greg Young - CQRS and Event Sourcing - Code on the Beach 2014: https://www.youtube.com/watch?v=JHGkaShoyNs ● Hexagonal Architecture: https://fideloper.com/hexagonal-architecture ● Versioning in an Event Sourced System: https://leanpub.com/esversioning ● CQRS Example in Go: https://github.com/jetbasrawi/go.cqrs ● CQRS Example in C#: https://github.com/gregoryyoung/m-r ● Distributed Sagas: A Protocol for Coordinating Microservices - Caitie McCaffrey: https://www.youtube.com/watch?v=0UTOLRTwOX0 

Questions? 

Thank you! Let’s keep in touch! @tourismgeek [email protected] tugberkugurlu.com