Designing Towards Event Sourcing

µCon 2019 (London, UK)

Event sourcing offers you many benefits from both business and pure programming angles. As it's such a fundamental design principle, one might argue that it should be introduced early in your designs. However, this doesn't need to be the case. Especially, when you are anticipating future requirements to be based on this but you don't have a budget to solve them yet as you are dealing with today's problems.

More: https://skillsmatter.com/legacy_profile/tugberk-ugurlu

Join Tugberk in this session where he will walk you though an evolvable software architecture based on Domain-driven Design and CQRS, which can let you optimise for today's problems at the same time allowing you to design towards event sourcing. You will also start understanding the concepts of Hexagonal Architecture pattern, which puts the Domain at the heart of the system and pushes the input and output at the edges.

This talk will give you some techniques to let you be tactical about your architectural approach, which will be especially handy when conflicts arises between business needs and refactoring costs.

Transcript

1. Designing Towards Event Sourcing Tugberk Ugurlu Senior Software Engineer Deliveroo
2. Hello, I’m Tugberk!
4. Sign up and get £10! https://roo.it/tugberku-dgfd
5. careers.deliveroo.co.uk We’re Growing! Unique challenges, amazing people and great food!
6. So, Event Sourcing....
7. Audit Log
8. Potential Changes to the Information Presentation
9. Growing complexity making it hard to diagnose issues
10. Event Sourcing
11. 1 BasketCreated ItemAdded ItemAdded ItemRemoved ItemQuantityIncreased ItemAdded BasketCompleted 2 BasketCreated ItemAdded ItemQuantityIncreased ItemAdded ItemAdded 3 BasketCreated ItemAdded ItemAdded BasketCompleted Event Streams
12. Write Side
13. Append Only Writes BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemAdded { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved { "itemId": "10", "time": 1559172286 } ItemQuantityIncreased { "itemId": "20", "increasedBy": 2, "time": 1559172325 } State { "id": 1, "items": [ { "id": "20", "quantity": 4 } ], "createdOn": 1559172146, "lastUpdatedOn": 1559172325, "version": 5 }
14. Changing Projections BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemAdded { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved { "itemId": "10", "time": 1559172286 } ItemQuantityIncreased { "itemId": "20", "increasedBy": 2, "time": 1559172325 } BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemRemoved { "itemId": "10", "time": 1559172286 } BasketCompleted { "itemId": "20", "increasedBy": 2, "time": 1559172325 } State { "completedBasketWithRemovedItems": "50%" } State { "averageItemRemoveDurationInSeconds": 70 } State { "removedItems": [ { "id": "10", "count": 2 } ] }
15. Consistency Checks BasketCreated { "id": "1", "time": 1559172146 } ItemAdded (v1) { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemAdded (v2) { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved (v3) { "itemId": "10", "time": 1559172286 } ItemQuantityIncreased (v3) { "itemId": "10", "increasedBy": 2, "time": 1559172325 } State { "id": 1, "items": [ { "id": "20", "quantity": 2 } ], "createdOn": 1559172146, "lastUpdatedOn": 1559172286, "version": 4 }
16. 1 BasketCreated ItemAdded ItemAdded ItemRemoved ItemQuantityIncreased ItemAdded BasketCompleted 2 BasketCreated ItemAdded ItemQuantityIncreased ItemAdded ItemAdded 3 BasketCreated ItemAdded ItemAdded BasketCompleted Naturally Partitioned
17. Read Side
18. Eventual Consistency • Read side needs to embrace eventual consistency • Unless you can reproject over the entire stream every time a read is requested • Which is especially very inefficient where your projection needs to work across streams
19. Consumer Driven Subscription • Consumer dictates where the delivery needs to start from • Unlike AMQP model, event messages can be requested over and over again
20. Event Order Matters • So that the state can be applied expectedly • Unless read models pay a special attention to the sequence of the messages and try to BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved { "itemId": "10", "time": 1559172286 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemQuantityIncreased { "itemId": "20", "increasedBy": 2, "time": 1559172325 }
21. Implementation
22. Two Sides Low level How to coordinate the flow from request arriving to response going out the door, for both reads and writes. High level How the implementation fits into our end-to-end solution, where the system resources defined and being used.
23. Low Level
24. Aggregate • Write model • Represents the consistency boundary • Stream of events are tied to an aggregate. Why? ◦ Aggregate represents our consistency boundary and we can enforce the consistency within an aggregate even when applying multiple events to its state • Not a strict relationship structure ◦ Could result in very large aggregates w/o necessarily needing all the resources • Repository handles aggregate construction by retrieving and applying all the events
25. Aggregate: Construction https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L22-L37
26. Aggregate: Construction https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L97-L118
27. Commands • Request to alter the state of a particular aggregate instance • Command is different than events ◦ commands can be rejected (e.g. due to invalidating business invariants) ◦ whereas events represent a state change which has already happened • Application of a command on an aggregate instance can cause one or multiple events to be emitted
28. Command Handler • Sits in between command sender and the aggregate • Abstract away the write model from the outside world • Performs invariant checks which an aggregate simply cannot do ◦ E.g. in cases where the check needs to query an external data outside the boundaries of an aggregate instance
29. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L29-L41
30. https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L19-L38
31. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L43-L46
32. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L43-L49
33. Queries • Construct a read model which satisfy the access pattern • Listening on events on one aggregate or multiple, across streams • Doesn’t have to project the data on the fly, can cache and serve data faster in that way • Projection can also happen offline
34. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L75-L111
35. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L128-L170
36. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L39-L58
37. High Level
38. Hexagonal Architecture (a.k.a. Ports and Adaptors) • Architectural approach to encapsulate changes • Prevents infrastructure code to leak into the domain • Consists of 3 layers: ◦ Domain ◦ Application ◦ Framework • Hexagonal Architecture != Layered Architecture http://tpierrain.blogspot.com/2016/04/hexagonal-layers.html
39. https://fideloper.com/hexagonal-architecture
40. Hexagonal Architecture and Even Sourcing: Test If you change your data storage system and this results in making a change in the domain layer, you failed! More accurately, your abstractions failed you!
41. More Resources • CQRS FAQ: https://cqrs.nu/Faq • Greg Young - CQRS and Event Sourcing - Code on the Beach 2014: https://www.youtube.com/watch?v=JHGkaShoyNs • Hexagonal Architecture: https://fideloper.com/hexagonal-architecture • Versioning in an Event Sourced System: https://leanpub.com/esversioning • CQRS Example in Go: https://github.com/jetbasrawi/go.cqrs • CQRS Example in C#: https://github.com/gregoryyoung/m-r • Distributed Sagas: A Protocol for Coordinating Microservices - Caitie McCaffrey: https://www.youtube.com/watch?v=0UTOLRTwOX0
42. Questions?
43. Thank you! Let’s keep in touch! @tourismgeek tugberk@outlook.com tugberkugurlu.com