Designing Towards Event Sourcing

Designing Towards Event Sourcing

µCon 2019 (London, UK)

Event sourcing offers you many benefits from both business and pure programming angles. As it's such a fundamental design principle, one might argue that it should be introduced early in your designs. However, this doesn't need to be the case. Especially, when you are anticipating future requirements to be based on this but you don't have a budget to solve them yet as you are dealing with today's problems.

More: https://skillsmatter.com/legacy_profile/tugberk-ugurlu

Join Tugberk in this session where he will walk you though an evolvable software architecture based on Domain-driven Design and CQRS, which can let you optimise for today's problems at the same time allowing you to design towards event sourcing. You will also start understanding the concepts of Hexagonal Architecture pattern, which puts the Domain at the heart of the system and pushes the input and output at the edges.

This talk will give you some techniques to let you be tactical about your architectural approach, which will be especially handy when conflicts arises between business needs and refactoring costs.

3a53a5136f530c121a196914aecf3936?s=128

Tugberk Ugurlu

May 30, 2019
Tweet

Transcript

  1. Designing Towards Event Sourcing Tugberk Ugurlu Senior Software Engineer Deliveroo

  2. Hello, I’m Tugberk!

  3. None
  4. Sign up and get £10! https://roo.it/tugberku-dgfd

  5. careers.deliveroo.co.uk We’re Growing! Unique challenges, amazing people and great food!

  6. So, Event Sourcing....

  7. Audit Log

  8. Potential Changes to the Information Presentation

  9. Growing complexity making it hard to diagnose issues

  10. Event Sourcing

  11. 1 BasketCreated ItemAdded ItemAdded ItemRemoved ItemQuantityIncreased ItemAdded BasketCompleted 2 BasketCreated

    ItemAdded ItemQuantityIncreased ItemAdded ItemAdded 3 BasketCreated ItemAdded ItemAdded BasketCompleted Event Streams
  12. Write Side

  13. Append Only Writes BasketCreated { "id": "1", "time": 1559172146 }

    ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemAdded { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved { "itemId": "10", "time": 1559172286 } ItemQuantityIncreased { "itemId": "20", "increasedBy": 2, "time": 1559172325 } State { "id": 1, "items": [ { "id": "20", "quantity": 4 } ], "createdOn": 1559172146, "lastUpdatedOn": 1559172325, "version": 5 }
  14. Changing Projections BasketCreated { "id": "1", "time": 1559172146 } ItemAdded

    { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemAdded { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved { "itemId": "10", "time": 1559172286 } ItemQuantityIncreased { "itemId": "20", "increasedBy": 2, "time": 1559172325 } BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemRemoved { "itemId": "10", "time": 1559172286 } BasketCompleted { "itemId": "20", "increasedBy": 2, "time": 1559172325 } State { "completedBasketWithRemovedItems": "50%" } State { "averageItemRemoveDurationInSeconds": 70 } State { "removedItems": [ { "id": "10", "count": 2 } ] }
  15. Consistency Checks BasketCreated { "id": "1", "time": 1559172146 } ItemAdded

    (v1) { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemAdded (v2) { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved (v3) { "itemId": "10", "time": 1559172286 } ItemQuantityIncreased (v3) { "itemId": "10", "increasedBy": 2, "time": 1559172325 } State { "id": 1, "items": [ { "id": "20", "quantity": 2 } ], "createdOn": 1559172146, "lastUpdatedOn": 1559172286, "version": 4 }
  16. 1 BasketCreated ItemAdded ItemAdded ItemRemoved ItemQuantityIncreased ItemAdded BasketCompleted 2 BasketCreated

    ItemAdded ItemQuantityIncreased ItemAdded ItemAdded 3 BasketCreated ItemAdded ItemAdded BasketCompleted Naturally Partitioned
  17. Read Side

  18. Eventual Consistency • Read side needs to embrace eventual consistency

    • Unless you can reproject over the entire stream every time a read is requested • Which is especially very inefficient where your projection needs to work across streams
  19. Consumer Driven Subscription • Consumer dictates where the delivery needs

    to start from • Unlike AMQP model, event messages can be requested over and over again
  20. Event Order Matters • So that the state can be

    applied expectedly • Unless read models pay a special attention to the sequence of the messages and try to BasketCreated { "id": "1", "time": 1559172146 } ItemAdded { "itemId": "20", "quantity": 2, "time": 1559172225 } ItemRemoved { "itemId": "10", "time": 1559172286 } ItemAdded { "itemId": "10", "quantity": 1, "time": 1559172216 } ItemQuantityIncreased { "itemId": "20", "increasedBy": 2, "time": 1559172325 }
  21. Implementation

  22. Two Sides Low level How to coordinate the flow from

    request arriving to response going out the door, for both reads and writes. High level How the implementation fits into our end-to-end solution, where the system resources defined and being used.
  23. Low Level

  24. Aggregate • Write model • Represents the consistency boundary •

    Stream of events are tied to an aggregate. Why? ◦ Aggregate represents our consistency boundary and we can enforce the consistency within an aggregate even when applying multiple events to its state • Not a strict relationship structure ◦ Could result in very large aggregates w/o necessarily needing all the resources • Repository handles aggregate construction by retrieving and applying all the events
  25. Aggregate: Construction https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L22-L37

  26. Aggregate: Construction https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L97-L118

  27. Commands • Request to alter the state of a particular

    aggregate instance • Command is different than events ◦ commands can be rejected (e.g. due to invalidating business invariants) ◦ whereas events represent a state change which has already happened • Application of a command on an aggregate instance can cause one or multiple events to be emitted
  28. Command Handler • Sits in between command sender and the

    aggregate • Abstract away the write model from the outside world • Performs invariant checks which an aggregate simply cannot do ◦ E.g. in cases where the check needs to query an external data outside the boundaries of an aggregate instance
  29. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L29-L41

  30. https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L19-L38

  31. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L43-L46

  32. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L43-L49

  33. Queries • Construct a read model which satisfy the access

    pattern • Listening on events on one aggregate or multiple, across streams • Doesn’t have to project the data on the fly, can cache and serve data faster in that way • Projection can also happen offline
  34. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L75-L111

  35. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L128-L170

  36. Example https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L39-L58

  37. High Level

  38. Hexagonal Architecture (a.k.a. Ports and Adaptors) • Architectural approach to

    encapsulate changes • Prevents infrastructure code to leak into the domain • Consists of 3 layers: ◦ Domain ◦ Application ◦ Framework • Hexagonal Architecture != Layered Architecture http://tpierrain.blogspot.com/2016/04/hexagonal-layers.html
  39. https://fideloper.com/hexagonal-architecture

  40. Hexagonal Architecture and Even Sourcing: Test If you change your

    data storage system and this results in making a change in the domain layer, you failed! More accurately, your abstractions failed you!
  41. More Resources • CQRS FAQ: https://cqrs.nu/Faq • Greg Young -

    CQRS and Event Sourcing - Code on the Beach 2014: https://www.youtube.com/watch?v=JHGkaShoyNs • Hexagonal Architecture: https://fideloper.com/hexagonal-architecture • Versioning in an Event Sourced System: https://leanpub.com/esversioning • CQRS Example in Go: https://github.com/jetbasrawi/go.cqrs • CQRS Example in C#: https://github.com/gregoryyoung/m-r • Distributed Sagas: A Protocol for Coordinating Microservices - Caitie McCaffrey: https://www.youtube.com/watch?v=0UTOLRTwOX0
  42. Questions?

  43. Thank you! Let’s keep in touch! @tourismgeek tugberk@outlook.com tugberkugurlu.com