Designing Towards Event Sourcing

µCon 2019 (London, UK)

Event sourcing offers you many benefits from both business and pure programming angles. As it's such a fundamental design principle, one might argue that it should be introduced early in your designs. However, this doesn't need to be the case. Especially, when you are anticipating future requirements to be based on this but you don't have a budget to solve them yet as you are dealing with today's problems.

More: https://skillsmatter.com/legacy_profile/tugberk-ugurlu

Join Tugberk in this session where he will walk you though an evolvable software architecture based on Domain-driven Design and CQRS, which can let you optimise for today's problems at the same time allowing you to design towards event sourcing. You will also start understanding the concepts of Hexagonal Architecture pattern, which puts the Domain at the heart of the system and pushes the input and output at the edges.

This talk will give you some techniques to let you be tactical about your architectural approach, which will be especially handy when conflicts arises between business needs and refactoring costs.

Transcript

1. Designing Towards Event
   Sourcing
   Tugberk Ugurlu
   Senior Software Engineer
   Deliveroo
   

   View Slide

2. Hello,
   I’m Tugberk!
   

   View Slide

3. View Slide

4. Sign up and get £10!
   https://roo.it/tugberku-dgfd
   

   View Slide

5. careers.deliveroo.co.uk
   We’re Growing!
   Unique
   challenges,
   amazing people
   and great food!
   

   View Slide

6. So, Event Sourcing....
   

   View Slide

7. Audit Log
   

   View Slide

8. Potential Changes to the Information
   Presentation
   

   View Slide

9. Growing complexity making it hard to
   diagnose issues
   

   View Slide

10. Event Sourcing
    

    View Slide

11. 1
    BasketCreated
    ItemAdded
    ItemAdded
    ItemRemoved
    ItemQuantityIncreased
    ItemAdded
    BasketCompleted
    2
    BasketCreated
    ItemAdded
    ItemQuantityIncreased
    ItemAdded
    ItemAdded
    3
    BasketCreated
    ItemAdded
    ItemAdded
    BasketCompleted
    Event Streams
    

    View Slide

12. Write Side
    

    View Slide

13. Append Only Writes
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemAdded
    {
    "itemId": "20",
    "quantity": 2,
    "time": 1559172225
    }
    ItemRemoved
    {
    "itemId": "10",
    "time": 1559172286
    }
    ItemQuantityIncreased
    {
    "itemId": "20",
    "increasedBy": 2,
    "time": 1559172325
    }
    State
    {
    "id": 1,
    "items": [
    {
    "id": "20",
    "quantity": 4
    }
    ],
    "createdOn": 1559172146,
    "lastUpdatedOn": 1559172325,
    "version": 5
    }
    

    View Slide

14. Changing Projections
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemAdded
    {
    "itemId": "20",
    "quantity": 2,
    "time": 1559172225
    }
    ItemRemoved
    {
    "itemId": "10",
    "time": 1559172286
    }
    ItemQuantityIncreased
    {
    "itemId": "20",
    "increasedBy": 2,
    "time": 1559172325
    }
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemRemoved
    {
    "itemId": "10",
    "time": 1559172286
    }
    BasketCompleted
    {
    "itemId": "20",
    "increasedBy": 2,
    "time": 1559172325
    }
    State
    {
    "completedBasketWithRemovedItems": "50%"
    }
    State
    {
    "averageItemRemoveDurationInSeconds": 70
    }
    State
    {
    "removedItems": [
    { "id": "10", "count": 2 }
    ]
    }
    

    View Slide

15. Consistency Checks
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded (v1)
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemAdded (v2)
    {
    "itemId": "20",
    "quantity": 2,
    "time": 1559172225
    }
    ItemRemoved (v3)
    {
    "itemId": "10",
    "time": 1559172286
    }
    ItemQuantityIncreased (v3)
    {
    "itemId": "10",
    "increasedBy": 2,
    "time": 1559172325
    }
    State
    {
    "id": 1,
    "items": [
    {
    "id": "20",
    "quantity": 2
    }
    ],
    "createdOn": 1559172146,
    "lastUpdatedOn": 1559172286,
    "version": 4
    }
    

    View Slide

16. 1
    BasketCreated
    ItemAdded
    ItemAdded
    ItemRemoved
    ItemQuantityIncreased
    ItemAdded
    BasketCompleted
    2
    BasketCreated
    ItemAdded
    ItemQuantityIncreased
    ItemAdded
    ItemAdded
    3
    BasketCreated
    ItemAdded
    ItemAdded
    BasketCompleted
    Naturally Partitioned
    

    View Slide

17. Read Side
    

    View Slide

18. Eventual Consistency
    ● Read side needs to embrace eventual consistency
    ● Unless you can reproject over the entire stream every time a read is
    requested
    ● Which is especially very inefficient where your projection needs to work
    across streams
    

    View Slide

19. Consumer Driven Subscription
    ● Consumer dictates where the delivery needs to start from
    ● Unlike AMQP model, event messages can be requested over and over again
    

    View Slide

20. Event Order Matters
    ● So that the state can be applied expectedly
    ● Unless read models pay a special attention to the sequence of the messages
    and try to
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded
    {
    "itemId": "20",
    "quantity": 2,
    "time": 1559172225
    }
    ItemRemoved
    {
    "itemId": "10",
    "time": 1559172286
    }
    ItemAdded
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemQuantityIncreased
    {
    "itemId": "20",
    "increasedBy": 2,
    "time": 1559172325
    }
    

    View Slide

21. Implementation
    

    View Slide

22. Two Sides
    Low level
    How to coordinate the flow from request arriving to response going out the
    door, for both reads and writes.
    High level
    How the implementation fits into our end-to-end solution, where the system
    resources defined and being used.
    

    View Slide

23. Low Level
    

    View Slide

24. Aggregate
    ● Write model
    ● Represents the consistency boundary
    ● Stream of events are tied to an aggregate. Why?
    ○ Aggregate represents our consistency boundary and we can enforce the consistency within an
    aggregate even when applying multiple events to its state
    ● Not a strict relationship structure
    ○ Could result in very large aggregates w/o necessarily needing all the resources
    ● Repository handles aggregate construction by retrieving and applying all the
    events
    

    View Slide

25. Aggregate: Construction
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L22-L37
    

    View Slide

26. Aggregate: Construction
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L97-L118
    

    View Slide

27. Commands
    ● Request to alter the state of a particular aggregate instance
    ● Command is different than events
    ○ commands can be rejected (e.g. due to invalidating business invariants)
    ○ whereas events represent a state change which has already happened
    ● Application of a command on an aggregate instance can cause one or
    multiple events to be emitted
    

    View Slide

28. Command Handler
    ● Sits in between command sender and the aggregate
    ● Abstract away the write model from the outside world
    ● Performs invariant checks which an aggregate simply cannot do
    ○ E.g. in cases where the check needs to query an external data outside the boundaries of an
    aggregate instance
    

    View Slide

29. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L29-L41
    

    View Slide

30. https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L19-L38
    

    View Slide

31. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L43-L46
    

    View Slide

32. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L43-L49
    

    View Slide

33. Queries
    ● Construct a read model which satisfy the access pattern
    ● Listening on events on one aggregate or multiple, across streams
    ● Doesn’t have to project the data on the fly, can cache and serve data faster in
    that way
    ● Projection can also happen offline
    

    View Slide

34. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L75-L111
    

    View Slide

35. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L128-L170
    

    View Slide

36. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L39-L58
    

    View Slide

37. High Level
    

    View Slide

38. Hexagonal Architecture (a.k.a. Ports and Adaptors)
    ● Architectural approach to encapsulate changes
    ● Prevents infrastructure code to leak into the domain
    ● Consists of 3 layers:
    ○ Domain
    ○ Application
    ○ Framework
    ● Hexagonal Architecture != Layered Architecture
    http://tpierrain.blogspot.com/2016/04/hexagonal-layers.html
    

    View Slide

39. https://fideloper.com/hexagonal-architecture
    

    View Slide

40. Hexagonal Architecture and Even Sourcing: Test
    If you change your data storage system and this results in making a change in the
    domain layer, you failed!
    More accurately, your abstractions failed you!
    

    View Slide

41. More Resources
    ● CQRS FAQ: https://cqrs.nu/Faq
    ● Greg Young - CQRS and Event Sourcing - Code on the Beach 2014:
    https://www.youtube.com/watch?v=JHGkaShoyNs
    ● Hexagonal Architecture: https://fideloper.com/hexagonal-architecture
    ● Versioning in an Event Sourced System: https://leanpub.com/esversioning
    ● CQRS Example in Go: https://github.com/jetbasrawi/go.cqrs
    ● CQRS Example in C#: https://github.com/gregoryyoung/m-r
    ● Distributed Sagas: A Protocol for Coordinating Microservices - Caitie
    McCaffrey: https://www.youtube.com/watch?v=0UTOLRTwOX0
    

    View Slide

42. Questions?
    

    View Slide

43. Thank you!
    Let’s keep in touch!
    @tourismgeek
    tugberk@outlook.com
    tugberkugurlu.com
    

    View Slide