$30 off During Our Annual Pro Sale. View Details »

Designing Towards Event Sourcing

Designing Towards Event Sourcing

µCon 2019 (London, UK)

Event sourcing offers you many benefits from both business and pure programming angles. As it's such a fundamental design principle, one might argue that it should be introduced early in your designs. However, this doesn't need to be the case. Especially, when you are anticipating future requirements to be based on this but you don't have a budget to solve them yet as you are dealing with today's problems.

More: https://skillsmatter.com/legacy_profile/tugberk-ugurlu

Join Tugberk in this session where he will walk you though an evolvable software architecture based on Domain-driven Design and CQRS, which can let you optimise for today's problems at the same time allowing you to design towards event sourcing. You will also start understanding the concepts of Hexagonal Architecture pattern, which puts the Domain at the heart of the system and pushes the input and output at the edges.

This talk will give you some techniques to let you be tactical about your architectural approach, which will be especially handy when conflicts arises between business needs and refactoring costs.

Tugberk Ugurlu

May 30, 2019
Tweet

More Decks by Tugberk Ugurlu

Other Decks in Programming

Transcript

  1. Designing Towards Event
    Sourcing
    Tugberk Ugurlu
    Senior Software Engineer
    Deliveroo

    View Slide

  2. Hello,
    I’m Tugberk!

    View Slide

  3. View Slide

  4. Sign up and get £10!
    https://roo.it/tugberku-dgfd

    View Slide

  5. careers.deliveroo.co.uk
    We’re Growing!
    Unique
    challenges,
    amazing people
    and great food!

    View Slide

  6. So, Event Sourcing....

    View Slide

  7. Audit Log

    View Slide

  8. Potential Changes to the Information
    Presentation

    View Slide

  9. Growing complexity making it hard to
    diagnose issues

    View Slide

  10. Event Sourcing

    View Slide

  11. 1
    BasketCreated
    ItemAdded
    ItemAdded
    ItemRemoved
    ItemQuantityIncreased
    ItemAdded
    BasketCompleted
    2
    BasketCreated
    ItemAdded
    ItemQuantityIncreased
    ItemAdded
    ItemAdded
    3
    BasketCreated
    ItemAdded
    ItemAdded
    BasketCompleted
    Event Streams

    View Slide

  12. Write Side

    View Slide

  13. Append Only Writes
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemAdded
    {
    "itemId": "20",
    "quantity": 2,
    "time": 1559172225
    }
    ItemRemoved
    {
    "itemId": "10",
    "time": 1559172286
    }
    ItemQuantityIncreased
    {
    "itemId": "20",
    "increasedBy": 2,
    "time": 1559172325
    }
    State
    {
    "id": 1,
    "items": [
    {
    "id": "20",
    "quantity": 4
    }
    ],
    "createdOn": 1559172146,
    "lastUpdatedOn": 1559172325,
    "version": 5
    }

    View Slide

  14. Changing Projections
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemAdded
    {
    "itemId": "20",
    "quantity": 2,
    "time": 1559172225
    }
    ItemRemoved
    {
    "itemId": "10",
    "time": 1559172286
    }
    ItemQuantityIncreased
    {
    "itemId": "20",
    "increasedBy": 2,
    "time": 1559172325
    }
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemRemoved
    {
    "itemId": "10",
    "time": 1559172286
    }
    BasketCompleted
    {
    "itemId": "20",
    "increasedBy": 2,
    "time": 1559172325
    }
    State
    {
    "completedBasketWithRemovedItems": "50%"
    }
    State
    {
    "averageItemRemoveDurationInSeconds": 70
    }
    State
    {
    "removedItems": [
    { "id": "10", "count": 2 }
    ]
    }

    View Slide

  15. Consistency Checks
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded (v1)
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemAdded (v2)
    {
    "itemId": "20",
    "quantity": 2,
    "time": 1559172225
    }
    ItemRemoved (v3)
    {
    "itemId": "10",
    "time": 1559172286
    }
    ItemQuantityIncreased (v3)
    {
    "itemId": "10",
    "increasedBy": 2,
    "time": 1559172325
    }
    State
    {
    "id": 1,
    "items": [
    {
    "id": "20",
    "quantity": 2
    }
    ],
    "createdOn": 1559172146,
    "lastUpdatedOn": 1559172286,
    "version": 4
    }

    View Slide

  16. 1
    BasketCreated
    ItemAdded
    ItemAdded
    ItemRemoved
    ItemQuantityIncreased
    ItemAdded
    BasketCompleted
    2
    BasketCreated
    ItemAdded
    ItemQuantityIncreased
    ItemAdded
    ItemAdded
    3
    BasketCreated
    ItemAdded
    ItemAdded
    BasketCompleted
    Naturally Partitioned

    View Slide

  17. Read Side

    View Slide

  18. Eventual Consistency
    ● Read side needs to embrace eventual consistency
    ● Unless you can reproject over the entire stream every time a read is
    requested
    ● Which is especially very inefficient where your projection needs to work
    across streams

    View Slide

  19. Consumer Driven Subscription
    ● Consumer dictates where the delivery needs to start from
    ● Unlike AMQP model, event messages can be requested over and over again

    View Slide

  20. Event Order Matters
    ● So that the state can be applied expectedly
    ● Unless read models pay a special attention to the sequence of the messages
    and try to
    BasketCreated
    {
    "id": "1",
    "time": 1559172146
    }
    ItemAdded
    {
    "itemId": "20",
    "quantity": 2,
    "time": 1559172225
    }
    ItemRemoved
    {
    "itemId": "10",
    "time": 1559172286
    }
    ItemAdded
    {
    "itemId": "10",
    "quantity": 1,
    "time": 1559172216
    }
    ItemQuantityIncreased
    {
    "itemId": "20",
    "increasedBy": 2,
    "time": 1559172325
    }

    View Slide

  21. Implementation

    View Slide

  22. Two Sides
    Low level
    How to coordinate the flow from request arriving to response going out the
    door, for both reads and writes.
    High level
    How the implementation fits into our end-to-end solution, where the system
    resources defined and being used.

    View Slide

  23. Low Level

    View Slide

  24. Aggregate
    ● Write model
    ● Represents the consistency boundary
    ● Stream of events are tied to an aggregate. Why?
    ○ Aggregate represents our consistency boundary and we can enforce the consistency within an
    aggregate even when applying multiple events to its state
    ● Not a strict relationship structure
    ○ Could result in very large aggregates w/o necessarily needing all the resources
    ● Repository handles aggregate construction by retrieving and applying all the
    events

    View Slide

  25. Aggregate: Construction
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L22-L37

    View Slide

  26. Aggregate: Construction
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L97-L118

    View Slide

  27. Commands
    ● Request to alter the state of a particular aggregate instance
    ● Command is different than events
    ○ commands can be rejected (e.g. due to invalidating business invariants)
    ○ whereas events represent a state change which has already happened
    ● Application of a command on an aggregate instance can cause one or
    multiple events to be emitted

    View Slide

  28. Command Handler
    ● Sits in between command sender and the aggregate
    ● Abstract away the write model from the outside world
    ● Performs invariant checks which an aggregate simply cannot do
    ○ E.g. in cases where the check needs to query an external data outside the boundaries of an
    aggregate instance

    View Slide

  29. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L29-L41

    View Slide

  30. https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inventoryitem.go#L19-L38

    View Slide

  31. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/inmem_repo.go#L43-L46

    View Slide

  32. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/command_handlers.go#L43-L49

    View Slide

  33. Queries
    ● Construct a read model which satisfy the access pattern
    ● Listening on events on one aggregate or multiple, across streams
    ● Doesn’t have to project the data on the fly, can cache and serve data faster in
    that way
    ● Projection can also happen offline

    View Slide

  34. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L75-L111

    View Slide

  35. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L128-L170

    View Slide

  36. Example
    https://github.com/jetbasrawi/go.cqrs/blob/e4d812d57f090ecede016aa36d70c73626a8eb17/examples/simplecqrs/simplecqrs/readmodel.go#L39-L58

    View Slide

  37. High Level

    View Slide

  38. Hexagonal Architecture (a.k.a. Ports and Adaptors)
    ● Architectural approach to encapsulate changes
    ● Prevents infrastructure code to leak into the domain
    ● Consists of 3 layers:
    ○ Domain
    ○ Application
    ○ Framework
    ● Hexagonal Architecture != Layered Architecture
    http://tpierrain.blogspot.com/2016/04/hexagonal-layers.html

    View Slide

  39. https://fideloper.com/hexagonal-architecture

    View Slide

  40. Hexagonal Architecture and Even Sourcing: Test
    If you change your data storage system and this results in making a change in the
    domain layer, you failed!
    More accurately, your abstractions failed you!

    View Slide

  41. More Resources
    ● CQRS FAQ: https://cqrs.nu/Faq
    ● Greg Young - CQRS and Event Sourcing - Code on the Beach 2014:
    https://www.youtube.com/watch?v=JHGkaShoyNs
    ● Hexagonal Architecture: https://fideloper.com/hexagonal-architecture
    ● Versioning in an Event Sourced System: https://leanpub.com/esversioning
    ● CQRS Example in Go: https://github.com/jetbasrawi/go.cqrs
    ● CQRS Example in C#: https://github.com/gregoryyoung/m-r
    ● Distributed Sagas: A Protocol for Coordinating Microservices - Caitie
    McCaffrey: https://www.youtube.com/watch?v=0UTOLRTwOX0

    View Slide

  42. Questions?

    View Slide

  43. Thank you!
    Let’s keep in touch!
    @tourismgeek
    [email protected]
    tugberkugurlu.com

    View Slide