Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dutch .NET Group Meetup - Building an event sourced system in .NET

Dutch .NET Group Meetup - Building an event sourced system in .NET

In this talk I will show how we are building a large ERP system in .NET, using CQRS and event sourcing. We will talk about the good stuff (modularization, scalability), the lessons learned (eventual consistency) and the challenges ahead (upgrades).

Michiel Overeem

April 19, 2018
Tweet

More Decks by Michiel Overeem

Other Decks in Technology

Transcript

  1. Building an event sourced system in .NET
    Michiel Overeem
    Lead Software Architect
    [email protected]
    @michielovereem

    View Slide

  2. 400+ employees (4 locations)
    10.500 customers (companies)
    AFAS Software

    View Slide

  3. HRM, CRM, finance, order
    management, project
    management, workflow, ...
    AFAS Profit

    View Slide

  4. 20 years of ERP
    10.000 customers
    90% cloud
    Web
    Scalable
    CQRS
    Event sourcing

    View Slide

  5. 20 years of ERP
    10.000 customers
    90% cloud
    Person
    Entity
    Customer
    Role
    Order
    Agreement
    party
    Spending limit
    Address
    Organisation
    Entity
    Delivery
    BusinessAc tivity
    party
    Invoice
    BusinessAc tivity
    party
    Address
    Address
    Spending limit
    Spending limit
    Own Organisation
    Workarea
    workarea
    Spending limit
    Put all our knowledge and
    experience in a model

    View Slide

  6. Customized
    Web
    Scalable
    CQRS
    Event sourcing
    Person
    Entity
    Customer
    Role
    Order
    Agreement
    party
    Spending limit
    Address
    Organisation
    Entity
    Delivery
    BusinessAc tivity
    party
    Invoice
    BusinessAc tivity
    party
    Address
    Address
    Spending limit
    Spending limit
    Own Organisation
    Workarea
    workarea
    Spending limit

    View Slide

  7. (2.830 tables with 126.705 columns)
    ERP software is inherently relational

    View Slide

  8. Command Query
    Responsibility Segregation
    Client
    Command system
    Query system
    Eventing system

    View Slide

  9. Command Query
    Responsibility Segregation
    Client
    Command system
    Query system
    Eventing system
    “PlaceOrderCommand”: {
    “OrderId”: “0b81b458-7671-4e9b-844b-9934255f5406”,
    “CustomerName”: “Linus Torvalds”,
    “OrderLines”: [ … ]
    }

    View Slide

  10. Command Query
    Responsibility Segregation
    Client
    Command system
    Query system
    Eventing system
    “GetOrders”: {
    “CustomerName”: “Linus Torvalds”,
    }

    View Slide

  11. Command Query
    Responsibility Segregation
    Client
    Command system
    Query system
    Eventing system

    View Slide

  12. DDD

    View Slide

  13. Aggregates - independent, self-contained business objects
    that handle commands and raise events.

    View Slide

  14. Aggregates

    View Slide

  15. Aggregates
    “PlaceOrderCommand”: {
    “OrderId”: “0b81b458-7671-4e9b-844b-9934255f5406”,
    “CustomerName”: “Linus Torvalds”,
    “OrderLines”: [ … ]
    }

    View Slide

  16. Aggregates
    “PlaceOrderCommand”: {
    “OrderId”: “0b81b458-7671-4e9b-844b-9934255f5406”,
    “CustomerName”: “Linus Torvalds”,
    “OrderLines”: [ … ]
    }
    “OrderPlacedEvent”: {
    “OrderId”: “0b81b458-7671-4e9b-844b-9934255f5406”,
    “CustomerName”: “Linus Torvalds”,
    “OrderLines”: [ … ]
    }

    View Slide

  17. Aggregates

    View Slide

  18. Aggregates

    View Slide

  19. Aggregates - independent, self-contained business objects
    that handle commands and raise events.
    That sounds a lot like actors ☺

    View Slide

  20. View Slide

  21. Events – Something that has happened.

    View Slide

  22. Events
    Events that need to
    be processed in the
    query system.

    View Slide

  23. Projectors – A process that builds a projection from events.

    View Slide

  24. Projectors

    View Slide

  25. Projectors
    Needs routing to find
    the correct store

    View Slide

  26. View Slide

  27. How to get the events from
    the command system
    to
    the query system?

    View Slide

  28. CQRS
    Client
    Command system
    Query system
    Eventing system
    Some kind of
    message bus?

    View Slide

  29. Event push

    View Slide

  30. View Slide

  31. T

    View Slide

  32. 2 phase commit Independence
    Command
    system
    Query system
    Eventing system

    View Slide

  33. Event push

    View Slide

  34. Event pull
    But we need persistence. And
    we still have a 2 phase commit.

    View Slide

  35. Use the emitted events as single
    source of truth
    Event sourcing

    View Slide

  36. CQRS
    Client
    Command system
    Query system
    Eventing system
    What is the data
    model used in the
    command side?

    View Slide

  37. CQRS
    Client
    Command system
    Query system
    Eventing system

    View Slide

  38. Event Store
    StreamId Stream
    Revision
    PayloadItemName Payload StreamType Sequence
    Number
    CommitTime AccountId
    Person1 1 PersonCreatedEvent { … } Person 1 … …
    Person1 2 PersonUpdatedEvent { … } Person 2 … …
    Person2 1 PersonCreatedEvent { … } Person 3 … …
    Order1 1 OrderCreatedEvent { … } Order 1 … …
    Order2 1 OrderCreatedEvent { … } Order 2 … …
    Order3 1 OrderCreatedEvent { … } Order 3 … …

    View Slide

  39. Aggregates loading
    Event Store
    StreamId Stream
    Revision
    PayloadItemName Payload StreamType Sequence
    Number
    CommitTime AccountId
    Person1 1 PersonCreatedEvent { … } Person 1 … …
    Person1 2 PersonUpdatedEvent { … } Person 2 … …
    Person2 1 PersonCreatedEvent { … } Person 3 … …
    Order1 1 OrderCreatedEvent { … } Order 1 … …
    Order2 1 OrderCreatedEvent { … } Order 2 … …
    Order3 1 OrderCreatedEvent { … } Order 3 … …

    View Slide

  40. Event processing
    Event Store
    StreamId Stream
    Revision
    PayloadItemName Payload StreamType Sequence
    Number
    CommitTime AccountId
    Person1 1 PersonCreatedEvent { … } Person 1 … …
    Person1 2 PersonUpdatedEvent { … } Person 2 … …
    Person2 1 PersonCreatedEvent { … } Person 3 … …
    Order1 1 OrderCreatedEvent { … } Order 1 … …
    Order2 1 OrderCreatedEvent { … } Order 2 … …
    Order3 1 OrderCreatedEvent { … } Order 3 … …

    View Slide

  41. Event Sourcing
    A
    B
    A
    E
    C
    D
    D
    F
    B
    B
    A
    C
    C
    G
    H
    D
    A
    B
    C
    A
    D
    A
    A
    A
    A
    A
    A
    B
    B
    C
    C
    1
    2
    3
    4
    5
    1
    2
    3
    4
    5
    1
    2
    3
    4
    5
    1
    1
    1
    2
    1
    1
    2
    3
    4
    5
    1
    1
    2
    1
    2

    View Slide

  42. Event Sourcing
    A
    B
    A
    E
    C
    D
    D
    F
    B
    B
    A
    C
    C
    G
    H
    D
    A
    B
    C
    A
    D
    A
    A
    A
    A
    A
    A
    B
    B
    C
    C
    1
    2
    3
    4
    5
    1
    2
    3
    4
    5
    1
    2
    3
    4
    5
    1
    1
    1
    2
    1
    1
    2
    3
    4
    5
    1
    1
    2
    1
    2
    Sequencenumbers for
    projectors

    View Slide

  43. Event Sourcing
    A
    B
    A
    E
    C
    D
    D
    F
    B
    B
    A
    C
    C
    G
    H
    D
    A
    B
    C
    A
    D
    A
    A
    A
    A
    A
    A
    B
    B
    C
    C
    1
    2
    3
    4
    5
    1
    2
    3
    4
    5
    1
    2
    3
    4
    5
    1
    1
    1
    2
    1
    1
    2
    3
    4
    5
    1
    1
    2
    1
    2
    Revisions for aggregate
    loading and concurrency
    control

    View Slide

  44. Event sourcing
    This is both the store as
    the messaging system.

    View Slide

  45. Eventual consistency

    View Slide

  46. Eventual
    consistency

    View Slide

  47. Eventual
    consistency
    This is an async process

    View Slide

  48. Eventual consistency
    Client
    Command system
    Query system
    Eventing system

    View Slide

  49. Notifications
    β
    α γ

    View Slide

  50. Notifications
    β
    α γ

    View Slide

  51. View Slide

  52. Projector optimization

    View Slide

  53. Projection
    Information from the
    customer stream
    Information from the
    order stream

    View Slide

  54. Projection
    A helper-table with customer info
    A table for the projection
    A customer update could
    mean that we need to
    update 1.000.000+ rows

    View Slide

  55. Projection:
    joins
    A table with customer info
    A table for the projection
    A query should join the
    two tables.

    View Slide

  56. Projection:
    duplication
    A table with customer info
    A table with order info
    A table with customer info
    A table with invoice info

    View Slide

  57. Projection:
    combine
    A table with customer info
    A table with order info
    A table with invoice info

    View Slide

  58. Projection:
    chaining

    View Slide

  59. So what have we seen?

    View Slide

  60. Summary

    View Slide

  61. Summary
    β
    α γ

    View Slide

  62. View Slide

  63. Building an event sourced system in .NET
    [email protected]
    @michielovereem

    View Slide