Dutch .NET Group Meetup - Building an event sourced system in .NET

Dutch .NET Group Meetup - Building an event sourced system in .NET

In this talk I will show how we are building a large ERP system in .NET, using CQRS and event sourcing. We will talk about the good stuff (modularization, scalability), the lessons learned (eventual consistency) and the challenges ahead (upgrades).

03dc1d993fa7d5de422c1f35d53f80e6?s=128

Michiel Overeem

April 19, 2018
Tweet

Transcript

  1. Building an event sourced system in .NET Michiel Overeem Lead

    Software Architect michiel.overeem@afas.nl @michielovereem
  2. 400+ employees (4 locations) 10.500 customers (companies) AFAS Software

  3. HRM, CRM, finance, order management, project management, workflow, ... AFAS

    Profit
  4. 20 years of ERP 10.000 customers 90% cloud Web Scalable

    CQRS Event sourcing
  5. 20 years of ERP 10.000 customers 90% cloud Person Entity

    Customer Role Order Agreement party Spending limit Address Organisation Entity Delivery BusinessAc tivity party Invoice BusinessAc tivity party Address Address Spending limit Spending limit Own Organisation Workarea workarea Spending limit Put all our knowledge and experience in a model
  6. Customized Web Scalable CQRS Event sourcing Person Entity Customer Role

    Order Agreement party Spending limit Address Organisation Entity Delivery BusinessAc tivity party Invoice BusinessAc tivity party Address Address Spending limit Spending limit Own Organisation Workarea workarea Spending limit
  7. (2.830 tables with 126.705 columns) ERP software is inherently relational

  8. Command Query Responsibility Segregation Client Command system Query system Eventing

    system
  9. Command Query Responsibility Segregation Client Command system Query system Eventing

    system “PlaceOrderCommand”: { “OrderId”: “0b81b458-7671-4e9b-844b-9934255f5406”, “CustomerName”: “Linus Torvalds”, “OrderLines”: [ … ] }
  10. Command Query Responsibility Segregation Client Command system Query system Eventing

    system “GetOrders”: { “CustomerName”: “Linus Torvalds”, }
  11. Command Query Responsibility Segregation Client Command system Query system Eventing

    system
  12. DDD

  13. Aggregates - independent, self-contained business objects that handle commands and

    raise events.
  14. Aggregates

  15. Aggregates “PlaceOrderCommand”: { “OrderId”: “0b81b458-7671-4e9b-844b-9934255f5406”, “CustomerName”: “Linus Torvalds”, “OrderLines”: [

    … ] }
  16. Aggregates “PlaceOrderCommand”: { “OrderId”: “0b81b458-7671-4e9b-844b-9934255f5406”, “CustomerName”: “Linus Torvalds”, “OrderLines”: [

    … ] } “OrderPlacedEvent”: { “OrderId”: “0b81b458-7671-4e9b-844b-9934255f5406”, “CustomerName”: “Linus Torvalds”, “OrderLines”: [ … ] }
  17. Aggregates

  18. Aggregates

  19. Aggregates - independent, self-contained business objects that handle commands and

    raise events. That sounds a lot like actors ☺
  20. None
  21. Events – Something that has happened.

  22. Events Events that need to be processed in the query

    system.
  23. Projectors – A process that builds a projection from events.

  24. Projectors

  25. Projectors Needs routing to find the correct store

  26. None
  27. How to get the events from the command system to

    the query system?
  28. CQRS Client Command system Query system Eventing system Some kind

    of message bus?
  29. Event push

  30. None
  31. T

  32. 2 phase commit Independence Command system Query system Eventing system

  33. Event push

  34. Event pull But we need persistence. And we still have

    a 2 phase commit.
  35. Use the emitted events as single source of truth Event

    sourcing
  36. CQRS Client Command system Query system Eventing system What is

    the data model used in the command side?
  37. CQRS Client Command system Query system Eventing system

  38. Event Store StreamId Stream Revision PayloadItemName Payload StreamType Sequence Number

    CommitTime AccountId Person1 1 PersonCreatedEvent { … } Person 1 … … Person1 2 PersonUpdatedEvent { … } Person 2 … … Person2 1 PersonCreatedEvent { … } Person 3 … … Order1 1 OrderCreatedEvent { … } Order 1 … … Order2 1 OrderCreatedEvent { … } Order 2 … … Order3 1 OrderCreatedEvent { … } Order 3 … …
  39. Aggregates loading Event Store StreamId Stream Revision PayloadItemName Payload StreamType

    Sequence Number CommitTime AccountId Person1 1 PersonCreatedEvent { … } Person 1 … … Person1 2 PersonUpdatedEvent { … } Person 2 … … Person2 1 PersonCreatedEvent { … } Person 3 … … Order1 1 OrderCreatedEvent { … } Order 1 … … Order2 1 OrderCreatedEvent { … } Order 2 … … Order3 1 OrderCreatedEvent { … } Order 3 … …
  40. Event processing Event Store StreamId Stream Revision PayloadItemName Payload StreamType

    Sequence Number CommitTime AccountId Person1 1 PersonCreatedEvent { … } Person 1 … … Person1 2 PersonUpdatedEvent { … } Person 2 … … Person2 1 PersonCreatedEvent { … } Person 3 … … Order1 1 OrderCreatedEvent { … } Order 1 … … Order2 1 OrderCreatedEvent { … } Order 2 … … Order3 1 OrderCreatedEvent { … } Order 3 … …
  41. Event Sourcing A B A E C D D F

    B B A C C G H D A B C A D A A A A A A B B C C 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 1 1 2 1 1 2 3 4 5 1 1 2 1 2
  42. Event Sourcing A B A E C D D F

    B B A C C G H D A B C A D A A A A A A B B C C 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 1 1 2 1 1 2 3 4 5 1 1 2 1 2 Sequencenumbers for projectors
  43. Event Sourcing A B A E C D D F

    B B A C C G H D A B C A D A A A A A A B B C C 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 1 1 2 1 1 2 3 4 5 1 1 2 1 2 Revisions for aggregate loading and concurrency control
  44. Event sourcing This is both the store as the messaging

    system.
  45. Eventual consistency

  46. Eventual consistency

  47. Eventual consistency This is an async process

  48. Eventual consistency Client Command system Query system Eventing system

  49. Notifications β α γ

  50. Notifications β α γ

  51. None
  52. Projector optimization

  53. Projection Information from the customer stream Information from the order

    stream
  54. Projection A helper-table with customer info A table for the

    projection A customer update could mean that we need to update 1.000.000+ rows
  55. Projection: joins A table with customer info A table for

    the projection A query should join the two tables.
  56. Projection: duplication A table with customer info A table with

    order info A table with customer info A table with invoice info
  57. Projection: combine A table with customer info A table with

    order info A table with invoice info
  58. Projection: chaining

  59. So what have we seen?

  60. Summary

  61. Summary β α γ

  62. None
  63. Building an event sourced system in .NET michiel.overeem@afas.nl @michielovereem