Upgrade to Pro — share decks privately, control downloads, hide ads and more …

EventSourcing.Live 2021 - A system with thousands of event types

Michiel Overeem
October 07, 2021
Technology
0
170

EventSourcing.Live 2021 - A system with thousands of event types

In the last 10 years we have applied event sourcing and CQRS in the development of our new ERP system. We now have almost two years of production experience. During these years we have run into interesting challenges that needed to be solved. In this session we share how we built a system with many thousands of event types, with the aim of serving thousands of companies and eventually millions of users.

Michiel Overeem

October 07, 2021
Tweet

More Decks by Michiel Overeem

See All by Michiel Overeem
Reminiscing a crazy journey: building an event sourced ERP system
overeemm
0
45
EventSourcing.Live 2021 - Event system evolution - A Scientific Study on Event Sourcing, Lessons from Industry
overeemm
0
150
DDDVienna - 103 years of event sourcing experience
overeemm
0
270
DDDVienna - Applying event sourcing and CQRS in a large ERP system
overeemm
1
790
Landelijk Architectuur Congres - The dark side of event sourcing: managing data conversion
overeemm
0
210
Dutch .NET Group Meetup - Building an event sourced system in .NET
overeemm
0
220
Drukwerkdeal.nl Developer Meetup - Event Sourcing After Launch
overeemm
0
140
DDDEurope 2018 - Event Sourcing After Launch
overeemm
5
2.1k
KanDDDinsky 2017 - The Dark Side of Event Sourcing: Managing Data Conversion
overeemm
2
950

Other Decks in Technology

See All in Technology
無理なく始めるGoでのユニットテストの並行化戦略
shohata
1
220
Oracle Database Technology Night #67 Oracle Database High Availability concept
oracle4engineer
PRO
0
570
microCMSのエンジニア組織と文化
microcms
0
460
ChatGPTをどう使うか？(JJUGナイトセミナー5/23)
shin_higuchi
0
730
MySQL Architectures - Design the Right Solution for Your Needs
lefred
1
110
組織の立ち上げと体制変更の1年
tarappo
2
520
C# の async/await は実際にどうやって動いているか
nenonaninu
3
11k
1人目QA奮闘記-2023年ver-/QA Engineer's Struggle 2023
mii3king
1
380
新リリース：microCMSテンプレート
microcms
1
650
Search Evolution - Keeping up with the hype?
spinscale
0
120
とあるQAエンジニアが、マイクロサービスの開発チームと、出会ったーー / Scrum Fest Niigata 2023
yoyakoba
0
130
フルComposeでTVアプリを作った話
sasakitomohiro
0
270

Featured

See All Featured
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
318
19k
Become a Pro
speakerdeck
PRO
7
3.4k
Gamification - CAS2011
davidbonilla
75
4.2k
Fashionably flexible responsive web design (full day workshop)
malarkey
395
64k
Visualization
eitanlees
130
12k
Support Driven Design
roundedbygravity
88
9k
Build The Right Thing And Hit Your Dates
maggiecrowley
23
1.5k
Statistics for Hackers
jakevdp
787
210k
Navigating Team Friction
lara
177
12k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
110
17k
Ruby is Unlike a Banana
tanoku
93
9.7k
Adopting Sorbet at Scale
ufuk
66
8k

Transcript

  1. A system with thousands of event types
    Our work at AFAS Software

    View Slide

  2. Michiel
    Lead Software Architect @ AFAS Software
    [email protected]
    @michielovereem
    https://movereem.nl
    Overeem

    View Slide

  3. 500+ employees (4 locations)
    12.000 customers (companies)
    AFAS Software
    500+ employees (4 locations)

    View Slide

  4. AFAS Profit

    View Slide

  5. Small business and
    accountants
    AFAS SB+
    500+ employees (4 locations)

    View Slide

  6. How do we move from Profit to SB+?

    View Slide

  7. Person
    Entity
    Customer
    Role
    Order
    Agreement
    party
    Spending limit
    Address
    Organisation
    Entity
    Delivery
    BusinessAc tivity
    party
    Invoice
    BusinessAc tivity
    party
    Address
    Address
    Spending limit
    Spending limit
    Own Organisation
    Workarea
    workarea
    Spending limit

    View Slide

  8. Person
    Entity
    Customer
    Role
    Order
    Agreement
    party
    Spending limit
    Address
    Organisation
    Entity
    Delivery
    BusinessAc tivity
    party
    Invoice
    BusinessAc tivity
    party
    Address
    Address
    Spending limit
    Spending limit
    Own Organisation
    Workarea
    workarea
    Spending limit
    • Functional model
    • Only suitable for ERP
    • Variability in the model
    • Contains 25 years of knowledge
    22
    25
    96

    View Slide

  9. Person
    Entity
    Customer
    Role
    Order
    Agreement
    party
    Spending limit
    Address
    Organisation
    Entity
    Delivery
    BusinessAc tivity
    party
    Invoice
    BusinessAc tivity
    party
    Address
    Address
    Spending limit
    Spending limit
    Own Organisation
    Workarea
    workarea
    Spending limit
    Generate ERP software

    View Slide

  10. Person
    Entity
    Customer
    Role
    Order
    Agreement
    party
    Spending limit
    Address
    Organisation
    Entity
    Delivery
    BusinessAc tivity
    party
    Invoice
    BusinessAc tivity
    party
    Address
    Address
    Spending limit
    Spending limit
    Own Organisation
    Workarea
    workarea
    Spending limit

    View Slide

  11. Data
    QueryModel Object
    StreamItem
    Aggregate
    EventHandler
    Projector
    Command
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Query
    Plain and simple CQRS & event sourcing

    View Slide

  12. StreamItem
    Aggregate
    StreamItem
    Aggregate
    StreamItem
    Aggregate
    Data
    QueryModel Object
    StreamItem
    Aggregate
    EventHandler
    Projector
    Command
    Event (Nr=1; Id=C)
    Event (Nr=1; Id=B)
    Event (Nr=1; Id=A)
    Query
    Event (Nr=1; Id=1)
    Event (Nr=1; Id=2)
    Event (Nr=2; Id=1)
    Aggregate
    Aggregate
    Aggregate
    StreamItem
    Aggregate
    Data
    QueryModel Object
    Data
    QueryModel Object
    EventHandler
    Projector
    Data
    QueryModel Object
    Data
    QueryModel Object
    Query
    Command

    View Slide

  13. Data
    QueryModel Object
    StreamItem
    Aggregate
    EventHandler
    Projector
    Command
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Query
    3687 11646
    4915
    1377
    1050

    View Slide

  14. View Slide

  15. Event Store stats
    34.106.336 events from 3469 event types
    Started with 8 customers, currently at 91 customers
    0,00
    5.000.000,00
    10.000.000,00
    15.000.000,00
    20.000.000,00
    25.000.000,00
    30.000.000,00
    35.000.000,00
    40.000.000,00
    Total # of events

    View Slide

  16. Person
    Entity
    Customer
    Role
    Order
    Agreement
    party
    Spending limit
    Address
    Organisation
    Entity
    Delivery
    BusinessAc tivity
    party
    Invoice
    BusinessAc tivity
    party
    Address
    Address
    Spending limit
    Spending limit
    Own Organisation
    Workarea
    workarea
    Spending limit
    A change
    here
    Or here …
    Or here …
    Impact!

    View Slide

  17. V1.1 V1.2

    View Slide

  18. V1.1

    View Slide

  19. V1.1 V1.2

    View Slide

  20. V1.1 V1.2

    View Slide

  21. V1.1 V1.2

    View Slide

  22. V1.2

    View Slide

  23. V1.1 V1.2
    This is where
    the magic
    happens…

    View Slide

  24. Repair/Upgrade
    scripts Default data
    Event rewrites
    Query
    StreamItem
    UpgradeScript
    Command
    Command
    StreamItem
    …
    Command

    View Slide

  25. Some additional patterns…

    View Slide

  26. Data
    QueryModel Object
    StreamItem
    Aggregate
    EventHandler
    Projector
    Command
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Query
    StreamItem
    ProcessManager
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Command
    EventHandler
    Router
    ProcessManagers

    View Slide

  27. EventHandler
    StreamProjector
    Data
    QueryModel Object
    Data
    QueryModel Object
    Data
    QueryModel Object
    Query
    StreamItem
    Aggregate
    Command
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Data
    QueryModel Object
    EventHandler
    Projector
    Query
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    StreamProjectors

    View Slide

  28. Data
    QueryModel Object
    EventHandler
    Projector
    Event (Nr=1; Id=C)
    Event (Nr=1; Id=B)
    Event (Nr=1; Id=A)
    Query
    Event (Nr=1; Id=1)
    Event (Nr=1; Id=2)
    Event (Nr=2; Id=1)
    Data
    QueryModel Object
    Data
    QueryModel Object
    EventHandler
    Projector
    Data
    QueryModel Object
    Data
    QueryModel Object
    Query
    Cross-Projector-Joins

    View Slide

  29. Data
    QueryModel Object
    EventHandler
    Projector
    Event (Nr=1; Id=C)
    Event (Nr=1; Id=B)
    Event (Nr=1; Id=A)
    Query
    Event (Nr=1; Id=1)
    Event (Nr=1; Id=2)
    Event (Nr=2; Id=1)
    Data
    QueryModel Object
    EventHandler
    Projector
    Data
    QueryModel Object
    Query

    View Slide

  30. Data
    QueryModel Object
    StreamItem
    Aggregate
    EventHandler
    Projector
    Command
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Query

    View Slide

  31. Data
    QueryModel Object
    StreamItem
    Aggregate
    EventHandler
    Projector
    Command
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Query
    Direct command results

    View Slide

  32. Data
    QueryModel Object
    StreamItem
    Aggregate
    EventHandler
    Projector
    Command
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Query
    Push notifications from projectors

    View Slide

  33. Data
    QueryModel Object
    EventHandler
    Projector
    Event (Nr=1; Id=C)
    Event (Nr=1; Id=B)
    Event (Nr=1; Id=A)
    Query
    Data
    QueryModel Object
    EventHandler
    Projector
    Data
    QueryModel Object
    Query
    Timers & Projectors-on-projectors

    View Slide

  34. Data
    QueryModel Object
    StreamItem
    Aggregate
    EventHandler
    Projector
    Command
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Query
    StreamItem
    ProcessManager
    Event (Nr=1)
    Event (Nr=2)
    Event (Nr=3)
    Command
    EventHandler
    Router
    Parallel & low-priority event processing

    View Slide

  35. View Slide

  36. Sync events

    View Slide

  37. [email protected]
    @michielovereem
    https://movereem.nl
    Low-code / Impact analysis / Upgrade process / Several specific patterns
    Person
    Entity
    Customer
    Role
    Order
    Agreement
    party
    Spending limit
    Address
    Organisation
    Entity
    Delivery
    BusinessAc tivity
    party
    Invoice
    BusinessAc tivity
    party
    Address
    Address
    Spending limit
    Spending limit
    Own Organisation
    Workarea
    workarea
    Spending limit

    View Slide