DDDVienna - Applying event sourcing and CQRS in a large ERP system

Slide 1

Slide 1 text

Applying event sourcing and CQRS in a large ERP system Our work at AFAS Software

Slide 2

Slide 2 text

400+ employees (4 locations) 10.500 customers (companies) AFAS Software

Slide 3

Slide 3 text

HRM, CRM, finance, order management, project management, workflow, ... AFAS Profit

Slide 4

Slide 4 text

1996 – AFAS starts 2011 (jan) – First steps in CQRS and event sourcing 2011 (dec) – Started at AFAS 2020 – Live Via JTeam/Trifork/Axon. 2016 – Started on evolution

Slide 5

Slide 5 text

20 years of ERP 10.000 customers 90% cloud Web Scalable CQRS Event sourcing

Slide 6

Slide 6 text

20 years of ERP 10.000 customers 90% cloud Person Entity Customer Role Order Agreement party Spending limit Address Organisation Entity Delivery BusinessAc tivity party Invoice BusinessAc tivity party Address Address Spending limit Spending limit Own Organisation Workarea workarea Spending limit Put all our knowledge and experience in a model

Slide 7

Slide 7 text

Customized Web Scalable CQRS Event sourcing Person Entity Customer Role Order Agreement party Spending limit Address Organisation Entity Delivery BusinessAc tivity party Invoice BusinessAc tivity party Address Address Spending limit Spending limit Own Organisation Workarea workarea Spending limit

Slide 8

Slide 8 text

(2.830 tables with 126.705 columns) ERP software is inherently relational

Slide 9

Slide 9 text

Our basic CQRS/Event Sourcing solution

Slide 10

Slide 10 text

Aggregate root Command Event store Event Projector Query store Event Queue Query request Query handler

Slide 11

Slide 11 text

Event store Aggregate root Command Event store Event Projector Query store Event Queue Query request Query handler Aggregate root Command Projector Event Queue Query store

Slide 12

Slide 12 text

Event store Aggregate root Command Event store Event Projector Query store Event Queue Query request Query handler Aggregate root Command Projector Event Queue Query store Data of different projectors can not be joined. Tables are fully inlined, no joins.

Slide 13

Slide 13 text

Aggregate root Command Event store Event Projector Query store Event Queue Query request Query handler • Aimed at CRUD • MongoDB, *SQL, DocumentDB • MS Orleans, MS Service Fabric • SPA-frontend

Slide 14

Slide 14 text

How about evolution? Part 1

Slide 15

Slide 15 text

Evolution as a feature of the model? Person Entity Customer Role Order Agreement party Spending limit Address Organisation Entity Delivery BusinessAc tivity party Invoice BusinessAc tivity party Address Address Spending limit Spending limit Own Organisation Workarea workarea Spending limit

Slide 16

Slide 16 text

Data upgrade Deployed with Executed by Executed by Data upgrade Application upgrade Lazy transformation Upcasting In place transformation Multiple versions Copy and transformation Basic & Complex Event Basic & Complex Stream Basic Store Complex Store Big Flip Rolling Upgrade Blue-Green Expand- Contract Blue- Green Deployed with Combined with Weak schema Expensive!

Slide 17

Slide 17 text

Event Publishing part 2

Slide 18

Slide 18 text

Aggregate root Command Event store Event Projector Query store Event Queue Query request Query handler

Slide 19

Slide 19 text

Aggregate root Command Event store Event Projector Query store Event Queue Query request Query handler Can give error (needs retry). Command-side in control over publishing.

Slide 20

Slide 20 text

Aggregate root Command Event store Event Projector Query store Query request Query handler

Slide 21

Slide 21 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Projector in control. More load on the event store.

Slide 22

Slide 22 text

When CRUD is not enough business processes

Slide 23

Slide 23 text

Aggregate root Command Event store Event Projector Query store Query request Query handler

Slide 24

Slide 24 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Process manager

Slide 25

Slide 25 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Process manager But what if these commands fail?

Slide 26

Slide 26 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Process manager Event store

Slide 27

Slide 27 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Process manager Event store Commands that cannot fail become events. We still have some internal commands.

Slide 28

Slide 28 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Process manager Event store Different instances to scale the work. Event sourced for state.

Slide 29

Slide 29 text

Aggregate root / Process manager Command Event store Event Projector Query store Query request Query handler Is this what we want?

Slide 30

Slide 30 text

Aggregate root / Process manager Command Event store Event Projector Query store Query request Query handler Is this what we want? Technically there is no difference between aggregates and processmanagers

Slide 31

Slide 31 text

The problem of relational data

Slide 32

Slide 32 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Process manager Event store

Slide 33

Slide 33 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Process manager Event store Process manager Process manager Process manager Aggregate root Aggregate root Aggregate root Aggregate root

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Process manager Event store Stream Projector Query store Event store

Slide 36

Slide 36 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Process manager Event store Stream Projector Query store Event store StreamProjectors are single-instanced, non- event-sourced processmanagers

Slide 37

Slide 37 text

Eventual consistency communication between client and server

Slide 38

Slide 38 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Client

Slide 39

Slide 39 text

Counter Process Manager Count Command Counted Event Event store Counter Projector Query store Get Counter Count Query handler Getting the next consecutive number in a series.

Slide 40

Slide 40 text

Counter Process Manager Count Command Counted Event Event store Counter Projector Query store Get Counter Count Query handler When is the value ready? Correlationid?

Slide 41

Slide 41 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Client Using websockets to notify the Client

Slide 42

Slide 42 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Client Returning values from the produced event directly

Slide 43

Slide 43 text

Counter Process Manager Count Command Counted Event Event store Counter Projector Query store Get Counter Count Query handler

Slide 44

Slide 44 text

How large is our system anyway?

Slide 45

Slide 45 text

Aggregate root Command Event store Event Projector Query store Query request Query handler Stream Projector Query store Event store 557 464 1927 4849 1075 2718 801 1479

Slide 46

Slide 46 text

Store (19) S M L Traffic (19) S M L Schema (14) S M L

Slide 47

Slide 47 text

Dealing with bounded contexts

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

Aggregate root Command Event store Event Aggregate root Event store Event

Slide 50

Slide 50 text

Aggregate root Command Event store Event Aggregate root Event store Event Team B Team A

Slide 51

Slide 51 text

Aggregate root Command Event store Event Aggregate root Event store Event These events now serve multiple goals. Team B Team A

Slide 52

Slide 52 text

Aggregate root Command Event store Event Aggregate root Event store Event Team B Team A Sync Event store Event

Slide 53

Slide 53 text

Aggregate root Command Event store Event Aggregate root Event store Event Team B Team A Sync Event store Event Introduce synchronization events.

Slide 54

Slide 54 text

How about evolution? Part 2

Slide 55

Slide 55 text

Versioned Events Weak Schema Upcasters In-Place Copy- Transform

Slide 56

Slide 56 text

We will go live in January… No guarantees are given ☺

Slide 57

Slide 57 text

Copyright Nasa Goddard Michiel Overeem @michielovereem • No event bus • Commands do have results • Projectors can publish events • Aggregates can listen to events • You can do what you want ☺ i.e. you do not have to listen to a consultant that does not know your context.