Tweet
Tweet

Slide 1

Slide 1 text

CQRS & EVENT SOURCING IN THE WILD Michiel Rook - @michieltcs

Slide 2

Slide 2 text

➤ Freelance Developer & Coach ➤ @michieltcs

Slide 3

Slide 3 text

YOU

Slide 4

Slide 4 text

RAISE YOUR HAND IF YOU HAVE

Slide 5

Slide 5 text

read CQRS / Event Sourcing theory RAISE YOUR HAND IF YOU HAVE

Slide 6

Slide 6 text

read CQRS / Event Sourcing theory followed a tutorial, built a hobby project RAISE YOUR HAND IF YOU HAVE

Slide 7

Slide 7 text

read CQRS / Event Sourcing theory followed a tutorial, built a hobby project used it in production RAISE YOUR HAND IF YOU HAVE

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

QUICK RECAP

Slide 10

Slide 10 text

' Event Sourcing ensures that all changes to application state are stored as a sequence of events. -Martin Fowler

Slide 11

Slide 11 text

ACTIVE RECORD VS. EVENT SOURCING Account Id Account number Balance 1234 12345678 €50,00 ... ... ... Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678 @michieltcs

Slide 12

Slide 12 text

COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00 @michieltcs 1 @Value 2 public class DepositMoney { 3 @TargetAggregateIdentifier 4 String accountId; 5 BigDecimal amount; 6 }

Slide 13

Slide 13 text

COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00 command
 handler @michieltcs 1 @CommandHandler 2 public void depositMoney(DepositMoney command) { 3 apply(new MoneyDeposited( 4 command.getAccountId(), 5 command.getAmount(), 6 ZonedDateTime.now())); 7 }

Slide 14

Slide 14 text

COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00 Money Deposited Account Id 1234 Amount €100,00 command
 handler @michieltcs 1 @Value 2 public class MoneyDeposited { 3 String accountId; 4 BigDecimal amount; 5 ZonedDateTime timestamp; 6 }

Slide 15

Slide 15 text

AGGREGATES @michieltcs 1 class BankAccount { 2 @AggregateIdentifier 3 private String accountId; 4 private String accountNumber; 5 private BigDecimal balance; 6 7 // ... 8 @EventHandler 9 public void accountOpened(AccountOpened event) { 10 this.accountId = event.getAccountId(); 11 this.accountNumber = event.getAccountNumber(); 12 this.balance = BigDecimal.valueOf(0); 13 } 14 15 @EventHandler 16 public void moneyDeposited(MoneyDeposited event) { 17 this.balance = this.balance.add(event.getAmount()); 18 } 19 }

Slide 16

Slide 16 text

AGGREGATE STATE Account number Balance 12345678 €0,00 Account number Balance 12345678 €100,00 Account number Balance 12345678 €50,00 event
 handler event
 handler event
 handler @michieltcs Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678

Slide 17

Slide 17 text

TESTING AGGREGATES @michieltcs 1 public class BankAccountTest { 2 private FixtureConfiguration fixture; 3 4 @Before 5 public void createFixture() { 6 fixture = new AggregateTestFixture<>(BankAccount.class); 7 } 8 9 @Test 10 public void noOverdraftsOnEmptyAccount() { 11 fixture.given(new AccountOpened(ACCOUNT_ID, ACCOUNT_NUMBER)) 12 .when(new WithdrawMoney(ACCOUNT_ID, new BigDecimal(20))) 13 .expectException(OverdraftDetectedException.class); 14 } 15 16 private final static String ACCOUNT_ID = "accountId"; 17 private final static String ACCOUNT_NUMBER = "accountNumber"; 18 }

Slide 18

Slide 18 text

CQRS

Slide 19

Slide 19 text

UI @michieltcs

Slide 20

Slide 20 text

Domain UI Command commands Aggregates @michieltcs

Slide 21

Slide 21 text

Domain UI Command Repository Event Store commands events Aggregates @michieltcs

Slide 22

Slide 22 text

Domain UI Event Bus Event Handlers Command Repository Database Database Event Store commands events events Aggregates @michieltcs

Slide 23

Slide 23 text

Domain UI Event Bus Event Handlers Command Repository Data Layer Database Database Event Store commands events events queries DTOs Aggregates @michieltcs

Slide 24

Slide 24 text

REPLAYS AND REBUILDS

Slide 25

Slide 25 text

ANSWERING QUERIES

Slide 26

Slide 26 text

BASED ON EVENTS

Slide 27

Slide 27 text

QUERIES Account Opened Account Opened Account Closed Number of active accounts? @michieltcs

Slide 28

Slide 28 text

QUERIES Money Deposited Money Withdrawn Interest Received Accounts with balance > €100? Money Deposited Money Withdrawn @michieltcs

Slide 29

Slide 29 text

PROJECTION Account Opened Event Handler # of active accounts +1 Account Closed Event Handler # of active accounts -1 @michieltcs

Slide 30

Slide 30 text

PROJECTION Events Event Handler(s) Storage @michieltcs

Slide 31

Slide 31 text

PROJECTION @michieltcs 1 public class BankAccountProjections { 2 private Map activeAccounts = new HashMap<>(); 3 4 @EventHandler 5 public void onAccountOpened(AccountOpened accountOpened) { 6 activeAccounts.put(accountOpened.getAccountId(), accountOpened. 7 getAccountNumber()); 8 } 9 10 @EventHandler 11 public void onAccountClosed(AccountClosed accountClosed) { 12 activeAccounts.remove(accountClosed.getAccountId()); 13 } 14 15 public String getAccountNumberForAccountId(String accountId) { 16 return activeAccounts.get(accountId); 17 } 18 }

Slide 32

Slide 32 text

PROJECTION @michieltcs 1 @GetMapping("accounts/{accountId}") 2 public String getAccountNumber(@PathVariable("accountId") String accountId) { 3 return bankAccountProjections.getAccountNumberForAccountId(accountId); 4 }

Slide 33

Slide 33 text

NEW PROJECTION

Slide 34

Slide 34 text

NEW STRUCTURE

Slide 35

Slide 35 text

BASED ON EXISTING EVENTS

Slide 36

Slide 36 text

REBUILDING Stop app Cleanup Loop over events Apply to projection Start app @michieltcs

Slide 37

Slide 37 text

ZERO DOWNTIME Loop over existing events Apply to new projection Use projection @michieltcs

Slide 38

Slide 38 text

ZERO DOWNTIME New events Queue Loop over existing events Apply to new projection Apply queued events Use projection @michieltcs

Slide 39

Slide 39 text

ZERO DOWNTIME Get next event Apply to new projection Last event? Use projection yes no @michieltcs

Slide 40

Slide 40 text

LONG RUNNING REBUILDS?

Slide 41

Slide 41 text

IN MEMORY

Slide 42

Slide 42 text

DISTRIBUTED

Slide 43

Slide 43 text

PARTIAL

Slide 44

Slide 44 text

BACKGROUND TASK

Slide 45

Slide 45 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 46

Slide 46 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 47

Slide 47 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 48

Slide 48 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 49

Slide 49 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 50

Slide 50 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event Token Store

Slide 51

Slide 51 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 52

Slide 52 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 53

Slide 53 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 54

Slide 54 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 55

Slide 55 text

TRACKING EVENT PROCESSOR @michieltcs 1 @Target(ElementType.TYPE) 2 @Retention(RetentionPolicy.RUNTIME) 3 public @interface RebuildableProjection { 4 String version() default ""; 5 boolean rebuild() default false; 6 }

Slide 56

Slide 56 text

TRACKING EVENT PROCESSOR @michieltcs 1 @Configuration 2 public class ProjectionsConfiguration { 3 @Autowired 4 private EventHandlingConfiguration eventHandlingConfiguration;

Slide 57

Slide 57 text

TRACKING EVENT PROCESSOR @michieltcs 1 @PostConstruct 2 public void startTrackingProjections() throws ClassNotFoundException { 3 ClassPathScanningCandidateComponentProvider scanner = 4 new ClassPathScanningCandidateComponentProvider(false); 5 scanner.addIncludeFilter(new AnnotationTypeFilter(RebuildableProjection. 6 class)); 7 8 for (BeanDefinition bd : scanner.findCandidateComponents("org.demo")) { 9 Class> aClass = Class.forName(bd.getBeanClassName()); 10 RebuildableProjection rebuildableProjection = aClass.getAnnotation( 11 RebuildableProjection. 12 class); 13 14 if (rebuildableProjection.rebuild()) { 15 registerRebuildableProjection(aClass, rebuildableProjection); 16 } 17 } 18 }

Slide 58

Slide 58 text

TRACKING EVENT PROCESSOR @michieltcs 1 private void registerRebuildableProjection(Class> aClass, 2 RebuildableProjection 3 rebuildableProjection) { 4 ProcessingGroup processingGroup = aClass.getAnnotation(ProcessingGroup. 5 class); 6 7 String name = Optional.ofNullable(processingGroup).map(ProcessingGroup:: 8 value) 9 .orElse(aClass.getName() + "/" + rebuildableProjection.version()); 10 11 eventHandlingConfiguration.assignHandlersMatching( 12 name, 13 Integer.MAX_VALUE, 14 (eventHandler) -> aClass.isAssignableFrom(eventHandler.getClass())); 15 16 eventHandlingConfiguration.registerTrackingProcessor(name); 17 }

Slide 59

Slide 59 text

EVENT VERSIONING

Slide 60

Slide 60 text

NEW BUSINESS REQUIREMENTS

Slide 61

Slide 61 text

CHANGING VIEW ON EVENTS

Slide 62

Slide 62 text

IRRELEVANT

Slide 63

Slide 63 text

DIFFERENT FIELDS

Slide 64

Slide 64 text

WRONG NAME

Slide 65

Slide 65 text

TOO COARSE

Slide 66

Slide 66 text

TOO FINE

Slide 67

Slide 67 text

SUPPORT YOUR LEGACY?

Slide 68

Slide 68 text

Commands can be renamed 1 @michieltcs

Slide 69

Slide 69 text

Commands can be renamed 1 Events are immutable 2 @michieltcs

Slide 70

Slide 70 text

Commands can be renamed 1 Events are immutable Correct old events with new events 2 3 @michieltcs

Slide 71

Slide 71 text

COMPENSATING ACTIONS class MoneyWithdrawn {
 String accountId;
 BigDecimal amount;
 } class WithdrawalRolledBack {
 String accountId;
 BigDecimal amount;
 } Typo: too much withdrawn!

Slide 72

Slide 72 text

COMPENSATING ACTIONS class AccountOpened {
 String accountId;
 String accountNumber;
 } class DuplicateAccountClosed {
 String accountId;
 } Duplicate account number!

Slide 73

Slide 73 text

UPCASTING

Slide 74

Slide 74 text

UPCASTING Event Store @michieltcs

Slide 75

Slide 75 text

UPCASTING @michieltcs 1 @Value 2 @Revision("1.0") 3 public class AccountOpened { 4 String accountId; 5 String accountNumber; 6 }

Slide 76

Slide 76 text

UPCASTING Event Store AccountOpened (1.0) Event Handler @michieltcs

Slide 77

Slide 77 text

UPCASTING @michieltcs 1 @Value 2 @Revision("2.0") 3 public class AccountOpened { 4 String accountId; 5 String accountNumberIban; 6 }

Slide 78

Slide 78 text

UPCASTING Event Store AccountOpened (1.0) Upcaster AccountOpened (2.0) Event Handler @michieltcs

Slide 79

Slide 79 text

UPCASTING @michieltcs 1 @Getter 2 public class AccountOpenedUpcaster implements EventUpcaster { 3 private final SerializedType typeConsumed 4 = new SimpleSerializedType(AccountOpened.class.getTypeName(), "1.0"); 5 private final SerializedType typeProduced 6 = new SimpleSerializedType(AccountOpened.class.getTypeName(), "2.0");

Slide 80

Slide 80 text

UPCASTING @michieltcs 1 @Override 2 public final Stream upcast( 3 Stream intermediateRepresentations) { 4 return intermediateRepresentations.map(evt -> { 5 if (evt.getType().equals(getTypeConsumed())) { 6 return evt.upcastPayload(getTypeProduced(), Document.class, 7 document -> { 8 Element rootElement = document.getRootElement(); 9 Element accountNumberElement = rootElement.element( 10 "accountNumber"); 11 rootElement.remove(accountNumberElement); 12 rootElement.addElement("accountNumberIban") 13 .setText(toIban(accountNumberElement.getText())); 14 return document; 15 }); 16 } else { 17 return evt; 18 } 19 }); 20 }

Slide 81

Slide 81 text

VERSIONED EVENT STORE

Slide 82

Slide 82 text

VERSIONED EVENT STORE events_v1 [
 {
 "id": "12345678",
 "type": "AccountOpened",
 "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "1.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ] @michieltcs

Slide 83

Slide 83 text

COPY & REPLACE

Slide 84

Slide 84 text

VERSIONED EVENT STORE Loop over existing events Apply upcaster Add queued events Use new event store New events Queue @michieltcs

Slide 85

Slide 85 text

VERSIONED EVENT STORE events_v2 [
 {
 "id": "12345678",
 "type": "AccountOpened",
 "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "2.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ] @michieltcs

Slide 86

Slide 86 text

GDPR

Slide 87

Slide 87 text

"RIGHT TO ERASURE"

Slide 88

Slide 88 text

PERSONALLY IDENTIFIABLE INFORMATION

Slide 89

Slide 89 text

ANONYMIZED OR REMOVED

Slide 90

Slide 90 text

PROCESSING GDPR ART. 17 REQUESTS @michieltcs ConsentWithdrawn Remove from event store Remove from read models Notify 3rd parties

Slide 91

Slide 91 text

PROCESSING GDPR ART. 17 REQUESTS @michieltcs ConsentWithdrawn Remove from event store ? Remove from read models Notify 3rd parties

Slide 92

Slide 92 text

IMMUTABLE EVENTS?

Slide 93

Slide 93 text

STORE PII EXTERNALLY

Slide 94

Slide 94 text

STORE PII EXTERNALLY @michieltcs AccountOpened External Storage 1 @Value 2 public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 John Doe ... ... ...

Slide 95

Slide 95 text

CRYPTO ERASURE

Slide 96

Slide 96 text

AXON GDPR MODULE @michieltcs 1 @Value 2 public class AccountOpened { 3 @DataSubjectId 4 String accountId; 5 6 @PersonalData 7 String accountNumber; 8 9 @PersonalData 10 String fullName; 11 }

Slide 97

Slide 97 text

CONCURRENCY

Slide 98

Slide 98 text

CONCURRENT COMMANDS Withdraw Money Account Id 1234 Amount €50,00 Deposit Money Account Id 1234 Amount €100,00 ? @michieltcs

Slide 99

Slide 99 text

PESSIMISTIC LOCKING Withdraw Money Account Id 1234 Amount €50,00 Deposit Money Account Id 1234 Amount €100,00 Account Id Balance 1234 €100,00 Account Id Balance 1234 €50,00 wait for lock lock @michieltcs

Slide 100

Slide 100 text

OPTIMISTIC LOCKING Withdraw Money Account Id 1234 Amount €50,00 version 1 Deposit Money Account Id 1234 Amount €100,00 version 1 Account Id Balance 1234 €100,00 version 2 ConcurrencyException @michieltcs

Slide 101

Slide 101 text

MULTIPLE INSTANCES

Slide 102

Slide 102 text

MULTIPLE INSTANCES Replica Replica Replica Commands Distributed
 Command Bus Command
 Handler Command
 Handler Command
 Handler Command
 Handler Command
 Handler Command
 Handler Command
 Handler Command
 Handler Command
 Handler @michieltcs

Slide 103

Slide 103 text

CLOSING WORDS

Slide 104

Slide 104 text

CQRS + ES = AWESOME

Slide 105

Slide 105 text

NO SILVER BULLET

Slide 106

Slide 106 text

COMPLEXITY

Slide 107

Slide 107 text

INFRASTRUCTURE

Slide 108

Slide 108 text

AUDIT TRAIL

Slide 109

Slide 109 text

SCALABILITY

Slide 110

Slide 110 text

TESTING

Slide 111

Slide 111 text

DOMAIN FIT

Slide 112

Slide 112 text

LITERATURE

Slide 113

Slide 113 text

THANK YOU! @michieltcs / michiel@michielrook.nl
 
 www.michielrook.nl