Tweet
Tweet

Slide 1

Slide 1 text

DEALING WITH CHANGE
 IN EVENT SOURCED APPLICATIONS Michiel Rook - @michieltcs

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

CHANGE

Slide 4

Slide 4 text

' Welcome changing requirements, even late in development. -Agile Manifesto

Slide 5

Slide 5 text

MODIFICATIONS

Slide 6

Slide 6 text

EVENT SOURCING

Slide 7

Slide 7 text

CHALLENGE!

Slide 8

Slide 8 text

YOU

Slide 9

Slide 9 text

RAISE YOUR HAND IF YOU HAVE

Slide 10

Slide 10 text

read CQRS / Event Sourcing theory RAISE YOUR HAND IF YOU HAVE

Slide 11

Slide 11 text

read CQRS / Event Sourcing theory followed a tutorial, built a hobby project RAISE YOUR HAND IF YOU HAVE

Slide 12

Slide 12 text

read CQRS / Event Sourcing theory followed a tutorial, built a hobby project used it in production RAISE YOUR HAND IF YOU HAVE

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

QUICK RECAP

Slide 15

Slide 15 text

' Event Sourcing ensures that all changes to application state are stored as a sequence of events. -Martin Fowler

Slide 16

Slide 16 text

ACTIVE RECORD VS. EVENT SOURCING Account Id Account number Balance 1234 12345678 €50,00 ... ... ... Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678 @michieltcs

Slide 17

Slide 17 text

COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00 @michieltcs 1 @Value 2 public class DepositMoney { 3 @TargetAggregateIdentifier 4 String accountId; 5 BigDecimal amount; 6 }

Slide 18

Slide 18 text

COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00 command
 handler @michieltcs 1 @CommandHandler 2 public void depositMoney(DepositMoney command) { 3 apply(new MoneyDeposited( 4 command.getAccountId(), 5 command.getAmount(), 6 ZonedDateTime.now())); 7 }

Slide 19

Slide 19 text

COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00 Money Deposited Account Id 1234 Amount €100,00 command
 handler @michieltcs 1 @Value 2 public class MoneyDeposited { 3 String accountId; 4 BigDecimal amount; 5 ZonedDateTime timestamp; 6 }

Slide 20

Slide 20 text

AGGREGATES @michieltcs an Aggregate handles Commands and generates Events based on the current state

Slide 21

Slide 21 text

AGGREGATES @michieltcs 1 class BankAccount { 2 @AggregateIdentifier 3 private String accountId; 4 private String accountNumber; 5 private BigDecimal balance; 6 7 // ... 8 @EventHandler 9 public void accountOpened(AccountOpened event) { 10 this.accountId = event.getAccountId(); 11 this.accountNumber = event.getAccountNumber(); 12 this.balance = BigDecimal.valueOf(0); 13 } 14 15 @EventHandler 16 public void moneyDeposited(MoneyDeposited event) { 17 this.balance = this.balance.add(event.getAmount()); 18 } 19 }

Slide 22

Slide 22 text

AGGREGATE STATE Account number Balance 12345678 €0,00 Account number Balance 12345678 €100,00 Account number Balance 12345678 €50,00 event
 handler event
 handler event
 handler @michieltcs Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678

Slide 23

Slide 23 text

VALIDATING COMMANDS @michieltcs 1 @CommandHandler 2 public void withdrawMoney(WithdrawMoney command) throws 3 OverdraftDetectedException { 4 if (balance.compareTo(command.getAmount()) >= 0) { 5 apply(new MoneyWithdrawn( 6 command.getAccountId(), 7 command.getAmount(), 8 ZonedDateTime.now())); 9 } else { 10 throw new OverdraftDetectedException(accountNumber, balance, command. 11 getAmount()); 12 } 13 }

Slide 24

Slide 24 text

TESTING AGGREGATES @michieltcs 1 public class BankAccountTest { 2 private FixtureConfiguration fixture; 3 4 @Before 5 public void createFixture() { 6 fixture = new AggregateTestFixture<>(BankAccount.class); 7 } 8 9 @Test 10 public void noOverdraftsOnEmptyAccount() { 11 fixture.given(new AccountOpened(ACCOUNT_ID, ACCOUNT_NUMBER)) 12 .when(new WithdrawMoney(ACCOUNT_ID, new BigDecimal(20))) 13 .expectException(OverdraftDetectedException.class); 14 } 15 16 private final static String ACCOUNT_ID = "accountId"; 17 private final static String ACCOUNT_NUMBER = "accountNumber"; 18 }

Slide 25

Slide 25 text

REPLAYS AND REBUILDS

Slide 26

Slide 26 text

ANSWERING QUERIES

Slide 27

Slide 27 text

BASED ON EVENTS

Slide 28

Slide 28 text

QUERIES Account Opened Account Opened Account Closed Number of active accounts? @michieltcs

Slide 29

Slide 29 text

QUERIES Money Deposited Money Withdrawn Interest Received Accounts with balance > €100? Money Deposited Money Withdrawn @michieltcs

Slide 30

Slide 30 text

PROJECTION Account Opened Event Handler # of active accounts +1 Account Closed Event Handler # of active accounts -1 @michieltcs

Slide 31

Slide 31 text

PROJECTION Events Event Handler(s) Storage @michieltcs

Slide 32

Slide 32 text

CQRS

Slide 33

Slide 33 text

UI @michieltcs

Slide 34

Slide 34 text

Domain UI Command commands Aggregates @michieltcs

Slide 35

Slide 35 text

Domain UI Command Repository Event Store commands events Aggregates @michieltcs

Slide 36

Slide 36 text

Domain UI Event Bus Event Handlers Command a Database Database Event Store Aggregates @michieltcs commands events events

Slide 37

Slide 37 text

Domain UI Event Bus Event Handlers Command Repository Data Layer Database Database Event Store commands events events queries DTOs Aggregates @michieltcs

Slide 38

Slide 38 text

MULTIPLE AGGREGATES Seller Event Seller Event Seller Event Seller Event Seller Event @michieltcs Listing Event Listing Event Listing Event Listing Event Listing Event Seller Listing Seller Name Listing Date Listing Description ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...

Slide 39

Slide 39 text

PROJECTION @michieltcs 1 public class BankAccountProjections { 2 private Map activeAccounts = new HashMap<>(); 3 4 @EventHandler 5 public void onAccountOpened(AccountOpened accountOpened) { 6 activeAccounts.put(accountOpened.getAccountId(), accountOpened. 7 getAccountNumber()); 8 } 9 10 @EventHandler 11 public void onAccountClosed(AccountClosed accountClosed) { 12 activeAccounts.remove(accountClosed.getAccountId()); 13 } 14 15 public String getAccountNumberForAccountId(String accountId) { 16 return activeAccounts.get(accountId); 17 } 18 }

Slide 40

Slide 40 text

PROJECTION @michieltcs 1 @GetMapping("accounts/{accountId}") 2 public String getAccountNumber(@PathVariable("accountId") String accountId) { 3 return bankAccountProjections.getAccountNumberForAccountId(accountId); 4 }

Slide 41

Slide 41 text

PROJECTION @michieltcs 1 public class BankAccountProjections { 2 private Map activeAccounts = new HashMap<>(); 3 4 @EventHandler 5 public void onAccountOpened(AccountOpened accountOpened) { 6 activeAccounts.put(accountOpened.getAccountId(), accountOpened. 7 getAccountNumber()); 8 } 9 10 @EventHandler 11 public void onAccountClosed(AccountClosed accountClosed) { 12 activeAccounts.remove(accountClosed.getAccountId()); 13 } 14 15 public int getNumberOfActiveAccounts() { 16 return activeAccounts.size(); 17 } 18 }

Slide 42

Slide 42 text

PROJECTION @michieltcs 1 @GetMapping("accounts/active") 2 public int getNumberOfActiveAccounts() { 3 return bankAccountProjections.getNumberOfActiveAccounts(); 4 }

Slide 43

Slide 43 text

NEW PROJECTION

Slide 44

Slide 44 text

NEW STRUCTURE

Slide 45

Slide 45 text

BASED ON EXISTING EVENTS

Slide 46

Slide 46 text

REBUILDING Stop app Cleanup Loop over events Apply to projection Start app @michieltcs

Slide 47

Slide 47 text

ZERO DOWNTIME Loop over existing events Apply to new projection Use projection @michieltcs

Slide 48

Slide 48 text

ZERO DOWNTIME New events Queue Loop over existing events Apply to new projection Apply queued events Use projection @michieltcs

Slide 49

Slide 49 text

LONG RUNNING REBUILDS?

Slide 50

Slide 50 text

IN MEMORY

Slide 51

Slide 51 text

DISTRIBUTED

Slide 52

Slide 52 text

DIVIDING THE WORK Event Event Event Event Event @michieltcs Aggregate Event

Slide 53

Slide 53 text

DIVIDING THE WORK Event Event Event Event Event @michieltcs Aggregate Instance Event Instance

Slide 54

Slide 54 text

DIVIDING THE WORK Event Event Event Event Event @michieltcs Aggregate Instance Event Instance

Slide 55

Slide 55 text

DIVIDING THE WORK Event Event Event Event Event @michieltcs Aggregate Instance Event Event Event Event Instance

Slide 56

Slide 56 text

DIVIDING THE WORK Event Event Event Event Event @michieltcs Aggregate Instance Event Event Event Event Instance Event Event Event

Slide 57

Slide 57 text

DIVIDING THE WORK Seller Event Seller Event Seller Event Seller Event Seller Event @michieltcs Listing Event Listing Event Listing Event Listing Event Listing Event Seller Listing Seller Name Listing Date Listing Description ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...

Slide 58

Slide 58 text

DIVIDING THE WORK Seller Event Seller Event Seller Event Seller Event Seller Event @michieltcs Listing Event Listing Event Listing Event Listing Event Listing Event Seller Listing Instance Instance

Slide 59

Slide 59 text

DIVIDING THE WORK Seller Event Seller Event Seller Event Seller Event Seller Event @michieltcs Listing Event Listing Event Listing Event Listing Event Listing Event Seller Listing Instance Instance ?

Slide 60

Slide 60 text

BACKGROUND TASK

Slide 61

Slide 61 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 62

Slide 62 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 63

Slide 63 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 64

Slide 64 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 65

Slide 65 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 66

Slide 66 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event Token Store

Slide 67

Slide 67 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 68

Slide 68 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 69

Slide 69 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 70

Slide 70 text

TRACKING EVENTS @michieltcs Event Event Event Event Event Event . . . . . . . . . . . . . . Event Event Event

Slide 71

Slide 71 text

TRACKING EVENT PROCESSOR Get next event Apply to new projection Last event? Use projection yes no @michieltcs

Slide 72

Slide 72 text

TRACKING EVENT PROCESSOR @michieltcs 1 @Target(ElementType.TYPE) 2 @Retention(RetentionPolicy.RUNTIME) 3 public @interface RebuildableProjection { 4 String version() default ""; 5 boolean rebuild() default false; 6 }

Slide 73

Slide 73 text

TRACKING EVENT PROCESSOR @michieltcs 1 @Configuration 2 public class ProjectionsConfiguration { 3 @Autowired 4 private EventHandlingConfiguration eventHandlingConfiguration;

Slide 74

Slide 74 text

TRACKING EVENT PROCESSOR @michieltcs 1 @PostConstruct 2 public void startTrackingProjections() throws ClassNotFoundException { 3 ClassPathScanningCandidateComponentProvider scanner = 4 new ClassPathScanningCandidateComponentProvider(false); 5 scanner.addIncludeFilter(new AnnotationTypeFilter(RebuildableProjection. 6 class)); 7 8 for (BeanDefinition bd : scanner.findCandidateComponents("org.demo")) { 9 Class> aClass = Class.forName(bd.getBeanClassName()); 10 RebuildableProjection rebuildableProjection = aClass.getAnnotation( 11 RebuildableProjection. 12 class); 13 14 if (rebuildableProjection.rebuild()) { 15 registerRebuildableProjection(aClass, rebuildableProjection); 16 } 17 } 18 }

Slide 75

Slide 75 text

TRACKING EVENT PROCESSOR @michieltcs 1 private void registerRebuildableProjection(Class> aClass, 2 RebuildableProjection 3 rebuildableProjection) { 4 ProcessingGroup processingGroup = aClass.getAnnotation(ProcessingGroup. 5 class); 6 7 String name = Optional.ofNullable(processingGroup).map(ProcessingGroup:: 8 value) 9 .orElse(aClass.getName() + "/" + rebuildableProjection.version()); 10 11 eventHandlingConfiguration.assignHandlersMatching( 12 name, 13 Integer.MAX_VALUE, 14 (eventHandler) -> aClass.isAssignableFrom(eventHandler.getClass())); 15 16 eventHandlingConfiguration.registerTrackingProcessor(name); 17 }

Slide 76

Slide 76 text

TRACKING EVENT PROCESSOR @michieltcs 1 private void registerRebuildableProjection(Class> aClass, 2 RebuildableProjection 3 rebuildableProjection) { 4 ProcessingGroup processingGroup = aClass.getAnnotation(ProcessingGroup. 5 class); 6 7 String name = Optional.ofNullable(processingGroup).map(ProcessingGroup:: 8 value) 9 .orElse(aClass.getName() + "/" + rebuildableProjection.version()); 10 11 eventHandlingConfiguration.assignHandlersMatching( 12 name, 13 Integer.MAX_VALUE, 14 (eventHandler) -> aClass.isAssignableFrom(eventHandler.getClass())); 15 16 eventHandlingConfiguration.registerTrackingProcessor(name); 17 }

Slide 77

Slide 77 text

EVENT VERSIONING

Slide 78

Slide 78 text

NEW BUSINESS REQUIREMENTS

Slide 79

Slide 79 text

CHANGING VIEW ON EVENTS

Slide 80

Slide 80 text

IRRELEVANT

Slide 81

Slide 81 text

DIFFERENT FIELDS

Slide 82

Slide 82 text

WRONG NAME

Slide 83

Slide 83 text

TOO COARSE

Slide 84

Slide 84 text

TOO FINE

Slide 85

Slide 85 text

SUPPORT YOUR LEGACY?

Slide 86

Slide 86 text

Commands can be renamed 1 @michieltcs

Slide 87

Slide 87 text

Commands can be renamed 1 Events are immutable 2 @michieltcs

Slide 88

Slide 88 text

Commands can be renamed 1 Events are immutable Correct old events with new events 2 3 @michieltcs

Slide 89

Slide 89 text

@michieltcs Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00

Slide 90

Slide 90 text

@michieltcs Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00 Ledger Entry Aug 14 Inventory €16500,00 Accounts Payable €16500,00

Slide 91

Slide 91 text

@michieltcs Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00 Ledger Entry Aug 14 Inventory €16500,00 Accounts Payable €16500,00 Ledger Correction Entry Aug 14 Inventory €900,00 Accounts Payable €900,00

Slide 92

Slide 92 text

COMPENSATING ACTIONS class MoneyWithdrawn {
 String accountId;
 BigDecimal amount;
 } class WithdrawalRolledBack {
 String accountId;
 BigDecimal amount;
 } Typo: too much withdrawn!

Slide 93

Slide 93 text

COMPENSATING ACTIONS class AccountOpened {
 String accountId;
 String accountNumber;
 } class DuplicateAccountClosed {
 String accountId;
 } Duplicate account number!

Slide 94

Slide 94 text

UPCASTING

Slide 95

Slide 95 text

UPCASTING Event Store @michieltcs

Slide 96

Slide 96 text

UPCASTING @michieltcs 1 @Value 2 @Revision("1.0") 3 public class AccountOpened { 4 String accountId; 5 String accountNumber; 6 }

Slide 97

Slide 97 text

UPCASTING @michieltcs 
 80f49161
 12345678

Slide 98

Slide 98 text

UPCASTING Event Store AccountOpened (1.0) Event Handler @michieltcs

Slide 99

Slide 99 text

UPCASTING @michieltcs 1 @Value 2 @Revision("2.0") 3 public class AccountOpened { 4 String accountId; 5 String accountNumberIban; 6 }

Slide 100

Slide 100 text

UPCASTING @michieltcs 
 80f49161
 NL00ABNA012345678


Slide 101

Slide 101 text

UPCASTING Event Store AccountOpened (1.0) Upcaster AccountOpened (2.0) Event Handler @michieltcs

Slide 102

Slide 102 text

UPCASTING @michieltcs 1 @Getter 2 public class AccountOpenedUpcaster implements EventUpcaster { 3 private final SerializedType typeConsumed 4 = new SimpleSerializedType(AccountOpened.class.getTypeName(), "1.0"); 5 private final SerializedType typeProduced 6 = new SimpleSerializedType(AccountOpened.class.getTypeName(), "2.0");

Slide 103

Slide 103 text

UPCASTING @michieltcs 1 @Override 2 public final Stream upcast( 3 Stream intermediateRepresentations) { 4 return intermediateRepresentations.map(evt -> { 5 if (evt.getType().equals(getTypeConsumed())) { 6 return evt.upcastPayload(getTypeProduced(), Document.class, 7 document -> { 8 Element rootElement = document.getRootElement(); 9 Element accountNumberElement = rootElement.element( 10 "accountNumber"); 11 rootElement.remove(accountNumberElement); 12 rootElement.addElement("accountNumberIban") 13 .setText(toIban(accountNumberElement.getText())); 14 return document; 15 }); 16 } else { 17 return evt; 18 } 19 }); 20 }

Slide 104

Slide 104 text

UPCASTING @michieltcs 1 @Override 2 public final Stream upcast( 3 Stream intermediateRepresentations) { 4 return intermediateRepresentations.map(evt -> { 5 if (evt.getType().equals(getTypeConsumed())) { 6 return evt.upcastPayload(getTypeProduced(), Document.class, 7 document -> { 8 Element rootElement = document.getRootElement(); 9 Element accountNumberElement = rootElement.element( 10 "accountNumber"); 11 rootElement.remove(accountNumberElement); 12 rootElement.addElement("accountNumberIban") 13 .setText(toIban(accountNumberElement.getText())); 14 return document; 15 }); 16 } else { 17 return evt; 18 } 19 }); 20 }

Slide 105

Slide 105 text

VERSIONED EVENT STORE

Slide 106

Slide 106 text

VERSIONED EVENT STORE events_v1 [
 {
 "id": "12345678",
 "type": "AccountOpened",
 "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "1.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ] @michieltcs

Slide 107

Slide 107 text

COPY & REPLACE

Slide 108

Slide 108 text

VERSIONED EVENT STORE Loop over existing events Apply upcaster Add queued events Use new event store New events Queue @michieltcs

Slide 109

Slide 109 text

VERSIONED EVENT STORE events_v2 [
 {
 "id": "12345678",
 "type": "AccountOpened",
 "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "2.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ] @michieltcs

Slide 110

Slide 110 text

GDPR

Slide 111

Slide 111 text

"RIGHT TO ERASURE"

Slide 112

Slide 112 text

' ... shall have the right to obtain ... the erasure of personal data concerning him or her without undue delay -GDPR, Article 17

Slide 113

Slide 113 text

PERSONALLY IDENTIFIABLE INFORMATION

Slide 114

Slide 114 text

REMOVED

Slide 115

Slide 115 text

ANONYMIZED?

Slide 116

Slide 116 text

PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Remove from event store Remove from read models Notify 3rd parties

Slide 117

Slide 117 text

PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Remove from event store ? Remove from read models Notify 3rd parties

Slide 118

Slide 118 text

IMMUTABLE EVENTS?

Slide 119

Slide 119 text

MODIFY DIRECTLY

Slide 120

Slide 120 text

COPY WITH FILTER

Slide 121

Slide 121 text

STORE PII EXTERNALLY

Slide 122

Slide 122 text

STORE PII EXTERNALLY @michieltcs AccountOpened External Storage 1 @Value 2 public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 John Doe ... ... ...

Slide 123

Slide 123 text

CRYPTO ERASURE

Slide 124

Slide 124 text

ENCRYPTING EVENTS @michieltcs 
 80f49161
 NL00ABNA012345678
 Foo
 Bar ...


Slide 125

Slide 125 text

ENCRYPTING EVENTS @michieltcs 
 80f49161
 2dqjHkY8Mc8+cek4vs/9hzgkob4J3fZJNIJh2sAXlJ0=
 N5Y27vd0UbKo6FIu5c7QGQ==
 OSKrzfuuuayuUNXYS5YUug== ...


Slide 126

Slide 126 text

ENCRYPTING EVENTS Generate event Find / create encryption key Encrypt payload values Store
 event @michieltcs

Slide 127

Slide 127 text

DECRYPTING EVENTS Load
 event Find associated encryption key Decrypt payload values Process
 event @michieltcs

Slide 128

Slide 128 text

SHEDDING THE KEY Load
 event Find associated encryption key Decrypt payload values Process
 event @michieltcs X

Slide 129

Slide 129 text

AXON GDPR MODULE @michieltcs 1 @Value 2 public class AccountOpened { 3 @DataSubjectId 4 String accountId; 5 6 @PersonalData 7 String accountNumberIban; 8 9 @PersonalData 10 String firstName; 11 12 @PersonalData 13 String lastName;
 14 }

Slide 130

Slide 130 text

CLOSING WORDS

Slide 131

Slide 131 text

NO SILVER BULLET

Slide 132

Slide 132 text

CHALLENGES

Slide 133

Slide 133 text

(IM)MUTABILITY

Slide 134

Slide 134 text

AUDIT TRAIL

Slide 135

Slide 135 text

SCALABILITY

Slide 136

Slide 136 text

TESTING

Slide 137

Slide 137 text

CQRS + ES = AWESOME

Slide 138

Slide 138 text

THANK YOU! @michieltcs / [email protected]
 
 www.michielrook.nl