Forget me, please? Event sourcing and the GDPR

In May 2018, a new piece of EU legislation called the General Data Protection Regulation (GDPR) has come into effect. The GDPR attempts to regulate data protection for individuals within the EU and has very interesting and specific implications for applications that use event sourcing. In that context, the most interesting and challenging part of the regulation is without a doubt Article 17, the ‘right to erasure’: the right for individuals to request that their data be removed from the databases of a company.

In this talk, I’ll discuss my thoughts on the GDPR and give a few helpful pointers and tips!

Transcript

1. FORGET ME PLEASE?
 EVENT SOURCING & THE GDPR Michiel Rook - @michieltcs
2. DISCLAIMER: I AM NOT A LAWYER
3. GDPR
4. GENERAL DATA PROTECTION REGULATION
5. ' Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (Data Protection Directive) -General Data Protection Regulation
6. A SHORT HISTORY
7. 1995 Data Protection Directive
8. 1995 Data Protection Directive 2012 GDPR proposal
9. 1995 Data Protection Directive 2012 GDPR proposal 2016 GDPR adopted
10. 1995 Data Protection Directive 2012 GDPR proposal 2016 GDPR adopted 25 May 2018 GDPR enforceable
11. REGULATION
12. PROTECTS EU CITIZENS
13. DATA PROTECTION ACT
14. BROAD & VAGUE
15. PRIVACY BY DESIGN
16. ' The controller shall implement appropriate technical and organisational measures for ensuring that, by default, only personal data which are necessary for each specific purpose of the processing are processed. -GDPR, Article 25
17. DATA PROTECTION OFFICER
18. SUPERVISORY AUTHORITY
19. FINES
20. €20 MILLION OR 4% OF ANNUAL TURNOVER
21. YOU
22. RAISE YOUR HAND IF YOU HAVE
23. read CQRS / Event Sourcing theory RAISE YOUR HAND IF YOU HAVE
24. read CQRS / Event Sourcing theory followed a tutorial, built a hobby project RAISE YOUR HAND IF YOU HAVE
25. read CQRS / Event Sourcing theory followed a tutorial, built a hobby project used it in production RAISE YOUR HAND IF YOU HAVE
26. Axon Framework Spring Boot
27. QUICK RECAP CQRS + EVENT SOURCING
28. CQRS
29. COMMAND QUERY RESPONSIBILITY SEGREGATION
30. STORAGE SIDE
 VS.
 QUERY SIDE
31. UI @michieltcs
32. Domain UI Command commands Aggregates @michieltcs
33. Domain UI Command Repository Event Store commands events Aggregates @michieltcs
34. Domain UI Event Bus Event Handlers Command Repository Database Database Event Store commands events events Aggregates @michieltcs
35. Domain UI Event Bus Event Handlers Command Repository Data Layer Database Database Event Store commands events events queries DTOs Aggregates @michieltcs
36. EVENT SOURCING
37. ' Event Sourcing ensures that all changes to application state are stored as a sequence of events. -Martin Fowler
38. ACTIVE RECORD VS. EVENT SOURCING Account Id Account number Balance 1234 12345678 €50,00 ... ... ... Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678 @michieltcs
39. COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00 @michieltcs 1 @Value 2 public class DepositMoney { 3 @TargetAggregateIdentifier 4 String accountId; 5 BigDecimal amount; 6 }
40. COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00 command
 handler @michieltcs 1 @CommandHandler 2 public void depositMoney(DepositMoney command) { 3 apply(new MoneyDeposited( 4 command.getAccountId(), 5 command.getAmount(), 6 ZonedDateTime.now())); 7 }
41. COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00 Money Deposited Account Id 1234 Amount €100,00 command
 handler @michieltcs 1 @Value 2 public class MoneyDeposited { 3 String accountId; 4 BigDecimal amount; 5 ZonedDateTime timestamp; 6 }
42. AGGREGATES @michieltcs an Aggregate handles Commands and generates Events based on the current state
43. AGGREGATES @michieltcs 1 class BankAccount { 2 @AggregateIdentifier 3 private String accountId; 4 private String accountNumber; 5 private BigDecimal balance; 6 7 // ... 8 @EventHandler 9 public void accountOpened(AccountOpened event) { 10 this.accountId = event.getAccountId(); 11 this.accountNumber = event.getAccountNumber(); 12 this.balance = BigDecimal.valueOf(0); 13 } 14 15 @EventHandler 16 public void moneyDeposited(MoneyDeposited event) { 17 this.balance = this.balance.add(event.getAmount()); 18 } 19 }
44. AGGREGATE STATE Account number Balance 12345678 €0,00 Account number Balance 12345678 €100,00 Account number Balance 12345678 €50,00 event
 handler event
 handler event
 handler @michieltcs Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678
45. VALIDATING COMMANDS @michieltcs 1 @CommandHandler 2 public void withdrawMoney(WithdrawMoney command) throws 3 OverdraftDetectedException { 4 if (balance.compareTo(command.getAmount()) >= 0) { 5 apply(new MoneyWithdrawn( 6 command.getAccountId(), 7 command.getAmount(), 8 ZonedDateTime.now())); 9 } else { 10 throw new OverdraftDetectedException(accountNumber, balance, command. 11 getAmount()); 12 } 13 }
46. TESTING AGGREGATES @michieltcs 1 public class BankAccountTest { 2 private FixtureConfiguration<BankAccount> fixture; 3 4 @Before 5 public void createFixture() { 6 fixture = new AggregateTestFixture<>(BankAccount.class); 7 } 8 9 @Test 10 public void noOverdraftsOnEmptyAccount() { 11 fixture.given(new AccountOpened(ACCOUNT_ID, ACCOUNT_NUMBER)) 12 .when(new WithdrawMoney(ACCOUNT_ID, new BigDecimal(20))) 13 .expectException(OverdraftDetectedException.class); 14 } 15 16 private final static String ACCOUNT_ID = "accountId"; 17 private final static String ACCOUNT_NUMBER = "accountNumber"; 18 }
47. EVENT SOURCING
 & GDPR
48. CONSENT
49. ' Where processing is based on consent, the controller shall be able to demonstrate that the data subject has consented to processing of his or her personal data. -GDPR, Article 7
50. REGISTERING CONSENT
51. ' ...the request for consent shall be presented in a manner which is clearly distinguishable from the other matters... -GDPR, Article 7
52. REVOKING CONSENT
53. ' The data subject shall have the right to withdraw his or her consent at any time. ... It shall be as easy to withdraw as to give consent. -GDPR, Article 7
54. WHY USE EVENT SOURCING / CQRS?
55. CAPTURE INTENT
56. DEMONSTRATING CONSENT
57. EVENT LOG
 AS AUDIT LOG
58. NEW READ MODELS
59. EASIER DEBUGGING
60. EVENT LOG AS AUDIT LOG @michieltcs ConsentedToNewsletters
61. EVENT LOG AS AUDIT LOG @michieltcs ConsentedToNewsletters ConsentedToDataGathering
62. EVENT LOG AS AUDIT LOG @michieltcs ConsentedToNewsletters ConsentedToDataGathering RevokedConsentToNewsletters
63. "RIGHT TO ACCESS"
64. ' The data subject shall have the right to obtain from the controller confirmation as to whether or not personal data ... are being processed, and ... access to the personal data ... -GDPR, Article 15
65. "RIGHT TO ERASURE"
66. ' The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay -GDPR, Article 17
67. PERSONALLY IDENTIFIABLE INFORMATION
68. ' ‘personal data’ means any information relating to an identified or identifiable natural person; an identifiable natural person is one who can be identified, directly or indirectly -GDPR, Article 4
69. GROUNDS
70. ' .. the personal data are no longer necessary .. the data subject withdraws consent on which the processing is based -GDPR, Article 17
71. EXCEPTIONS
72. ' .. to comply with a legal obligation .. for the establishment, exercise or defence of legal claims (*) -GDPR, Article 17
73. UNDUE DELAY
74. INFORM 3RD PARTIES
75. BACKUPS?
76. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked
77. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Notify 3rd parties
78. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Remove from read models Notify 3rd parties
79. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Remove from event store Remove from read models Notify 3rd parties
80. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Remove from event store ? Remove from read models Notify 3rd parties
81. IMMUTABLE EVENTS?
82. COMPENSATING ACTIONS
83. @michieltcs Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00
84. @michieltcs Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00 Ledger Entry Aug 14 Inventory €16500,00 Accounts Payable €16500,00
85. @michieltcs Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00 Ledger Entry Aug 14 Inventory €16500,00 Accounts Payable €16500,00 Ledger Correction Entry Aug 14 Inventory €900,00 Accounts Payable €900,00
86. COMPENSATING ACTIONS class MoneyWithdrawn {
 String accountId;
 BigDecimal amount;
 } class WithdrawalRolledBack {
 String accountId;
 BigDecimal amount;
 } Typo: too much withdrawn!
87. COMPENSATING ACTIONS class AccountOpened {
 String accountId;
 String accountNumber;
 } class DuplicateAccountClosed {
 String accountId;
 } Duplicate account number!
88. GDPR?
89. STRATEGIES
90. ONLY REMOVE FROM PROJECTION?
91. LEGAL DEFENCE?
92. ' .. adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimisation’) -GDPR, Article 5
93. UPCASTING?
94. UPCASTING Event Store Event_V1 Upcaster Event_V2 Event Handler @michieltcs
95. UPCASTING Event Store Event_V1 Upcaster Event_V2 Event Handler @michieltcs Event_V2 = f(Event_V1)
96. UPCASTING Event Store Event_V1 Upcaster Event_V2 Event Handler @michieltcs Event_V2 = f(Event_V1)
97. DELETING EVENTS
98. DELETING EVENTS
99. MODIFYING EVENTS
100. MODIFYING EVENTS
101. COPY & FILTER
102. VERSIONED EVENT STORE
103. VERSIONED EVENT STORE events_v1 [
 {
 "id": "12345678",
 "type": "AccountOpened",
 "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "1.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ] @michieltcs
104. COPY & REPLACE
105. VERSIONED EVENT STORE Loop over existing events Apply upcaster Add queued events Use new event store New events Queue @michieltcs
106. VERSIONED EVENT STORE events_v2 [
 {
 "id": "12345678",
 "type": "AccountOpened",
 "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "2.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ] @michieltcs
107. STORE PII EXTERNALLY
108. STORE PII EXTERNALLY @michieltcs 1 @Value 2 public class AccountOpened { 3 String accountId; 4 String accountNumber; 5 String name; 6 }
109. STORE PII EXTERNALLY @michieltcs 1 @Value 2 public class AccountOpened { 3 String accountId; 4 }
110. STORE PII EXTERNALLY @michieltcs AccountOpened External Storage 1 @Value 2 public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 John Doe ... ... ...
111. STORE PII EXTERNALLY @michieltcs AccountOpened External Storage 1 @Value 2 public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 ANON ... ... ...
112. STORE PII EXTERNALLY @michieltcs AccountOpened External Storage 1 @Value 2 public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 ANON ... ... ...
113. CRYPTO ERASURE
114. ENCRYPT EVENTS
115. DECRYPT EVENTS
116. ENCRYPT FIELD VALUES
117. DECRYPT FIELD VALUES
118. ENCRYPTING EVENTS @michieltcs <org.demo.AccountOpened>
 <accountId>80f49161</accountId>
 <accountNumberIban>NL00ABNA012345678</accountNumberIban>
 <firstName>Foo</firstName>
 <lastName>Bar</lastName> ...
 </org.demo.AccountOpened>
119. ENCRYPTING EVENTS @michieltcs <org.demo.AccountOpened>
 <accountId>80f49161</accountId>
 <accountNumberIban>2dqjHkY8Mc8+cek4vs/9hzgkob4J3fZJNIJh2sAXlJ0=</accountNumberIban>
 <firstName>N5Y27vd0UbKo6FIu5c7QGQ==</firstName>
 <lastName>OSKrzfuuuayuUNXYS5YUug==</lastName> ...
 </org.demo.AccountOpened>
120. ENCRYPTING EVENTS Generate event Find / create encryption key Encrypt payload values Store
 event @michieltcs
121. DECRYPTING EVENTS Load
 event Find associated encryption key Decrypt payload values Process
 event @michieltcs
122. SHEDDING THE KEY Load
 event Find associated encryption key Decrypt payload values Process
 event @michieltcs X
123. AXON GDPR MODULE @michieltcs 1 @Value 2 public class AccountOpened { 3 @DataSubjectId 4 String accountId; 5 6 @PersonalData 7 String accountNumberIban; 8 9 @PersonalData 10 String firstName; 11 12 @PersonalData 13 String lastName;
 14 }
124. KEY MANAGEMENT
125. PERFORMANCE
126. RE-ENCRYPT DATA AT REST
127. CLOSING WORDS
128. GDPR
129. CHALLENGES
130. FRAMEWORK SUPPORT
131. (IM)MUTABILITY
132. AUDIT TRAIL
133. DEMONSTRATING CONSENT
134. FUTURE?
135. THANK YOU! @michieltcs / michiel@michielrook.nl
 
 www.michielrook.nl