Forget me, please? Event sourcing and the GDPR

Forget me, please? Event sourcing and the GDPR

In May 2018, a new piece of EU legislation called the General Data Protection Regulation (GDPR) has come into effect. The GDPR attempts to regulate data protection for individuals within the EU and has very interesting and specific implications for applications that use event sourcing. In that context, the most interesting and challenging part of the regulation is without a doubt Article 17, the ‘right to erasure’: the right for individuals to request that their data be removed from the databases of a company.

In this talk, I’ll discuss my thoughts on the GDPR and give a few helpful pointers and tips!

2f4800411154a8c66dde489448a044d2?s=128

Michiel Rook

October 10, 2018
Tweet

Transcript

  1. FORGET ME PLEASE?
 EVENT SOURCING & THE GDPR Michiel Rook

    - @michieltcs
  2. DISCLAIMER: I AM NOT A LAWYER

  3. GDPR

  4. GENERAL DATA PROTECTION REGULATION

  5. ' Regulation (EU) 2016/679 of the European Parliament and of

    the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (Data Protection Directive) -General Data Protection Regulation
  6. A SHORT HISTORY

  7. 1995 Data Protection Directive

  8. 1995 Data Protection Directive 2012 GDPR proposal

  9. 1995 Data Protection Directive 2012 GDPR proposal 2016 GDPR adopted

  10. 1995 Data Protection Directive 2012 GDPR proposal 2016 GDPR adopted

    25 May 2018 GDPR enforceable
  11. REGULATION

  12. PROTECTS EU CITIZENS

  13. DATA PROTECTION ACT

  14. BROAD & VAGUE

  15. PRIVACY BY DESIGN

  16. ' The controller shall implement appropriate technical and organisational measures

    for ensuring that, by default, only personal data which are necessary for each specific purpose of the processing are processed. -GDPR, Article 25
  17. DATA PROTECTION OFFICER

  18. SUPERVISORY AUTHORITY

  19. FINES

  20. €20 MILLION OR 4% OF ANNUAL TURNOVER

  21. YOU

  22. RAISE YOUR HAND IF YOU HAVE

  23. read CQRS / Event Sourcing theory RAISE YOUR HAND IF

    YOU HAVE
  24. read CQRS / Event Sourcing theory followed a tutorial, built

    a hobby project RAISE YOUR HAND IF YOU HAVE
  25. read CQRS / Event Sourcing theory followed a tutorial, built

    a hobby project used it in production RAISE YOUR HAND IF YOU HAVE
  26. Axon Framework Spring Boot

  27. QUICK RECAP CQRS + EVENT SOURCING

  28. CQRS

  29. COMMAND QUERY RESPONSIBILITY SEGREGATION

  30. STORAGE SIDE
 VS.
 QUERY SIDE

  31. UI @michieltcs

  32. Domain UI Command commands Aggregates @michieltcs

  33. Domain UI Command Repository Event Store commands events Aggregates @michieltcs

  34. Domain UI Event Bus Event Handlers Command Repository Database Database

    Event Store commands events events Aggregates @michieltcs
  35. Domain UI Event Bus Event Handlers Command Repository Data Layer

    Database Database Event Store commands events events queries DTOs Aggregates @michieltcs
  36. EVENT SOURCING

  37. ' Event Sourcing ensures that all changes to application state

    are stored as a sequence of events. -Martin Fowler
  38. ACTIVE RECORD VS. EVENT SOURCING Account Id Account number Balance

    1234 12345678 €50,00 ... ... ... Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678 @michieltcs
  39. COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00

    @michieltcs 1 @Value 2 public class DepositMoney { 3 @TargetAggregateIdentifier 4 String accountId; 5 BigDecimal amount; 6 }
  40. COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00

    command
 handler @michieltcs 1 @CommandHandler 2 public void depositMoney(DepositMoney command) { 3 apply(new MoneyDeposited( 4 command.getAccountId(), 5 command.getAmount(), 6 ZonedDateTime.now())); 7 }
  41. COMMANDS TO EVENTS Deposit Money Account Id 1234 Amount €100,00

    Money Deposited Account Id 1234 Amount €100,00 command
 handler @michieltcs 1 @Value 2 public class MoneyDeposited { 3 String accountId; 4 BigDecimal amount; 5 ZonedDateTime timestamp; 6 }
  42. AGGREGATES @michieltcs an Aggregate handles Commands and generates Events based

    on the current state
  43. AGGREGATES @michieltcs 1 class BankAccount { 2 @AggregateIdentifier 3 private

    String accountId; 4 private String accountNumber; 5 private BigDecimal balance; 6 7 // ... 8 @EventHandler 9 public void accountOpened(AccountOpened event) { 10 this.accountId = event.getAccountId(); 11 this.accountNumber = event.getAccountNumber(); 12 this.balance = BigDecimal.valueOf(0); 13 } 14 15 @EventHandler 16 public void moneyDeposited(MoneyDeposited event) { 17 this.balance = this.balance.add(event.getAmount()); 18 } 19 }
  44. AGGREGATE STATE Account number Balance 12345678 €0,00 Account number Balance

    12345678 €100,00 Account number Balance 12345678 €50,00 event
 handler event
 handler event
 handler @michieltcs Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678
  45. VALIDATING COMMANDS @michieltcs 1 @CommandHandler 2 public void withdrawMoney(WithdrawMoney command)

    throws 3 OverdraftDetectedException { 4 if (balance.compareTo(command.getAmount()) >= 0) { 5 apply(new MoneyWithdrawn( 6 command.getAccountId(), 7 command.getAmount(), 8 ZonedDateTime.now())); 9 } else { 10 throw new OverdraftDetectedException(accountNumber, balance, command. 11 getAmount()); 12 } 13 }
  46. TESTING AGGREGATES @michieltcs 1 public class BankAccountTest { 2 private

    FixtureConfiguration<BankAccount> fixture; 3 4 @Before 5 public void createFixture() { 6 fixture = new AggregateTestFixture<>(BankAccount.class); 7 } 8 9 @Test 10 public void noOverdraftsOnEmptyAccount() { 11 fixture.given(new AccountOpened(ACCOUNT_ID, ACCOUNT_NUMBER)) 12 .when(new WithdrawMoney(ACCOUNT_ID, new BigDecimal(20))) 13 .expectException(OverdraftDetectedException.class); 14 } 15 16 private final static String ACCOUNT_ID = "accountId"; 17 private final static String ACCOUNT_NUMBER = "accountNumber"; 18 }
  47. EVENT SOURCING
 & GDPR

  48. CONSENT

  49. ' Where processing is based on consent, the controller shall

    be able to demonstrate that the data subject has consented to processing of his or her personal data. -GDPR, Article 7
  50. REGISTERING CONSENT

  51. ' ...the request for consent shall be presented in a

    manner which is clearly distinguishable from the other matters... -GDPR, Article 7
  52. REVOKING CONSENT

  53. ' The data subject shall have the right to withdraw

    his or her consent at any time. ... It shall be as easy to withdraw as to give consent. -GDPR, Article 7
  54. WHY USE EVENT SOURCING / CQRS?

  55. CAPTURE INTENT

  56. DEMONSTRATING CONSENT

  57. EVENT LOG
 AS AUDIT LOG

  58. NEW READ MODELS

  59. EASIER DEBUGGING

  60. EVENT LOG AS AUDIT LOG @michieltcs ConsentedToNewsletters

  61. EVENT LOG AS AUDIT LOG @michieltcs ConsentedToNewsletters ConsentedToDataGathering

  62. EVENT LOG AS AUDIT LOG @michieltcs ConsentedToNewsletters ConsentedToDataGathering RevokedConsentToNewsletters

  63. "RIGHT TO ACCESS"

  64. ' The data subject shall have the right to obtain

    from the controller confirmation as to whether or not personal data ... are being processed, and ... access to the personal data ... -GDPR, Article 15
  65. "RIGHT TO ERASURE"

  66. ' The data subject shall have the right to obtain

    from the controller the erasure of personal data concerning him or her without undue delay -GDPR, Article 17
  67. PERSONALLY IDENTIFIABLE INFORMATION

  68. ' ‘personal data’ means any information relating to an identified

    or identifiable natural person; an identifiable natural person is one who can be identified, directly or indirectly -GDPR, Article 4
  69. GROUNDS

  70. ' .. the personal data are no longer necessary ..

    the data subject withdraws consent on which the processing is based -GDPR, Article 17
  71. EXCEPTIONS

  72. ' .. to comply with a legal obligation .. for

    the establishment, exercise or defence of legal claims (*) -GDPR, Article 17
  73. UNDUE DELAY

  74. INFORM 3RD PARTIES

  75. BACKUPS?

  76. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked

  77. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Notify 3rd parties

  78. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Remove from read

    models Notify 3rd parties
  79. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Remove from event

    store Remove from read models Notify 3rd parties
  80. PROCESSING GDPR ART. 17 REQUESTS @michieltcs RightToErasureInvoked Remove from event

    store ? Remove from read models Notify 3rd parties
  81. IMMUTABLE EVENTS?

  82. COMPENSATING ACTIONS

  83. @michieltcs Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00

  84. @michieltcs Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00

    Ledger Entry Aug 14 Inventory €16500,00 Accounts Payable €16500,00
  85. @michieltcs Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00

    Ledger Entry Aug 14 Inventory €16500,00 Accounts Payable €16500,00 Ledger Correction Entry Aug 14 Inventory €900,00 Accounts Payable €900,00
  86. COMPENSATING ACTIONS class MoneyWithdrawn {
 String accountId;
 BigDecimal amount;
 }

    class WithdrawalRolledBack {
 String accountId;
 BigDecimal amount;
 } Typo: too much withdrawn!
  87. COMPENSATING ACTIONS class AccountOpened {
 String accountId;
 String accountNumber;
 }

    class DuplicateAccountClosed {
 String accountId;
 } Duplicate account number!
  88. GDPR?

  89. STRATEGIES

  90. ONLY REMOVE FROM PROJECTION?

  91. LEGAL DEFENCE?

  92. ' .. adequate, relevant and limited to what is necessary

    in relation to the purposes for which they are processed (‘data minimisation’) -GDPR, Article 5
  93. UPCASTING?

  94. UPCASTING Event Store Event_V1 Upcaster Event_V2 Event Handler @michieltcs

  95. UPCASTING Event Store Event_V1 Upcaster Event_V2 Event Handler @michieltcs Event_V2

    = f(Event_V1)
  96. UPCASTING Event Store Event_V1 Upcaster Event_V2 Event Handler @michieltcs Event_V2

    = f(Event_V1)
  97. DELETING EVENTS

  98. DELETING EVENTS

  99. MODIFYING EVENTS

  100. MODIFYING EVENTS

  101. COPY & FILTER

  102. VERSIONED EVENT STORE

  103. VERSIONED EVENT STORE events_v1 [
 {
 "id": "12345678",
 "type": "AccountOpened",


    "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "1.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ] @michieltcs
  104. COPY & REPLACE

  105. VERSIONED EVENT STORE Loop over existing events Apply upcaster Add

    queued events Use new event store New events Queue @michieltcs
  106. VERSIONED EVENT STORE events_v2 [
 {
 "id": "12345678",
 "type": "AccountOpened",


    "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "2.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ] @michieltcs
  107. STORE PII EXTERNALLY

  108. STORE PII EXTERNALLY @michieltcs 1 @Value 2 public class AccountOpened

    { 3 String accountId; 4 String accountNumber; 5 String name; 6 }
  109. STORE PII EXTERNALLY @michieltcs 1 @Value 2 public class AccountOpened

    { 3 String accountId; 4 }
  110. STORE PII EXTERNALLY @michieltcs AccountOpened External Storage 1 @Value 2

    public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 John Doe ... ... ...
  111. STORE PII EXTERNALLY @michieltcs AccountOpened External Storage 1 @Value 2

    public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 ANON ... ... ...
  112. STORE PII EXTERNALLY @michieltcs AccountOpened External Storage 1 @Value 2

    public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 ANON ... ... ...
  113. CRYPTO ERASURE

  114. ENCRYPT EVENTS

  115. DECRYPT EVENTS

  116. ENCRYPT FIELD VALUES

  117. DECRYPT FIELD VALUES

  118. ENCRYPTING EVENTS @michieltcs <org.demo.AccountOpened>
 <accountId>80f49161</accountId>
 <accountNumberIban>NL00ABNA012345678</accountNumberIban>
 <firstName>Foo</firstName>
 <lastName>Bar</lastName> ...
 </org.demo.AccountOpened>

  119. ENCRYPTING EVENTS @michieltcs <org.demo.AccountOpened>
 <accountId>80f49161</accountId>
 <accountNumberIban>2dqjHkY8Mc8+cek4vs/9hzgkob4J3fZJNIJh2sAXlJ0=</accountNumberIban>
 <firstName>N5Y27vd0UbKo6FIu5c7QGQ==</firstName>
 <lastName>OSKrzfuuuayuUNXYS5YUug==</lastName> ...
 </org.demo.AccountOpened>

  120. ENCRYPTING EVENTS Generate event Find / create encryption key Encrypt

    payload values Store
 event @michieltcs
  121. DECRYPTING EVENTS Load
 event Find associated encryption key Decrypt payload

    values Process
 event @michieltcs
  122. SHEDDING THE KEY Load
 event Find associated encryption key Decrypt

    payload values Process
 event @michieltcs X
  123. AXON GDPR MODULE @michieltcs 1 @Value 2 public class AccountOpened

    { 3 @DataSubjectId 4 String accountId; 5 6 @PersonalData 7 String accountNumberIban; 8 9 @PersonalData 10 String firstName; 11 12 @PersonalData 13 String lastName;
 14 }
  124. KEY MANAGEMENT

  125. PERFORMANCE

  126. RE-ENCRYPT DATA AT REST

  127. CLOSING WORDS

  128. GDPR

  129. CHALLENGES

  130. FRAMEWORK SUPPORT

  131. (IM)MUTABILITY

  132. AUDIT TRAIL

  133. DEMONSTRATING CONSENT

  134. FUTURE?

  135. THANK YOU! @michieltcs / michiel@michielrook.nl
 
 www.michielrook.nl