Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Event sourcing, GDPR Article 17, and the AxonIQ GDPR Module

AxonIQ
February 02, 2018

Event sourcing, GDPR Article 17, and the AxonIQ GDPR Module

Practical guide on how to implement Article 17 of General Data Protection Regulation (GDPR) in the systems using event sourcing pattern.

The event sourcing pattern (which entails storing state as an immutable series of past events rather than just the current state)
has become popular over the past years. How does this affect GDPR compliance?

In event sourcing, the list of past events is (at least conceptually) immutable. This also implies that data can’t be deleted from it, which is exactly what you must be able to do according to article 17’s right to erasure (also known as ‘right to be forgotten’).

How to reconcile this?

The AxonIQ GDPR module ensures that personal data within events is stored in an encrypted form. The encryption keys are automatically generated and are unique to the data subject or the aggregate identifer. The module provides an API to delete this key afterwards. Deletion of this key is equivalent to deleting the personal data fields itself, but doesn’t require any changes to the list of past events. This solves the problem of erasing data in an immutable event list.

AxonIQ

February 02, 2018
Tweet

More Decks by AxonIQ

Other Decks in Technology

Transcript

  1. Contact me: [email protected] Follow me: @Frans_vanBuul Event sourcing, GDPR Article

    17, and the AxonIQ GDPR Module Frans van Buul Evangelist, AxonIQ
  2. Contact me: [email protected] Follow me: @Frans_vanBuul Event-driven architecture “a software

    architecture pattern promoting the production, detection, consumption of, and reaction to events” https://en.wikipedia.org/wiki/Event-driven_architecture
  3. Contact me: [email protected] Follow me: @Frans_vanBuul Event sourced persistence A

    mechanism to store and retrieve data from a program’s working memory into something more permanent, like a database.
  4. Contact me: [email protected] Follow me: @Frans_vanBuul Traditional persistence Event sourced

    persistence • Store the current state directly to database • CRUD operations
  5. Contact me: [email protected] Follow me: @Frans_vanBuul Traditional persistence Event Sourcing

    Event sourced persistence • Store the current state directly to database • CRUD operations • All state change through events. • Events are distributed and persisted to an event store. • To access current state, replay events. • Events are immutable and undeletable.
  6. Contact me: [email protected] Follow me: @Frans_vanBuul Traditional persistence Example Event

    Sourcing Order customer: Frans 1 pizza 1 soda OrderCreated(customer Frans) ProductAdded(pizza) ProductAdded(icecream) ProductAdded(soda) ProductRemoved(icecream) OrderConfirmed
  7. Contact me: [email protected] Follow me: @Frans_vanBuul Business reasons Technical reasons

    Why use event sourcing? • Auditing / compliance / transparency • Data mining, analytics: value from data • Guaranteed completeness of raised events • Single source of truth • Concurrency / conflict resolution • Facilitates debugging • Replay into new read models (CQRS) • Easily capturing intent
  8. Contact me: [email protected] Follow me: @Frans_vanBuul Article 17 GDPR Right

    to erasure (‘right to be forgotten’) 1. The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay and the controller shall have the obligation to erase personal data without undue delay where one of the following grounds applies: ……
  9. Contact me: [email protected] Follow me: @Frans_vanBuul GDPR vs Event Sourcing

    • Generally, very good match because of transparency • There’s also a challenge Erasure is sometimes mandatory Events are immutable and undeletable ? GDPR Event Sourcing Let’s have a look at some ideas!
  10. Contact me: [email protected] Follow me: @Frans_vanBuul Idea Deleting/modifying events •

    Delete events anyhow Reconciling mandatory erasure and the immutable event store
  11. Contact me: [email protected] Follow me: @Frans_vanBuul Idea Problems Deleting/modifying events

    • Delete events anyhow • Modify events anyhow, e.g. by overwriting field values • Difficult/expensive to implement consistently • Expensive to execute, especially on a true append- only event store. • Diminishes value/reliability of event store Reconciling mandatory erasure and the immutable event store
  12. Contact me: [email protected] Follow me: @Frans_vanBuul Idea Re-interpret the law

    Reconciling mandatory erasure and the immutable event store • “The law only requires deletion from operational storage, so that’s read models in CQRS. An event store is like a back-up and that’s excluded from the law.” • “Data needed for legal claims is excluded from art 17 (by 17.3.c). I intend to use my events to support legal claims. Therefore my event store is excluded.”
  13. Contact me: [email protected] Follow me: @Frans_vanBuul Idea Problems Re-interpret the

    law • No. That violates the minimization principle. Reconciling mandatory erasure and the immutable event store • No. There’s simply nothing like that in the law. • “The law only requires deletion from operational storage, so that’s read models in CQRS. An event store is like a back-up and that’s excluded from the law. ” • “Data needed for legal claims is excluded from art 17 (by 17.3.c). I intend to use my events to support legal claims. Therefore my event store is excluded.”
  14. Contact me: [email protected] Follow me: @Frans_vanBuul Idea Separate contexts •

    Store non-personal data in an event sourced context. • Store personal data in an non-event sourced context. • Bring them together when needed in read-model projections. Reconciling mandatory erasure and the immutable event store
  15. Contact me: [email protected] Follow me: @Frans_vanBuul Idea Problems Separate contexts

    • Store non-personal data in an event sourced context. • Store personal data in an non-event sourced context. • Bring them together when needed in read-model projections. • Lots of complexity to keep the mapping in place. • Huge refactoring in existing applications. • Leads to unnatural domain models in many situations. • Loses all ES benefits for personal data. Reconciling mandatory erasure and the immutable event store
  16. Contact me: [email protected] Follow me: @Frans_vanBuul GDPR vs Event Sourcing

    • Generally, very good match because of transparency • There’s also a challenge Erasure is sometimes mandatory Events are immutable and undeletable ? GDPR Event Sourcing Existing ideas have major downsides
  17. Contact me: [email protected] Follow me: @Frans_vanBuul Cryptographic erasure Core idea:

    • Store data in encrypted form • To delete the data, throw away the key A.k.a. “crypto erase”, “crypto- shredding”, “digital shredding”
  18. Contact me: [email protected] Follow me: @Frans_vanBuul Not a novel idea!

    By Evan-Amos - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=27940250 As data sanitization strategy: • Built into “Self Encrypting Drives” and enterprise-grade storage arrays. • Supported by storage security standards: • ISO/IEC 27040 • NIST SP 800-88 Rev 1
  19. Contact me: [email protected] Follow me: @Frans_vanBuul GDPR Module design goals

    Implement crypto-erasure on the application level, for Java • Easily usable. No changes to business logic. Integration with Axon. • Generally usable: • With Axon 3, Axon 2 or without Axon at all. • Independent of serialization and event store choices. • Fine grained control. Decide what you want encrypted/erased. • Highest possible, long-term security.
  20. Contact me: [email protected] Follow me: @Frans_vanBuul uses as delegate API

    Overview <<class>> FieldEncrypter void encrypt(Object obj) void decrypt(Object obj) <<interface>> CryptoEngine SecretKey getOrCreateKey(String id) SecretKey getKey(String id) Cipher getCipher(SecretKey key) void deleteKey(String id) uses Implementations: • In memory • JPA-backed • JKS • PKCS#11 <<class>> FieldEncryptingSerializer <<interface>> Serializer implements uses Driven by annotations on the object’s fields Enables seamless Axon Framework integration. Three versions: axon3, axon2, core.
  21. Contact me: [email protected] Follow me: @Frans_vanBuul Common questions • Today's

    encryption may be hacked in the future – how can you guarantee security? • Is this allowed under GDPR? • How about key management? Are keys secured? • Is this open source?
  22. Contact me: [email protected] Follow me: @Frans_vanBuul Long-term security • There

    are big difference in crypto algorithms here: we use the symmetric AES algorithm, which is very different from e.g. RSA • We recommend and default to using 256-bit keysize • Best known cryptanalysis attack (Biclique) reduces this to 254.4 bits • This will withstand any classic brute-force attack (given size of keyspace). • This will also withstand the best known quantum attack (Grover's algorithm) This provides the long-term security that is the basis for current use for disk sanitization and its inclusion in NIST and ISO standards.
  23. Contact me: [email protected] Follow me: @Frans_vanBuul Is this allowed under

    GDPR? • GDPR is explicitly "technology neutral" • Is doesn't define "erasure" This gives room to different technical implementations of erasure. • The strengths and weaknesses of crypto erasure should be compared against other practices. • Regular SQL DELETE doesn't event delete data from disk Crypto erasure is at least as strong as traditional DELETE and therefore a reasonable technical implementation of "erasure".
  24. Contact me: [email protected] Follow me: @Frans_vanBuul Are keys secured? •

    The module offers various options to store your keys, including in a database (with regular database security) or a HSM. You can also define your own by implementing an interface. • But, from a risk analysis point of view, these crypto-erasure keys are much like regular data!
  25. Contact me: [email protected] Follow me: @Frans_vanBuul Without cryptoerasure With cryptoerasure

    Are keys secured? Cleartext event data record Encrypted event data record Key Get access to this, and you have the event data Get access to this, and you have the event data
  26. Contact me: [email protected] Follow me: @Frans_vanBuul Is the GDPR Module

    open source? • No, it's a commercial, closed source offering. • Flexible pricing scheme with both enterprise-wide licenses as well as single project licenses on offer. Contact us to discuss. • We are 100% transparent about what the module does. • The cryptographic scheme that we use is fully described in the documentation and may e.g. be provided to your security auditors. • The scheme is not patented and may be implemented by others. Free as in beer: No Open and transparent: Yes
  27. Contact me: [email protected] Follow me: @Frans_vanBuul Example scheme documentation SecureRandom

    pre_iv (int) big-endian enc. pre_iv (byte[]) MD5 AES/CBC/PKCS#5 cleartext byte[] encrypted byte[] MD5 digest byte[] AES key first 8 big-end dec AES/ECB/NoPad first 8 iv data key data key 1 2 big-end dec EncryptedFieldData int32 version = 1 1 fixed32 pre_iv= 2 bytes encrypted_value= 3 fixed64 digest= 4 fixed64 enc_digest= 5 bytes partial_value= 6 ReplacementValueProvider #partialValueForStorage Protobuf 3 encoding encrypted byte[]
  28. Contact me: [email protected] Follow me: @Frans_vanBuul Where does this fit?

    Part of a strategy to deal with mandatory erasure (driven by GDPR or otherwise) Easy to implement solution for the erasure vs. immutability problem Request for deletion Valid? Yes Raise specific deletion event Delete from event store Delete from read models Inform 3rd parties Keystore Read model database AxonIQ GDPR Module Request for deletion Inform data subject Deletion acknowledge- ment
  29. Contact me: [email protected] Follow me: @Frans_vanBuul Finally Please contact me

    directly in case you have any questions after this webinar [email protected] Want to learn about more about Axon Framework? • We offer training: https://axoniq.io/services/axon-training.html On September 21th, we'll have our annual conference: https://www.eventbrite.com/o/axoniq-15996042943 • Visit https://axoniq.io and subscribe to newsletter.