A presentation by @stuherbert
for @GanbaroDigital
Event-Sourcing & GDPR
When Immutability Meets Reality
Slide 2
Slide 2 text
Industry veteran: architect, engineer,
leader, manager, mentor
F/OSS contributor since 1994
Talking and writing about PHP
since 2004
Chief Software Archaeologist
Building Quality @GanbaroDigital
About Stuart
Slide 3
Slide 3 text
Follow me
I do tweet a lot about
non-tech stuff though :)
@stuherbert
Slide 4
Slide 4 text
@GanbaroDigital
??
??
Do you currently
use event-sourcing?
Slide 5
Slide 5 text
@GanbaroDigital
??
??
Are you planning
on adopting event-sourcing?
Slide 6
Slide 6 text
@GanbaroDigital
??
??
Do you currently work
in a regulated industry?
Slide 7
Slide 7 text
@GanbaroDigital
In This Talk
1. Event Sourcing
2. GDPR
3. How GDPR Impacts Event Sourcing
4. Summary
Slide 8
Slide 8 text
@GanbaroDigital
Please ask questions
as we go!
Slide 9
Slide 9 text
@GanbaroDigital
Event Sourcing
Slide 10
Slide 10 text
@GanbaroDigital
What Is It?
Slide 11
Slide 11 text
@GanbaroDigital
Event Sourcing
is a data architecture.
Slide 12
Slide 12 text
@GanbaroDigital
All state changes
are represented as events.
Slide 13
Slide 13 text
@GanbaroDigital
“
An event
is something
that has happened.
Slide 14
Slide 14 text
@GanbaroDigital
Some Example Events
• User added item to basket
• User completed basket checkout
• User paid for order
• Order shipped
Slide 15
Slide 15 text
@GanbaroDigital
Slide 16
Slide 16 text
@GanbaroDigital
UI
Slide 17
Slide 17 text
@GanbaroDigital
UI
API
Slide 18
Slide 18 text
@GanbaroDigital
Business
Model
&
Data
Model
UI
API
Slide 19
Slide 19 text
@GanbaroDigital
So far,
that looks like
traditional software systems.
Slide 20
Slide 20 text
@GanbaroDigital
In a traditional software system,
the database holds
the current state.
Slide 21
Slide 21 text
@GanbaroDigital
Current state
is the result
of all the operations
that have already happened.
Slide 22
Slide 22 text
@GanbaroDigital
The database
stores the result
of what has happened.
It doesn't store
what has happened.
Slide 23
Slide 23 text
@GanbaroDigital
Business
Model
&
Data
Model
UI
API
Slide 24
Slide 24 text
@GanbaroDigital
Business
Model
&
Data
Model
UI
API
Database
Slide 25
Slide 25 text
@GanbaroDigital
Event Source systems
store events
in the database ...
... not the current state
(and not the operations either).
Slide 26
Slide 26 text
@GanbaroDigital
Business
Model
&
Data
Model
UI
API
Slide 27
Slide 27 text
@GanbaroDigital
Business
Model
&
Data
Model
UI
API
Event Store
Slide 28
Slide 28 text
@GanbaroDigital
Current state
isn't stored in the Event Store.
It has to be built.
Slide 29
Slide 29 text
@GanbaroDigital
Current state is built
by playback of
the stored events.
Slide 30
Slide 30 text
@GanbaroDigital
Business
Model
&
Data
Model
UI
API
Event Store
Slide 31
Slide 31 text
@GanbaroDigital
Event
Validation
UI
Event Store
API
Event
Playback
Slide 32
Slide 32 text
@GanbaroDigital
“ Event-Sourcing
guarantees
that you can build
any state at any time
through event playback.
Slide 33
Slide 33 text
@GanbaroDigital
We're going
to put that guarantee
under a microscope
later in this talk.
Slide 34
Slide 34 text
@GanbaroDigital
Events are stored in,
and played back from,
the Event Store.
Slide 35
Slide 35 text
@GanbaroDigital
Event
Validation
UI
Event Store
API
Event
Playback
Slide 36
Slide 36 text
@GanbaroDigital
An Event Store
is, ultimately, a database.
It may be
a general purpose RDBMS,
a NoSQL datastore,
or a specialist ESDB.
Slide 37
Slide 37 text
@GanbaroDigital
The Event Store
is subject to
the same performance constraints
that govern all databases.
Slide 38
Slide 38 text
@GanbaroDigital
Performance Constraints
• IOPS in production
• Network bandwidth & latency
• Concurrency
• Maintenance operations
Slide 39
Slide 39 text
@GanbaroDigital
Performance Constraints
• IOPS in production
• Network bandwidth & latency
• Concurrency
• Maintenance operations
Slide 40
Slide 40 text
@GanbaroDigital
Performance Constraints
• IOPS in production
• Network bandwidth & latency
• Concurrency
• Maintenance operations
Slide 41
Slide 41 text
@GanbaroDigital
Performance Constraints
• IOPS in production
• Network bandwidth & latency
• Concurrency
• Maintenance operations
Slide 42
Slide 42 text
@GanbaroDigital
One way to minimise
these performance constraints
is to use
an append-only / log datastore.
Slide 43
Slide 43 text
@GanbaroDigital
Append-only / log datastores
can be
immutable.
Slide 44
Slide 44 text
@GanbaroDigital
Event playback
is too slow, too expensive
to use all the time.
@GanbaroDigital
Breaking Down GDPR
• Obligations on Organisations
• Rights of Individuals
Slide 98
Slide 98 text
@GanbaroDigital
Breaking Down GDPR
• Obligations on Organisations
• Rights of Individuals
Slide 99
Slide 99 text
@GanbaroDigital
Obligations on
Organisations
Slide 100
Slide 100 text
@GanbaroDigital
It is illegal
to hold personal data
without a lawful basis.
Slide 101
Slide 101 text
@GanbaroDigital
Identify the lawful basis
for each piece of personal data.
Slide 102
Slide 102 text
@GanbaroDigital
Maintain records
of personal data.
Slide 103
Slide 103 text
@GanbaroDigital
Maintain records
of processing activities.
Slide 104
Slide 104 text
@GanbaroDigital
Use personal data
in a way that is fair.
Slide 105
Slide 105 text
@GanbaroDigital
Consent
is one lawful basis
for storing personal data.
Slide 106
Slide 106 text
@GanbaroDigital
Use personal data
only for what you have
explicit consent for.
Slide 107
Slide 107 text
@GanbaroDigital
Obtain new consent
if you want to use personal data
for new purposes.
Slide 108
Slide 108 text
@GanbaroDigital
Only collect personal data
that you need
for the processing
you have consent for.
Slide 109
Slide 109 text
@GanbaroDigital
Correct personal data
that is factually inaccurate
or misleading.
Or delete it.
Slide 110
Slide 110 text
@GanbaroDigital
You must not keep
personal data
any longer than required.
Slide 111
Slide 111 text
@GanbaroDigital
Delete all personal data
that you no longer need.
Slide 112
Slide 112 text
@GanbaroDigital
The personal data
must be erased from
backups and archives too.
Slide 113
Slide 113 text
@GanbaroDigital
Inform all third-parties
that you have deleted personal data
that you have passed to them.
And tell the individual
about those third-parties.
Slide 114
Slide 114 text
@GanbaroDigital
Take appropriate security measures
to protect personal data.
Slide 115
Slide 115 text
@GanbaroDigital
Have evidence
to demonstrate
your compliance with GDPR.
Slide 116
Slide 116 text
@GanbaroDigital
Rights of
Individuals
Slide 117
Slide 117 text
@GanbaroDigital
• Right to be informed
• Right of access
• Right to rectification
• Right to erasure
• Right to restrict
processing
• Right to data
portability
• Right to object
• Rights related to
automated
processing
Individual Rights
Slide 118
Slide 118 text
@GanbaroDigital
Provide individuals
with privacy information
at the point of collection.
Slide 119
Slide 119 text
@GanbaroDigital
If you obtain personal data
from third-party sources,
you must* provide individuals
with your privacy information
within 1 month.
Slide 120
Slide 120 text
@GanbaroDigital
Provide subject access
to personal data
within 1 month of a request.
Slide 121
Slide 121 text
@GanbaroDigital
Make sure a subject access request
does not disclose personal data
about anyone else.
Slide 122
Slide 122 text
@GanbaroDigital
Correct factually inaccurate
personal data
within 1 month
of a rectification request.
Slide 123
Slide 123 text
@GanbaroDigital
Erase all personal data
you can no longer hold
within 1 month
of an erasure request.
Slide 124
Slide 124 text
@GanbaroDigital
The 'right to be forgotten'
has stronger obligations
if the personal data
is about children.
Slide 125
Slide 125 text
@GanbaroDigital
Do not use personal data
that is subject to
a processing restriction request.
But you can still store it.
Slide 126
Slide 126 text
@GanbaroDigital
Provide personal data
in commonly-used
machine-readable formats*.
Slide 127
Slide 127 text
@GanbaroDigital
*but only when
lawful basis is consent or by contract,
and only when
personal data is processed
by automated means.
Slide 128
Slide 128 text
@GanbaroDigital
We'll look at the
Right to Object
in a moment.
Slide 129
Slide 129 text
@GanbaroDigital
Provide individuals
with information about
solely-automated decision making.
Slide 130
Slide 130 text
@GanbaroDigital
Provide individuals
with the means
to request human intervention.
Slide 131
Slide 131 text
@GanbaroDigital
Provide individuals
with the means to challenge
solely-automated decisions.
Slide 132
Slide 132 text
@GanbaroDigital
Perform regular checks
to ensure solely-automated decisions
are working as intended.
Slide 133
Slide 133 text
@GanbaroDigital
Exemptions
Slide 134
Slide 134 text
@GanbaroDigital
Individuals have the right to object
about the data held
and how it is being used.