Securing
Kafka Connect Pipelines
with Client-Side
field level Cryptography
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
Slide 2
Slide 2 text
Why should we care?
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
2
Slide 3
Slide 3 text
61%
of breaches involved
credential data1
1 Verzion DBIR 2021 - https://www.verizon.com/dbir
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
3
Slide 4
Slide 4 text
85%
of breaches involved
the human element1
1 Verzion DBIR 2021 - https://www.verizon.com/dbir
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
4
Slide 5
Slide 5 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
5
Slide 6
Slide 6 text
compromised external
cloud assets
more common than
on-premises assets1
1 Verzion DBIR 2021 - https://www.verizon.com/dbir
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
6
Slide 7
Slide 7 text
Don't
forget about the price tag
of data breaches.
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
7
Slide 8
Slide 8 text
Don't
forget about the price tag
of data breaches.
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
8
Slide 9
Slide 9 text
$4.24M
average cost of data
breach2
2 IBM Cost of Data Breach Report - https://www.ibm.com/security/data-breach
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
9
Slide 10
Slide 10 text
$180
per record cost of
customer pii2
2 IBM Cost of Data Breach Report - https://www.ibm.com/security/data-breach
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
10
Slide 11
Slide 11 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
11
Slide 12
Slide 12 text
!
But Kafka related? Yes!
3
3 https://spectralops.io/blog/misconfigured-kafdrop-puts-companies-apache-kafka-completely-exposed/
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
12
Slide 13
Slide 13 text
!
They found it "all" ...
3
3 https://spectralops.io/blog/misconfigured-kafdrop-puts-companies-apache-kafka-completely-exposed/
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
13
Slide 14
Slide 14 text
unhappy
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
14
Slide 15
Slide 15 text
Core Kafka
Security Mechanisms
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
15
Slide 16
Slide 16 text
Table Stakes ?
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
16
Slide 17
Slide 17 text
over-the-wire encryption
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
17
Slide 18
Slide 18 text
authentication
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
18
Slide 19
Slide 19 text
authorization
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
19
Slide 20
Slide 20 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
20
Slide 21
Slide 21 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
21
Slide 22
Slide 22 text
disturbing
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
22
Slide 23
Slide 23 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
23
Slide 24
Slide 24 text
Core Security
Necessary !
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
24
Slide 25
Slide 25 text
Core Security
Sufficient ?
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
25
Slide 26
Slide 26 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
26
Slide 27
Slide 27 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
27
Slide 28
Slide 28 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
28
Slide 29
Slide 29 text
?
in use by brokers
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
29
Slide 30
Slide 30 text
brokers
see everything ...
and so does
any legitimate
Kafka client
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
30
Slide 31
Slide 31 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
31
Slide 32
Slide 32 text
human promise
is NOT
technical promise
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
32
Slide 33
Slide 33 text
end-to-end
encryption
? ? ?
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
33
Slide 34
Slide 34 text
Community Project
Kryptonite for Kafka
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
34
Slide 35
Slide 35 text
client-side
field level
cryptography
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
35
Slide 36
Slide 36 text
Client-Side Cryptography
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
36
Slide 37
Slide 37 text
Client-Side Cryptography
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
37
Slide 38
Slide 38 text
Field Level Encryption
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
38
Slide 39
Slide 39 text
Field Level Encryption
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
39
Slide 40
Slide 40 text
Field Level Decryption
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
40
Slide 41
Slide 41 text
Field Level Decryption
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
41
Slide 42
Slide 42 text
Kafka Connect
Single Message
Transform
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
42
Slide 43
Slide 43 text
CSFLC with Source Connectors
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
43
Slide 44
Slide 44 text
CSFLC with Source Connectors
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
44
Slide 45
Slide 45 text
CSFLC with Source Connectors
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
45
Slide 46
Slide 46 text
CSFLC with Source Connectors
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
46
Slide 47
Slide 47 text
CSFLC with Sink Connectors
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
47
Slide 48
Slide 48 text
CSFLC with Sink Connectors
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
48
Slide 49
Slide 49 text
CSFLC with Sink Connectors
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
49
Slide 50
Slide 50 text
CSFLC with Sink Connectors
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
50
Slide 51
Slide 51 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
51
Slide 52
Slide 52 text
Demo Scenario 1
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
52
Slide 53
Slide 53 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
53
Slide 54
Slide 54 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
54
Slide 55
Slide 55 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
55
Slide 56
Slide 56 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
56
Slide 57
Slide 57 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
57
Slide 58
Slide 58 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
58
Slide 59
Slide 59 text
Demo Scenario 2
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
59
Slide 60
Slide 60 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
60
Slide 61
Slide 61 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
61
Slide 62
Slide 62 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
62
Slide 63
Slide 63 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
63
Slide 64
Slide 64 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
64
Slide 65
Slide 65 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
65
Slide 66
Slide 66 text
Behind the Curtain ?
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
66
Slide 67
Slide 67 text
Cryptography
• Tink by Google
• AEAD based on AES GCM
• DAEAD based on AES SIV
• key rotation support
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
67
Slide 68
Slide 68 text
Keyset
Management
• within SMT config (not recommended)
• externalized to separate file (okayish)
• remote / cloud KMS (recommended)
• currently Azure Key Vault
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
68
Slide 69
Slide 69 text
!
Little Ideas
!
• wildcard / regex matching for field names
• dynamic keyset selection based on payload
• additional KMS providers (GCP, AWS, ...)
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
69
Slide 70
Slide 70 text
!
Bigger Ideas
!
• add further cryptography options (e.g. FPE)
• language / runtime agnostic data serialization
• extend scope beyond Kafka Connect
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
70
Slide 71
Slide 71 text
data should continue
to be a valuable
asset not become
a costly liability
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
71
Slide 72
Slide 72 text
twitter
@hpgrahsl
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
72
Slide 73
Slide 73 text
Wanna try this?
• Project Code
https://bit.ly/ks22-ldn-k4k
• Demo Scenarios
https://bit.ly/ks22-ldn-demo
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
73
Slide 74
Slide 74 text
@hpgrahsl | #KafkaSummit April 25-26, 2022 | London
Slide 75
Slide 75 text
Photo Credits
in order of appearance
(c) Parsoa Khorsand - https://unsplash.com/photos/Dd6n63H9szw
(c) Wolf Zimmermann - https://unsplash.com/photos/6sf5rf8QYFE
(c) Jason Leung - https://unsplash.com/photos/SAYzxuS1O3M
(c) Dev Asangbam - https://unsplash.com/photos/sh9vkVbVgo
(c) Keenan Constance - https://unsplash.com/photos/VTLcvV6UVaI
(c) Steve Johnson - https://unsplash.com/photos/hokONTrHIAQ
(c) Pete Linforth - https://pixabay.com/illustrations/biometrics-access-identification-4503187/
(c) Miguel Á. Padriñán - https://www.pexels.com/photo/close-up-shot-of-keys-on-a-red-surface-2882687/
(c) Camila Quintero Franco - https://unsplash.com/photos/mC852jACK1g
(c) Gerd Altmann - https://pixabay.com/illustrations/board-excuse-me-excuse-discharge-1848736/
(c) Vijaya narasimha - https://pixabay.com/photos/crevasse-sand-stone-hills-rock-399957/
(c) Gerd Altmann - https://pixabay.com/photos/trust-man-hood-map-prompt-4321822/
(c) Matheo JBT - https://unsplash.com/photos/HLhvZ9HRAwo
(c) Rob Laughter - https://unsplash.com/photos/WW1jsInXgwM
(c) Markus Spiske - https://unsplash.com/photos/iar-afB0QQw
(c) Nerene Grobler - https://unsplash.com/photos/sLxcfdsqLQ
(c) Wilhelm Gunkel - https://unsplash.com/photos/L04Kczg_Jvs
(c) Matt Walsh - https://unsplash.com/photos/tVkdGtEe2C4