End-to-End Encryption for Streaming Data Pipelines @ Berlin Buzzwords 2024

Slide 1

Slide 1 text

End-to-End Encryption for Streaming Data Pipelines @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin

Slide 2

Slide 2 text

! whoami ! • Hans-Peter Grahsl • Developer ! Advocate @ Red Hat • Open-Source Enthusiast • Confluent Community Catalyst • MongoDB Champion since 2020 @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 2

Slide 3

Slide 3 text

Core Kafka Security Mechanisms @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 3

Slide 4

Slide 4 text

over-the-wire encryption @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 4

Slide 5

Slide 5 text

authentication @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 5

Slide 6

Slide 6 text

authorization @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 6

Slide 7

Slide 7 text

table stakes @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 7

Slide 8

Slide 8 text

core security ! necessary ! @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 8

Slide 9

Slide 9 text

core security ? sufficient ? @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 9

Slide 10

Slide 10 text

@hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 10

Slide 11

Slide 11 text

@hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 11

Slide 12

Slide 12 text

@hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 12

Slide 13

Slide 13 text

? data in use by BROKERS @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 13

Slide 14

Slide 14 text

BROKERS see everything ... and so does any legitimate Kafka client @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 14

Slide 15

Slide 15 text

BROKERS see everything ... and so does any legitimate Kafka client @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 15

Slide 16

Slide 16 text

@hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 16

Slide 17

Slide 17 text

real end-to-end encryption @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 17

Slide 18

Slide 18 text

Open-Source Community Project Kryptonite for Kafka @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 18

Slide 19

Slide 19 text

client-side field level cryptography @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 19

Slide 20

Slide 20 text

Client-Side Cryptography @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 20

Slide 21

Slide 21 text

Field Level Cryptography @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 21

Slide 22

Slide 22 text

Field Level Cryptography @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 22

Slide 23

Slide 23 text

Kafka Connect Sources Single Message Transform @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 23

Slide 24

Slide 24 text

CSFLC with Source Connectors @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 24

Slide 25

Slide 25 text

Demo Part 1 @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 25

Slide 26

Slide 26 text

@hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 26

Slide 27

Slide 27 text

Flink SQL User-Defined Function @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 27

Slide 28

Slide 28 text

CSFLC with Streaming SQL @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 28

Slide 29

Slide 29 text

CSFLC with Streaming SQL @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 29

Slide 30

Slide 30 text

Demo Part 2 @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 30

Slide 31

Slide 31 text

experimental integration @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 31

Slide 32

Slide 32 text

@hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 32

Slide 33

Slide 33 text

Kafka Connect Sinks Single Message Transform @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 33

Slide 34

Slide 34 text

CSFLC with Sink Connectors @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 34