Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Client-Side Field-Level Encryption for Apache Kafka Connect @ VoxxedDays Luxembourg 2022

Client-Side Field-Level Encryption for Apache Kafka Connect @ VoxxedDays Luxembourg 2022

Abstract:
Apache Kafka offers several security features ranging from authentication and authorisation mechanisms to over-the-wire encryption. This notwithstanding, data encryption performed at the client-side, which leads to explicit data-at-rest protection in topics at the broker's side, can still be considered a blind spot.

After highlighting the main benefits for data-at-rest protection, this session discusses in-depth how to selectively encrypt and decrypt sensitive payload fields in the context of Apache Kafka Connect pipelines. In particular, Kryptonite is introduced which is an ecosystem community project written and open-sourced by the speaker in 2021.

During this talk, you will learn how to benefit from a configurable single message transformation that lets you perform encryption and decryption operations in Kafka Connect worker nodes without any custom code. Client-side cryptography makes your Kafka-based data integration scenarios more secure by safeguarding the most sensitive and precious data against any form of uncontrolled or illegal access once it hits the Apache Kafka brokers.

Recording:

https://www.youtube.com/watch?v=4FmbWir_abM

Code:

- Kryptonite for Kafka Project: https://github.com/hpgrahsl/kryptonite-for-kafka
- Demo Scenarios: https://github.com/hpgrahsl/vdlux22-k4k-demo

Hans-Peter Grahsl

June 21, 2022
Tweet

More Decks by Hans-Peter Grahsl

Other Decks in Programming

Transcript

  1. 61% of breaches involved credential data1 1 Verzion DBIR 2021

    - https://www.verizon.com/dbir @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 3
  2. 85% of breaches involved the human element1 1 Verzion DBIR

    2021 - https://www.verizon.com/dbir @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 4
  3. compromised external cloud assets more common than on-premises assets1 1

    Verzion DBIR 2021 - https://www.verizon.com/dbir @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 6
  4. Don't forget about the price tag of data breaches. @hpgrahsl

    | #VoxxedDays Luxembourg | June 21-22, 2022 7
  5. Don't forget about the price tag of data breaches. @hpgrahsl

    | #VoxxedDays Luxembourg | June 21-22, 2022 8
  6. $4.24M average cost of data breach2 2 IBM Cost of

    Data Breach Report - https://www.ibm.com/security/data-breach @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 9
  7. $180 per record cost of customer pii2 2 IBM Cost

    of Data Breach Report - https://www.ibm.com/security/data-breach @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 10
  8. It's me ... ! • technical trainer at NETCONOMY •

    independent engineer & consultant • Confluent Community Catalyst • MongoDB Champion @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 12
  9. brokers see everything ... and so does any legitimate Kafka

    client @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 32
  10. Cryptography • Tink by Google • AEAD based on AES

    GCM • DAEAD based on AES SIV • key rotation support @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 69
  11. Keyset Management • within SMT config (not recommended) • externalized

    to separate file (okayish) • remote / cloud KMS (recommended) • currently Azure Key Vault @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 70
  12. ! Little Ideas ! • wildcard / regex matching for

    field names • dynamic keyset selection based on payload • additional KMS providers (GCP, AWS, ...) @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 71
  13. ! Bigger Ideas ! • add further cryptography options (e.g.

    FPE) • language / runtime agnostic data serialization • extend scope beyond Kafka Connect @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 72
  14. data should continue to be a valuable asset not become

    a costly liability @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 73
  15. Go check it out ! • Project Code https://bit.ly/vdlux22-k4k •

    Demo Scenarios https://bit.ly/vdlux22-demo @hpgrahsl | #VoxxedDays Luxembourg | June 21-22, 2022 75
  16. Photo Credits in order of appearance (c) Chunli Ju -

    https://unsplash.com/photos/8fs1X0JFgFE (c) Wolf Zimmermann - https://unsplash.com/photos/6sf5rf8QYFE (c) Jason Leung - https://unsplash.com/photos/SAYzxuS1O3M (c) Dev Asangbam - https://unsplash.com/photos/sh9vkVbVgo (c) Keenan Constance - https://unsplash.com/photos/VTLcvV6UVaI (c) Steve Johnson - https://unsplash.com/photos/hokONTrHIAQ (c) Pete Linforth - https://pixabay.com/illustrations/biometrics-access-identification-4503187/ (c) Miguel Á. Padriñán - https://www.pexels.com/photo/close-up-shot-of-keys-on-a-red-surface-2882687/ (c) Camila Quintero Franco - https://unsplash.com/photos/mC852jACK1g (c) Gerd Altmann - https://pixabay.com/illustrations/board-excuse-me-excuse-discharge-1848736/ (c) Vijaya narasimha - https://pixabay.com/photos/crevasse-sand-stone-hills-rock-399957/ (c) Gerd Altmann - https://pixabay.com/photos/trust-man-hood-map-prompt-4321822/ (c) Matheo JBT - https://unsplash.com/photos/HLhvZ9HRAwo (c) Rob Laughter - https://unsplash.com/photos/WW1jsInXgwM (c) Markus Spiske - https://unsplash.com/photos/iar-afB0QQw (c) Nerene Grobler - https://unsplash.com/photos/sLxcfdsqLQ (c) Wilhelm Gunkel - https://unsplash.com/photos/L04Kczg_Jvs (c) Matt Walsh - https://unsplash.com/photos/tVkdGtEe2C4