Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Securing Kafka Connect Pipelines with Client-Side Field Level Cryptography @ Kafka Summit London 2022

Securing Kafka Connect Pipelines with Client-Side Field Level Cryptography @ Kafka Summit London 2022

Abstract:
Apache Kafka offers several security features ranging from authentication and authorisation mechanisms to over-the-wire encryption. This notwithstanding, data encryption performed at the client-side, which leads to explicit data-at-rest protection in topics at the broker's side, can still be considered a blind spot.
After highlighting the main benefits for data-at-rest protection, this session discusses in-depth how to selectively encrypt and decrypt sensitive payload fields in the context of Apache Kafka Connect pipelines. In particular, an ecosystem community project with codename Kryptonite - written and open-sourced by the speaker earlier this year - is introduced.
During this demo-driven talk, you will learn how to benefit from a configurable single message transformation that lets you perform encryption and decryption operations in Kafka Connect worker nodes without any custom code. Client-side cryptography makes your integration scenarios more secure by safeguarding the most sensitive and precious data against any form of uncontrolled or illegal access once it hits the Apache Kafka brokers.

Recording:
https://www.confluent.io/events/kafka-summit-london-2022/securing-kafka-connect-pipelines-with-client-side-field-level-cryptography/

Code:

- Kryptonite for Kafka Project: https://github.com/hpgrahsl/kryptonite-for-kafka
- Demo Scenarios: https://github.com/hpgrahsl/ks22-ldn-k4k-demo

Hans-Peter Grahsl

April 26, 2022
Tweet

More Decks by Hans-Peter Grahsl

Other Decks in Programming

Transcript

  1. 61% of breaches involved credential data1 1 Verzion DBIR 2021

    - https://www.verizon.com/dbir @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 3
  2. 85% of breaches involved the human element1 1 Verzion DBIR

    2021 - https://www.verizon.com/dbir @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 4
  3. compromised external cloud assets more common than on-premises assets1 1

    Verzion DBIR 2021 - https://www.verizon.com/dbir @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 6
  4. Don't forget about the price tag of data breaches. @hpgrahsl

    | #KafkaSummit April 25-26, 2022 | London 7
  5. Don't forget about the price tag of data breaches. @hpgrahsl

    | #KafkaSummit April 25-26, 2022 | London 8
  6. $4.24M average cost of data breach2 2 IBM Cost of

    Data Breach Report - https://www.ibm.com/security/data-breach @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 9
  7. $180 per record cost of customer pii2 2 IBM Cost

    of Data Breach Report - https://www.ibm.com/security/data-breach @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 10
  8. brokers see everything ... and so does any legitimate Kafka

    client @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 30
  9. Cryptography • Tink by Google • AEAD based on AES

    GCM • DAEAD based on AES SIV • key rotation support @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 67
  10. Keyset Management • within SMT config (not recommended) • externalized

    to separate file (okayish) • remote / cloud KMS (recommended) • currently Azure Key Vault @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 68
  11. ! Little Ideas ! • wildcard / regex matching for

    field names • dynamic keyset selection based on payload • additional KMS providers (GCP, AWS, ...) @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 69
  12. ! Bigger Ideas ! • add further cryptography options (e.g.

    FPE) • language / runtime agnostic data serialization • extend scope beyond Kafka Connect @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 70
  13. data should continue to be a valuable asset not become

    a costly liability @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 71
  14. Wanna try this? • Project Code https://bit.ly/ks22-ldn-k4k • Demo Scenarios

    https://bit.ly/ks22-ldn-demo @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 73
  15. Photo Credits in order of appearance (c) Parsoa Khorsand -

    https://unsplash.com/photos/Dd6n63H9szw (c) Wolf Zimmermann - https://unsplash.com/photos/6sf5rf8QYFE (c) Jason Leung - https://unsplash.com/photos/SAYzxuS1O3M (c) Dev Asangbam - https://unsplash.com/photos/sh9vkVbVgo (c) Keenan Constance - https://unsplash.com/photos/VTLcvV6UVaI (c) Steve Johnson - https://unsplash.com/photos/hokONTrHIAQ (c) Pete Linforth - https://pixabay.com/illustrations/biometrics-access-identification-4503187/ (c) Miguel Á. Padriñán - https://www.pexels.com/photo/close-up-shot-of-keys-on-a-red-surface-2882687/ (c) Camila Quintero Franco - https://unsplash.com/photos/mC852jACK1g (c) Gerd Altmann - https://pixabay.com/illustrations/board-excuse-me-excuse-discharge-1848736/ (c) Vijaya narasimha - https://pixabay.com/photos/crevasse-sand-stone-hills-rock-399957/ (c) Gerd Altmann - https://pixabay.com/photos/trust-man-hood-map-prompt-4321822/ (c) Matheo JBT - https://unsplash.com/photos/HLhvZ9HRAwo (c) Rob Laughter - https://unsplash.com/photos/WW1jsInXgwM (c) Markus Spiske - https://unsplash.com/photos/iar-afB0QQw (c) Nerene Grobler - https://unsplash.com/photos/sLxcfdsqLQ (c) Wilhelm Gunkel - https://unsplash.com/photos/L04Kczg_Jvs (c) Matt Walsh - https://unsplash.com/photos/tVkdGtEe2C4