Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Securing Kafka Connect Pipelines with Client-Side Field Level Cryptography @ Kafka Summit London 2022

Securing Kafka Connect Pipelines with Client-Side Field Level Cryptography @ Kafka Summit London 2022

Abstract:
Apache Kafka offers several security features ranging from authentication and authorisation mechanisms to over-the-wire encryption. This notwithstanding, data encryption performed at the client-side, which leads to explicit data-at-rest protection in topics at the broker's side, can still be considered a blind spot.
After highlighting the main benefits for data-at-rest protection, this session discusses in-depth how to selectively encrypt and decrypt sensitive payload fields in the context of Apache Kafka Connect pipelines. In particular, an ecosystem community project with codename Kryptonite - written and open-sourced by the speaker earlier this year - is introduced.
During this demo-driven talk, you will learn how to benefit from a configurable single message transformation that lets you perform encryption and decryption operations in Kafka Connect worker nodes without any custom code. Client-side cryptography makes your integration scenarios more secure by safeguarding the most sensitive and precious data against any form of uncontrolled or illegal access once it hits the Apache Kafka brokers.

Recording:
https://www.confluent.io/events/kafka-summit-london-2022/securing-kafka-connect-pipelines-with-client-side-field-level-cryptography/

Code:

- Kryptonite for Kafka Project: https://github.com/hpgrahsl/kryptonite-for-kafka
- Demo Scenarios: https://github.com/hpgrahsl/ks22-ldn-k4k-demo

744f1c2c6cbea2ff5104b0ac512936bd?s=128

Hans-Peter Grahsl

April 26, 2022
Tweet

More Decks by Hans-Peter Grahsl

Other Decks in Programming

Transcript

  1. Securing Kafka Connect Pipelines with Client-Side field level Cryptography @hpgrahsl

    | #KafkaSummit April 25-26, 2022 | London
  2. Why should we care? @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 2
  3. 61% of breaches involved credential data1 1 Verzion DBIR 2021

    - https://www.verizon.com/dbir @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 3
  4. 85% of breaches involved the human element1 1 Verzion DBIR

    2021 - https://www.verizon.com/dbir @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 4
  5. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 5

  6. compromised external cloud assets more common than on-premises assets1 1

    Verzion DBIR 2021 - https://www.verizon.com/dbir @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 6
  7. Don't forget about the price tag of data breaches. @hpgrahsl

    | #KafkaSummit April 25-26, 2022 | London 7
  8. Don't forget about the price tag of data breaches. @hpgrahsl

    | #KafkaSummit April 25-26, 2022 | London 8
  9. $4.24M average cost of data breach2 2 IBM Cost of

    Data Breach Report - https://www.ibm.com/security/data-breach @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 9
  10. $180 per record cost of customer pii2 2 IBM Cost

    of Data Breach Report - https://www.ibm.com/security/data-breach @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 10
  11. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 11

  12. ! But Kafka related? Yes! 3 3 https://spectralops.io/blog/misconfigured-kafdrop-puts-companies-apache-kafka-completely-exposed/ @hpgrahsl |

    #KafkaSummit April 25-26, 2022 | London 12
  13. ! They found it "all" ... 3 3 https://spectralops.io/blog/misconfigured-kafdrop-puts-companies-apache-kafka-completely-exposed/ @hpgrahsl

    | #KafkaSummit April 25-26, 2022 | London 13
  14. unhappy @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 14

  15. Core Kafka Security Mechanisms @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 15
  16. Table Stakes ? @hpgrahsl | #KafkaSummit April 25-26, 2022 |

    London 16
  17. over-the-wire encryption @hpgrahsl | #KafkaSummit April 25-26, 2022 | London

    17
  18. authentication @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 18

  19. authorization @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 19

  20. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 20

  21. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 21

  22. disturbing @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 22

  23. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 23

  24. Core Security Necessary ! @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 24
  25. Core Security Sufficient ? @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 25
  26. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 26

  27. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 27

  28. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 28

  29. ? in use by brokers @hpgrahsl | #KafkaSummit April 25-26,

    2022 | London 29
  30. brokers see everything ... and so does any legitimate Kafka

    client @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 30
  31. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 31

  32. human promise is NOT technical promise @hpgrahsl | #KafkaSummit April

    25-26, 2022 | London 32
  33. end-to-end encryption ? ? ? @hpgrahsl | #KafkaSummit April 25-26,

    2022 | London 33
  34. Community Project Kryptonite for Kafka @hpgrahsl | #KafkaSummit April 25-26,

    2022 | London 34
  35. client-side field level cryptography @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 35
  36. Client-Side Cryptography @hpgrahsl | #KafkaSummit April 25-26, 2022 | London

    36
  37. Client-Side Cryptography @hpgrahsl | #KafkaSummit April 25-26, 2022 | London

    37
  38. Field Level Encryption @hpgrahsl | #KafkaSummit April 25-26, 2022 |

    London 38
  39. Field Level Encryption @hpgrahsl | #KafkaSummit April 25-26, 2022 |

    London 39
  40. Field Level Decryption @hpgrahsl | #KafkaSummit April 25-26, 2022 |

    London 40
  41. Field Level Decryption @hpgrahsl | #KafkaSummit April 25-26, 2022 |

    London 41
  42. Kafka Connect Single Message Transform @hpgrahsl | #KafkaSummit April 25-26,

    2022 | London 42
  43. CSFLC with Source Connectors @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 43
  44. CSFLC with Source Connectors @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 44
  45. CSFLC with Source Connectors @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 45
  46. CSFLC with Source Connectors @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 46
  47. CSFLC with Sink Connectors @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 47
  48. CSFLC with Sink Connectors @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 48
  49. CSFLC with Sink Connectors @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 49
  50. CSFLC with Sink Connectors @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 50
  51. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 51

  52. Demo Scenario 1 @hpgrahsl | #KafkaSummit April 25-26, 2022 |

    London 52
  53. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 53

  54. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 54

  55. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 55

  56. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 56

  57. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 57

  58. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 58

  59. Demo Scenario 2 @hpgrahsl | #KafkaSummit April 25-26, 2022 |

    London 59
  60. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 60

  61. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 61

  62. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 62

  63. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 63

  64. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 64

  65. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 65

  66. Behind the Curtain ? @hpgrahsl | #KafkaSummit April 25-26, 2022

    | London 66
  67. Cryptography • Tink by Google • AEAD based on AES

    GCM • DAEAD based on AES SIV • key rotation support @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 67
  68. Keyset Management • within SMT config (not recommended) • externalized

    to separate file (okayish) • remote / cloud KMS (recommended) • currently Azure Key Vault @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 68
  69. ! Little Ideas ! • wildcard / regex matching for

    field names • dynamic keyset selection based on payload • additional KMS providers (GCP, AWS, ...) @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 69
  70. ! Bigger Ideas ! • add further cryptography options (e.g.

    FPE) • language / runtime agnostic data serialization • extend scope beyond Kafka Connect @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 70
  71. data should continue to be a valuable asset not become

    a costly liability @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 71
  72. twitter @hpgrahsl @hpgrahsl | #KafkaSummit April 25-26, 2022 | London

    72
  73. Wanna try this? • Project Code https://bit.ly/ks22-ldn-k4k • Demo Scenarios

    https://bit.ly/ks22-ldn-demo @hpgrahsl | #KafkaSummit April 25-26, 2022 | London 73
  74. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London

  75. Photo Credits in order of appearance (c) Parsoa Khorsand -

    https://unsplash.com/photos/Dd6n63H9szw (c) Wolf Zimmermann - https://unsplash.com/photos/6sf5rf8QYFE (c) Jason Leung - https://unsplash.com/photos/SAYzxuS1O3M (c) Dev Asangbam - https://unsplash.com/photos/sh9vkVbVgo (c) Keenan Constance - https://unsplash.com/photos/VTLcvV6UVaI (c) Steve Johnson - https://unsplash.com/photos/hokONTrHIAQ (c) Pete Linforth - https://pixabay.com/illustrations/biometrics-access-identification-4503187/ (c) Miguel Á. Padriñán - https://www.pexels.com/photo/close-up-shot-of-keys-on-a-red-surface-2882687/ (c) Camila Quintero Franco - https://unsplash.com/photos/mC852jACK1g (c) Gerd Altmann - https://pixabay.com/illustrations/board-excuse-me-excuse-discharge-1848736/ (c) Vijaya narasimha - https://pixabay.com/photos/crevasse-sand-stone-hills-rock-399957/ (c) Gerd Altmann - https://pixabay.com/photos/trust-man-hood-map-prompt-4321822/ (c) Matheo JBT - https://unsplash.com/photos/HLhvZ9HRAwo (c) Rob Laughter - https://unsplash.com/photos/WW1jsInXgwM (c) Markus Spiske - https://unsplash.com/photos/iar-afB0QQw (c) Nerene Grobler - https://unsplash.com/photos/sLxcfdsqLQ (c) Wilhelm Gunkel - https://unsplash.com/photos/L04Kczg_Jvs (c) Matt Walsh - https://unsplash.com/photos/tVkdGtEe2C4