Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Securing Kafka Connect Pipelines with Client-Side Field Level Cryptography @ Kafka Summit London 2022

Securing Kafka Connect Pipelines with Client-Side Field Level Cryptography @ Kafka Summit London 2022

Abstract:
Apache Kafka offers several security features ranging from authentication and authorisation mechanisms to over-the-wire encryption. This notwithstanding, data encryption performed at the client-side, which leads to explicit data-at-rest protection in topics at the broker's side, can still be considered a blind spot.
After highlighting the main benefits for data-at-rest protection, this session discusses in-depth how to selectively encrypt and decrypt sensitive payload fields in the context of Apache Kafka Connect pipelines. In particular, an ecosystem community project with codename Kryptonite - written and open-sourced by the speaker earlier this year - is introduced.
During this demo-driven talk, you will learn how to benefit from a configurable single message transformation that lets you perform encryption and decryption operations in Kafka Connect worker nodes without any custom code. Client-side cryptography makes your integration scenarios more secure by safeguarding the most sensitive and precious data against any form of uncontrolled or illegal access once it hits the Apache Kafka brokers.

Recording:
https://www.confluent.io/events/kafka-summit-london-2022/securing-kafka-connect-pipelines-with-client-side-field-level-cryptography/

Code:

- Kryptonite for Kafka Project: https://github.com/hpgrahsl/kryptonite-for-kafka
- Demo Scenarios: https://github.com/hpgrahsl/ks22-ldn-k4k-demo

Hans-Peter Grahsl

April 26, 2022
Tweet

More Decks by Hans-Peter Grahsl

Other Decks in Programming

Transcript

  1. Securing
    Kafka Connect Pipelines
    with Client-Side
    field level Cryptography
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London

    View Slide

  2. Why should we care?
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    2

    View Slide

  3. 61%
    of breaches involved
    credential data1
    1 Verzion DBIR 2021 - https://www.verizon.com/dbir
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    3

    View Slide

  4. 85%
    of breaches involved
    the human element1
    1 Verzion DBIR 2021 - https://www.verizon.com/dbir
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    4

    View Slide

  5. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    5

    View Slide

  6. compromised external
    cloud assets
    more common than
    on-premises assets1
    1 Verzion DBIR 2021 - https://www.verizon.com/dbir
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    6

    View Slide

  7. Don't
    forget about the price tag
    of data breaches.
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    7

    View Slide

  8. Don't
    forget about the price tag
    of data breaches.
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    8

    View Slide

  9. $4.24M
    average cost of data
    breach2
    2 IBM Cost of Data Breach Report - https://www.ibm.com/security/data-breach
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    9

    View Slide

  10. $180
    per record cost of
    customer pii2
    2 IBM Cost of Data Breach Report - https://www.ibm.com/security/data-breach
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    10

    View Slide

  11. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    11

    View Slide

  12. !
    But Kafka related? Yes!
    3
    3 https://spectralops.io/blog/misconfigured-kafdrop-puts-companies-apache-kafka-completely-exposed/
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    12

    View Slide

  13. !
    They found it "all" ...
    3
    3 https://spectralops.io/blog/misconfigured-kafdrop-puts-companies-apache-kafka-completely-exposed/
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    13

    View Slide

  14. unhappy
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    14

    View Slide

  15. Core Kafka
    Security Mechanisms
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    15

    View Slide

  16. Table Stakes ?
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    16

    View Slide

  17. over-the-wire encryption
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    17

    View Slide

  18. authentication
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    18

    View Slide

  19. authorization
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    19

    View Slide

  20. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    20

    View Slide

  21. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    21

    View Slide

  22. disturbing
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    22

    View Slide

  23. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    23

    View Slide

  24. Core Security
    Necessary !
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    24

    View Slide

  25. Core Security
    Sufficient ?
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    25

    View Slide

  26. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    26

    View Slide

  27. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    27

    View Slide

  28. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    28

    View Slide

  29. ?
    in use by brokers
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    29

    View Slide

  30. brokers
    see everything ...
    and so does
    any legitimate
    Kafka client
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    30

    View Slide

  31. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    31

    View Slide

  32. human promise
    is NOT
    technical promise
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    32

    View Slide

  33. end-to-end
    encryption
    ? ? ?
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    33

    View Slide

  34. Community Project
    Kryptonite for Kafka
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    34

    View Slide

  35. client-side
    field level
    cryptography
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    35

    View Slide

  36. Client-Side Cryptography
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    36

    View Slide

  37. Client-Side Cryptography
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    37

    View Slide

  38. Field Level Encryption
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    38

    View Slide

  39. Field Level Encryption
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    39

    View Slide

  40. Field Level Decryption
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    40

    View Slide

  41. Field Level Decryption
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    41

    View Slide

  42. Kafka Connect
    Single Message
    Transform
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    42

    View Slide

  43. CSFLC with Source Connectors
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    43

    View Slide

  44. CSFLC with Source Connectors
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    44

    View Slide

  45. CSFLC with Source Connectors
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    45

    View Slide

  46. CSFLC with Source Connectors
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    46

    View Slide

  47. CSFLC with Sink Connectors
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    47

    View Slide

  48. CSFLC with Sink Connectors
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    48

    View Slide

  49. CSFLC with Sink Connectors
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    49

    View Slide

  50. CSFLC with Sink Connectors
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    50

    View Slide

  51. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    51

    View Slide

  52. Demo Scenario 1
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    52

    View Slide

  53. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    53

    View Slide

  54. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    54

    View Slide

  55. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    55

    View Slide

  56. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    56

    View Slide

  57. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    57

    View Slide

  58. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    58

    View Slide

  59. Demo Scenario 2
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    59

    View Slide

  60. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    60

    View Slide

  61. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    61

    View Slide

  62. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    62

    View Slide

  63. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    63

    View Slide

  64. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    64

    View Slide

  65. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    65

    View Slide

  66. Behind the Curtain ?
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    66

    View Slide

  67. Cryptography
    • Tink by Google
    • AEAD based on AES GCM
    • DAEAD based on AES SIV
    • key rotation support
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    67

    View Slide

  68. Keyset
    Management
    • within SMT config (not recommended)
    • externalized to separate file (okayish)
    • remote / cloud KMS (recommended)
    • currently Azure Key Vault
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    68

    View Slide

  69. !
    Little Ideas
    !
    • wildcard / regex matching for field names
    • dynamic keyset selection based on payload
    • additional KMS providers (GCP, AWS, ...)
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    69

    View Slide

  70. !
    Bigger Ideas
    !
    • add further cryptography options (e.g. FPE)
    • language / runtime agnostic data serialization
    • extend scope beyond Kafka Connect
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    70

    View Slide

  71. data should continue
    to be a valuable
    asset not become
    a costly liability
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    71

    View Slide

  72. twitter
    @hpgrahsl
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    72

    View Slide

  73. Wanna try this?
    • Project Code
    https://bit.ly/ks22-ldn-k4k
    • Demo Scenarios
    https://bit.ly/ks22-ldn-demo
    @hpgrahsl | #KafkaSummit April 25-26, 2022 | London
    73

    View Slide

  74. @hpgrahsl | #KafkaSummit April 25-26, 2022 | London

    View Slide

  75. Photo Credits
    in order of appearance
    (c) Parsoa Khorsand - https://unsplash.com/photos/Dd6n63H9szw
    (c) Wolf Zimmermann - https://unsplash.com/photos/6sf5rf8QYFE
    (c) Jason Leung - https://unsplash.com/photos/SAYzxuS1O3M
    (c) Dev Asangbam - https://unsplash.com/photos/sh9vkVbVgo
    (c) Keenan Constance - https://unsplash.com/photos/VTLcvV6UVaI
    (c) Steve Johnson - https://unsplash.com/photos/hokONTrHIAQ
    (c) Pete Linforth - https://pixabay.com/illustrations/biometrics-access-identification-4503187/
    (c) Miguel Á. Padriñán - https://www.pexels.com/photo/close-up-shot-of-keys-on-a-red-surface-2882687/
    (c) Camila Quintero Franco - https://unsplash.com/photos/mC852jACK1g
    (c) Gerd Altmann - https://pixabay.com/illustrations/board-excuse-me-excuse-discharge-1848736/
    (c) Vijaya narasimha - https://pixabay.com/photos/crevasse-sand-stone-hills-rock-399957/
    (c) Gerd Altmann - https://pixabay.com/photos/trust-man-hood-map-prompt-4321822/
    (c) Matheo JBT - https://unsplash.com/photos/HLhvZ9HRAwo
    (c) Rob Laughter - https://unsplash.com/photos/WW1jsInXgwM
    (c) Markus Spiske - https://unsplash.com/photos/iar-afB0QQw
    (c) Nerene Grobler - https://unsplash.com/photos/sLxcfdsqLQ
    (c) Wilhelm Gunkel - https://unsplash.com/photos/L04Kczg_Jvs
    (c) Matt Walsh - https://unsplash.com/photos/tVkdGtEe2C4

    View Slide