Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Lambda Sidecar Pattern for Event-Driven Kubernetes

Ken Collins
December 07, 2022
89

The Lambda Sidecar Pattern for Event-Driven Kubernetes

In this session we are going to share a top to bottom (architecture → tooling) story on how we at Custom Ink are using real cloud-native & serverless solutions for our Kubernetes workloads during our migration from AMQP to EventBridge. Like many teams, we are slowly marching to a more event-driven architecture (EDA) and we cant wait to share how Lambda, SQS, & EventBridge are a key part of our journey. In our time together, we want to explore the following topics.
A product review on the current state of open-soruce solutions for containerized event handling.

Ken Collins

December 07, 2022
Tweet

Transcript

  1. The Lambda Sidecar
    Pattern for Event-Driven
    Kubernetes (YMMV)
    2022-12-07 by Ken Collins

    View Slide

  2. Our Need to Change?

    View Slide

  3. 🐰
    What is AMQP?
    AKA RabbitMQ, how is it used at Custom Ink?

    View Slide

  4. 🐰 [email protected] -> Topics




    polling process
    polling process

    View Slide

  5. ● Service owns the data being published.
    ● Other service(s) interested in data create/updates.
    ● Service does not depend on receipt of data/event.
    Our AMQP Patterns 🪢

    View Slide

  6. ● Custom in-house software 11 years old. No active
    development in 4 years for supporting software like
    AMQP & EventMachine.
    ● Infrastructure is not managed nor multi-region.
    Application integrations are not cloud-native.
    ● EDA is part of our future-ready platform’s composability,
    SaaS integration, & real-time data needs.
    Our Need to Change?

    View Slide

  7. Cloud-Native EDA w/Kubernetes ⁉
    ● Containerization has always promised us future
    opportunities. Can we start cashing in on that now?
    ● Can this be done via simple & viable means? Are the
    integration capabilities compelling?

    View Slide

  8. Cloud-Native EDA w/Kubernetes ⁉

    View Slide

  9. Cloud-Native EDA w/Kubernetes ⁉

    View Slide

  10. Deep Dive into
    this Architecture
    🤿
    Down the C4 Model

    View Slide


  11. Our Custom
    Kubernetes Tooling

    View Slide

  12. GitOps Tooling Flow (Before)

    View Slide

  13. GitOps Tooling Flow (Before)

    View Slide

  14. Kubernetes
    for each Project
    🦈

    View Slide

  15. Typical Kubernetes Project

    View Slide

  16. Typical Kubernetes Project

    View Slide

  17. With The “Lambda Sidecar” Pattern

    View Slide

  18. Kubernetes
    AWS Resources
    🪸

    View Slide

  19. For Each EventBridge Rule (simple)

    View Slide

  20. Tooling EventBridge Interface

    View Slide

  21. GitOps Tooling Flow (Before)

    View Slide

  22. GitOps Tooling Flow (After)

    View Slide

  23. AWS::Lambda::Function ImageConfig
    https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-imageconfig.html

    View Slide

  24. Runtime Interface Client (RIC)
    https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html

    View Slide

  25. GitOps CloudFormation Snippet

    View Slide

  26. GitOps Tooling Flow (After)

    View Slide

  27. Diving Deeper

    View Slide

  28. Secret Environment Variables?
    https://external-secrets.io/v0.7.0-rc1/

    View Slide

  29. Hello Crypteia & LD_PRELOAD
    https://github.com/customink/crypteia

    View Slide

  30. For Each EventBridge Rule (simple)

    View Slide

  31. For Each EventBridge Rule (full)

    View Slide

  32. Multiple EventBridge Rules (full)

    View Slide

  33. Typical Kubernetes Project (Before)

    View Slide

  34. Typical Kubernetes Project (After)

    View Slide

  35. Questions &
    Answers

    View Slide