Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Lambda Sidecar Pattern for Event-Driven Kubernetes

Ken Collins
December 07, 2022
190

The Lambda Sidecar Pattern for Event-Driven Kubernetes

In this session we are going to share a top to bottom (architecture → tooling) story on how we at Custom Ink are using real cloud-native & serverless solutions for our Kubernetes workloads during our migration from AMQP to EventBridge. Like many teams, we are slowly marching to a more event-driven architecture (EDA) and we cant wait to share how Lambda, SQS, & EventBridge are a key part of our journey. In our time together, we want to explore the following topics.
A product review on the current state of open-soruce solutions for containerized event handling.

Ken Collins

December 07, 2022
Tweet

Transcript

  1. The Lambda Sidecar
    Pattern for Event-Driven
    Kubernetes (YMMV)
    2022-12-07 by Ken Collins

    View full-size slide

  2. Our Need to Change?

    View full-size slide

  3. 🐰
    What is AMQP?
    AKA RabbitMQ, how is it used at Custom Ink?

    View full-size slide

  4. 🐰 AMQP@CustomInk -> Topics




    polling process
    polling process

    View full-size slide

  5. ● Service owns the data being published.
    ● Other service(s) interested in data create/updates.
    ● Service does not depend on receipt of data/event.
    Our AMQP Patterns 🪢

    View full-size slide

  6. ● Custom in-house software 11 years old. No active
    development in 4 years for supporting software like
    AMQP & EventMachine.
    ● Infrastructure is not managed nor multi-region.
    Application integrations are not cloud-native.
    ● EDA is part of our future-ready platform’s composability,
    SaaS integration, & real-time data needs.
    Our Need to Change?

    View full-size slide

  7. Cloud-Native EDA w/Kubernetes ⁉
    ● Containerization has always promised us future
    opportunities. Can we start cashing in on that now?
    ● Can this be done via simple & viable means? Are the
    integration capabilities compelling?

    View full-size slide

  8. Cloud-Native EDA w/Kubernetes ⁉

    View full-size slide

  9. Cloud-Native EDA w/Kubernetes ⁉

    View full-size slide

  10. Deep Dive into
    this Architecture
    🤿
    Down the C4 Model

    View full-size slide


  11. Our Custom
    Kubernetes Tooling

    View full-size slide

  12. GitOps Tooling Flow (Before)

    View full-size slide

  13. GitOps Tooling Flow (Before)

    View full-size slide

  14. Kubernetes
    for each Project
    🦈

    View full-size slide

  15. Typical Kubernetes Project

    View full-size slide

  16. Typical Kubernetes Project

    View full-size slide

  17. With The “Lambda Sidecar” Pattern

    View full-size slide

  18. Kubernetes
    AWS Resources
    🪸

    View full-size slide

  19. For Each EventBridge Rule (simple)

    View full-size slide

  20. Tooling EventBridge Interface

    View full-size slide

  21. GitOps Tooling Flow (Before)

    View full-size slide

  22. GitOps Tooling Flow (After)

    View full-size slide

  23. AWS::Lambda::Function ImageConfig
    https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-imageconfig.html

    View full-size slide

  24. Runtime Interface Client (RIC)
    https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html

    View full-size slide

  25. GitOps CloudFormation Snippet

    View full-size slide

  26. GitOps Tooling Flow (After)

    View full-size slide

  27. Diving Deeper

    View full-size slide

  28. Secret Environment Variables?
    https://external-secrets.io/v0.7.0-rc1/

    View full-size slide

  29. Hello Crypteia & LD_PRELOAD
    https://github.com/customink/crypteia

    View full-size slide

  30. For Each EventBridge Rule (simple)

    View full-size slide

  31. For Each EventBridge Rule (full)

    View full-size slide

  32. Multiple EventBridge Rules (full)

    View full-size slide

  33. Typical Kubernetes Project (Before)

    View full-size slide

  34. Typical Kubernetes Project (After)

    View full-size slide

  35. Questions &
    Answers

    View full-size slide