Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Event Driven Autoscaling with KEDA - 1 Sept 2021 - Microsoft Reactor

9e33a1d43a88f23f6c545c1e0f07f4b5?s=47 Nilesh Gule
September 01, 2021

Event Driven Autoscaling with KEDA - 1 Sept 2021 - Microsoft Reactor

Slide deck related to the Microsoft Reactor Benagluru event on September 1 2021.

During the session following topics were covered
- Scaling options in Kubernetes including HPA, Cluster Autoscaler, Manual scaling
- Need for Kubernetes based Event Driven Autoscaling (KEDA)
- Live demos of Producer and Consumer for RabbitMQ queue
- KEDA Architecture
- KEDA scalers and event sources


Nilesh Gule

September 01, 2021

More Decks by Nilesh Gule

Other Decks in Technology


  1. Event Driven Autoscaling with KEDA

  2. $whoami { “name” : “Nilesh Gule”, “website” : “https://www.HandsOnArchitect.com", “github”

    : “https://GitHub.com/NileshGule" “twitter” : “@nileshgule”, “linkedin” : “https://www.linkedin.com/in/nileshgule”, “likes” : “Technical Evangelism, Cricket”, “co-organizer” : “Azure Singapore UG” } https://bit.ly/AzureSingaporeSessions
  3. None
  4. AGENDA ▸ Scaling options with Kubernetes ▸ KEDA overview ▸

    TechTalks demo with RabbitMQ ▸ Autoscaling with KEDA ▸ KEDA Architecture 4
  5. Pre-requisites • Self contained application with all its dependencies Docker

    • Orchestrates containers • Self healing • Service discovery • Scaling Kubernetes • Dockerhub / private container registry • Repository for storing container images Container Registry • Managed Kubernetes service on Azure • Native integration with other Azure services like Azure Active Directory (AAD), Azure Container registry (ACR), Azure Container Instances (ACI), Azure Monitor etc. Azure Kubernetes Service (AKS)
  6. @nileshgule Kubernetes Scaling Options 1 2 3 Horizontal Pod Autoscaler

    Cluster Autoscaler Manual scaling
  7. None
  8. TechTalks Demo 8

  9. @nileshgule TechTalks Producer setup https://github.com/NileshGule/pd-tech-fest-2019  Techtalks producer .Net Core

    API  2 replicas  Exposed as Kubernetes service using LoadBalancer
  10. @nileshgule TechTalks Consumer setup https://github.com/NileshGule/pd-tech-fest-2019  Techtalks consumer .Net Core

    exe  Not exposed outside Kubernetes cluster  Consumes configured number of messages in a batch
  11. @nileshgule RabbitMQ setup https://github.com/NileshGule/pd-tech-fest-2019  RabbitMQ deployed using Helm chart

     Exposed as Kubernetes service
  12. @nileshgule TechTalks Demo setup 1000 (configurable) messages Batch of 50

    (configurable) messages https://github.com/NileshGule/pd-tech-fest-2019
  13. @nileshgule TechTalks Demo – Kubernetes deployment https://github.com/NileshGule/pd-tech-fest-2019  Manifest files

    describe Deployment and Service  Helm charts to deploy third-party apps / services  Octant – visualize k8s cluster  Kubectl to deploy Manifest  Postman to generate workload
  14. Autoscaling Demo 15

  15. None
  16. None
  17. None
  18. @nileshgule How KEDA works? KEDA moves from the CNCF Sandbox

    to become an incubating project | Cloud Native Computing Foundation
  19. apiVersion: keda.k8s.io/v1alpha1 kind: ScaledObject metadata: name: rabbitmq-consumer-scaled-object namespace: default labels:

    deploymentName: rabbitmq-consumer-deployment spec: scaleTargetRef: deploymentName: rabbitmq-consumer pollingInterval: 5 # Optional. Default: 30 seconds cooldownPeriod: 30 # Optional. Default: 300 seconds minReplicaCount: 1 # Optional. Default: 0 maxReplicaCount: 30 # Optional. Default: 100 triggers: - type: rabbitmq metadata: queueName: hello queueLength : ‘5’ authenticationRef: name: trigger-auth-rabbitmq-host
  20. @nileshgule KEDA ScaledObject https://github.com/NileshGule/pd-tech-fest-2019

  21. None
  22. Azure Blob Storage Azure Event Hub Azure Log Analytics Azure

    Service Bus Queues and Topics Azure Storage Queues ActiveMQ Artemis Kafka Prometheus RabbitMQ Redis Streams Redis Lists Liiklus NATS Huawei Cloudeye AWS CloudWatch AWS Kinesis Stream AWS Simple Queue Service GCP PubSub Azure functions MySQL PostgreSQL https://keda.sh/docs/2.4/scalers/
  23. Summary ▸ Single purpose lightweight component ▸ Any Kubernetes cluster

    (1.16+) ▸ Extends Kubernetes with event driven capabilities ▸ No overwriting or duplication of existing functionalities ▸ Maps an event source to Deployment, StatefulSet, Custom Resource or Job for scaling ▸ Flexible and safe to run alongside any other Kubernetes framework or application ▸ Support 35+ auto scalers out of the box 25
  24. Play with Docker Play with Kubernetes Linux Academy learning paths

  25. KEDA AKS cluster-autoscalar Horizontal pod autoscale https://azure.microsoft.com/mediahandler/files/resourcefiles/kubernetes-learning- path/Kubernetes%20Learning%20Path%20version%201.0.pdf

  26. Azure Kubernetes Service Workshop References

  27. References https://bit.ly/youtube-nileshgule

  28. https://github.com/NileshGule/pd-tech-fest-2019 reactor-demo-september-1-2021.md

  29. Thank you very much Code with Passion and Strive for

    Excellence https://www.slideshare.net/nileshgule/presentations https://speakerdeck.com/nileshgule/
  30. Nilesh Gule ARCHITECT | MICROSOFT MVP “Code with Passion and

    Strive for Excellence” nileshgule @nileshgule Nilesh Gule NileshGule www.handsonarchitect.com https://bit.ly/youtube-nileshgule
  31. Q&A