Distributed event driven systems with KEDA

Slides related to the Distributed event driven systems with KEDA for Devtron webinar. The demo focusses on using kubernetes bades Event driven Autoscaling to dynamically scale the microservice based on the number of messages in RabbitMQ queue.

Transcript

1. Distributed Event Driven Systems with KEDA Nilesh Gule @nileshgule
2. $whoami { “name” : “Nilesh Gule”, “website” : “https://www.HandsOnArchitect.com", “github” : “https://GitHub.com/NileshGule" “twitter” : “@nileshgule”, “linkedin” : “https://www.linkedin.com/in/nileshgule”, “YouTube” : “https://www.YouTube.com/@nilesh-gule” “likes” : “Technical Evangelism, Cricket”, “co-organizer” : “Azure Singapore UG” }
4. @nileshgule KEDA Kubernets based Event Driven Autoscaling Scaling in Kubernetes KEDA Overview TechTalks demo with RabbitMQ Autoscaling with KEDA KEDA Architecture Agenda
5. Pre-requisites • Self contained application with all its dependencies Docker • Orchestrates containers • Self healing • Service discovery • Scaling Kubernetes • Dockerhub / private container registry • Repository for storing container images Container Registry • Managed Kubernetes service on Azure • Native integration with other Azure services like Azure Active Directory (AAD), Azure Container registry (ACR), Azure Container Instances (ACI), Azure Monitor etc. Azure Kubernetes Service (AKS)
6. Kubernetes scaling options Horizontal Pod Autoscaler Vertical Pod Autoscaler Cluster Autoscaler Manual
8. @nileshgule Demo setup 1000 (configurable) messages Batch of 50 (configurable) messages https://github.com/NileshGule/pd-tech-fest-2019
9. @nileshgule TechTalks Demo – Kubernetes deployment https://github.com/NileshGule/pd-tech-fest-2019  Manifest files describe Deployment and Service  Helm charts to deploy third-party apps / services  Octant / Lens – visualize k8s cluster  Kubectl to deploy Manifest  Postman / RESTClient to generate workload
10. @nileshgule Generate workload
11. @nileshgule Demo KEDA in action
13. apiVersion: keda.k8s.io/v1alpha1 kind: ScaledObject metadata: name: rabbitmq-consumer namespace: default labels: deploymentName: rabbitmq-consumer-deployment spec: scaleTargetRef: deploymentName: rabbitmq-consumer pollingInterval: 5 # Optional. Default: 30 seconds cooldownPeriod: 30 # Optional. Default: 300 seconds minReplicaCount: 1 # Optional. Default: 0 maxReplicaCount: 30 # Optional. Default: 100 triggers: - type: rabbitmq metadata: queueName: hello queueLength : ‘5’ authenticationRef: name: trigger-auth-rabbitmq-host
15. Run Containers at scale ❖KEDA supported scale triggers including Azure Event Hub, Apache Kafka, RabbitMQ, MongoDB, MySQL, PostgreSQL etc. ❖Robust autoscaling without managing complex infrastructure ❖Scale to zero and pay only for use by the second https://keda.sh/docs/2.9/scalers/
16. Summary • Single purpose lightweight component • Any Kubernetes cluster (1.16+) • Extends Kubernetes with event driven capabilities • No overwriting or duplication of existing functionalities • Maps an event source to Deployment, StatefulSet, Custom Resource or Job for scaling • Flexible and safe to run alongside any other Kubernetes framework or application • Support 50+ auto scalers out of the box KEDA
17. Play with Docker Play with Kubernetes Linux Academy learning paths
18. KEDA AKS cluster-autoscalar Horizontal pod autoscale https://azure.microsoft.com/mediahandler/files/resourcefiles/kubernetes-learning- path/Kubernetes%20Learning%20Path%20version%201.0.pdf
19. Azure Kubernetes Service Workshop References
20. References https://www.youtube.com/@nilesh-gule
21. https://github.com/NileshGule/pd-tech-fest-2019
22. Thank you very much Code with Passion and Strive for Excellence https://www.slideshare.net/nileshgule/presentations https://speakerdeck.com/nileshgule/
23. Nilesh Gule ARCHITECT | MICROSOFT MVP “Code with Passion and Strive for Excellence” nileshgule @nileshgule Nilesh Gule NileshGule www.handsonarchitect.com https://bit.ly/youtube-nileshgule
24. Q&A