The Hitchhiker's Guide to Kubernetes Addons

Are you tapping into Kubernetes' true potential? Beyond the core, the real magic happens with add-ons, controllers, and CRDs. From automating workflows to enforcing ironclad security, these tools are the key to optimizing Kubernetes for your unique needs. But with scale comes complexity...
Don’t Panic—We’ve Got You Covered!

Transcript

1. The Hitchhiker’s Guide to K8s Addons Kubernetes For Humans February 2025
2. Agenda Introductions Cert-manager External-DNS Helm Argo Workflows Karpenter Kyverno Summary Komodor Demo
3. Meet our speakers 👋 Udi Hofesh K8s Advocate Itiel Shwartz Co-Founding CTO Amit Bar Oz Product Architect
4. Kubernetes is a Complete Ecosystem. Pods and deployments are the tip of the iceberg, there are dozens of components that are required to run Kubernetes in production effectively and at scale.
5. Cert-manager Overview Cert-manager is a Kubernetes add-on that automates the management and issuance of TLS certificates from various sources Advantages • Automation • Integration • Security Challenges • Complexity • Resource management • Controller upgrades Best Practices • Implementing network policies • Dedicated node pools, and • Configuring in a highly available way
6. External-DNS Overview External-DNS is a Kubernetes add-on that synchronizes exposed Kubernetes Services and Ingresses with DNS providers, making Kubernetes resources discoverable via public and or private DNS servers. Advantages • Automation • Supports multiple DNS providers as well as a custom provider • Dynamic updates Challenges • Performance bottlenecks • API rate limits • Configuration complexity • Security risks Best Practices • Clear ownership, define which resources should be managed by External-DNS • Configure for high availability • Use separate DNS zones for different environments • Set appropriate TTL values for DNS records to balance propagation time and update frequency.
7. HELM Overview Helm is a package manager for Kubernetes that simplifies the deployment and management of applications by using "charts," which are pre-configured application templates. It enables users to define, install, and upgrade even the most complex Kubernetes applications. Advantages • Simplified Deployment • Version control • Community support Challenges • Hard to customize third party charts • Chart quality varies • Managing state Best Practices • Regularly update your helm charts to ensure latest security patches are applied
8. Argo Workflows Overview An open-source container-native workflow engine designed to orchestrate parallel jobs on Kubernetes. It allows users to define multi-step workflows as a sequence of tasks or as a directed acyclic graph (DAG), with each step represented as a container. Advantages • Kubernetes native • Scalability • Flexibility Challenges • Troubleshooting • Complexity • Resource management • Security Best Practices • Templates - utilize to create reusable and modular workflow components • Implement retry strategies for individual steps to handle transient failures gracefully
9. Karpenter / Cluster Autoscaler Overview Both node autoscalers are open-source projects built for Kubernetes. They observe pods that the Kubernetes scheduler has marked as unschedulable and provisions nodes that meet the specific requirements of those pods. They also observe nodes to identify opportunities to scale down resources and reduce the cluster footprint. Advantages • Dynamic scaling • Cost efficiency • Simplified operations Challenges • Configuration complexity • Resource starvation if reaching the configured limitations • Can lead to unnecessary costs/spend if not configured properly Best Practices • Define resources requests for your pods to enable the autoscalers to make smart scaling decisions for node provisioning • Utilize various node groups (types, lifecycles) for different workload types
10. Kyverno Overview Kyverno is a policy engine designed specifically for Kubernetes. It allows users to define, validate, and enforce policies for Kubernetes, facilitating security and operational best practices. Advantages • K8s native • Declarative policies • Dynamic admission control Challenges • Performance overhead • Policy conflicts • Learning curve Best Practices • Start small and simple, expand and introduce complexity as needed • Break down complex policies into smaller, reusable rules • Store your policies in version control to track changes and collaborate effectively
11. Recap of K8s Addon Challenges Operational Toil: Managing a production-grade Kubernetes stack at scale imposes significant toil, including handling numerous Custom Resource Definitions (CRDs). Complex Correlation: Teams must manage tasks like autoscaling, network policies, and service meshes (e.g., Istio), consistently applying best practices and optimizing configurations to ensure seamless service availability across the cluster. Risk of Downtime: Misconfigurations or delays with critical components like storage solutions, networking plugins and other core services can lead to cluster-wide downtime or degraded performance. Lack of Visibility and Proactiveness: as CRDs are custom built, there is no standards for visibility error tracking and alerting. This results in many issues go undetected and reactive firefighting. Impact on Productivity: The burden of managing these core services is time consuming and diverts focus from strategic initiatives to tactic ‘keeping the head above the water’.
12. Komodor to the Rescue Visualize Operate Detect Automated Investigation Optimize Autoscaling Networking Policies & IAC Core Services Packages AI / MLOps & Batch Processing Storage IAC Policies Native Networking Service Mesh API Gateway Cluster Autoscaler Streaming Node Scaling Pod Scaling VCS & CI/CD
13. Demo Time