Let Karpenter manage our nodes !

Transcript

1. Let Karpenter manage our nodes! solidnerd NiclasMietz Niclas Mietz - Sr. Engineer@Spherity
2. Kubernetes Scaling Pod Scaling Kubernetes Autoscaling solidnerd
3. Karpenter vs Autoscaler Cluster Autoscaler Karpenter solidnerd
4. Karpenter • Uses EC2 Fleet with a Combination of Launch Templates • No Node Groups at all ! • Expects that Cloud Provider to fulfill the requirements from the Pod • Supports Custom AMI’s through Custom Launch Templates • Direct Support for Amazon Linux 2 or BottleRocket • Native Spot Termination Handling (Since 0.19.x) • Node Expiry (ttlSecondsUntilExpired) • Empty Nodes (ttlSecondsAfterEmpty) solidnerd
5. TTL for the WIN solidnerd Example View from a Dynamic Cluster
6. Requirements apiVersion: karpenter.sh/v1alpha5 kind: Provisioner metadata: name: small-categories spec: requirements: - key: "karpenter.k8s.aws/instance-category" operator: In values: ["c", "m", "r", "p", "g", "dl"] solidnerd karpenter.k8s.aws/instance-category c karpenter.k8s.aws/instance-cpu 2 karpenter.k8s.aws/instance-family c5ad karpenter.k8s.aws/instance-generation 5 karpenter.k8s.aws/instance-hypervisor nitro karpenter.k8s.aws/instance-local-nvme 75 karpenter.k8s.aws/instance-memory 4096 karpenter.k8s.aws/instance-pods 29 karpenter.k8s.aws/instance-size large kubernetes.io/arch amd64 kubernetes.io/os linux node.kubernetes.io/instance-type c5ad.large Available Types
7. Scheduling a Pod Normal Nominated 42s karpenter Pod should schedule on ip-192-168-77-141.eu-west-1.compute.internal solidnerd 2022-11-17T14:35:50.306Z INFO controller.provisioning Found 1 provisionable pod(s) {"commit": "ea5dc14"} 2022-11-17T14:35:50.306Z INFO controller.provisioning Computed 1 new node(s) will fit 1 pod(s) {"commit": "ea5dc14"} 2022-11-17T14:35:50.316Z INFO controller.provisioning Launching node with 1 pods requesting {"cpu":"1155m","memory":"372880Ki","pods":"4"} from types c5d.large, c5ad.large, c5a.large, c5.large, c6i.large and 29 other(s) {"commit": "ea5dc14", "provisioner": "default"} 2022-11-17T14:35:50.710Z DEBUG controller.provisioning.cloudprovider Created launch template, Karpenter-awsug3-cologne-8011056993201718368 {"commit": "ea5dc14", "provisioner": "default"} 2022-11-17T14:35:52.803Z INFO controller.provisioning.cloudprovider Launched instance: i-0fd93815e22b3934d, hostname: ip-192-168-77-141.eu-west-1.compute.internal, type: c5a.large, zone: eu-west-1c, capacityType: spot {"commit": "ea5dc14", "provisioner": "default"}
8. DEMO Time 🎉
9. Whats next ? • Optimize your CPU/Memory requests of your Workloads to be more efficient ◦ Be Data Driven ◦ Analyze them manual or with Recommendation Algorithms ◦ Apply your Recommandations solidnerd
10. Summary Did you like the resources on this template? Get them for free at our other websites: • Schedule Pods to different capacity Types (OnDemand,Spot) • Use different Provisioners to allow best practices for your Type of Workload • Use Spot whenever your workload allows it • Use Scheduling Techniques like NodeSelector,ToplogySpeadConstraints,Taints and Tolerations in combination with Provisioners • Karpenter provides the foundation for Scaling • It’s can be combined with Pod Scaling (HPA,KEDA) solidnerd
11. CREDITS: This presentation template was created by Slidesgo, and includes icons by Flaticon, and infographics & images by Freepik Thanks! Do you have any questions? Please keep this slide for attribution solidnerd NiclasMietz
12. Additional Resources General Information • karpenter.sh • Karpenter Best Practices • Karpenter Spot Workshop • AWS Node Termination Handler Cool Ideas and Support • Karpenter Support for AKS • Karpenter k3d • Minimum Capacity Draft • Kubelet Graceful Node Shutdown Support solidnerd