EKS and EC2 Spot Workshop - 241019

Whether you run k8s yourself or with EKS, come and learn how to save *up to* 90% on your worker nodes

Transcript

1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Optimizing Kubernetes Clusters on Spot Instances for Cost and Performance August 2019
2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda • Amazon Containers Landscape & EKS Recap • Amazon EC2 Spot Instances: Overview and Best Practices • Applying Spot best practices to K8s/EKS • Adding Spot Instances to your EKS clusters • Handling Spot Interruptions to avoid application impact • Scaling mechanisms for application and cluster elasticity • Taints, Tolerations and Affinity & Tools • Main Takeaways
3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Container Services Landscape MANAGEMENT Deployment, Scheduling, Scaling & Management of containerized applications HOSTING Where the containers run Amazon Elastic Container Service Amazon Elastic Container Service for Kubernetes Amazon EC2 AWS Fargate IMAGE REGISTRY Container Image Repository Amazon Elastic Container Registry
4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. E L A S T I C C O N TA I N E R S E RV I C E F O R K U B E R N E T E S (EKS)
5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. —CNCF survey
6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How are customer using Amazon EKS? Microservices PaaS Platform-as-a-Service Enterprise App Migration Machine Learning
7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Tenet 3: If EKS customers want to use additional AWS services, the integrations are seamless and eliminate undifferentiated heavy lifting Tenet 4: The EKS team actively contributes to the Kubernetes project Tenet 1: EKS is a platform for enterprises to run production-grade workloads Tenet 2: EKS provides a native and upstream Kubernetes experience EKS is Kubernetes certified
8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. mycluster.eks.amazonaws.com Availability Zone 1 Availability Zone 2 Availability Zone 3 Kubectl
9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. API server Cloud controller Controller manager Scheduler Add-ons KubeDNS Kubernetes master
10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. mycluster.eks.amazonaws.com EKS Workers Kubectl Amazon EKS AZ 1 AZ 2 AZ 3 Your AWS account
11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Nginx Pod Java Pod ENI Secondary IPs: 10.0.0.1 10.0.0.2 Veth IP: 10.0.0.1 Veth IP: 10.0.0.2 Nginx Pod Java Pod ENI Veth IP: 10.0.0.20 Veth IP: 10.0.0.22 Secondary IPs: 10.0.0.20 10.0.0.22 ec2.associateaddress() VPC Subnet – 10.0.0.0/24 Instance 1 Instance 2
12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon EC2 purchase options Spot Instances Spare EC2 capacity at off On-Demand prices Fault-tolerant, flexible, stateless workloads Reserved Instances Make a 1 or 3-year commitment and receive a off On-Demand prices Committed & steady-state usage On-Demand Pay for compute capacity with no long-term commitments Spiky workloads, to define needs
14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon EC2 Spot Instances
15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Spare capacity at scale single HPC cluster of 1 million vCPUs
16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. $0.27 $0.29 $0.50 1b 1c 1a 8XL $0.30 $0.16 $0.21 4XL $0.07 $0.08 $0.08 2XL $0.05 $0.04 $0.04 XL $0.01 $0.04 $0.01 L C4 $1.76 On Demand $0.88 $0.44 $0.22 $0.11 EC2 Spot pools – instance flexibility Each instance family Each instance size Each Availability Zone (66) In every region (21) Is a separate Spot pool R5 M4 C5 I3 M5d R4 D2 C5n
17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Auto Scaling Groups EMR Instance Fleets
18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Spot is easy Price changes infrequently based on long term supply and demand of spare capacity in each pool independently Just request capacity and pay the current rate. No Bidding Interruptions only happen when OD needs capacity. No outbidding
19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What about interruptions?
20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Containers + Spot = match made in heaven Skyscanner is a travel fare aggregator website and travel metasearch engine based in Edinburgh, Scotland
21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EKS & EC2 Spot instances 1. Acquiring capacity 2. Handling interruptions: Daemonsets 3. Scaling Mechanisms 4. Taints, Tolerations and affinity 5. Tools
22. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. eksctl – adding a diversified Spot nodegroup
23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EKS & EC2 Spot instances 1. Acquiring capacity 2. Handling interruptions, Daemonsets 3. Scaling Mechanisms 4. Taints, Tolerations and affinity 5. Tools
24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Handling Interruptions & Daemonset https://github.com/kube-aws/kube-spot-termination-notice-handler
25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EKS & EC2 Spot instances 1. Acquiring capacity 2. Handling interruptions, Daemonsets 3. Scaling Mechanisms 4. Taints, Tolerations and affinity 5. Tools
26. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Auto Scaling the App & Cluster • HPA (horizontal pod autoscaler) Ø Autoscales the number of pods in a Deployment/ReplicaSet • CA (cluster-autoscaler) Ø Autoscales the number of worker nodes in the cluster when: o Pods cannot be scheduled due to lack of resources (Pending state) o Nodes are underutilized and important pods can be rescheduled elsewhere
27. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Auto Scaling the App & Cluster • CA Nodegroups are still expected to be homogeneous. Implement diversification !
28. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EKS & EC2 Spot instances 1. Acquiring capacity 2. Handling interruptions, Daemonsets 3. Scaling Mechanisms 4. Taints, Tolerations and affinity 5. Tools
29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Taints, Toleration & Affinity Multi-tenant cluster. Group affinity Life-cycle affinity & Toleration
30. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EKS & EC2 Spot instances 1. Acquiring capacity 2. Handling interruptions, Daemonsets 3. Scaling Mechanisms 4. Taints, Tolerations and affinity 5. Tools
31. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Tools • eksctl : https://github.com/weaveworks/eksctl • Cloudwatch Container Insights • Descheduler: https://github.com/kubernetes-incubator/descheduler • K8s-node-drainer: https://github.com/aws-samples/amazon-k8s-node-drainer • Overprovisioner: https://github.com/helm/charts/tree/master/stable/cluster- overprovisioner
32. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Main Takeaways • Understand Spot best practices: Pricing model, Termination, Instance diversification, Spot Instance Advisor, Launch Template, ASGs • Apply Spot best practices to K8: Instance diversification, HPA, CA. • Apply instance termination daemonset, taints, tolerations, affinities • Know your tools: cluster autoscaler, eksctl • AWS EKS/ECS Roadmap : https://github.com/aws/containers- roadmap/projects/1
33. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you The definitive guide to running EC2 Spot Instances as Kubernetes worker nodes: http://bit.ly/DefintiveSpotK8sGuide EKS workshop: https://eksworkshop.com/ EC2 Spot instance Workshops: https://ec2spotworkshops.com