Mastering Kubernetes on AWS

Short presentation about the Amazon Elastic Container Service for Kubernetes (EKS) at the Kubernetes Munich Meetup on 03/13/2019.

1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Christoph Kassen Sr. Solutions Architect – AWS @christoph_k Mastering Kubernetes on AWS
2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customer use cases Microservices PaaS Platform-as-a-Service Enterprise App Migration Machine Learning
3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS container services landscape Management Deployment, Scheduling, Scaling & Management of containerized applications Hosting Where the containers run Amazon Elastic Container Service Amazon Elastic Container Service for Kubernetes Amazon EC2 AWS Fargate Image Registry Container Image Repository Amazon Elastic Container Registry
4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. New: AWS Application management services Service discovery for all your cloud resources Increase developer productivity Integration with Amazon container services AWS Cloud Map Observability & traffic control Works across clusters and container services AWS built and run
5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customers adopting Kubernetes on AWS
6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark “Run Kubernetes for me.” 51%of Kubernetes workloads run on AWS today — Cloud Native Computing Foundation
7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Kubernetes on AWS Highly available Scalable Secure 3x Kubernetes API Servers for HA 3x Kubernetes etcd Servers for HA
8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark EKS is Kubernetes Certified Kubernetes Conformance • Guaranteed Portability and Interoperability • Timely Updates • Confirmability
9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Availability Zone 1 Availability Zone 2 Availability Zone 3 API Server API Server API Server etcd etcd etcd Worker nodes Worker nodes Worker nodes EKS Overview AWS Managed Customer Managed
10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. [mycluster].eks.amazonaws.com Availability Zone 1 Availability Zone 2 Availability Zone 3 Kubectl AWS Cloud VPC EKS Overview
11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Creating a Cluster AWS Console AWS CloudFormation Type: "AWS::EKS::Cluster” Properties:` Name: "prod" Version: "1.11“ RoleArn: "arn:aws:iam::012345678910:role/eks- service-role" ResourcesVpcConfig: SecurityGroupIds: ["sg-6979fe18"] SubnetIds: ["subnet-6782e71e", "subnet- e7e761ac"]
12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark eksctl Tooling to provision EKS clusters - Manages worker node groups - Configures CLI https://eksctl.io - Open source project - started by Weaveworks eksctl create cluster --name=cluster-1 --nodes=4 --region=eu-central-1
13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Kubernetes version 1.11 available, 1.12 coming soon. Amazon EKS will support up to three versions of Kubernetes at once “Deprecation” will prevent new cluster creation on old versions
14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Amazon EKS platform version Platform version revisions represent API server configuration changes or Kubernetes patches Platform versions increment within a Kubernetes version only
15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Deployment
16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS & K8s - CI/CD Controls • AWS CodeBuild • Amazon ECR • Spinnaker • AWS partners • GitLab • Shippable • CircleCI • Codeship • … • Helm • Kustomize • Image scanning • coreos/clair • Twistlock • Aquasec
17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Visibility & Monitoring • Amazon CloudWatch Metrics • VPC / ALB / Amazon EC2 / ASG / Amazon EKS Control plane • Custom metrics • CloudWatch Logs • VPC / ALB /Amazon EC2 / ASG / Amazon EKS Control plane • AWS CloudTrail • K8s Scaling metrics • HPA – Horizontal Pod Autoscaler • CPU / Memory • Custom Metrics • Cluster Autoscaler • Cluster wide metrics • Integrates with AWS AutoScaling • Reactive
18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Networking: Pod to pod
19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark • All the pods can communicate with each other directly without NAT • All the nodes can communicate with all pods (and vice versa) without NAT • The IP that a pod sees itself as is the same IP that others see it as The three rules of Kubernetes networking…
20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS VPC CNI plugin ENI Secondary IPs: 10.0.0.1 10.0.0.2 10.0.0.1 10.0.0.2 ENI 10.0.0.20 10.0.0.22 Secondary IPs: 10.0.0.20 10.0.0.22 ec2.associateaddress() VPC Subnet – 10.0.0.0/24 Instance 1 Instance 2 VPC
21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS VPC CNI plugin - configurability • Custom Network Configs • SNAT / External SNAT • Configurable warm pool
22. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Typical 3-tier application : Traffic flow constraints Web server pods Application server pods Reporting server pods DB instance read replica X Hello app Users X
23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark STAGE SEPARATION “TENANT” SEPARATION FINE-GRAINED FIREWALLS COMPLIANCE Namespaces – without network policy, they are not network isolated Reduce attack surface within microservice-based applications Isolate dev, test, and prod E.g., PCI, HIPAA
24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Network Policy kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: access-appserver namespace: sample-policy spec: podSelector: matchLabels: run: app-server ingress: - from: - podSelector: matchLabels: run: web-server Select affected Pods Define traffic that is allowed
25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Networking: Pod to service
26. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Service load balancer: NLB apiVersion: v1 kind: Service metadata: name: nginx namespace: default labels: app: nginx annotations: service.beta.kubernetes.io/aws-load-balancer-type: "nlb" spec: externalTrafficPolicy: Local ports: - name: http port: 80 protocol: TCP targetPort: 80 selector: app: nginx type: LoadBalancer
27. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark • exposes HTTP/HTTPS routes to services within the cluster • Many implementations: ALB, Nginx, F5, HAProxy etc • Default Service Type: ClusterIP Kubernetes Ingress Object
28. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ALB Ingress Controller AWS Resources Kubernetes Cluster Node Node Kubernetes API Server ALB Ingress Controller Node HTTP Listener HTTPS Listener Rule: /cheeses Rule: /charcuterie TargetGroup: Green (IP Mode) TargetGroup: Blue (Instance Mode) NodePort NodePort
29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Security: Runtime
30. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Security is layered Network
31. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Kubernetes specifics Network Policy Controller Use RBAC Pod Security Policy
32. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark I want to give a pod permissions to an AWS service kube2iam kiam iam4kube kube-aws- iam- controller method instance profile (metadata) credentials pod mounted secrets race conditions yes causing app to get invalid creds – yes no leading app to access to node creds - no prefetch no yes yes by design Annotation on pod pod service account pod production good mediocre minimal minimal Read more:
33. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Logging: workers
34. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Implementing logging with EFK - DIY EKS Worker pod fluentd daemonset
35. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Implementing logging with EFK - DIY EKS Worker pod fluentd daemonset
36. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Resources
37. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Resources Helm | https://helm.sh/ Kustomize | https://kustomize.io/ EKS blog posts: https://aws.amazon.com/blogs/compute/category/compute/am azon-elastic-container-service-for-kubernetes/ Public Roadmap https://github.com/aws/containers- roadmap/projects/1?card_filter_query=label%3A+eks
38. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Thank You! Christoph Kassen Sr. Solutions Architect @christoph_k