AWS Loft Berlin 2018 - EKS Workshop

EKS Workshop at the Loft Berlin 2018 on 10/16.

Transcript

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EKS - Kubernetes on AWS Christoph Kassen Solutions Architect Jonas Wagner Solutions Architect
2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What can you expect?
3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. WHY DO WE LOVE CONTAINERS? Packaging Distribution Immutable infrastructure
4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Open source container management platform Helps you run containers at scale Gives you primitives for building modern applications What is Kubernetes?
6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. W h y d e v e l o p e r s l o v e K u b e r n e t e s Vibrant and growing community of users and contributors
7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Why developers love Kubernetes A single extensible API S C A L E P E R F O R M A N C E B R E A D T H
8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cloud-native applications M I C R O S E R V I C E T O O L I N G N A T I V E A P P L I C A T I O N S
9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Run Kubernetes for me.” “Native AWS integrations” “An open source Kubernetes experience.”
10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. ELASTIC CONTAINER SERVICE FOR KUBERNETES
11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Container Services
12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EKS is Kubernetes Certified
13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. API server Cloud controller Controller manager Scheduler Add-ons KubeDNS EKS control plane
14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Open Source Kubernetes Community Kubernetes https://github.com/kubernetes/kuber netes CNI plugin https://github.com/aws/amazon-vpc- cni-k8s Heptio AWS Authenticator https://github.com/heptio/authentic ator Virtual Kubelet https://github.com/virtual- kubelet/virtual-kubelet/ SIG AWS https://github.com/kubernetes/com munity/tree/master/sig-aws Cloud Provider Working Group https://github.com/kubernetes/com munity/tree/master/wg-cloud- provider External-DNS https://github.com/kubernetes- incubator/external-dns CoreOS ALB Ingress https://github.com/coreos/alb- ingress-controller CODE REVIEWS FIXING BUGS IMPLEMENTING NEW FEATURES
15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. mycluster.eks.amazonaws.com Availability Zone 1 Availability Zone 2 Availability Zone 3 Kubectl Workers
16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Worker Nodes EKS Control Plane Customer VPC EKS VPC Network Load Balancer ENI API Access Kubectl Exec/Logs TLS Static IPs ENI Attachment Autoscaling Group EKS Architecture
18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Metrics Nodes Node exporter Pod/Container Kube-state-metrics cAdvisor Application /metrics JMX Cluster-wide Aggregator Prometheus, Heapster Visualizer Grafana, Kibana, Dashboard Data Model InfluxDB, Graphite Alerting AlertManager, Kapacitor
20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IAM authentication with Kubernetes
21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ~/.kube/config (with IAM) apiVersion: v1 clusters: - cluster: certificate-authority-data: REDACTED server: https://1234.sk1.us-west-2.eks.amazonaws.com name: eks contexts: - context: cluster: eks user: eks name: eks current-context: eks kind: Config users: - name: eks user: exec: apiVersion: client.authentication.k8s.io/v1alpha1 command: heptio-authenticator-aws args: - token - -i - eks Config file is no longer user-specific J
22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Access and Authentication IAM ROLE User X IAM ROLE Service Account Y kubectl → K8s APIs → CRUD Operations on K8s aws-cli → EKS Service APIs → CRUD Operations on Infra K8s Master Nodes K8s Master Nodes K8s Master Nodes API Server Controller Mgr kubelet etcd Cloud Controller Mgr. Scheduler Authentication Webhook Tokens Authorization RBAC Mode Admission Control NamespaceLifecyle,LimitRanger ServiceAccount,DefaultStorageClass, ResourceQuota AWS STS client side Heptio-aws-authenticator server side
23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Worker provisioning k u b e c t l A W S A u t h c o n f i g m a p & R B A C W o r k e r s R o l e R o l e config map
25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Native VPC networking with CNI plugin Pods have the same VPC address inside the pod as on the VPC Simple, secure networking Open source and on Github … { }
27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Networking with CNI plugin 172.16.0.0/16 User X Service Account Y Kubectl K8s Node 2 K8s Node 1 kubelet kube-proxy kubelet kube-proxy VPC Subnet per AZ 172.16.0.1/24 ENI ENI Primary Private IP: 172.16.1.118 Secondary IPs: 172.16.1.147, 172.16.1.224…. Service: Front end POD 2 POD 3 eth0 Service: Back end POD 1 POD 4 eth0 ec2.associateaddress() L3 RouteTable veth0 B veth0 A eth0 172.16.1.147/32 eth0 172.16.1.224/32 CNI K8s Master Nodes K8s Master Nodes K8s Master Nodes API Server Controller Manager kubelet etcd Scheduler kube-proxy Cloud Controller Mgr.
28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DNS, Services and ELB 172.16.0.0/16 User X Service Account Y K8s Node 2 K8s Node 1 kubelet kube-proxy kubelet kube-proxy VPC Subnet per AZ - 172.16.0.1/24 ENI ENI Service: Front end POD 2 POD 3 Service: Back end POD 1 POD 4 CNI K8s Master Nodes K8s Master Nodes K8s Master Nodes API Server Controller Manager kubelet etcd Scheduler kube-proxy Cloud Controller Mgr. DNS kubedns dnsmasq healthz DNS Service – Static IP POD 2 POD 2 kind: Service type: LoadBalancer
29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes Network Policies enforce network security rules Calico is the leading implementation of the network policy API Open source, active development (>100 contributors) Commercial support available from Tigera
30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. S T A G E S E P A R A T I O N “ T E N A N T ” S E P A R A T I O N F I N E - G R A I N E D F I R E W A L L S C O M P L I A N C E Namespaces – without network policy, they are not network isolated Reduce attack surface within microservice-based applications Isolate dev, test, and prod E.g., PCI, HIPAA
31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Network Policy kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: web-allow-prod spec: podSelector: matchLabels: app: web ingress: - from: - namespaceSelector: matchLabels: purpose: production Select affected Pods Define traffic that is allowed
32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lab 1
33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hands-on! – Lab 1 AWS Workshop for Kubernetes 1. Create an EKS cluster 1. Follow the instructions at https://eksworkshop.com (beginner) 2. Use eksctl https://eksctl.io (advanced) 3. Follow the EKS docs https://docs.aws.amazon.com/eks/latest/userguide/getting- started.html (the “hard“ way)
34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Additional resources https://github.com/ramitsurana/awesome-kubernetes https://discuss.kubernetes.io/ TGIK Playlist: https://www.youtube.com/playlist?list=PLvmPtYZtoXOEN HJiAQc6HmV2jmuexKfrJ https://aws.amazon.com/blogs/compute/tag/containers/
35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes Concepts
36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes concepts port 8080 port 8080 ReplicaSet #Pods—2 label selector: v1 ReplicaSet #Pods—1 label selector: v2 Node Docker Pod Containers
37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes concepts URI: /svc1/* URI: /svc2/* DaemonSet Daemon pod
38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Application Deployment
39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pod definition example apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: selector: matchLabels: app: nginx replicas: 2 # tells deployment to run 2 pods matching the template template: # create pods using pod definition in this template metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80
40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Deployments ❯ kubectl create -f nginx.yaml --record ❯ kubectl rollout history deployment/nginx ❯ kubectl expose deployment/nginx --port=80 --target-port=80 -- name=nginx --type=LoadBalancer ❯ kubectl describe svc nginx ❯ curl http://xyz.us-west-2.elb.amazonaws.com
41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Deployments - Helm • Package management for k8s • Chart - collection of files • Description of k8s resources • Flexible templating • Tiller • Installed on k8s cluster • Client • Cli tool to install/update Charts
42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Using helm ❯ helm search nginx NAME CHART VERSION APP VERSION DESCRIPTION stable/nginx-ingress 0.19.2 0.14.0 An nginx Ingress controller that uses ConfigMap... stable/nginx-lego 0.3.1 Chart for nginx-ingress-controller and kube-lego stable/gcloud-endpoints 0.1.0 Develop, deploy, protect and monitor your APIs ... ❯ helm install stable/nginx-ingress --name nginx-ingress --set rbac.create=true [displays README + information about deployment] ❯ helm list NAME REVISION UPDATED STATUS CHART NAMESPACE Nginx-ingress 1 Mon May 21 18:30:17 2018 DEPLOYED nginx-ingress-0.19.2 default
43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hosting Helm repositories • Anywhere that serves HTTP can host a helm repo • Host private Helm Repo with Chartmuseum https://github.com/kubernetes-helm/chartmuseum • There’s also a handy plugin for S3! • This means IAM Role = auth for your repo J • https://github.com/hypnoglow/helm-s3
44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Deploying Helm on EKS Helm 2.9+ works with EKS out of the box Helm needs RBAC permissions ❯ kubectl -n kube-system create serviceaccount tiller ❯ kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller ❯ helm init --service-account=tiller
45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Understanding deployments # Columnar output ❯ kubectl get services # List all services in the namespace ❯ kubectl get pods --all-namespaces # List all pods in all namespaces ❯ kubectl get pods -o wide # List all pods in the namespace, with details ❯ kubectl get rc <rc-name> # Get a replication controller # Verbose output ❯ kubectl describe nodes <node-name> ❯ kubectl describe pods <pod-name> ❯ kubectl describe pods/<pod-name> # Equivalent to previous ❯ kubectl describe pods <rc-name> # Lists pods created by<rc-name>using common prefix # List Services Sorted by Name ❯ kubectl get services --sort-by=.metadata.name # Get ExternalIPs of all nodes ❯ kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=="ExternalIP")].address}'
46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Interacting with Pods ❯ kubectl logs <pod-name> # dump pod logs (stdout) ❯ kubectl logs –f <pod-name> # stream pod logs (stdout) ❯ kubectl run –i --tty busybox—image=busybox -- sh # Run pod as interactive shell ❯ kubectl attach <podname> -i # Attach to Running Container ❯ kubectl port-forward <podname> <local>:<remote> # Forward port of Pod to localhost ❯ kubectl port-forward <servicename><port> # Forward port to service ❯ kubectl exec <pod-name> -- ls / # Run command in existing pod (1 container case) ❯ kubectl exec <pod-name> -c <container-name> -- ls /
47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Deployment Strategies
48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Rolling Update apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app labels: app: my-app spec: replicas: 10 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 # Numeric or percentage based value maxUnavailable: 0 [...]
49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Blue / Green Deployment apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app-blue labels: app: my-app spec: replicas: 3 template: metadata: labels: app: my-app version: blue [...] Blue apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app-green labels: app: my-app spec: replicas: 3 template: metadata: labels: app: my-app version: green [...] Green
50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Blue / Green Deployment Blue kind: Service metadata: name: my-app labels: app: my-app spec: type: LoadBalancer ports: - name: http port: 80 targetPort: http selector: app: my-app version: blue kind: Service metadata: name: my-app labels: app: my-app spec: type: LoadBalancer ports: - name: http port: 80 targetPort: http selector: app: my-app version: green Green kubectl patch service my-app -p '{"spec":{"selector":{"version":"green"}}}'
51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Canary Deployment Production apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app-prod labels: app: my-app spec: replicas: 9 template: metadata: labels: app: my-app spec: containers: - name: my-app image: images/container:v1 [...] apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app-canary labels: app: my-app spec: replicas: 1 template: metadata: labels: app: my-app spec: containers: - name: my-app image: images/container:v2 [...] More examples at https://container-solutions.com/kubernetes-deployment-strategies/ Canary
52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Load Balancing
53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Nginx Pods EC2 instances kube-proxy :32002 nginx-service :32001 Internet 10001:8080 10002:8080 10003:8080 Request to NGINX Pod {NLB}:443 NLB NLB Forwards to the node {node:32001} Service Type – LoadBalancer (NLB) k8s service ClusterIP receives request kube-proxy load balances to pods
54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Network Load Balancer apiVersion: v1 kind: Service metadata: name: nginx namespace: default labels: app: nginx annotations: service.beta.kubernetes.io/aws-load-balancer-type: "nlb" spec: type: LoadBalancer externalTrafficPolicy: Local ports: - name: http port: 80 protocol: TCP targetPort: 80 selector: app: nginx More options: • Draining • Logging • SSL Certs • Tagging • Security groups • Health checks https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws /aws.go
55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Nginx Pods EC2 instances kube-proxy :32001 nginx-service :32003 Internet Request to NGINX Pod {ALB}:443 ALB ALB Routes based on the path. /api /home 10002:8080 Webapp Pods 10002:8080 Installation: https://github.com/pahud/eks-alb-ingress Ingress Type – CoreOS ALB Ingress kube-proxy :32002 webapp-service :32004 Load Balances to pods Proxies request to the k8s service ClusterIP
56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DNS
57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automatic Route53 DNS creation for services apiVersion: v1 kind: Service metadata: name: nginx annotations: # Uses https://github.com/kubernetes-incubator/external-dns external-dns.alpha.kubernetes.io/hostname: nginx.highlyavailable.systems. spec: type: LoadBalancer ports: - port: 80 name: http targetPort: 80 selector: app: nginx
58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automatic Route53 DNS creation for Ingress apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: nginx.highlyavailable.systems http: paths: - backend: serviceName: nginx servicePort: 80
59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lab 2
60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lab 2 1. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/01-path-basics/103-kubernetes- concepts 2. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/03-path-application- development/303-app-update 3. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/04-path-security-and- networking/405-ingress-controllers#alb-ingress-controller 4. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/03-path-application- development/307-app-management-with-helm
61. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Scheduling
62. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Controlling scheduling Resource requirements Resource filters
63. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Limit resource usage Container A Container B limit request 900m 600m limit request 800m 400m ⎲ ⎳ Pod CPU and memory resources
64. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Resource Quotas apiVersion: v1 kind: Pod metadata: name: production spec: containers: - name: nginx-pod image: nginx resources: limits: memory: "800Mi" cpu: "800m" # 0.8 vCPU requests: memory: "600Mi" cpu: "400m“ # 0.4 vCPU Applied per Namespace apiVersion: v1 kind: ResourceQuota metadata: name: production spec: hard: requests.cpu: "1" requests.memory: 1Gi limits.cpu: "2" limits.memory: 2Gi ResourceQuota defined both, so Pod must define both Pod Resource Request
65. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Controlling scheduling Resource requirements Constraints • Taints Node-level • Tolerations Pod-level Topology filters
66. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Taints and Tolerations # Taint node $ kubectl taint nodes ip-10-0-32-12.us-west-2.compute.internal \ skynet=false:NoSchedule # Tolerations kind: Pod spec: tolerations: - key: skynet operator: Equal value: “false” effect: NoSchedule [...] Match taint to schedule onto tainted node
67. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Controlling scheduling Resource requirements Constraints • Taints Node-level • Tolerations Pod-level Affinity/Anti-Affinity Topology filters
68. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Affinity / Anti-Affinity • Control scheduling onto nodes ◦ Combine with Taints & Tolerations • Distribute Pods across cluster affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: "beta.kubernetes.io/instance-type" operator: In values: [“r4.large",“r4.xlarge"]
69. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. App Auto-Scaling
70. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. App Auto-Scaling # Install heapster ❯ kubectl create -f templates/heapster/heapster.yaml ❯ kubectl run webapp --image=trevorrobertsjr/webapp --requests=cpu=50m -- expose --port=8080 ❯ kubectl autoscale deployment webapp --cpu-percent=5 --min=1 --max=10 ❯ kubectl run -i --tty load-generator --image=busybox -- /bin/sh –c “while true; do wget -q -O- http://webapp.default.svc.cluster.local:8080; done“ ❯ kubectl get hpa –o wide ❯ kubectl delete hpa webapp ❯ kubectl delete deployment/load-generator deployment/webapp
71. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Monitoring and Operations
72. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes Dashboard • General purpose web-based UI for Kubernetes clusters • Manage applications running in the cluster • Manage the cluster
73. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes Dashboard
74. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes Dashboard
75. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Prometheus, Node exporter, and Grafana • Prometheus: • Open-source systems monitoring and alerting toolkit • Collects metrics from monitored targets by scraping metrics from HTTP endpoints • Dynamically scrape new targets by adding a ServiceMonitor • Grafana: • Open source, feature rich metrics dashboard and graph editor • Node exporter:
76. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Prometheus, Node exporter, and Grafana
77. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Prometheus installation cd cluster-monitoring kubectl apply -f templates/prometheus/prometheus-bundle.yaml kubectl rollout status deployment/prometheus-operator -n monitoring kubectl apply -f templates/prometheus/prometheus.yaml kubectl get po -l prometheus=prometheus -n monitoring kubectl port-forward $(kubectl get po -l prometheus=prometheus -n monitoring -o jsonpath="{.items[0].metadata.name}") 9090 -n monitoring open http://localhost:9090
78. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Grafana installation kubectl apply -f templates/prometheus/grafana-bundle.yaml kubectl rollout status deployment/grafana -n monitoring kubectl port-forward $(kubectl get pod -l app=grafana -o jsonpath="{.items[0].metadata.name}" -n monitoring) 3000 -n monitoring open http://localhost:3000/?orgId=1 kubectl delete -f templates/prometheus/prometheus-bundle.yaml
79. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lab 3
80. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hands-on! - Lab 3 1. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/02-path-working-with- clusters/205-cluster-autoscaling 2. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/02-path-working-with- clusters/201-cluster-monitoring
81. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CI/CD pipelines
82. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Jenkins – CI/CD with Kubernetes
83. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS CodePipeline – CI/CD with Kubernetes
84. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS CodePipeline – CI/CD with Kubernetes • Continuous Deployment to Kubernetes using AWS CodePipeline, AWS CodeCommit, AWS CodeBuild, Amazon ECR and AWS Lambda • CodeSuite - Continuous Deployment Reference Architecture for Kubernetes
85. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Deployment Tools • Spinnaker • Skaffold • Jenkins X • Argo • ... • kubectl Overview: https://engineering.opsgenie.com/cloud-native- continuous-integration-and-delivery-tools-for-kubernetes- e6ea34d308c
86. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Distributed Tracing
87. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS X-Ray for Kubernetes
88. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS X-Ray tracing
89. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS X-Ray segment info
90. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. X-Ray for Kubernetes AWS X-Ray X-Ray DaemonSet X-Ray trace k8s nodes running pods Service A Service B Client HTTP Requests AWS Console
91. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hands-on
92. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hands-on! - Lab 4 1. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/03-path-application- development/308-cicd-workflows/308-1-codesuite 2. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/03-path-application- development/305-app-tracing-with-jaeger-and-x- ray/x-ray
93. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Make sure to stop&delete resources!
94. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you! Please leave 1-Minute feedback!