AWS Loft Berlin 2018 - EKS Workshop

© 2018, Amazon Web Services, Inc. or its Affiliates. All
rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EKS - Kubernetes on AWS Christoph Kassen Solutions Architect Jonas Wagner Solutions Architect

rights reserved. What can you expect?

rights reserved.

rights reserved. Open source container management platform Helps you run containers at scale Gives you primitives for building modern applications What is Kubernetes?

rights reserved. W h y d e v e l o p e r s l o v e K u b e r n e t e s Vibrant and growing community of users and contributors

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Open Source Kubernetes Community Kubernetes https://github.com/kubernetes/kuber netes CNI plugin https://github.com/aws/amazon-vpc- cni-k8s Heptio AWS Authenticator https://github.com/heptio/authentic ator Virtual Kubelet https://github.com/virtual- kubelet/virtual-kubelet/ SIG AWS https://github.com/kubernetes/com munity/tree/master/sig-aws Cloud Provider Working Group https://github.com/kubernetes/com munity/tree/master/wg-cloudprovider External-DNS https://github.com/kubernetes- incubator/external-dns CoreOS ALB Ingress https://github.com/coreos/alb- ingress-controller CODE REVIEWS FIXING BUGS IMPLEMENTING NEW FEATURES

rights reserved.

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC2 Worker Nodes EKS Control Plane Customer VPC EKS VPC Network Load Balancer ENI API Access Kubectl Exec/Logs TLS Static IPs ENI Attachment Autoscaling Group EKS Architecture

rights reserved.

rights reserved. Metrics Nodes Node exporter Pod/Container Kube-state-metrics cAdvisor Application /metrics JMX Cluster-wide Aggregator Prometheus, Heapster Visualizer Grafana, Kibana, Dashboard Data Model InfluxDB, Graphite Alerting AlertManager, Kapacitor

rights reserved. IAM authentication with Kubernetes

rights reserved. ~/.kube/config (with IAM) apiVersion: v1 clusters: - cluster: certificate-authority-data: REDACTED server: https://1234.sk1.us-west-2.eks.amazonaws.com name: eks contexts: - context: cluster: eks user: eks name: eks current-context: eks kind: Config users: - name: eks user: exec: apiVersion: client.authentication.k8s.io/v1alpha1 command: heptio-authenticator-aws args: - token - -i - eks Config file is no longer user-specific J

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Access and Authentication IAM ROLE User X IAM ROLE Service Account Y kubectl → K8s APIs → CRUD Operations on K8s aws-cli → EKS Service APIs → CRUD Operations on Infra K8s Master Nodes K8s Master Nodes K8s Master Nodes API Server Controller Mgr kubelet etcd Cloud Controller Mgr. Scheduler Authentication Webhook Tokens Authorization RBAC Mode Admission Control NamespaceLifecyle,LimitRanger ServiceAccount,DefaultStorageClass, ResourceQuota AWS STS client side Heptio-aws-authenticator server side

rights reserved.

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Worker provisioning k u b e c t l A W S A u t h c o n f i g m a p & R B A C W o r k e r s R o l e R o l e config map

rights reserved.

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Native VPC networking with CNI plugin Pods have the same VPC address inside the pod as on the VPC Simple, secure networking Open source and on Github … { }

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Networking with CNI plugin 172.16.0.0/16 User X Service Account Y Kubectl K8s Node 2 K8s Node 1 kubelet kube-proxy kubelet kube-proxy VPC Subnet per AZ 172.16.0.1/24 ENI ENI Primary Private IP: 172.16.1.118 Secondary IPs: 172.16.1.147, 172.16.1.224…. Service: Front end POD 2 POD 3 eth0 Service: Back end POD 1 POD 4 eth0 ec2.associateaddress() L3 RouteTable veth0 B veth0 A eth0 172.16.1.147/32 eth0 172.16.1.224/32 CNI K8s Master Nodes K8s Master Nodes K8s Master Nodes API Server Controller Manager kubelet etcd Scheduler kube-proxy Cloud Controller Mgr.

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DNS, Services and ELB 172.16.0.0/16 User X Service Account Y K8s Node 2 K8s Node 1 kubelet kube-proxy kubelet kube-proxy VPC Subnet per AZ - 172.16.0.1/24 ENI ENI Service: Front end POD 2 POD 3 Service: Back end POD 1 POD 4 CNI K8s Master Nodes K8s Master Nodes K8s Master Nodes API Server Controller Manager kubelet etcd Scheduler kube-proxy Cloud Controller Mgr. DNS kubedns dnsmasq healthz DNS Service – Static IP POD 2 POD 2 kind: Service type: LoadBalancer

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kubernetes Network Policies enforce network security rules Calico is the leading implementation of the network policy API Open source, active development (>100 contributors) Commercial support available from Tigera

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. S T A G E S E P A R A T I O N “ T E N A N T ” S E P A R A T I O N F I N E - G R A I N E D F I R E W A L L S C O M P L I A N C E Namespaces – without network policy, they are not network isolated Reduce attack surface within microservice-based applications Isolate dev, test, and prod E.g., PCI, HIPAA

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Network Policy kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: web-allow-prod spec: podSelector: matchLabels: app: web ingress: - from: - namespaceSelector: matchLabels: purpose: production Select affected Pods Define traffic that is allowed

rights reserved. Hands-on! – Lab 1 AWS Workshop for Kubernetes 1. Create an EKS cluster 1. Follow the instructions at https://eksworkshop.com (beginner) 2. Use eksctl https://eksctl.io (advanced) 3. Follow the EKS docs https://docs.aws.amazon.com/eks/latest/userguide/getting- started.html (the “hard“ way)

rights reserved. Additional resources https://github.com/ramitsurana/awesome-kubernetes https://discuss.kubernetes.io/ TGIK Playlist: https://www.youtube.com/playlist?list=PLvmPtYZtoXOEN HJiAQc6HmV2jmuexKfrJ https://aws.amazon.com/blogs/compute/tag/containers/

rights reserved. Kubernetes concepts port 8080 port 8080 ReplicaSet #Pods—2 label selector: v1 ReplicaSet #Pods—1 label selector: v2 Node Docker Pod Containers

rights reserved. Kubernetes concepts URI: /svc1/* URI: /svc2/* DaemonSet Daemon pod

rights reserved. Pod definition example apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: selector: matchLabels: app: nginx replicas: 2 # tells deployment to run 2 pods matching the template template: # create pods using pod definition in this template metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80

rights reserved. Deployments ❯ kubectl create -f nginx.yaml --record ❯ kubectl rollout history deployment/nginx ❯ kubectl expose deployment/nginx --port=80 --target-port=80 -- name=nginx --type=LoadBalancer ❯ kubectl describe svc nginx ❯ curl http://xyz.us-west-2.elb.amazonaws.com

rights reserved. Deployments - Helm • Package management for k8s • Chart - collection of files • Description of k8s resources • Flexible templating • Tiller • Installed on k8s cluster • Client • Cli tool to install/update Charts

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Using helm ❯ helm search nginx NAME CHART VERSION APP VERSION DESCRIPTION stable/nginx-ingress 0.19.2 0.14.0 An nginx Ingress controller that uses ConfigMap... stable/nginx-lego 0.3.1 Chart for nginx-ingress-controller and kube-lego stable/gcloud-endpoints 0.1.0 Develop, deploy, protect and monitor your APIs ... ❯ helm install stable/nginx-ingress --name nginx-ingress --set rbac.create=true [displays README + information about deployment] ❯ helm list NAME REVISION UPDATED STATUS CHART NAMESPACE Nginx-ingress 1 Mon May 21 18:30:17 2018 DEPLOYED nginx-ingress-0.19.2 default

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hosting Helm repositories • Anywhere that serves HTTP can host a helm repo • Host private Helm Repo with Chartmuseum https://github.com/kubernetes-helm/chartmuseum • There’s also a handy plugin for S3! • This means IAM Role = auth for your repo J • https://github.com/hypnoglow/helm-s3

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Deploying Helm on EKS Helm 2.9+ works with EKS out of the box Helm needs RBAC permissions ❯ kubectl -n kube-system create serviceaccount tiller ❯ kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller ❯ helm init --service-account=tiller

rights reserved. Understanding deployments # Columnar output ❯ kubectl get services # List all services in the namespace ❯ kubectl get pods --all-namespaces # List all pods in all namespaces ❯ kubectl get pods -o wide # List all pods in the namespace, with details ❯ kubectl get rc <rc-name> # Get a replication controller # Verbose output ❯ kubectl describe nodes <node-name> ❯ kubectl describe pods <pod-name> ❯ kubectl describe pods/<pod-name> # Equivalent to previous ❯ kubectl describe pods <rc-name> # Lists pods created by<rc-name>using common prefix # List Services Sorted by Name ❯ kubectl get services --sort-by=.metadata.name # Get ExternalIPs of all nodes ❯ kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=="ExternalIP")].address}'

rights reserved. Interacting with Pods ❯ kubectl logs <pod-name> # dump pod logs (stdout) ❯ kubectl logs –f <pod-name> # stream pod logs (stdout) ❯ kubectl run –i --tty busybox—image=busybox -- sh # Run pod as interactive shell ❯ kubectl attach <podname> -i # Attach to Running Container ❯ kubectl port-forward <podname> <local>:<remote> # Forward port of Pod to localhost ❯ kubectl port-forward <servicename><port> # Forward port to service ❯ kubectl exec <pod-name> -- ls / # Run command in existing pod (1 container case) ❯ kubectl exec <pod-name> -c <container-name> -- ls /

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Rolling Update apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app labels: app: my-app spec: replicas: 10 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 # Numeric or percentage based value maxUnavailable: 0 [...]

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Blue / Green Deployment apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app-blue labels: app: my-app spec: replicas: 3 template: metadata: labels: app: my-app version: blue [...] Blue apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app-green labels: app: my-app spec: replicas: 3 template: metadata: labels: app: my-app version: green [...] Green

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Blue / Green Deployment Blue kind: Service metadata: name: my-app labels: app: my-app spec: type: LoadBalancer ports: - name: http port: 80 targetPort: http selector: app: my-app version: blue kind: Service metadata: name: my-app labels: app: my-app spec: type: LoadBalancer ports: - name: http port: 80 targetPort: http selector: app: my-app version: green Green kubectl patch service my-app -p '{"spec":{"selector":{"version":"green"}}}'

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Canary Deployment Production apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app-prod labels: app: my-app spec: replicas: 9 template: metadata: labels: app: my-app spec: containers: - name: my-app image: images/container:v1 [...] apiVersion: extensions/v1beta1 kind: Deployment metadata: name: my-app-canary labels: app: my-app spec: replicas: 1 template: metadata: labels: app: my-app spec: containers: - name: my-app image: images/container:v2 [...] More examples at https://container-solutions.com/kubernetes-deployment-strategies/ Canary

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Nginx Pods EC2 instances kube-proxy :32002 nginx-service :32001 Internet 10001:8080 10002:8080 10003:8080 Request to NGINX Pod {NLB}:443 NLB NLB Forwards to the node {node:32001} Service Type – LoadBalancer (NLB) k8s service ClusterIP receives request kube-proxy load balances to pods

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Network Load Balancer apiVersion: v1 kind: Service metadata: name: nginx namespace: default labels: app: nginx annotations: service.beta.kubernetes.io/aws-load-balancer-type: "nlb" spec: type: LoadBalancer externalTrafficPolicy: Local ports: - name: http port: 80 protocol: TCP targetPort: 80 selector: app: nginx More options: • Draining • Logging • SSL Certs • Tagging • Security groups • Health checks https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws /aws.go

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Nginx Pods EC2 instances kube-proxy :32001 nginx-service :32003 Internet Request to NGINX Pod {ALB}:443 ALB ALB Routes based on the path. /api /home 10002:8080 Webapp Pods 10002:8080 Installation: https://github.com/pahud/eks-alb-ingress Ingress Type – CoreOS ALB Ingress kube-proxy :32002 webapp-service :32004 Load Balances to pods Proxies request to the k8s service ClusterIP

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automatic Route53 DNS creation for services apiVersion: v1 kind: Service metadata: name: nginx annotations: # Uses https://github.com/kubernetes-incubator/external-dns external-dns.alpha.kubernetes.io/hostname: nginx.highlyavailable.systems. spec: type: LoadBalancer ports: - port: 80 name: http targetPort: 80 selector: app: nginx

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automatic Route53 DNS creation for Ingress apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: nginx.highlyavailable.systems http: paths: - backend: serviceName: nginx servicePort: 80

rights reserved. Lab 2 1. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/01-path-basics/103-kubernetes- concepts 2. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/03-path-application- development/303-app-update 3. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/04-path-security-and- networking/405-ingress-controllers#alb-ingress-controller 4. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/03-path-application- development/307-app-management-with-helm

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Limit resource usage Container A Container B limit request 900m 600m limit request 800m 400m ⎲ ⎳ Pod CPU and memory resources

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Resource Quotas apiVersion: v1 kind: Pod metadata: name: production spec: containers: - name: nginx-pod image: nginx resources: limits: memory: "800Mi" cpu: "800m" # 0.8 vCPU requests: memory: "600Mi" cpu: "400m“ # 0.4 vCPU Applied per Namespace apiVersion: v1 kind: ResourceQuota metadata: name: production spec: hard: requests.cpu: "1" requests.memory: 1Gi limits.cpu: "2" limits.memory: 2Gi ResourceQuota defined both, so Pod must define both Pod Resource Request

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Controlling scheduling Resource requirements Constraints • Taints Node-level • Tolerations Pod-level Topology filters

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Taints and Tolerations # Taint node $ kubectl taint nodes ip-10-0-32-12.us-west-2.compute.internal \ skynet=false:NoSchedule # Tolerations kind: Pod spec: tolerations: - key: skynet operator: Equal value: “false” effect: NoSchedule [...] Match taint to schedule onto tainted node

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Controlling scheduling Resource requirements Constraints • Taints Node-level • Tolerations Pod-level Affinity/Anti-Affinity Topology filters

rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Affinity / Anti-Affinity • Control scheduling onto nodes ◦ Combine with Taints & Tolerations • Distribute Pods across cluster affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: "beta.kubernetes.io/instance-type" operator: In values: [“r4.large",“r4.xlarge"]

rights reserved. App Auto-Scaling # Install heapster ❯ kubectl create -f templates/heapster/heapster.yaml ❯ kubectl run webapp --image=trevorrobertsjr/webapp --requests=cpu=50m -- expose --port=8080 ❯ kubectl autoscale deployment webapp --cpu-percent=5 --min=1 --max=10 ❯ kubectl run -i --tty load-generator --image=busybox -- /bin/sh –c “while true; do wget -q -O- http://webapp.default.svc.cluster.local:8080; done“ ❯ kubectl get hpa –o wide ❯ kubectl delete hpa webapp ❯ kubectl delete deployment/load-generator deployment/webapp

rights reserved. Kubernetes Dashboard • General purpose web-based UI for Kubernetes clusters • Manage applications running in the cluster • Manage the cluster

rights reserved. Kubernetes Dashboard

rights reserved. Prometheus, Node exporter, and Grafana • Prometheus: • Open-source systems monitoring and alerting toolkit • Collects metrics from monitored targets by scraping metrics from HTTP endpoints • Dynamically scrape new targets by adding a ServiceMonitor • Grafana: • Open source, feature rich metrics dashboard and graph editor • Node exporter:

rights reserved. Prometheus, Node exporter, and Grafana

rights reserved. Prometheus installation cd cluster-monitoring kubectl apply -f templates/prometheus/prometheus-bundle.yaml kubectl rollout status deployment/prometheus-operator -n monitoring kubectl apply -f templates/prometheus/prometheus.yaml kubectl get po -l prometheus=prometheus -n monitoring kubectl port-forward $(kubectl get po -l prometheus=prometheus -n monitoring -o jsonpath="{.items[0].metadata.name}") 9090 -n monitoring open http://localhost:9090

rights reserved. Grafana installation kubectl apply -f templates/prometheus/grafana-bundle.yaml kubectl rollout status deployment/grafana -n monitoring kubectl port-forward $(kubectl get pod -l app=grafana -o jsonpath="{.items[0].metadata.name}" -n monitoring) 3000 -n monitoring open http://localhost:3000/?orgId=1 kubectl delete -f templates/prometheus/prometheus-bundle.yaml

rights reserved. Hands-on! - Lab 3 1. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/02-path-working-with- clusters/205-cluster-autoscaling 2. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/02-path-working-with- clusters/201-cluster-monitoring

rights reserved. Jenkins – CI/CD with Kubernetes

rights reserved. AWS CodePipeline – CI/CD with Kubernetes

rights reserved. AWS CodePipeline – CI/CD with Kubernetes • Continuous Deployment to Kubernetes using AWS CodePipeline, AWS CodeCommit, AWS CodeBuild, Amazon ECR and AWS Lambda • CodeSuite - Continuous Deployment Reference Architecture for Kubernetes

rights reserved. Deployment Tools • Spinnaker • Skaffold • Jenkins X • Argo • ... • kubectl Overview: https://engineering.opsgenie.com/cloud-native- continuous-integration-and-delivery-tools-for-kubernetes- e6ea34d308c

rights reserved. AWS X-Ray for Kubernetes

rights reserved. AWS X-Ray tracing

rights reserved. AWS X-Ray segment info

rights reserved. X-Ray for Kubernetes AWS X-Ray X-Ray DaemonSet X-Ray trace k8s nodes running pods Service A Service B Client HTTP Requests AWS Console

rights reserved. Hands-on! - Lab 4 1. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/03-path-application- development/308-cicd-workflows/308-1-codesuite 2. https://github.com/aws-samples/aws-workshop-for- kubernetes/tree/master/03-path-application- development/305-app-tracing-with-jaeger-and-x- ray/x-ray

AWS Loft Berlin 2018 - EKS Workshop

AWS Loft Berlin 2018 - EKS Workshop

More Decks by Christoph Kassen

Other Decks in Technology

Featured

Transcript