Containing Chaos with Kubernetes

An Introduction to the whys and hows of Kubernetes.

Transcript

1. ‹#› @tpryan Terry Ryan Developer Advocate Containing Chaos with Kubernetes
2. ‹#› @tpryan Who are you?
3. ‹#› @tpryan 01 Introduction Why Kubernetes?
4. ‹#› @tpryan What problem are 
 we trying to solve?
5. ‹#› @tpryan # FRONTEND AND SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/
6. ‹#› @tpryan # BACKEND FROM ubuntu:12.04 ADD ./mysql-setup.sh /tmp/mysql-setup.sh RUN /bin/sh /tmp/mysql-setup.sh EXPOSE 3306 CMD ["/usr/sbin/mysqld"]
7. ‹#› @tpryan
8. ‹#› @tpryan # FRONTEND AND SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/ # JUST SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/
9. ‹#› @tpryan # FRONTEND AND SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/ # FRONTEND FROM nginx COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/
10. ‹#› @tpryan
11. ‹#› @tpryan # BACKEND FROM ubuntu:12.04 ADD ./mysql-setup.sh /tmp/mysql-setup.sh RUN /bin/sh /tmp/mysql-setup.sh EXPOSE 3306 CMD ["/usr/sbin/mysqld"] # BACKEND FROM ubuntu:12.04 ADD ./mysql-setup.sh /tmp/mysql-setup.sh RUN /bin/sh /tmp/mysql-setup.sh EXPOSE 3306 VOLUME ["/etc/mysql", "/var/lib/mysql"] CMD ["/usr/sbin/mysqld"]
12. ‹#› @tpryan
13. ‹#› @tpryan
14. ‹#› @tpryan
15. ‹#› @tpryan
16. ‹#› @tpryan
17. ‹#› @tpryan That’s a lot to manage.
18. ‹#› @tpryan 4 3 2
19. ‹#› @tpryan Kubernetes • Container Orchestration System • Open Source • Started by Google • Contributed to by others
20. ‹#› @tpryan 02 Concepts Philosophies
21. ‹#› @tpryan Cattle, not Pets
22. ‹#› @tpryan Cattle • Has a number • One is much like any other • Run as a group • If it gets ill, you make hamburgers Pet • Has a name • Is unique or rare • Personal Attention • If it gets ill… you make it better
23. ‹#› @tpryan Desired State
24. ‹#› @tpryan Build Script ./create_docker_images.sh ./launch_backend.sh x 1 ./launch_services.sh x 2 ./launch_frontend.sh x 3
25. ‹#› @tpryan Build Script ./create_docker_images.sh ./launch_backend.sh x 1 ./launch_services.sh x 2 ./launch_frontend.sh x 3
26. ‹#› @tpryan Desired State There should be: 3 Frontends 2 Services 1 Backend
27. ‹#› @tpryan Employees, not Children
28. ‹#› @tpryan Child • Go upstairs • Get undressed • Put on pajamas • Brush your teeth • Pick out 2 stories Employee • “We had a tough day, go home and get some sleep”
29. ‹#› @tpryan 03 Components What makes up Kubernetes?
30. ‹#› @tpryan Nodes • Machines that run Kubernetes • Containers will run on these • Can by hardware or virtual
31. ‹#› @tpryan Containers • Subatomic particles of Kubernetes • Dockerfiles just like you are used to.
32. ‹#› @tpryan Pods • Atomic component of Kubernetes • Made from one or more containers • Share • IP Address • Local Storage • Namespace • It’s okay to have just one container • Examples • Sidecar (Webserver + File sync) • Ambassador • Adaptor • Converting an all in one box
33. ‹#› @tpryan Pods apiVersion: v1 kind: Pod metadata: name: php labels: name: php spec: containers: - image: nginx-php-fpm:latest name: php ports: - containerPort: 80 name: http apiVersion: v1 kind: Pod metadata: name: php labels: name: php spec: containers: - image: nginx-php-fpm:latest name: php ports: - containerPort: 80 name: http
34. ‹#› @tpryan Controllers • Handle turning current state into desired state • Example • Replication Controllers Observe Diff Act
35. ‹#› @tpryan Controllers kind: "ReplicationController" apiVersion: "v1" id: fe-rc-1 metadata: name: "frontend-controller" labels: state: "serving" spec: replicas: 2 selector: app: "todotodo-fe" version: v1 template: metadata: labels: app: "todotodo-fe" version: v1 spec: volumes: null containers: - name: "php" image: "nginx-php-fpm:latest" ports: - containerPort: 80 protocol: "TCP" imagePullPolicy: "IfNotPresent" restartPolicy: "Always" dnsPolicy: "ClusterFirst" kind: "ReplicationController" apiVersion: "v1" id: fe-rc-1 metadata: name: "frontend-controller" labels: state: "serving" spec: replicas: 2 selector: app: "todotodo-fe" version: v1 template: metadata: labels: app: "todotodo-fe" version: v1 spec: volumes: null containers: - name: "php" image: "nginx-php-fpm:latest" ports: - containerPort: 80 protocol: "TCP" imagePullPolicy: "IfNotPresent" restartPolicy: "Always" dnsPolicy: "ClusterFirst"
36. ‹#› @tpryan Replica Set • Everything that Replication Controllers do • Can do set-based selector • Lot of the docs will refer to Replication Controllers but you can move to Replica sets.
37. ‹#› @tpryan Deployments • An improvement over previous rolling updates. • Allow for easy updates to application pieces.
38. ‹#› @tpryan apiVersion: extensions/v1beta1 kind: Deployment metadata: name: frontend-deployment spec: replicas: 2 strategy: type: RollingUpdate template: metadata: labels: app: todotodo-fe spec: containers: - name: php image: nginx-php-fpm:latest ports: - containerPort: 80 Deployments apiVersion: extensions/v1beta1 kind: Deployment metadata: name: frontend-deployment spec: replicas: 2 strategy: type: RollingUpdate template: metadata: labels: app: todotodo-fe spec: containers: - name: php image: nginx-php-fpm:latest ports: - containerPort: 80
39. ‹#› @tpryan Services • Defines an endpoint from which to access applications • Gets a virtual IP address • Can get a public load balancer • Used for exposing an application • Other Kubernetes clients • Non-Kubernetes clients 192.168.99.100
40. ‹#› @tpryan Services apiVersion: v1 kind: Service metadata: labels: name: frontend name: frontend spec: type: LoadBalancer ports: - port: 80 targetPort: 80 protocol: TCP selector: app: "todotodo-fe" apiVersion: v1 kind: Service metadata: labels: name: frontend name: frontend spec: type: LoadBalancer ports: - port: 80 targetPort: 80 protocol: TCP selector: app: "todotodo-fe"
41. ‹#› @tpryan Labels & Selectors • Metadata for Objects • Select sections of your infrastructure App Tier Env todo frontend stage App Tier Env todo frontend prod App Tier Env todo frontend dev App Tier Env todo frontend test App Tier Env todo api prod App Tier Env todo backend prod App Tier Env todo api stage App Tier Env todo api test App Tier Env todo api dev App Tier Env todo backend stage App Tier Env todo backend test App Tier Env todo backend dev
42. ‹#› @tpryan Labels & Selectors & Services App todo-fe apiVersion: extensions/v1beta1 kind: Deployment metadata: name: fe-deployment spec: replicas: 4 strategy: type: RollingUpdate template: metadata: labels: app: todo-fe spec: containers: - name: php image: app ports: - containerPort: 80 App todo-fe App todo-fe App todo-fe apiVersion: v1 kind: Service metadata: labels: name: frontend name: frontend spec: type: LoadBalancer ports: - port: 80 targetPort: 80 protocol: TCP selector: app: "todo-fe" 192.168.99.100 130.91.xxx.xxx Public Private
43. ‹#› @tpryan Networking • Pod IPs are routable • Docker default is private IP • Pods can reach each other without NAT • even across Kubernetes nodes
44. ‹#› @tpryan Sum up Service Replication Controller Replica Set Deployment Pod Container
45. ‹#› @tpryan Why?
46. ‹#› @tpryan
47. ‹#› @tpryan Demo: Kubernetes in Action
48. ‹#› @tpryan Before the demo
49. ‹#› @tpryan PHP & Apache Frontend & API Mysql Backend
50. ‹#› @tpryan
51. ‹#› @tpryan Demo: Kubernetes in Action
52. ‹#› @tpryan Demo: Larger Kubernetes install
53. ‹#› @tpryan Is this the right way to set up a database?
54. ‹#› @tpryan Not really
55. ‹#› @tpryan Sum up Service Replication Controller Replica Set Deployment Pod Container
56. ‹#› @tpryan Replica Set Service Stateful Set Service replicas: 1
57. ‹#› @tpryan apiVersion: apps/v1beta1 kind: StatefulSet metadata: name: mysql labels: name: mysql spec: serviceName: "mysql" replicas: 1 template: metadata: labels: name: mysql Stateful Set spec: terminationGracePeriodSeconds: 0 containers: - name: mysql image: "gcr.io/gke-test-tpryan/mysql-php" ports: - containerPort: 3306 name: mysql volumeMounts: - name: mysqlpet mountPath: /var/lib/mysql volumeClaimTemplates: - metadata: name: mysqlpet spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi apiVersion: apps/v1beta1 kind: StatefulSet metadata: name: mysql labels: name: mysql spec: serviceName: "mysql" replicas: 1 template: metadata: labels: name: mysql spec: terminationGracePeriodSeconds: 0 containers: - name: mysql image: "gcr.io/gke-test-tpryan/mysql-php" ports: - containerPort: 3306 name: mysql volumeMounts: - name: mysqlpet mountPath: /var/lib/mysql volumeClaimTemplates: - metadata: name: mysqlpet spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi
58. ‹#› @tpryan Why isn’t yours using Stateful Set
59. ‹#› @tpryan Not a huge difference for MySQL
60. ‹#› @tpryan We also talked about • Rolling Updates • Persistent Volumes
61. ‹#› @tpryan Secrets • Secrets interface for sensitive data • Can be mounted as files • Can be imported directly to ENV
62. ‹#› @tpryan apiVersion: v1 kind: Secret metadata: name: mysecret type: Opaque data: password: MWYyZDFlMmU2N2Rm username: YWRtaW4= Secrets apiVersion: v1 kind: Secret metadata: name: mysecret type: Opaque data: password: MWYyZDFlMmU2N2Rm username: YWRtaW4=
63. ‹#› @tpryan containers: - name: php image: gcr.io/gke-test-tpryan/php ports: - containerPort: 80 env: - name: USERNAME valueFrom: secretKeyRef: name: mysecret key: username - name: PASSWORD valueFrom: secretKeyRef: name: mysecret key: password Secrets containers: - name: php image: gcr.io/gke-test-tpryan/php ports: - containerPort: 80 env: - name: USERNAME valueFrom: secretKeyRef: name: mysecret key: username - name: PASSWORD valueFrom: secretKeyRef: name: mysecret key: password
64. ‹#› @tpryan Horizontal Autoscaler • Upscales or downscales as necessary • Processor utilization • Custom Application metrics • Allows for pods to autoscale but not nodes.
65. ‹#› @tpryan apiVersion: extensions/v1beta1 kind: HorizontalPodAutoscaler metadata: name: frontend-deployment spec: cpuUtilization: targetPercentage: 80 maxReplicas: 5 minReplicas: 1 scaleRef: apiVersion: extensions/v1beta1 kind: Deployment name: frontend-deployment subresource: scale Horizontal Autoscaler apiVersion: extensions/v1beta1 kind: HorizontalPodAutoscaler metadata: name: frontend-deployment spec: cpuUtilization: targetPercentage: 80 maxReplicas: 5 minReplicas: 1 scaleRef: apiVersion: extensions/v1beta1 kind: Deployment name: frontend-deployment subresource: scale
66. ‹#› @tpryan There’s More • Logging • Monitoring • Events • Web Interface • Configmaps • Jobs • Ubernetes
67. ‹#› @tpryan 04 Comparisons What about Docker Compose, or Swarm, or …
68. ‹#› @tpryan Kubernetes Docker Swarm Docker Compose Docker Machine Launch Container hosts in several clouds Cluster of Container Hosts Replication Orchestration Scheduling Routable Network Scheduled Jobs Stateful Set Autoscaling Secrets Config Maps Multiple containers on same localhost Manage Remote Container Hosts Docker
69. ‹#› @tpryan But
70. ‹#› @tpryan Docker’s logos are much cooler than ours.
71. ‹#› @tpryan Kubernetes • Management software for containers • Has strong opinions • Service Discovery • Logging • Can run on top of Mesos Mesos • Multi machine kernel • Turns datacenter (or all installed machines) into a single logical system • Can do containers • Can do other distributed jobs
72. ‹#› @tpryan 05 Container Engine Hosted Kubernetes
73. ‹#› @tpryan I’ve mostly talked about developing on Kubernetes
74. ‹#› @tpryan Setting up a cluster
75. ‹#› @tpryan Setting up a cluster • Choose an infrastructure: • Google Cloud Platform, AWS, Azure, Rackspace, on-premises, …
76. ‹#› @tpryan Setting up a cluster • Choose an infrastructure: • Google Cloud Platform, AWS, Azure, Rackspace, on-premises, … • Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ...
77. ‹#› @tpryan Setting up a cluster • Choose an infrastructure: • Google Cloud Platform, AWS, Azure, Rackspace, on-premises, … • Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ... • Provision machines: • Boot VMs, install and run kube components, ...
78. ‹#› @tpryan Setting up a cluster • Choose an infrastructure: • Google Cloud Platform, AWS, Azure, Rackspace, on-premises, … • Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ... • Provision machines: • Boot VMs, install and run kube components, ... • Configure networking: • IP ranges for Pods, Services, SDN, ...
79. ‹#› @tpryan Setting up a cluster • Choose an infrastructure: • Google Cloud Platform, AWS, Azure, Rackspace, on-premises, … • Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ... • Provision machines: • Boot VMs, install and run kube components, ... • Configure networking: • IP ranges for Pods, Services, SDN, ... • Start cluster services: • DNS, logging, monitoring, ...
80. ‹#› @tpryan Setting up a cluster • Choose an infrastructure: • Google Cloud Platform, AWS, Azure, Rackspace, on-premises, … • Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ... • Provision machines: • Boot VMs, install and run kube components, ... • Configure networking: • IP ranges for Pods, Services, SDN, ... • Start cluster services: • DNS, logging, monitoring, ... • Manage nodes: • kernel upgrades, OS updates, hardware failures...
81. ‹#› @tpryan Or…
82. ‹#› @tpryan
83. ‹#› @tpryan Container Engine • Hosted Kubernetes • A few smart defaults set • Allow for dipping your feet in • Allows for node autoscaling
84. ‹#› @tpryan Container Registry • Hosted • Private • Can be used with GKE or not
85. ‹#› @tpryan 06 Conclusions Bring it home
86. ‹#› @tpryan Kubernetes is Open Source We want your help! • http://kubernetes.io • https://github.com/kubernetes/kubernetes • irc.freenode.net #google-containers • @kubernetesio
87. ‹#› @tpryan Roadmap Kubernetes 1.5 Released: December 2016 • Stateful State beta • Pod Disruption Budget beta • Federated Kubernetes CLI • Windows Containers alpha http://blog.kubernetes.io/2016/12/kubernetes-1.5-supporting-production-workloads.html
88. ‹#› @tpryan Roadmap Kubernetes 1.6 Target: March 2017 https://github.com/kubernetes/kubernetes/milestones/
89. ‹#› @tpryan If you like Kubernetes, but wonder “What the hell am I getting into here?”
90. ‹#› @tpryan Container Engine can make dipping your toes in a little easier.
91. ‹#› @tpryan There’s also Minikube https://github.com/kubernetes/minikube
92. Google has been developing and using containers to manage our applications for over 10 years.
93. ‹#› @tpryan Everything at Google runs on Containers: • Gmail, Web Search, Maps, ... • MapReduce, batch, ... • GFS, Colossus, ... • Even Google’s Cloud Platform: VMs run in containers! We launch 2 Billion Containers a week
94. ‹#› @tpryan We think containers are the way to manage scale.
95. ‹#› @tpryan You should carefully consider whether running everything on containers is right for you.
96. ‹#› @tpryan You should run everything on containers.
97. ‹#› @tpryan You should carefully consider whether running everything on containers is right for you.
98. ‹#› @tpryan Thank You terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-chaos