cluster management, cognitive shifts, &
 kubernetes seattle.rb @juliaferraioli, kubernautical advocate

cluster management with Borg

cluster management 
 
 with Borg the system we internally call

@juliaferraioli Image by Connie Zhou

@juliaferraioli developer view

@juliaferraioli job hello_world = { developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks } developer view

@juliaferraioli job hello_world = { runtime = { cell = 'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks } 10000 developer view

@juliaferraioli Image by Connie Zhou Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world!

@juliaferraioli Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Image by Connie Zhou Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world!

@juliaferraioli developer view

@juliaferraioli developer view

@juliaferraioli task-eviction rates and causes failures

@juliaferraioli advanced bin- packing algorithms experimental placement of production VM workload, July 2014 one
 machine efficiency

@juliaferraioli advanced bin- packing algorithms experimental placement of production VM workload, July 2014 available resources one
 machine efficiency

@juliaferraioli advanced bin- packing algorithms experimental placement of production VM workload, July 2014 stranded resources available resources one
 machine efficiency

@juliaferraioli Borg paper: http://goo.gl/1C4nuo Images by Connie Zhou observations: 1. resiliency is achieved only by ruthless attention to detail a. ubiquitous software fault tolerance b. persistent, declarative specs
 2. we get efficiency by: a. sharing resources b. reclaiming unused allocations 1. containers make users more productive

@juliaferraioli app libs kernel libs app app kernel app libs libs kernel kernel not so long ago: virtual machines

@juliaferraioli libs app kernel libs app libs app libs app the new kid in town: containers

A fundamentally different way of managing applications

enter kubernetes, stage center

@juliaferraioli κυβερνήτης: “Helmsman”; root of “Governor” and “Cybernetic” manage applications, not machines kubernetes (or k8s)

demo, part 1

@juliaferraioli

@juliaferraioli demo overview Server Debian Docker Engine PHP libs Redis libs

@juliaferraioli A loop that drives current state towards desired state A small group of tightly coupled containers Pod Replication Controller Service A set of running pods that work together Arbitrary metadata to organize components Labels kubernetes cluster

@juliaferraioli Dashboard show: type = FE Object Pod frontend Object type = FE type = FE ● metadata with semantic meaning ● membership identifier ● the only grouping mechanism behavior benefits ➔ allow for intent of many users (e.g. dashboards) ➔ build higher level systems ➔ queryable by selectors labels

@juliaferraioli Dashboard show: type = FE Object Pod frontend Pod frontend Object Object Dashboard show: version = v2 type = FE version = v2 type = FE type = FE version = v2 ● metadata with semantic meaning ● membership identifier ● the only grouping mechanism behavior benefits ➔ allow for intent of many users (e.g. dashboards) ➔ build higher level systems ➔ queryable by selectors labels

@juliaferraioli A loop that drives current state towards desired state A set of running pods that work together Replication Controller Service Arbitrary metadata to organize components Labels A small group of tightly coupled containers Pod kubernetes cluster

@juliaferraioli small group of containers & volumes containers are tightly coupled the atom of cluster scheduling shared namespace • Shared network IP and port namespace ephemeral • can die and be replaced Pod Site generator Web Server Volume Consumers Content Manager pods

@juliaferraioli pods template:
 metadata:
 labels:
 name: frontend
 spec:
 containers:
 - name: php-redis
 image: kubernetes/example-guestbook-php-redis:v2
 ports:
 - containerPort: 80

@juliaferraioli A set of running pods that work together Service Arbitrary metadata to organize components Labels A small group of tightly coupled containers A loop that drives current state towards desired state Pod Replication Controller kubernetes cluster

@juliaferraioli Replication Controller Pod Pod frontend Pod frontend Pod Pod Replication Controller #pods = 1 version = v2 show: version = v2 version= v1 version = v1 version = v2 Replication Controller #pods = 2 version = v1 show: version = v2 Behavior Benefits ● keeps pods running ● gives direct control of pod #s ● grouped by label selector ➔ recreates pods, maintains desired state ➔ fine-grained control for scaling ➔ standard grouping semantics replication controllers

@juliaferraioli Replication Controller Replication Controller - Name = “frontend” - Selector = {“Name”: “frontend”} - PodTemplate = { ... } - NumReplicas = 3 API Server 2 Start 1 more OK 3 How many? How many? canonical example of control loops have one job: ensure N copies of a pod replicated pods are fungible replication controllers

@juliaferraioli replication controllers kind: ReplicationController
 metadata:
 name: frontend
 labels:
 name: frontend
 spec:
 replicas: 3
 selector:
 name: frontend
 template: $ kubectl create -f frontend-controller.yaml

@juliaferraioli kubernetes cluster Arbitrary metadata to organize components Labels A small group of tightly coupled containers A loop that drives current state towards desired state A set of running pods that work together Pod Replication Controller Service

@juliaferraioli Portal (VIP) Client Pod Container Container Container Container Pod Container Container Container Container Pod Container Container Container Container a group of pods that act as one == Service defines access policy gets a stable virtual IP and port VIP is captured by kube-proxy services

@juliaferraioli Service Label selectors: version = 1.0 type = Frontend Service Label selectors: version = v1 type = FE Replication Controller Pod Pod frontend Pod version= v1 version = v1 Replication Controller #pods = 2 show: version = v2 type = FE type = FE VIP services

@juliaferraioli services kind: Service
 metadata:
 name: frontend
 labels:
 name: frontend
 spec:
 type: LoadBalancer
 ports:
 - port: 80
 selector:
 name: frontend $ kubectl create -f frontend-service.yaml

...in technicolor! what’s doing what...

@juliaferraioli Master APIs Scheduling REST (pods, services, controllers) AuthN Scheduler Replication Controller Node3 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container Node3 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container Node1 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container $ kubectl proxy --www=k8s-visualizer/ visualizing kubernetes

demo, part 2

@juliaferraioli

@juliaferraioli open sourced in June, 2014 Google launched Google Container Engine (GKE) • https://cloud.google.com/container-engine/ roadmap: • https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/roadmap.md where we are today http://www.kuberneteslaunch.com/

@juliaferraioli find us online… http://kubernetes.io https://github.com/GoogleCloudPlatform/kubernetes irc.freenode.net #google-containers @kubernetesio

@juliaferraioli thanks to... Brian Dorsey - Developer Advocate, Google Aja Hammerly - Developer Advocate, Google Amy Unruh - Developer Programs Engineer, Google Mandy Waite - Developer Advocate, Google John Wilkes - Principal Engineer, Google

https://flic.kr/p/6jk7ZL