Share
Embed
cluster management, cognitive shifts, & kubernetes
How did we get into the cluster management game? This briefly covers some of the lessons that we have learned over the years from the system internally known as Borg. Then, we step through an application running on kubernetes and see how those lessons are present in what kubernetes does and how it's designed.
juliaferraioli
July 07, 2015
Transcript
56 slides
-
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks } developer view -
@juliaferraioli job hello_world = { runtime = { cell =
'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks } 10000 developer view -
@juliaferraioli Image by Connie Zhou Hello world! Hello world! Hello
world! Hello world! Hello world! Hello world! Hello world! Hello world! -
@juliaferraioli Hello world! Hello world! Hello world! Hello world! Hello
world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Image by Connie Zhou Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! -
@juliaferraioli advanced bin- packing algorithms experimental placement of production VM
workload, July 2014 one machine efficiency -
@juliaferraioli advanced bin- packing algorithms experimental placement of production VM
workload, July 2014 available resources one machine efficiency -
@juliaferraioli advanced bin- packing algorithms experimental placement of production VM
workload, July 2014 stranded resources available resources one machine efficiency -
@juliaferraioli Borg paper: http://goo.gl/1C4nuo Images by Connie Zhou observations: 1.
resiliency is achieved only by ruthless attention to detail a. ubiquitous software fault tolerance b. persistent, declarative specs 2. we get efficiency by: a. sharing resources b. reclaiming unused allocations 1. containers make users more productive -
@juliaferraioli app libs kernel libs app app kernel app libs
libs kernel kernel not so long ago: virtual machines -
@juliaferraioli libs app kernel libs app libs app libs app
the new kid in town: containers -
@juliaferraioli κυβερνήτης: “Helmsman”; root of “Governor” and “Cybernetic” manage applications,
not machines kubernetes (or k8s) -
@juliaferraioli A loop that drives current state towards desired state
A small group of tightly coupled containers Pod Replication Controller Service A set of running pods that work together Arbitrary metadata to organize components Labels kubernetes cluster -
@juliaferraioli Dashboard show: type = FE Object Pod frontend Object
type = FE type = FE • metadata with semantic meaning • membership identifier • the only grouping mechanism behavior benefits ➔ allow for intent of many users (e.g. dashboards) ➔ build higher level systems ➔ queryable by selectors labels -
@juliaferraioli Dashboard show: type = FE Object Pod frontend Pod
frontend Object Object Dashboard show: version = v2 type = FE version = v2 type = FE type = FE version = v2 • metadata with semantic meaning • membership identifier • the only grouping mechanism behavior benefits ➔ allow for intent of many users (e.g. dashboards) ➔ build higher level systems ➔ queryable by selectors labels -
@juliaferraioli A loop that drives current state towards desired state
A set of running pods that work together Replication Controller Service Arbitrary metadata to organize components Labels A small group of tightly coupled containers Pod kubernetes cluster -
@juliaferraioli small group of containers & volumes containers are tightly
coupled the atom of cluster scheduling shared namespace • Shared network IP and port namespace ephemeral • can die and be replaced Pod Site generator Web Server Volume Consumers Content Manager pods -
@juliaferraioli pods template: metadata: labels: name: frontend spec: containers: -
name: php-redis image: kubernetes/example-guestbook-php-redis:v2 ports: - containerPort: 80 -
@juliaferraioli A set of running pods that work together Service
Arbitrary metadata to organize components Labels A small group of tightly coupled containers A loop that drives current state towards desired state Pod Replication Controller kubernetes cluster -
@juliaferraioli Replication Controller Pod Pod frontend Pod frontend Pod Pod
Replication Controller #pods = 1 version = v2 show: version = v2 version= v1 version = v1 version = v2 Replication Controller #pods = 2 version = v1 show: version = v2 Behavior Benefits • keeps pods running • gives direct control of pod #s • grouped by label selector ➔ recreates pods, maintains desired state ➔ fine-grained control for scaling ➔ standard grouping semantics replication controllers -
@juliaferraioli Replication Controller Replication Controller - Name = “frontend” -
Selector = {“Name”: “frontend”} - PodTemplate = { ... } - NumReplicas = 3 API Server 2 Start 1 more OK 3 How many? How many? canonical example of control loops have one job: ensure N copies of a pod replicated pods are fungible replication controllers -
@juliaferraioli replication controllers kind: ReplicationController metadata: name: frontend labels: name:
frontend spec: replicas: 3 selector: name: frontend template: <snip!> $ kubectl create -f frontend-controller.yaml -
@juliaferraioli kubernetes cluster Arbitrary metadata to organize components Labels A
small group of tightly coupled containers A loop that drives current state towards desired state A set of running pods that work together Pod Replication Controller Service -
@juliaferraioli Portal (VIP) Client Pod Container Container Container Container Pod
Container Container Container Container Pod Container Container Container Container a group of pods that act as one == Service defines access policy gets a stable virtual IP and port VIP is captured by kube-proxy services -
@juliaferraioli Service Label selectors: version = 1.0 type = Frontend
Service Label selectors: version = v1 type = FE Replication Controller Pod Pod frontend Pod version= v1 version = v1 Replication Controller #pods = 2 show: version = v2 type = FE type = FE VIP services -
@juliaferraioli services kind: Service metadata: name: frontend labels: name: frontend
spec: type: LoadBalancer ports: - port: 80 selector: name: frontend $ kubectl create -f frontend-service.yaml -
@juliaferraioli Master APIs Scheduling REST (pods, services, controllers) AuthN Scheduler
Replication Controller Node3 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container Node3 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container Node1 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container $ kubectl proxy --www=k8s-visualizer/ visualizing kubernetes -
@juliaferraioli open sourced in June, 2014 Google launched Google Container
Engine (GKE) • https://cloud.google.com/container-engine/ roadmap: • https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/roadmap.md where we are today http://www.kuberneteslaunch.com/ -
@juliaferraioli thanks to... Brian Dorsey - Developer Advocate, Google Aja
Hammerly - Developer Advocate, Google Amy Unruh - Developer Programs Engineer, Google Mandy Waite - Developer Advocate, Google John Wilkes - Principal Engineer, Google