Upgrade to Pro — share decks privately, control downloads, hide ads and more …

cluster management, cognitive shifts, & 
kubernetes

cluster management, cognitive shifts, & 
kubernetes

How did we get into the cluster management game? This briefly covers some of the lessons that we have learned over the years from the system internally known as Borg. Then, we step through an application running on kubernetes and see how those lessons are present in what kubernetes does and how it's designed.

62b2249a42d624dc93357931f0f5d2f1?s=128

juliaferraioli

July 07, 2015
Tweet

More Decks by juliaferraioli

Other Decks in Technology

Transcript

  1. cluster management, cognitive shifts, &
 kubernetes seattle.rb @juliaferraioli, kubernautical advocate

  2. cluster management with Borg

  3. cluster management 
 
 with Borg the system we internally

    call
  4. @juliaferraioli Image by Connie Zhou

  5. @juliaferraioli developer view

  6. @juliaferraioli job hello_world = { developer view

  7. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in developer view
  8. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run developer view
  9. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters developer view
  10. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements developer view
  11. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M developer view
  12. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M developer view
  13. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 developer view
  14. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } developer view
  15. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks developer view
  16. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks } developer view
  17. @juliaferraioli job hello_world = { runtime = { cell =

    'ic' } // Cell (cluster) to run in binary = '.../hello_world_webserver' // Program to run args = { port = '%port%' } // Command line parameters requirements = { // Resource requirements ram = 100M disk = 100M cpu = 0.1 } replicas = 5 // Number of tasks } 10000 developer view
  18. @juliaferraioli Image by Connie Zhou Hello world! Hello world! Hello

    world! Hello world! Hello world! Hello world! Hello world! Hello world!
  19. @juliaferraioli Hello world! Hello world! Hello world! Hello world! Hello

    world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Image by Connie Zhou Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world! Hello world!
  20. @juliaferraioli developer view

  21. @juliaferraioli developer view

  22. @juliaferraioli task-eviction rates and causes failures

  23. @juliaferraioli advanced bin- packing algorithms experimental placement of production VM

    workload, July 2014 one
 machine efficiency
  24. @juliaferraioli advanced bin- packing algorithms experimental placement of production VM

    workload, July 2014 available resources one
 machine efficiency
  25. @juliaferraioli advanced bin- packing algorithms experimental placement of production VM

    workload, July 2014 stranded resources available resources one
 machine efficiency
  26. @juliaferraioli Borg paper: http://goo.gl/1C4nuo Images by Connie Zhou observations: 1.

    resiliency is achieved only by ruthless attention to detail a. ubiquitous software fault tolerance b. persistent, declarative specs
 2. we get efficiency by: a. sharing resources b. reclaiming unused allocations 1. containers make users more productive
  27. @juliaferraioli app libs kernel libs app app kernel app libs

    libs kernel kernel not so long ago: virtual machines
  28. @juliaferraioli libs app kernel libs app libs app libs app

    the new kid in town: containers
  29. A fundamentally different way of managing applications

  30. enter kubernetes, stage center

  31. @juliaferraioli κυβερνήτης: “Helmsman”; root of “Governor” and “Cybernetic” manage applications,

    not machines kubernetes (or k8s)
  32. demo, part 1

  33. @juliaferraioli

  34. @juliaferraioli demo overview Server Debian Docker Engine PHP libs Redis

    libs
  35. @juliaferraioli A loop that drives current state towards desired state

    A small group of tightly coupled containers Pod Replication Controller Service A set of running pods that work together Arbitrary metadata to organize components Labels kubernetes cluster
  36. @juliaferraioli Dashboard show: type = FE Object Pod frontend Object

    type = FE type = FE • metadata with semantic meaning • membership identifier • the only grouping mechanism behavior benefits ➔ allow for intent of many users (e.g. dashboards) ➔ build higher level systems ➔ queryable by selectors labels
  37. @juliaferraioli Dashboard show: type = FE Object Pod frontend Pod

    frontend Object Object Dashboard show: version = v2 type = FE version = v2 type = FE type = FE version = v2 • metadata with semantic meaning • membership identifier • the only grouping mechanism behavior benefits ➔ allow for intent of many users (e.g. dashboards) ➔ build higher level systems ➔ queryable by selectors labels
  38. @juliaferraioli A loop that drives current state towards desired state

    A set of running pods that work together Replication Controller Service Arbitrary metadata to organize components Labels A small group of tightly coupled containers Pod kubernetes cluster
  39. @juliaferraioli small group of containers & volumes containers are tightly

    coupled the atom of cluster scheduling shared namespace • Shared network IP and port namespace ephemeral • can die and be replaced Pod Site generator Web Server Volume Consumers Content Manager pods
  40. @juliaferraioli pods template:
 metadata:
 labels:
 name: frontend
 spec:
 containers:
 -

    name: php-redis
 image: kubernetes/example-guestbook-php-redis:v2
 ports:
 - containerPort: 80
  41. @juliaferraioli A set of running pods that work together Service

    Arbitrary metadata to organize components Labels A small group of tightly coupled containers A loop that drives current state towards desired state Pod Replication Controller kubernetes cluster
  42. @juliaferraioli Replication Controller Pod Pod frontend Pod frontend Pod Pod

    Replication Controller #pods = 1 version = v2 show: version = v2 version= v1 version = v1 version = v2 Replication Controller #pods = 2 version = v1 show: version = v2 Behavior Benefits • keeps pods running • gives direct control of pod #s • grouped by label selector ➔ recreates pods, maintains desired state ➔ fine-grained control for scaling ➔ standard grouping semantics replication controllers
  43. @juliaferraioli Replication Controller Replication Controller - Name = “frontend” -

    Selector = {“Name”: “frontend”} - PodTemplate = { ... } - NumReplicas = 3 API Server 2 Start 1 more OK 3 How many? How many? canonical example of control loops have one job: ensure N copies of a pod replicated pods are fungible replication controllers
  44. @juliaferraioli replication controllers kind: ReplicationController
 metadata:
 name: frontend
 labels:
 name:

    frontend
 spec:
 replicas: 3
 selector:
 name: frontend
 template: <snip!> $ kubectl create -f frontend-controller.yaml
  45. @juliaferraioli kubernetes cluster Arbitrary metadata to organize components Labels A

    small group of tightly coupled containers A loop that drives current state towards desired state A set of running pods that work together Pod Replication Controller Service
  46. @juliaferraioli Portal (VIP) Client Pod Container Container Container Container Pod

    Container Container Container Container Pod Container Container Container Container a group of pods that act as one == Service defines access policy gets a stable virtual IP and port VIP is captured by kube-proxy services
  47. @juliaferraioli Service Label selectors: version = 1.0 type = Frontend

    Service Label selectors: version = v1 type = FE Replication Controller Pod Pod frontend Pod version= v1 version = v1 Replication Controller #pods = 2 show: version = v2 type = FE type = FE VIP services
  48. @juliaferraioli services kind: Service
 metadata:
 name: frontend
 labels:
 name: frontend


    spec:
 type: LoadBalancer
 ports:
 - port: 80
 selector:
 name: frontend $ kubectl create -f frontend-service.yaml
  49. ...in technicolor! what’s doing what...

  50. @juliaferraioli Master APIs Scheduling REST (pods, services, controllers) AuthN Scheduler

    Replication Controller Node3 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container Node3 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container Node1 Kubelet Proxy Pod Container Container Container Container Pod Container Container Container Container $ kubectl proxy --www=k8s-visualizer/ visualizing kubernetes
  51. demo, part 2

  52. @juliaferraioli

  53. @juliaferraioli open sourced in June, 2014 Google launched Google Container

    Engine (GKE) • https://cloud.google.com/container-engine/ roadmap: • https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/roadmap.md where we are today http://www.kuberneteslaunch.com/
  54. @juliaferraioli find us online… http://kubernetes.io https://github.com/GoogleCloudPlatform/kubernetes irc.freenode.net #google-containers @kubernetesio

  55. @juliaferraioli thanks to... Brian Dorsey - Developer Advocate, Google Aja

    Hammerly - Developer Advocate, Google Amy Unruh - Developer Programs Engineer, Google Mandy Waite - Developer Advocate, Google John Wilkes - Principal Engineer, Google
  56. https://flic.kr/p/6jk7ZL