Scheduling and Resource Management in Kubernetes

Scheduling and Resource Management in Kubernetes

Effective management of shared resources (CPU, memory, etc.) is critical to achieving high utilization and strong isolation in shared clusters. This talk will describe the Kubernetes features that allow you to simultaneously achieve your utilization, isolation, and availability goals. We will also examine the new features in Kubernetes 1.6 that give you fine-grained control over placement of your containers, and describe how to use them to achieve various concrete policy goals. Lastly we will discuss the feature roadmap in this area for the next few releases.

01dc8e954957a10b428aa60b28c89d52?s=128

Ian Lewis

July 12, 2017
Tweet

Transcript

  1. Kubernetes Scheduling Features Or How Can I Make the System

    Do What I Want? Ian Lewis, Developer Advocate, Google
  2. Scheduling ?

  3. Scheduling Predicates Priorities

  4. Scheduling Predicates

  5. Scheduling Priorities 10 5

  6. Scheduling

  7. Scheduling Predicates Priorities

  8. Predicates deep dive

  9. Prevent overcommit ?

  10. Prevent overcommit ?

  11. Prevent co-scheduling (pod anti-affinity) ?

  12. Prevent co-scheduling (pod anti-affinity)

  13. Prevent co-scheduling (pod anti-affinity) ? PodAntiAffinity: { TopologyKey: "hostname", LabelSelector:

    "type:cat" }
  14. Prevent co-scheduling (pod anti-affinity)

  15. Prevent co-scheduling (pod anti-affinity)

  16. Prevent co-scheduling (pod anti-affinity)

  17. Prevent co-scheduling (pod anti-affinity)

  18. Prevent co-scheduling (pod anti-affinity) PodAntiAffinity: { TopologyKey: "hostname", LabelSelector: "type:cat"

    }
  19. Force co-scheduling (pod affinity) ?

  20. Force co-scheduling (pod affinity) ? PodAffinity: { TopologyKey: "hostname", LabelSelector:

    "type:sapling" }
  21. Force co-scheduling (pod affinity)

  22. Dedicated machines (Taints)

  23. Dedicated machines (Taints)

  24. Dedicated machines (Taints) Taint: { TaintEffect: "NoSchedule", Key: "color", Value:

    "blue" } Toleration: { Key: "color", Value: "blue", Operator: "Equal", TaintEffect: "NoSchedule" }
  25. Dedicated machines (Taints)

  26. Predicate summary • Prevent overcommit • Prevent co-scheduling • Force

    co-scheduling • Dedicated Nodes
  27. Priorities deep dive

  28. Best fit vs worst fit ?

  29. Best fit vs worst fit Worst fit

  30. Best fit vs worst fit Best fit

  31. Selector Spreading ?

  32. Selector Spreading

  33. Node Affinity ?

  34. Node Affinity

  35. Pod Affinity ?

  36. Pod Affinity

  37. Pod Anti-Affinity ?

  38. Pod Anti-Affinity

  39. How to combine scores? ?

  40. Linear combination Σa i p i

  41. https://commons.wikimedia.org/wiki/File:Messina_Straits_Chauliodus_sloani.jpg Dangers of depending on priorities

  42. Decision tree Node Affinity score Node Affinity score ? Spreading

    score Spreading score ? ... > < = 1000 * (Node Affinity score) + 100 * (Spreading score) + ... > < =
  43. Priorities are powerful … but require deep understanding

  44. • [beta in 1.6] Multi-scheduler support • [beta in 1.6]

    Node affinity • [beta in 1.6] Pod affinity/anti-affinity • [beta in 1.6] Tains & tolerations Feature state as of 1.6
  45. • Priorities & preemptions • Rescheduler • Resource estimation Future

    plans
  46. SIG Scheduling • Meeting every other Monday 2pm PST •

    #sig-scheduling Slack channel • kubernetes-sig-scheduling@ mailing group Join community