Scheduling and Resource Management in Kubernetes

Effective management of shared resources (CPU, memory, etc.) is critical to achieving high utilization and strong isolation in shared clusters. This talk will describe the Kubernetes features that allow you to simultaneously achieve your utilization, isolation, and availability goals. We will also examine the new features in Kubernetes 1.6 that give you fine-grained control over placement of your containers, and describe how to use them to achieve various concrete policy goals. Lastly we will discuss the feature roadmap in this area for the next few releases.

Transcript

1. Kubernetes Scheduling Features Or How Can I Make the System Do What I Want? Ian Lewis, Developer Advocate, Google
2. Scheduling ?
3. Scheduling Predicates Priorities
4. Scheduling Predicates
5. Scheduling Priorities 10 5
6. Scheduling
7. Scheduling Predicates Priorities
8. Predicates deep dive
9. Prevent overcommit ?
10. Prevent overcommit ?
11. Prevent co-scheduling (pod anti-affinity) ?
12. Prevent co-scheduling (pod anti-affinity)
13. Prevent co-scheduling (pod anti-affinity) ? PodAntiAffinity: { TopologyKey: "hostname", LabelSelector: "type:cat" }
14. Prevent co-scheduling (pod anti-affinity)
15. Prevent co-scheduling (pod anti-affinity)
16. Prevent co-scheduling (pod anti-affinity)
17. Prevent co-scheduling (pod anti-affinity)
18. Prevent co-scheduling (pod anti-affinity) PodAntiAffinity: { TopologyKey: "hostname", LabelSelector: "type:cat" }
19. Force co-scheduling (pod affinity) ?
20. Force co-scheduling (pod affinity) ? PodAffinity: { TopologyKey: "hostname", LabelSelector: "type:sapling" }
21. Force co-scheduling (pod affinity)
22. Dedicated machines (Taints)
23. Dedicated machines (Taints)
24. Dedicated machines (Taints) Taint: { TaintEffect: "NoSchedule", Key: "color", Value: "blue" } Toleration: { Key: "color", Value: "blue", Operator: "Equal", TaintEffect: "NoSchedule" }
25. Dedicated machines (Taints)
26. Predicate summary • Prevent overcommit • Prevent co-scheduling • Force co-scheduling • Dedicated Nodes
27. Priorities deep dive
28. Best fit vs worst fit ?
29. Best fit vs worst fit Worst fit
30. Best fit vs worst fit Best fit
31. Selector Spreading ?
32. Selector Spreading
33. Node Affinity ?
34. Node Affinity
35. Pod Affinity ?
36. Pod Affinity
37. Pod Anti-Affinity ?
38. Pod Anti-Affinity
39. How to combine scores? ?
40. Linear combination Σa i p i
41. https://commons.wikimedia.org/wiki/File:Messina_Straits_Chauliodus_sloani.jpg Dangers of depending on priorities
42. Decision tree Node Affinity score Node Affinity score ? Spreading score Spreading score ? ... > < = 1000 * (Node Affinity score) + 100 * (Spreading score) + ... > < =
43. Priorities are powerful … but require deep understanding
44. • [beta in 1.6] Multi-scheduler support • [beta in 1.6] Node affinity • [beta in 1.6] Pod affinity/anti-affinity • [beta in 1.6] Tains & tolerations Feature state as of 1.6
45. • Priorities & preemptions • Rescheduler • Resource estimation Future plans
46. SIG Scheduling • Meeting every other Monday 2pm PST • #sig-scheduling Slack channel • kubernetes-sig-scheduling@ mailing group Join community