Scheduling and Resource Management in Kubernetes

Effective management of shared resources (CPU, memory, etc.) is critical to achieving high utilization and strong isolation in shared clusters. This talk will describe the Kubernetes features that allow you to simultaneously achieve your utilization, isolation, and availability goals. We will also examine the new features in Kubernetes 1.6 that give you fine-grained control over placement of your containers, and describe how to use them to achieve various concrete policy goals. Lastly we will discuss the feature roadmap in this area for the next few releases.

Transcript

1. Kubernetes Scheduling Features
   Or
   How Can I Make the System Do What I Want?
   Ian Lewis, Developer Advocate, Google
   

   View Slide

2. Scheduling
   ?
   

   View Slide

3. Scheduling
   Predicates Priorities
   

   View Slide

4. Scheduling
   Predicates
   

   View Slide

5. Scheduling
   Priorities
   10
   5
   

   View Slide

6. Scheduling
   

   View Slide

7. Scheduling
   Predicates Priorities
   

   View Slide

8. Predicates deep dive
   

   View Slide

9. Prevent overcommit
   ?
   

   View Slide

10. Prevent overcommit
    ?
    

    View Slide

11. Prevent co-scheduling (pod anti-affinity)
    ?
    

    View Slide

12. Prevent co-scheduling (pod anti-affinity)
    

    View Slide

13. Prevent co-scheduling (pod anti-affinity)
    ?
    PodAntiAffinity: {
    TopologyKey: "hostname",
    LabelSelector: "type:cat"
    }
    

    View Slide

14. Prevent co-scheduling (pod anti-affinity)
    

    View Slide

15. Prevent co-scheduling (pod anti-affinity)
    

    View Slide

16. Prevent co-scheduling (pod anti-affinity)
    

    View Slide

17. Prevent co-scheduling (pod anti-affinity)
    

    View Slide

18. Prevent co-scheduling (pod anti-affinity)
    PodAntiAffinity: {
    TopologyKey: "hostname",
    LabelSelector: "type:cat"
    }
    

    View Slide

19. Force co-scheduling (pod affinity)
    ?
    

    View Slide

20. Force co-scheduling (pod affinity)
    ?
    PodAffinity: {
    TopologyKey: "hostname",
    LabelSelector: "type:sapling"
    }
    

    View Slide

21. Force co-scheduling (pod affinity)
    

    View Slide

22. Dedicated machines (Taints)
    

    View Slide

23. Dedicated machines (Taints)
    

    View Slide

24. Dedicated machines (Taints)
    Taint: {
    TaintEffect: "NoSchedule",
    Key: "color",
    Value: "blue"
    }
    Toleration: {
    Key: "color",
    Value: "blue",
    Operator: "Equal",
    TaintEffect: "NoSchedule"
    }
    

    View Slide

25. Dedicated machines (Taints)
    

    View Slide

26. Predicate summary
    ● Prevent overcommit
    ● Prevent co-scheduling
    ● Force co-scheduling
    ● Dedicated Nodes
    

    View Slide

27. Priorities deep dive
    

    View Slide

28. Best fit vs worst fit
    ?
    

    View Slide

29. Best fit vs worst fit
    Worst fit
    

    View Slide

30. Best fit vs worst fit
    Best fit
    

    View Slide

31. Selector Spreading
    ?
    

    View Slide

32. Selector Spreading
    

    View Slide

33. Node Affinity
    ?
    

    View Slide

34. Node Affinity
    

    View Slide

35. Pod Affinity
    ?
    

    View Slide

36. Pod Affinity
    

    View Slide

37. Pod Anti-Affinity
    ?
    

    View Slide

38. Pod Anti-Affinity
    

    View Slide

39. How to combine scores?
    ?
    

    View Slide

40. Linear combination
    Σa
    i
    p
    i
    

    View Slide

41. https://commons.wikimedia.org/wiki/File:Messina_Straits_Chauliodus_sloani.jpg
    Dangers of depending on priorities
    

    View Slide

42. Decision tree
    Node Affinity score Node Affinity score
    ?
    Spreading score Spreading score
    ?
    ...
    > <
    =
    1000 * (Node Affinity score) +
    100 * (Spreading score) +
    ...
    > <
    =
    

    View Slide

43. Priorities are powerful
    … but require deep understanding
    

    View Slide

44. • [beta in 1.6] Multi-scheduler support
    • [beta in 1.6] Node affinity
    • [beta in 1.6] Pod affinity/anti-affinity
    • [beta in 1.6] Tains & tolerations
    Feature state as of 1.6
    

    View Slide

45. • Priorities & preemptions
    • Rescheduler
    • Resource estimation
    Future plans
    

    View Slide

46. SIG Scheduling
    • Meeting every other Monday 2pm PST
    • #sig-scheduling Slack channel
    • kubernetes-sig-scheduling@ mailing group
    Join community
    

    View Slide