Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scheduling and Resource Management in Kubernetes

Scheduling and Resource Management in Kubernetes

Effective management of shared resources (CPU, memory, etc.) is critical to achieving high utilization and strong isolation in shared clusters. This talk will describe the Kubernetes features that allow you to simultaneously achieve your utilization, isolation, and availability goals. We will also examine the new features in Kubernetes 1.6 that give you fine-grained control over placement of your containers, and describe how to use them to achieve various concrete policy goals. Lastly we will discuss the feature roadmap in this area for the next few releases.

Ian Lewis

July 12, 2017
Tweet

More Decks by Ian Lewis

Other Decks in Technology

Transcript

  1. Kubernetes Scheduling Features
    Or
    How Can I Make the System Do What I Want?
    Ian Lewis, Developer Advocate, Google

    View Slide

  2. Scheduling
    ?

    View Slide

  3. Scheduling
    Predicates Priorities

    View Slide

  4. Scheduling
    Predicates

    View Slide

  5. Scheduling
    Priorities
    10
    5

    View Slide

  6. Scheduling

    View Slide

  7. Scheduling
    Predicates Priorities

    View Slide

  8. Predicates deep dive

    View Slide

  9. Prevent overcommit
    ?

    View Slide

  10. Prevent overcommit
    ?

    View Slide

  11. Prevent co-scheduling (pod anti-affinity)
    ?

    View Slide

  12. Prevent co-scheduling (pod anti-affinity)

    View Slide

  13. Prevent co-scheduling (pod anti-affinity)
    ?
    PodAntiAffinity: {
    TopologyKey: "hostname",
    LabelSelector: "type:cat"
    }

    View Slide

  14. Prevent co-scheduling (pod anti-affinity)

    View Slide

  15. Prevent co-scheduling (pod anti-affinity)

    View Slide

  16. Prevent co-scheduling (pod anti-affinity)

    View Slide

  17. Prevent co-scheduling (pod anti-affinity)

    View Slide

  18. Prevent co-scheduling (pod anti-affinity)
    PodAntiAffinity: {
    TopologyKey: "hostname",
    LabelSelector: "type:cat"
    }

    View Slide

  19. Force co-scheduling (pod affinity)
    ?

    View Slide

  20. Force co-scheduling (pod affinity)
    ?
    PodAffinity: {
    TopologyKey: "hostname",
    LabelSelector: "type:sapling"
    }

    View Slide

  21. Force co-scheduling (pod affinity)

    View Slide

  22. Dedicated machines (Taints)

    View Slide

  23. Dedicated machines (Taints)

    View Slide

  24. Dedicated machines (Taints)
    Taint: {
    TaintEffect: "NoSchedule",
    Key: "color",
    Value: "blue"
    }
    Toleration: {
    Key: "color",
    Value: "blue",
    Operator: "Equal",
    TaintEffect: "NoSchedule"
    }

    View Slide

  25. Dedicated machines (Taints)

    View Slide

  26. Predicate summary
    ● Prevent overcommit
    ● Prevent co-scheduling
    ● Force co-scheduling
    ● Dedicated Nodes

    View Slide

  27. Priorities deep dive

    View Slide

  28. Best fit vs worst fit
    ?

    View Slide

  29. Best fit vs worst fit
    Worst fit

    View Slide

  30. Best fit vs worst fit
    Best fit

    View Slide

  31. Selector Spreading
    ?

    View Slide

  32. Selector Spreading

    View Slide

  33. Node Affinity
    ?

    View Slide

  34. Node Affinity

    View Slide

  35. Pod Affinity
    ?

    View Slide

  36. Pod Affinity

    View Slide

  37. Pod Anti-Affinity
    ?

    View Slide

  38. Pod Anti-Affinity

    View Slide

  39. How to combine scores?
    ?

    View Slide

  40. Linear combination
    Σa
    i
    p
    i

    View Slide

  41. https://commons.wikimedia.org/wiki/File:Messina_Straits_Chauliodus_sloani.jpg
    Dangers of depending on priorities

    View Slide

  42. Decision tree
    Node Affinity score Node Affinity score
    ?
    Spreading score Spreading score
    ?
    ...
    > <
    =
    1000 * (Node Affinity score) +
    100 * (Spreading score) +
    ...
    > <
    =

    View Slide

  43. Priorities are powerful
    … but require deep understanding

    View Slide

  44. • [beta in 1.6] Multi-scheduler support
    • [beta in 1.6] Node affinity
    • [beta in 1.6] Pod affinity/anti-affinity
    • [beta in 1.6] Tains & tolerations
    Feature state as of 1.6

    View Slide

  45. • Priorities & preemptions
    • Rescheduler
    • Resource estimation
    Future plans

    View Slide

  46. SIG Scheduling
    • Meeting every other Monday 2pm PST
    • #sig-scheduling Slack channel
    • kubernetes-sig-scheduling@ mailing group
    Join community

    View Slide