Upgrade to Pro — share decks privately, control downloads, hide ads and more …

図で理解する Descheduler #k8sjp #ymju / Introduction to Descheduler

図で理解する Descheduler #k8sjp #ymju / Introduction to Descheduler

Kubernetes Meetup Tokyo 19 (Yahoo! JAPAN MEETUP共催) で発表した資料です
https://k8sjp.connpass.com/event/126207/

d-kuro

May 31, 2019
Tweet

More Decks by d-kuro

Other Decks in Technology

Transcript

  1. ਤͰཧղ͢Δ
    Descheduler
    19/05/31 Kubernetes Meetup Tokyo #19 / Yahoo! JAPAN MEETUP #32

    @ponde_m

    View full-size slide

  2. Descheduler ?
    ?

    View full-size slide

  3. Scheduler
    ͷ͓͞Β͍

    View full-size slide

  4. Kubernetes ͷΞʔΩςΫνϟ
    API Server
    Contoroller

    Manager
    Scheduler
    etcd

    View full-size slide

  5. Kubernetes ͷΞʔΩςΫνϟ
    API Server
    Contoroller

    Manager
    Scheduler
    etcd
    Scheduler
    Pod Λ Node ΁ׂΓ౰ͯΔ
    Scheduler

    View full-size slide

  6. Scheduler
    • Pod ͷ࡞੒࣌ʹεέδϡʔϦϯάΛߦ͏
    • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ
    • Node ͷϦιʔεʹภΓ͕ग़Δ
    • taints, label ͕มߋ͞Εͯ΋

    εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node Ͱো֐͕ൃੜ͠, 

    Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ··

    View full-size slide

  7. Scheduler
    • Pod ͷ࡞੒࣌ʹεέδϡʔϦϯάΛߦ͏
    • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ
    • Node ͷϦιʔεʹภΓ͕ग़Δ
    • taints, label ͕มߋ͞Εͯ΋

    εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node Ͱো֐͕ൃੜ͠, 

    Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ··
    • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ
    • Node ͷϦιʔεʹภΓ͕ग़Δ
    • taints, label ͕มߋ͞Εͯ΋

    εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node Ͱো֐͕ൃੜ͠, 

    Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ··
    Pod ͷ࡞੒࣌ʹ͔͠

    εέδϡʔϦϯά

    ͯ͘͠Εͳ͍ͷͰ

    ͜ͷΑ͏ͳ໰୊͕ग़ͯ͘Δ

    View full-size slide

  8. Node Node
    Node * 2

    View full-size slide

  9. Pod
    Pod
    Pod Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    Pod Pod
    ֤ϊʔυʹಉ͡
    Pod ͕ 6 ͭ

    View full-size slide

  10. Node
    Pod
    Pod
    Pod Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    Pod Pod
    NEW!!

    View full-size slide

  11. Node
    Pod
    Pod
    Pod Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    Pod Pod
    Node
    طʹ Node ʹ

    εέδϡʔϦϯά͞Ε͍ͯΔ
    Pod ͸Ҡಈͯ͘͠Εͳ͍

    View full-size slide

  12. Pod
    Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    ͜͏ͳͬͯ΄͍͠

    View full-size slide

  13. Descheduler
    • github.com/kubernetes-incubator/descheduler
    • ҠಈՄೳͳ Pod Λݟ͚ͭͯ࡟আͯ͘͠ΕΔ
    • ࠶εέδϡʔϧ͸ Scheduler ͕ߦ͏
    • Kubernetes ͷ Job ͱ࣮ͯ͠ߦͤ͞Δ

    View full-size slide

  14. Descheduler
    • github.com/kubernetes-incubator/descheduler
    • ҠಈՄೳͳ Pod Λݟ͚ͭͯ࡟আͯ͘͠ΕΔ
    • ࠶εέδϡʔϧ͸ Scheduler ͕ߦ͏
    • Kubernetes ͷ Job ͱ࣮ͯ͠ߦͤ͞Δ
    ҠಈՄೳͳ Pod
    ର৅ͷ Pod Λ
    Ͳ͏΍ܾͬͯΊΔͷ͔

    View full-size slide

  15. Policy and Strategies
    • Descheduler Ͱ͸, ࡟আ͢Δ Pod ͷϙϦγʔΛ

    ϑΝΠϧͰఆٛ͢Δ
    • 4 ͭͷετϥςδΛ૊Έ߹Θͤͯઃఆ͢Δ
    • RemoveDuplicates
    • LowNodeUtilization
    • RemovePodsViolatingInterPodAntiAffinity
    • RemovePodsViolatingNodeAffinity

    View full-size slide

  16. RemoveDuplicates
    Pod Pod Pod
    • Replica Set / Replication Controller / Deployment / Job

    ʹඥ͍͍ͮͯΔ Pod ͕֤ Node Ͱ 1 ͚ͭͩͰ͋Δ͜ͱΛ֬ೝ͢Δ
    • ෳ਺഑ஔ͞ΕͯͨΒ࡟আ͢Δ

    View full-size slide

  17. RemoveDuplicates
    Pod Pod Pod

    View full-size slide

  18. RemoveDuplicates
    Pod
    Pod
    Pod
    Pod ͕

    ଞͷ Node ʹҠಈ

    View full-size slide

  19. RemoveDuplicates
    Pod
    Pod
    Node
    Pod
    Node ͕ੜ͖ฦͬͯ΋

    Pod ͸໭ͬͯ͜ͳ͍

    View full-size slide

  20. RemoveDuplicates
    Pod
    Pod
    Pod
    Pod ͕

    ࠶εέδϡʔϦϯά͞ΕΔ
    Pod
    Run Descheduler

    View full-size slide

  21. LowNodeUtilization
    • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁

    Ϧιʔεͷ࢖༻཰͕ߴ͍ Node ͔Β

    Pod Λ࠶εέδϡʔϧ͢Δ͜ͱΛ໨తͱ͍ͯ͠Δ
    • Pod ͷ Resource requests ͕ܭࢉʹ༻͍ΒΕΔ
    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ

    Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ
    • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ

    Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ

    View full-size slide

  22. LowNodeUtilization
    • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁

    Ϧιʔεͷ࢖༻཰͕ߴ͍ Node ͔Β

    Pod Λ࠶εέδϡʔϧ͢Δ
    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ

    Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ
    • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ

    Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ
    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ

    Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ
    CPU / RAM / Pod ਺ ͷᮢ஋

    ԼճΔͱϦιʔε࢖༻཰͕௿͍
    Node ͱ൑ఆ͞ΕΔ

    View full-size slide

  23. LowNodeUtilization
    • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁

    Ϧιʔεͷར༻཰͕ߴ͍ Node ͔Β

    Pod Λ࠶εέδϡʔϧ͢Δ
    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ

    Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ
    • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ

    Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ
    CPU / RAM / Pod ਺ ͷᮢ஋

    ্ճΔͱϦιʔε࢖༻཰͕ߴ͍
    Node ͱ൑ఆ͞ΕΔ
    • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ

    Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ

    View full-size slide

  24. LowNodeUtilization
    Pod
    Pod
    Pod
    Pod
    Pod
    Pod
    thresholds:
    “pods": 2
    targetThresholds:
    “pods”: 2

    View full-size slide

  25. LowNodeUtilization
    Pod
    Pod
    Pod
    Pod
    Pod
    Pod
    thresholds:
    “pods": 2
    targetThresholds:
    “pods”: 2
    Pod
    Pod
    Pod
    targetThresholds:
    “pods”: 2
    Pod ਺ 2 Λ௒͑ͨΒ

    ௥͍ग़͠ର৅

    View full-size slide

  26. LowNodeUtilization
    Pod
    Pod
    Pod
    Pod
    Pod
    Pod
    thresholds:
    “pods": 2
    targetThresholds:
    “pods”: 2
    Pod
    thresholds:
    “pods": 2
    Pod ਺ 2 ະຬͳΒ

    ଞͷ Node ͔Β

    Pod ΛҠಈͤͯ͞΋͍͍

    View full-size slide

  27. LowNodeUtilization
    Pod
    Pod
    Pod
    Pod
    Pod
    thresholds:
    “pods": 2
    targetThresholds:
    “pods”: 2
    Pod
    Pod
    Run Descheduler

    View full-size slide

  28. RemovePodsViolatingInterPodAntiAffinity
    • Inter-Pod Anti-Affinity ʹ

    ҧ൓͍ͯ͠Δ Pod ͕࡟আ͞ΕΔ
    • Inter-Pod Anti-Affinity
    • ಛఆͷϥϕϧΛ࣋ͬͨ Pod ͸ಉ͡ Node ʹ

    εέδϡʔϦϯά͠ͳ͍Α͏ʹ͢Δ΍ͭ

    View full-size slide

  29. Pod C
    RemovePodsViolatingInterPodAntiAffinity
    Pod B
    Pod A
    Node

    View full-size slide

  30. Pod C
    RemovePodsViolatingInterPodAntiAffinity
    Pod B
    Pod A
    Node
    Pod C
    Pod B
    Pod A
    Pod ͕ Node Ͱ

    ಈ͍͍ͯΔঢ়ଶͰ
    Pod B ͱ Pod C ʹ

    Pod A ͱಉ͡ϊʔυͰͷ

    ࣮ߦΛېࢭ͢Δ 

    Anti-Affinity rule Λ

    ௥Ճ͢Δ

    View full-size slide

  31. Pod C
    RemovePodsViolatingInterPodAntiAffinity
    Pod B
    Pod A
    Node
    Anti-Affinity ͸

    εέδϡʔϧ͞ΕΔ͔࣌͠

    ݟͯ͘Εͳ͍ͷͰ

    Pod ͸Ҡಈͯ͘͠Εͳ͍

    View full-size slide

  32. Pod C
    RemovePodsViolatingInterPodAntiAffinity
    Pod B
    Pod A
    Pod ͕

    ࠶εέδϡʔϦϯά͞ΕΔ
    Pod A
    Run Descheduler

    View full-size slide

  33. RemovePodsViolatingNodeAffinity
    • Node Affinity ʹҧ൓͍ͯ͠Δ Pod ͕࡟আ͞ΕΔ
    • Node Affinity
    • Pod Λಛఆͷ Node ʹ

    εέδϡʔϦϯά͢ΔΑ͏ʹ͢Δ΍ͭ

    View full-size slide

  34. Pod C
    Pod B
    Pod A
    Node
    RemovePodsViolatingNodeAffinity
    node: gopher-a node: gopher-b

    View full-size slide

  35. Pod C
    Pod B
    Pod A
    Node
    RemovePodsViolatingNodeAffinity
    node: gopherA node: gopherB
    Pod A
    node: gopher-a
    Pod C
    Pod B
    Pod A ͸
    node: gopher-a
    ͷϥϕϧͷ Node ʹ

    εέδϡʔϧ͢Δ

    Node Affinity

    View full-size slide

  36. Pod C
    Pod B
    Pod A
    Node
    RemovePodsViolatingNodeAffinity
    node: gopher-a
    node: gopher-b
    Pod C
    Pod B
    Pod A
    Node
    node: gopher-a
    node: gopher-b
    ϥϕϧͷ෇͚ସ͑Λߦ͏

    View full-size slide

  37. RemovePodsViolatingNodeAffinity
    Pod C
    Pod B
    Pod A
    Node
    node: gopher-a
    node: gopher-b
    Node Affinity ͸

    εέδϡʔϧ͞ΕΔ͔࣌͠

    ݟͯ͘Εͳ͍ͷͰ

    Pod ͸Ҡಈͯ͘͠Εͳ͍

    View full-size slide

  38. RemovePodsViolatingNodeAffinity
    Pod C
    Pod B
    Pod A
    node: gopher-a
    node: gopher-b
    Pod A
    Run Descheduler Pod ͕

    ࠶εέδϡʔϦϯά͞ΕΔ

    View full-size slide

  39. Pod ͷཱͪୀ͖
    • ҎԼͷϧʔϧͰߦΘΕΔ
    • Critical Pods ͸ର৅֎
    • Replica Set / Replication Controller / Deployment / Job

    ʹඥ͍͍ͮͯͳ͍ Pod ͸ର৅֎
    • DaemonSet ͷ Pod ͸ର৅֎
    • ϩʔΧϧετϨʔδΛ࣋ͭ Pod ͸ର৅֎
    • QoS classes ͕ BestEffort ͷ Pod ͸ Guaranteed/ Burstable

    ΑΓ΋༏ઌͯ͠࡟আ͞ΕΔ

    View full-size slide

  40. ͓͠·͍
    Gopher ͘Μ͸͔͜͜Β: https://gopherize.me/

    View full-size slide