Upgrade to Pro — share decks privately, control downloads, hide ads and more …

図で理解する Descheduler #k8sjp #ymju / Introduction to Descheduler

図で理解する Descheduler #k8sjp #ymju / Introduction to Descheduler

Kubernetes Meetup Tokyo 19 (Yahoo! JAPAN MEETUP共催) で発表した資料です
https://k8sjp.connpass.com/event/126207/

d-kuro

May 31, 2019
Tweet

More Decks by d-kuro

Other Decks in Technology

Transcript

  1. ਤͰཧղ͢Δ
    Descheduler
    19/05/31 Kubernetes Meetup Tokyo #19 / Yahoo! JAPAN MEETUP #32

    @ponde_m

    View Slide

  2. View Slide

  3. Descheduler ?
    ?

    View Slide

  4. Scheduler
    ͷ͓͞Β͍

    View Slide

  5. Kubernetes ͷΞʔΩςΫνϟ
    API Server
    Contoroller

    Manager
    Scheduler
    etcd

    View Slide

  6. Kubernetes ͷΞʔΩςΫνϟ
    API Server
    Contoroller

    Manager
    Scheduler
    etcd
    Scheduler
    Pod Λ Node ΁ׂΓ౰ͯΔ
    Scheduler

    View Slide

  7. Scheduler
    • Pod ͷ࡞੒࣌ʹεέδϡʔϦϯάΛߦ͏
    • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ
    • Node ͷϦιʔεʹภΓ͕ग़Δ
    • taints, label ͕มߋ͞Εͯ΋

    εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node Ͱো֐͕ൃੜ͠, 

    Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ··

    View Slide

  8. Scheduler
    • Pod ͷ࡞੒࣌ʹεέδϡʔϦϯάΛߦ͏
    • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ
    • Node ͷϦιʔεʹภΓ͕ग़Δ
    • taints, label ͕มߋ͞Εͯ΋

    εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node Ͱো֐͕ൃੜ͠, 

    Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ··
    • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ
    • Node ͷϦιʔεʹภΓ͕ग़Δ
    • taints, label ͕มߋ͞Εͯ΋

    εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ··
    • Node Ͱো֐͕ൃੜ͠, 

    Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ··
    Pod ͷ࡞੒࣌ʹ͔͠

    εέδϡʔϦϯά

    ͯ͘͠Εͳ͍ͷͰ

    ͜ͷΑ͏ͳ໰୊͕ग़ͯ͘Δ

    View Slide

  9. Node Node
    Node * 2

    View Slide

  10. Pod
    Pod
    Pod Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    Pod Pod
    ֤ϊʔυʹಉ͡
    Pod ͕ 6 ͭ

    View Slide

  11. Node
    Pod
    Pod
    Pod Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    Pod Pod
    NEW!!

    View Slide

  12. Node
    Pod
    Pod
    Pod Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    Pod Pod
    Node
    طʹ Node ʹ

    εέδϡʔϦϯά͞Ε͍ͯΔ
    Pod ͸Ҡಈͯ͘͠Εͳ͍

    View Slide

  13. Pod
    Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    Pod
    Pod
    Pod Pod
    ͜͏ͳͬͯ΄͍͠

    View Slide

  14. Descheduler

    View Slide

  15. Descheduler
    • github.com/kubernetes-incubator/descheduler
    • ҠಈՄೳͳ Pod Λݟ͚ͭͯ࡟আͯ͘͠ΕΔ
    • ࠶εέδϡʔϧ͸ Scheduler ͕ߦ͏
    • Kubernetes ͷ Job ͱ࣮ͯ͠ߦͤ͞Δ

    View Slide

  16. Descheduler
    • github.com/kubernetes-incubator/descheduler
    • ҠಈՄೳͳ Pod Λݟ͚ͭͯ࡟আͯ͘͠ΕΔ
    • ࠶εέδϡʔϧ͸ Scheduler ͕ߦ͏
    • Kubernetes ͷ Job ͱ࣮ͯ͠ߦͤ͞Δ
    ҠಈՄೳͳ Pod
    ର৅ͷ Pod Λ
    Ͳ͏΍ܾͬͯΊΔͷ͔

    View Slide

  17. Policy and Strategies
    • Descheduler Ͱ͸, ࡟আ͢Δ Pod ͷϙϦγʔΛ

    ϑΝΠϧͰఆٛ͢Δ
    • 4 ͭͷετϥςδΛ૊Έ߹Θͤͯઃఆ͢Δ
    • RemoveDuplicates
    • LowNodeUtilization
    • RemovePodsViolatingInterPodAntiAffinity
    • RemovePodsViolatingNodeAffinity

    View Slide

  18. RemoveDuplicates
    Pod Pod Pod
    • Replica Set / Replication Controller / Deployment / Job

    ʹඥ͍͍ͮͯΔ Pod ͕֤ Node Ͱ 1 ͚ͭͩͰ͋Δ͜ͱΛ֬ೝ͢Δ
    • ෳ਺഑ஔ͞ΕͯͨΒ࡟আ͢Δ

    View Slide

  19. RemoveDuplicates
    Pod Pod Pod

    View Slide

  20. RemoveDuplicates
    Pod
    Pod
    Pod
    Pod ͕

    ଞͷ Node ʹҠಈ

    View Slide

  21. RemoveDuplicates
    Pod
    Pod
    Node
    Pod
    Node ͕ੜ͖ฦͬͯ΋

    Pod ͸໭ͬͯ͜ͳ͍

    View Slide

  22. RemoveDuplicates
    Pod
    Pod
    Pod
    Pod ͕

    ࠶εέδϡʔϦϯά͞ΕΔ
    Pod
    Run Descheduler

    View Slide

  23. LowNodeUtilization
    • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁

    Ϧιʔεͷ࢖༻཰͕ߴ͍ Node ͔Β

    Pod Λ࠶εέδϡʔϧ͢Δ͜ͱΛ໨తͱ͍ͯ͠Δ
    • Pod ͷ Resource requests ͕ܭࢉʹ༻͍ΒΕΔ
    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ

    Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ
    • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ

    Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ

    View Slide

  24. LowNodeUtilization
    • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁

    Ϧιʔεͷ࢖༻཰͕ߴ͍ Node ͔Β

    Pod Λ࠶εέδϡʔϧ͢Δ
    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ

    Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ
    • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ

    Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ
    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ

    Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ
    CPU / RAM / Pod ਺ ͷᮢ஋

    ԼճΔͱϦιʔε࢖༻཰͕௿͍
    Node ͱ൑ఆ͞ΕΔ

    View Slide

  25. LowNodeUtilization
    • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁

    Ϧιʔεͷར༻཰͕ߴ͍ Node ͔Β

    Pod Λ࠶εέδϡʔϧ͢Δ
    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ

    Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ
    • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ

    Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ
    CPU / RAM / Pod ਺ ͷᮢ஋

    ্ճΔͱϦιʔε࢖༻཰͕ߴ͍
    Node ͱ൑ఆ͞ΕΔ
    • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ

    Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ

    View Slide

  26. LowNodeUtilization
    Pod
    Pod
    Pod
    Pod
    Pod
    Pod
    thresholds:
    “pods": 2
    targetThresholds:
    “pods”: 2

    View Slide

  27. LowNodeUtilization
    Pod
    Pod
    Pod
    Pod
    Pod
    Pod
    thresholds:
    “pods": 2
    targetThresholds:
    “pods”: 2
    Pod
    Pod
    Pod
    targetThresholds:
    “pods”: 2
    Pod ਺ 2 Λ௒͑ͨΒ

    ௥͍ग़͠ର৅

    View Slide

  28. LowNodeUtilization
    Pod
    Pod
    Pod
    Pod
    Pod
    Pod
    thresholds:
    “pods": 2
    targetThresholds:
    “pods”: 2
    Pod
    thresholds:
    “pods": 2
    Pod ਺ 2 ະຬͳΒ

    ଞͷ Node ͔Β

    Pod ΛҠಈͤͯ͞΋͍͍

    View Slide

  29. LowNodeUtilization
    Pod
    Pod
    Pod
    Pod
    Pod
    thresholds:
    “pods": 2
    targetThresholds:
    “pods”: 2
    Pod
    Pod
    Run Descheduler

    View Slide

  30. RemovePodsViolatingInterPodAntiAffinity
    • Inter-Pod Anti-Affinity ʹ

    ҧ൓͍ͯ͠Δ Pod ͕࡟আ͞ΕΔ
    • Inter-Pod Anti-Affinity
    • ಛఆͷϥϕϧΛ࣋ͬͨ Pod ͸ಉ͡ Node ʹ

    εέδϡʔϦϯά͠ͳ͍Α͏ʹ͢Δ΍ͭ

    View Slide

  31. Pod C
    RemovePodsViolatingInterPodAntiAffinity
    Pod B
    Pod A
    Node

    View Slide

  32. Pod C
    RemovePodsViolatingInterPodAntiAffinity
    Pod B
    Pod A
    Node
    Pod C
    Pod B
    Pod A
    Pod ͕ Node Ͱ

    ಈ͍͍ͯΔঢ়ଶͰ
    Pod B ͱ Pod C ʹ

    Pod A ͱಉ͡ϊʔυͰͷ

    ࣮ߦΛېࢭ͢Δ 

    Anti-Affinity rule Λ

    ௥Ճ͢Δ

    View Slide

  33. Pod C
    RemovePodsViolatingInterPodAntiAffinity
    Pod B
    Pod A
    Node
    Anti-Affinity ͸

    εέδϡʔϧ͞ΕΔ͔࣌͠

    ݟͯ͘Εͳ͍ͷͰ

    Pod ͸Ҡಈͯ͘͠Εͳ͍

    View Slide

  34. Pod C
    RemovePodsViolatingInterPodAntiAffinity
    Pod B
    Pod A
    Pod ͕

    ࠶εέδϡʔϦϯά͞ΕΔ
    Pod A
    Run Descheduler

    View Slide

  35. RemovePodsViolatingNodeAffinity
    • Node Affinity ʹҧ൓͍ͯ͠Δ Pod ͕࡟আ͞ΕΔ
    • Node Affinity
    • Pod Λಛఆͷ Node ʹ

    εέδϡʔϦϯά͢ΔΑ͏ʹ͢Δ΍ͭ

    View Slide

  36. Pod C
    Pod B
    Pod A
    Node
    RemovePodsViolatingNodeAffinity
    node: gopher-a node: gopher-b

    View Slide

  37. Pod C
    Pod B
    Pod A
    Node
    RemovePodsViolatingNodeAffinity
    node: gopherA node: gopherB
    Pod A
    node: gopher-a
    Pod C
    Pod B
    Pod A ͸
    node: gopher-a
    ͷϥϕϧͷ Node ʹ

    εέδϡʔϧ͢Δ

    Node Affinity

    View Slide

  38. Pod C
    Pod B
    Pod A
    Node
    RemovePodsViolatingNodeAffinity
    node: gopher-a
    node: gopher-b
    Pod C
    Pod B
    Pod A
    Node
    node: gopher-a
    node: gopher-b
    ϥϕϧͷ෇͚ସ͑Λߦ͏

    View Slide

  39. RemovePodsViolatingNodeAffinity
    Pod C
    Pod B
    Pod A
    Node
    node: gopher-a
    node: gopher-b
    Node Affinity ͸

    εέδϡʔϧ͞ΕΔ͔࣌͠

    ݟͯ͘Εͳ͍ͷͰ

    Pod ͸Ҡಈͯ͘͠Εͳ͍

    View Slide

  40. RemovePodsViolatingNodeAffinity
    Pod C
    Pod B
    Pod A
    node: gopher-a
    node: gopher-b
    Pod A
    Run Descheduler Pod ͕

    ࠶εέδϡʔϦϯά͞ΕΔ

    View Slide

  41. Pod ͷཱͪୀ͖
    • ҎԼͷϧʔϧͰߦΘΕΔ
    • Critical Pods ͸ର৅֎
    • Replica Set / Replication Controller / Deployment / Job

    ʹඥ͍͍ͮͯͳ͍ Pod ͸ର৅֎
    • DaemonSet ͷ Pod ͸ର৅֎
    • ϩʔΧϧετϨʔδΛ࣋ͭ Pod ͸ର৅֎
    • QoS classes ͕ BestEffort ͷ Pod ͸ Guaranteed/ Burstable

    ΑΓ΋༏ઌͯ͠࡟আ͞ΕΔ

    View Slide

  42. ͓͠·͍
    Gopher ͘Μ͸͔͜͜Β: https://gopherize.me/

    View Slide