図で理解する Descheduler #k8sjp #ymju / Introduction to Descheduler

図で理解する Descheduler #k8sjp #ymju / Introduction to Descheduler

Kubernetes Meetup Tokyo 19 (Yahoo! JAPAN MEETUP共催) で発表した資料です
https://k8sjp.connpass.com/event/126207/

Cad656ed619672b702191833dc819943?s=128

d-kuro

May 31, 2019
Tweet

Transcript

  1. 2.
  2. 7.

    Scheduler • Pod ͷ࡞੒࣌ʹεέδϡʔϦϯάΛߦ͏ • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ • Node ͷϦιʔεʹภΓ͕ग़Δ

    • taints, label ͕มߋ͞Εͯ΋
 εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node Ͱো֐͕ൃੜ͠, 
 Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ··
  3. 8.

    Scheduler • Pod ͷ࡞੒࣌ʹεέδϡʔϦϯάΛߦ͏ • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ • Node ͷϦιʔεʹภΓ͕ग़Δ

    • taints, label ͕มߋ͞Εͯ΋
 εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node Ͱো֐͕ൃੜ͠, 
 Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ·· • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ • Node ͷϦιʔεʹภΓ͕ग़Δ • taints, label ͕มߋ͞Εͯ΋
 εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node Ͱো֐͕ൃੜ͠, 
 Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ·· Pod ͷ࡞੒࣌ʹ͔͠
 εέδϡʔϦϯά
 ͯ͘͠Εͳ͍ͷͰ
 ͜ͷΑ͏ͳ໰୊͕ग़ͯ͘Δ
  4. 10.

    Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod

    Pod Pod ֤ϊʔυʹಉ͡ Pod ͕ 6 ͭ
  5. 12.

    Node Pod Pod Pod Pod Pod Pod Pod Pod Pod

    Pod Pod Pod Node طʹ Node ʹ
 εέδϡʔϦϯά͞Ε͍ͯΔ Pod ͸Ҡಈͯ͘͠Εͳ͍
  6. 13.

    Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod

    Pod Pod ͜͏ͳͬͯ΄͍͠
  7. 16.

    Descheduler • github.com/kubernetes-incubator/descheduler • ҠಈՄೳͳ Pod Λݟ͚ͭͯ࡟আͯ͘͠ΕΔ • ࠶εέδϡʔϧ͸ Scheduler

    ͕ߦ͏ • Kubernetes ͷ Job ͱ࣮ͯ͠ߦͤ͞Δ ҠಈՄೳͳ Pod ର৅ͷ Pod Λ Ͳ͏΍ܾͬͯΊΔͷ͔
  8. 17.

    Policy and Strategies • Descheduler Ͱ͸, ࡟আ͢Δ Pod ͷϙϦγʔΛ
 ϑΝΠϧͰఆٛ͢Δ

    • 4 ͭͷετϥςδΛ૊Έ߹Θͤͯઃఆ͢Δ • RemoveDuplicates • LowNodeUtilization • RemovePodsViolatingInterPodAntiAffinity • RemovePodsViolatingNodeAffinity
  9. 18.

    RemoveDuplicates Pod Pod Pod • Replica Set / Replication Controller

    / Deployment / Job
 ʹඥ͍͍ͮͯΔ Pod ͕֤ Node Ͱ 1 ͚ͭͩͰ͋Δ͜ͱΛ֬ೝ͢Δ • ෳ਺഑ஔ͞ΕͯͨΒ࡟আ͢Δ
  10. 23.

    LowNodeUtilization • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁
 Ϧιʔεͷ࢖༻཰͕ߴ͍ Node ͔Β
 Pod Λ࠶εέδϡʔϧ͢Δ͜ͱΛ໨తͱ͍ͯ͠Δ

    • Pod ͷ Resource requests ͕ܭࢉʹ༻͍ΒΕΔ • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ
 Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ
 Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ
  11. 24.

    LowNodeUtilization • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁
 Ϧιʔεͷ࢖༻཰͕ߴ͍ Node ͔Β
 Pod Λ࠶εέδϡʔϧ͢Δ

    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ
 Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ
 Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ
 Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ CPU / RAM / Pod ਺ ͷᮢ஋
 ԼճΔͱϦιʔε࢖༻཰͕௿͍ Node ͱ൑ఆ͞ΕΔ
  12. 25.

    LowNodeUtilization • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁
 Ϧιʔεͷར༻཰͕ߴ͍ Node ͔Β
 Pod Λ࠶εέδϡʔϧ͢Δ

    • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ
 Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ
 Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ CPU / RAM / Pod ਺ ͷᮢ஋
 ্ճΔͱϦιʔε࢖༻཰͕ߴ͍ Node ͱ൑ఆ͞ΕΔ • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ
 Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ
  13. 27.

    LowNodeUtilization Pod Pod Pod Pod Pod Pod thresholds: “pods": 2

    targetThresholds: “pods”: 2 Pod Pod Pod targetThresholds: “pods”: 2 Pod ਺ 2 Λ௒͑ͨΒ
 ௥͍ग़͠ର৅
  14. 28.

    LowNodeUtilization Pod Pod Pod Pod Pod Pod thresholds: “pods": 2

    targetThresholds: “pods”: 2 Pod thresholds: “pods": 2 Pod ਺ 2 ະຬͳΒ
 ଞͷ Node ͔Β
 Pod ΛҠಈͤͯ͞΋͍͍
  15. 30.

    RemovePodsViolatingInterPodAntiAffinity • Inter-Pod Anti-Affinity ʹ
 ҧ൓͍ͯ͠Δ Pod ͕࡟আ͞ΕΔ • Inter-Pod

    Anti-Affinity • ಛఆͷϥϕϧΛ࣋ͬͨ Pod ͸ಉ͡ Node ʹ
 εέδϡʔϦϯά͠ͳ͍Α͏ʹ͢Δ΍ͭ
  16. 32.

    Pod C RemovePodsViolatingInterPodAntiAffinity Pod B Pod A Node Pod C

    Pod B Pod A Pod ͕ Node Ͱ
 ಈ͍͍ͯΔঢ়ଶͰ Pod B ͱ Pod C ʹ
 Pod A ͱಉ͡ϊʔυͰͷ
 ࣮ߦΛېࢭ͢Δ 
 Anti-Affinity rule Λ
 ௥Ճ͢Δ
  17. 33.

    Pod C RemovePodsViolatingInterPodAntiAffinity Pod B Pod A Node Anti-Affinity ͸


    εέδϡʔϧ͞ΕΔ͔࣌͠
 ݟͯ͘Εͳ͍ͷͰ
 Pod ͸Ҡಈͯ͘͠Εͳ͍
  18. 35.
  19. 37.

    Pod C Pod B Pod A Node RemovePodsViolatingNodeAffinity node: gopherA

    node: gopherB Pod A node: gopher-a Pod C Pod B Pod A ͸ node: gopher-a ͷϥϕϧͷ Node ʹ
 εέδϡʔϧ͢Δ
 Node Affinity
  20. 38.

    Pod C Pod B Pod A Node RemovePodsViolatingNodeAffinity node: gopher-a

    node: gopher-b Pod C Pod B Pod A Node node: gopher-a node: gopher-b ϥϕϧͷ෇͚ସ͑Λߦ͏
  21. 39.

    RemovePodsViolatingNodeAffinity Pod C Pod B Pod A Node node: gopher-a

    node: gopher-b Node Affinity ͸
 εέδϡʔϧ͞ΕΔ͔࣌͠
 ݟͯ͘Εͳ͍ͷͰ
 Pod ͸Ҡಈͯ͘͠Εͳ͍
  22. 40.

    RemovePodsViolatingNodeAffinity Pod C Pod B Pod A node: gopher-a node:

    gopher-b Pod A Run Descheduler Pod ͕
 ࠶εέδϡʔϦϯά͞ΕΔ
  23. 41.

    Pod ͷཱͪୀ͖ • ҎԼͷϧʔϧͰߦΘΕΔ • Critical Pods ͸ର৅֎ • Replica

    Set / Replication Controller / Deployment / Job
 ʹඥ͍͍ͮͯͳ͍ Pod ͸ର৅֎ • DaemonSet ͷ Pod ͸ର৅֎ • ϩʔΧϧετϨʔδΛ࣋ͭ Pod ͸ର৅֎ • QoS classes ͕ BestEffort ͷ Pod ͸ Guaranteed/ Burstable
 ΑΓ΋༏ઌͯ͠࡟আ͞ΕΔ