図で理解する Descheduler #k8sjp #ymju / Introduction to Descheduler

by d-kuro

Slide 1

Slide 1 text

ਤͰཧղ͢Δ Descheduler 19/05/31 Kubernetes Meetup Tokyo #19 / Yahoo! JAPAN MEETUP #32  @ponde_m

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Descheduler ? ?

Slide 4

Slide 4 text

Scheduler ͷ͓͞Β͍

Slide 5

Slide 5 text

Kubernetes ͷΞʔΩςΫνϟ API Server Contoroller  Manager Scheduler etcd

Slide 6

Slide 6 text

Kubernetes ͷΞʔΩςΫνϟ API Server Contoroller  Manager Scheduler etcd Scheduler Pod Λ Node ΁ׂΓ౰ͯΔ Scheduler

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Scheduler • Pod ͷ࡞੒࣌ʹεέδϡʔϦϯάΛߦ͏ • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ • Node ͷϦιʔεʹภΓ͕ग़Δ • taints, label ͕มߋ͞Εͯ΋  εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node Ͱো֐͕ൃੜ͠,   Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ·· • Kubernetes Ϋϥελ͸ಈతʹมԽ͢Δ • Node ͷϦιʔεʹภΓ͕ग़Δ • taints, label ͕มߋ͞Εͯ΋  εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node ͕૿͑ͯ΋εέδϡʔϦϯά͞Εͨ Pod ͸ͦͷ·· • Node Ͱো֐͕ൃੜ͠,   Pod ͕ଞͷ Node ʹҠಈͨ͋͠ͱͦͷ·· Pod ͷ࡞੒࣌ʹ͔͠  εέδϡʔϦϯά  ͯ͘͠Εͳ͍ͷͰ  ͜ͷΑ͏ͳ໰୊͕ग़ͯ͘Δ

Slide 9

Slide 9 text

Node Node Node * 2

Slide 10

Slide 10 text

Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod ֤ϊʔυʹಉ͡ Pod ͕ 6 ͭ

Slide 11

Slide 11 text

Node Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod NEW!!

Slide 12

Slide 12 text

Node Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Node طʹ Node ʹ  εέδϡʔϦϯά͞Ε͍ͯΔ Pod ͸Ҡಈͯ͘͠Εͳ͍

Slide 13

Slide 13 text

Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod Pod ͜͏ͳͬͯ΄͍͠

Slide 14

Slide 14 text

Descheduler

Slide 15

Slide 15 text

Descheduler • github.com/kubernetes-incubator/descheduler • ҠಈՄೳͳ Pod Λݟ͚ͭͯ࡟আͯ͘͠ΕΔ • ࠶εέδϡʔϧ͸ Scheduler ͕ߦ͏ • Kubernetes ͷ Job ͱ࣮ͯ͠ߦͤ͞Δ

Slide 16

Slide 16 text

Descheduler • github.com/kubernetes-incubator/descheduler • ҠಈՄೳͳ Pod Λݟ͚ͭͯ࡟আͯ͘͠ΕΔ • ࠶εέδϡʔϧ͸ Scheduler ͕ߦ͏ • Kubernetes ͷ Job ͱ࣮ͯ͠ߦͤ͞Δ ҠಈՄೳͳ Pod ର৅ͷ Pod Λ Ͳ͏΍ܾͬͯΊΔͷ͔

Slide 17

Slide 17 text

Policy and Strategies • Descheduler Ͱ͸, ࡟আ͢Δ Pod ͷϙϦγʔΛ  ϑΝΠϧͰఆٛ͢Δ • 4 ͭͷετϥςδΛ૊Έ߹Θͤͯઃఆ͢Δ • RemoveDuplicates • LowNodeUtilization • RemovePodsViolatingInterPodAntiAfﬁnity • RemovePodsViolatingNodeAfﬁnity

Slide 18

Slide 18 text

RemoveDuplicates Pod Pod Pod • Replica Set / Replication Controller / Deployment / Job  ʹඥ͍͍ͮͯΔ Pod ͕֤ Node Ͱ 1 ͚ͭͩͰ͋Δ͜ͱΛ֬ೝ͢Δ • ෳ਺഑ஔ͞ΕͯͨΒ࡟আ͢Δ

Slide 19

Slide 19 text

RemoveDuplicates Pod Pod Pod

Slide 20

Slide 20 text

RemoveDuplicates Pod Pod Pod Pod ͕  ଞͷ Node ʹҠಈ

Slide 21

Slide 21 text

RemoveDuplicates Pod Pod Node Pod Node ͕ੜ͖ฦͬͯ΋  Pod ͸໭ͬͯ͜ͳ͍

Slide 22

Slide 22 text

RemoveDuplicates Pod Pod Pod Pod ͕  ࠶εέδϡʔϦϯά͞ΕΔ Pod Run Descheduler

Slide 23

Slide 23 text

LowNodeUtilization • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁  Ϧιʔεͷ࢖༻཰͕ߴ͍ Node ͔Β  Pod Λ࠶εέδϡʔϧ͢Δ͜ͱΛ໨తͱ͍ͯ͠Δ • Pod ͷ Resource requests ͕ܭࢉʹ༻͍ΒΕΔ • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ  Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ  Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ

Slide 24

Slide 24 text

LowNodeUtilization • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁  Ϧιʔεͷ࢖༻཰͕ߴ͍ Node ͔Β  Pod Λ࠶εέδϡʔϧ͢Δ • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ  Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ  Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ  Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ CPU / RAM / Pod ਺ ͷᮢ஋  ԼճΔͱϦιʔε࢖༻཰͕௿͍ Node ͱ൑ఆ͞ΕΔ

Slide 25

Slide 25 text

LowNodeUtilization • Ϧιʔεͷ࢖༻཰͕௿͍ Node ΁  Ϧιʔεͷར༻཰͕ߴ͍ Node ͔Β  Pod Λ࠶εέδϡʔϧ͢Δ • thresholds ʹઃఆͨ͠ᮢ஋ΛԼճΔͱ  Node ͷϦιʔε࢖༻཰͕௿͍ͱΈͳ͞ΕΔ • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ  Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ CPU / RAM / Pod ਺ ͷᮢ஋  ্ճΔͱϦιʔε࢖༻཰͕ߴ͍ Node ͱ൑ఆ͞ΕΔ • targetThresholds ʹઃఆͨ͠ᮢ஋Λ্ճΔͱ  Node ͷϦιʔε࢖༻཰͕ߴ͍ͱΈͳ͞ΕΔ

Slide 26

Slide 26 text

LowNodeUtilization Pod Pod Pod Pod Pod Pod thresholds: “pods": 2 targetThresholds: “pods”: 2

Slide 27

Slide 27 text

LowNodeUtilization Pod Pod Pod Pod Pod Pod thresholds: “pods": 2 targetThresholds: “pods”: 2 Pod Pod Pod targetThresholds: “pods”: 2 Pod ਺ 2 Λ௒͑ͨΒ  ௥͍ग़͠ର৅

Slide 28

Slide 28 text

LowNodeUtilization Pod Pod Pod Pod Pod Pod thresholds: “pods": 2 targetThresholds: “pods”: 2 Pod thresholds: “pods": 2 Pod ਺ 2 ະຬͳΒ  ଞͷ Node ͔Β  Pod ΛҠಈͤͯ͞΋͍͍

Slide 29

Slide 29 text

LowNodeUtilization Pod Pod Pod Pod Pod thresholds: “pods": 2 targetThresholds: “pods”: 2 Pod Pod Run Descheduler

Slide 30

Slide 30 text

RemovePodsViolatingInterPodAntiAffinity • Inter-Pod Anti-Affinity ʹ  ҧ൓͍ͯ͠Δ Pod ͕࡟আ͞ΕΔ • Inter-Pod Anti-Affinity • ಛఆͷϥϕϧΛ࣋ͬͨ Pod ͸ಉ͡ Node ʹ  εέδϡʔϦϯά͠ͳ͍Α͏ʹ͢Δ΍ͭ

Slide 31

Slide 31 text

Pod C RemovePodsViolatingInterPodAntiAfﬁnity Pod B Pod A Node

Slide 32

Slide 32 text

Pod C RemovePodsViolatingInterPodAntiAfﬁnity Pod B Pod A Node Pod C Pod B Pod A Pod ͕ Node Ͱ  ಈ͍͍ͯΔঢ়ଶͰ Pod B ͱ Pod C ʹ  Pod A ͱಉ͡ϊʔυͰͷ  ࣮ߦΛېࢭ͢Δ   Anti-Afﬁnity rule Λ  ௥Ճ͢Δ

Slide 33

Slide 33 text

Pod C RemovePodsViolatingInterPodAntiAfﬁnity Pod B Pod A Node Anti-Afﬁnity ͸  εέδϡʔϧ͞ΕΔ͔࣌͠  ݟͯ͘Εͳ͍ͷͰ  Pod ͸Ҡಈͯ͘͠Εͳ͍

Slide 34

Slide 34 text

Pod C RemovePodsViolatingInterPodAntiAfﬁnity Pod B Pod A Pod ͕  ࠶εέδϡʔϦϯά͞ΕΔ Pod A Run Descheduler

Slide 35

Slide 35 text

RemovePodsViolatingNodeAffinity • Node Affinity ʹҧ൓͍ͯ͠Δ Pod ͕࡟আ͞ΕΔ • Node Affinity • Pod Λಛఆͷ Node ʹ  εέδϡʔϦϯά͢ΔΑ͏ʹ͢Δ΍ͭ

Slide 36

Slide 36 text

Pod C Pod B Pod A Node RemovePodsViolatingNodeAfﬁnity node: gopher-a node: gopher-b

Slide 37

Slide 37 text

Pod C Pod B Pod A Node RemovePodsViolatingNodeAfﬁnity node: gopherA node: gopherB Pod A node: gopher-a Pod C Pod B Pod A ͸ node: gopher-a ͷϥϕϧͷ Node ʹ  εέδϡʔϧ͢Δ  Node Afﬁnity

Slide 38

Slide 38 text

Pod C Pod B Pod A Node RemovePodsViolatingNodeAfﬁnity node: gopher-a node: gopher-b Pod C Pod B Pod A Node node: gopher-a node: gopher-b ϥϕϧͷ෇͚ସ͑Λߦ͏

Slide 39

Slide 39 text

RemovePodsViolatingNodeAfﬁnity Pod C Pod B Pod A Node node: gopher-a node: gopher-b Node Afﬁnity ͸  εέδϡʔϧ͞ΕΔ͔࣌͠  ݟͯ͘Εͳ͍ͷͰ  Pod ͸Ҡಈͯ͘͠Εͳ͍

Slide 40

Slide 40 text

RemovePodsViolatingNodeAfﬁnity Pod C Pod B Pod A node: gopher-a node: gopher-b Pod A Run Descheduler Pod ͕  ࠶εέδϡʔϦϯά͞ΕΔ

Slide 41

Slide 41 text

Pod ͷཱͪୀ͖ • ҎԼͷϧʔϧͰߦΘΕΔ • Critical Pods ͸ର৅֎ • Replica Set / Replication Controller / Deployment / Job  ʹඥ͍͍ͮͯͳ͍ Pod ͸ର৅֎ • DaemonSet ͷ Pod ͸ର৅֎ • ϩʔΧϧετϨʔδΛ࣋ͭ Pod ͸ର৅֎ • QoS classes ͕ BestEffort ͷ Pod ͸ Guaranteed/ Burstable  ΑΓ΋༏ઌͯ͠࡟আ͞ΕΔ

Slide 42

Slide 42 text

͓͠·͍ Gopher ͘Μ͸͔͜͜Β: https://gopherize.me/