Dynamic VM Scheduling  in OpenStack

by Kazuhiko Yamashita

Slide 1

Slide 1 text

Dynamic VM Scheduling   in OpenStack ʙ͜ͷલɺθΫγΟಡΜ͚ͩͲɺRFCΑΓ೉͔ͬͨ͠ฤʙ

Slide 2

Slide 2 text

ࢁԼ࿨඙!QZBNB (.0ϖύϘٕज़ج൫νʔϜ γχΞɾϓϦϯγύϧ UFOTOBQPODPN QZBNBGVO TUOTKQ

Slide 3

Slide 3 text

45/4 -JOVY/444FSWFS TUOTKQ

Slide 4

Slide 4 text

45/4

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

CNDT ٱ͠ͿΓʹΦϑϥΠϯͰ   ࠷ߴ!!1   Ͱ΋͜Ε͸࿥ը͚ͩͲͶ

Slide 7

Slide 7 text

ϖύϘͷΠϯϑϥߏ੒ CPU StealʹىҼͨ͠ɺྡਓ໰୊ εʔύʔˑϥΠϒϚΠάϨʔγϣϯ ·ͱΊͱɺ݁ࠗʹ͓͚Δେࣄͳ͜ͱ42બ ࠓ೔࿩͢͜ͱ

Slide 8

Slide 8 text

ϖύϘͷΠϯϑϥߏ੒ 0QFO4UBDL #BSFNFUBM "84 IFSPLV ($1 ෺ཧαʔό୆7.ن໛ ϗεςΟϯάܥͷαʔϏε΍ɺ֤छج൫αʔϏεͰར༻ &,4 3%4 &MBTUJDYYYܥͷϚωʔδυαʔϏεΛத৺ʹར༻ (,& #JH2VFSZΛத৺ʹ"84ͷ%3ɺσʔλΤϯδχΞϦϯάͷج൫ͱͯ͠ར༻

Slide 9

Slide 9 text

0QFO4UBDL 0QFO4UBDLͰ؅ཧ͢Δ7.ͷ্ʹ   ࣗࣾ։ൃͷΤϯδϯΛར༻ͯ͠ ,VCFSOFUFTΛσϓϩΠ 0QFO4UBDL #BSFNFUBM #BSFNFUBM #BSFNFUBM 7. 7. 7. ,VCFSOFUFT $POUBJOFS $POUBJOFS $POUBJOFS ,VCFSOFUFT

Slide 10

Slide 10 text

CPU StealʹىҼͨ͠ɺ ྡਓ໰୊

Slide 11

Slide 11 text

ॠؒɺ৺ɺॏͶͯ ೔ຊ࣌ؒͰαʔϏε΍ͬͯΔ͔Β ϐʔΫλΠϜ͕ॏෳ 9:00 12:00 20:00 ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ

Slide 12

Slide 12 text

CPU Steal Ϧιʔε͕ୣ͍߹͍ʹͳΓɺCPU Steal͕ൃੜ

Slide 13

Slide 13 text

CPU Stealͷ֓೦ Hypervisor VM1 CPU࢖͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU࢖͍͍ͨʂ 10ඵ͋͛Δ

Slide 14

Slide 14 text

VM؀ڥʹ͓͚ΔCPU Stealͷ֓೦ Hypervisor VM1 CPU࢖͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU࢖͍͍ͨʂ 10ඵ͋͛Δ ࣮ࡍ͸8ඵ͔͋͛͠ΒΕͳ͔ͬͨ ࠩ෼ͷ2ඵ͕CPU Steal

Slide 15

Slide 15 text

CPU Steal͕ى͖Δͱʁ αʔόॲཧೳྗͷ௿Լ αʔόϝτϦΫεͷѱԽ HTTPͷϨΠςϯγͷѱԽ ઀ଓ਺ͷ૿Ճ ϩʔυΞϕϨʔδ஋ͷ্ঢ

Slide 16

Slide 16 text

εʔύʔˑϥΠϒϚΠάϨʔγϣϯ

Slide 17

Slide 17 text

ϥΠϒϚΠάϨʔγϣϯ Hypervisor VM1 VM2 10ඵ CPU 2ඵ CPU Hypervisor VM1 10ඵ CPU VMΛఀࢭ͢Δ͜ͱͳ͘HypervisorؒΛҠಈ

Slide 18

Slide 18 text

ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1 σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍৔߹

Slide 19

Slide 19 text

ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 2 ϝϞϦΛಉظͯ͠੩ࢭ఺Λ଴ͭ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍৔߹ Memory

Slide 20

Slide 20 text

OpenStackͷϥΠϒϚΠάϨʔγϣϯ https://docs.openstack.org/nova/pike/admin/con fi guring-migrations.html

Slide 21

Slide 21 text

ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk Memory STEP 3 ੩ࢭ఺͕औΕͨΒ৽͍͠ΠϯελϯεΛ  ༗ޮʹͯ͠ɺωοτϫʔΫܦ࿏Λ੾Γସ͑Δ ͓લ͕ࠓ೔͔ΒVM1ͩ

Slide 22

Slide 22 text

εʔύʔˑϥΠϒϚΠάϨʔγϣϯ

Slide 23

Slide 23 text

εʔύʔˑϥΠϒϚΠάϨʔγϣϯ 1. VMʹ͓͍ͯStealͷൃੜΛݕ஌͍ͨ͠ 2.Stealͷൃੜ͍ͯ͠ͳ͍HypervisorʹεέδϡʔϦϯά͍ͨ͠ 3.μ΢ϯλΠϜΛൃੜͤͨ͘͞ͳ͍ 4.ࣗಈͰ৸ͯΔؒʹ͢΂ͯऴΘͬͯ΄͍͠ ͱ͍͏ͷΛɺRubyͰ࣮૷ͨ͠

Slide 24

Slide 24 text

VMͷStealൃੜݕ஌ Load5͕͖͍͠஋Ҏ্͔ͭɺSteal͕ൃੜ͍ͯ͠Δ͜ͱ Steal͕ൃੜͯ͠΋ɺॲཧ͕஗Ԇ͍ͯ͠ͳ͍ͳΒ͹໰୊ͳ͍

Slide 25

Slide 25 text

εέδϡʔϦϯά HypervisorͷϝτϦΫεͰSteal͕͖͍͠஋ҎԼͰɺΑΓۭ͖͕͋Δ ಉ͡ϩʔϧͷVM͕ಉҰαʔόʹ͖͍͠஋Ҏ্ऩ༰͞Ε͍ͯͳ͍ ಉ͡AZͷαʔόͰ͋Δ

Slide 26

Slide 26 text

Availability Zone LB AZ-1 VM1 AZ-2 VM2 LB AZ-1 VM1 VM2 AZ-2 AZΛ·͍ͨͰεέδϡʔϦϯά͢Δͱ ଱ো֐ੑ͕ഁ໓!!1

Slide 27

Slide 27 text

μ΢ϯλΠϜΛൃੜͤ͞ͳ͍ ΦʔέετϨʔλʔͰ޻෉͢Δ $ kubectl drain example.local --force --ignore-daemonsets —delete-emptydir-data # after migrated $ kubectl uncordon example.local $ consul maint -enable # after migrated $ consul maint -disable Kubernetes Consul

Slide 28

Slide 28 text

kubectl drain node-1 pod-1 Scheduling Disabled node-2 pod-2 pod-1 node΁ͷPodͷׂ౰Λఀࢭͯ͠ɺ௥͍ग़͢

Slide 29

Slide 29 text

consul node-1   (10.0.0.1) nginx node-2   (10.0.0.2) nginx node-3 (10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 10.0.0.3 ϔϧενΣοΫͳͲΛར༻ͯ͠ɺαʔϏεσΟεΧόϦ͕Ͱ͖Δ DNSͷଞʹ΋ɺConsul TemplateΛར༻ͨ͠ ߏ੒ϑΝΠϧͷϨϯμϦϯάͳͲͰ׆༻͍ͯ͠Δ

Slide 30

Slide 30 text

consul maint -enable node-1   (10.0.0.1) nginx node-2   (10.0.0.2) nginx node-3 (10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 Ϋϥελ͔Β੾Γ཭͠ consul maint -enable

Slide 31

Slide 31 text

ॲཧܥ·ͱΊ Stealͷൃੜ͔ͭ ϩʔυΞϕϨʔδͷ্ঢ ࠷దͳ฼؋ͷબఆ Ϋϥελ͔Βͷ੾Γ཭͠ ϥΠϒϚΠάϨʔγϣϯ Ϋϥελ΁ͷ૊ΈࠐΈ ࣮ߦ؀ڥͱͯ͠͸OpenStackͷ্Ͱಈ͘ Kubernetesͷதͷpodͱͯ͠ɺ Reconciliation loopͬΆ͘ಈ͍͍ͯΔ

Slide 32

Slide 32 text

ຊ౰ʹ͋ͬͨා͍࿩

Slide 33

Slide 33 text

ʮεʔύʔˑϥΠϒϚΠάϨʔγϣϯ   ಈ͍ͯΔΜͰ͚͢ͲɺϨΠςϯγ   ߴ͍ΜͰ͢ΑͶʯ

Slide 34

Slide 34 text

ϥΠϒϚΠάϨʔγϣϯ͕ଳҬΛ৯͍੾Δ ͓ன͔Βഁ໓!!1

Slide 35

Slide 35 text

ϥΠϒϚΠάϨʔγϣϯ(࠶ܝ) Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1 σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍৔߹ ʊਓਓਓਓʊ ʼɹ500Gɹʻ ʉY^Y^Y^Y^ʉ

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

·ͱΊ ྡਓ໰୊Λࣗ෼Ͱεέδϡʔϥʔॻ͘͜ͱͰղܾͨ͠ OSSʹ͢ΔͱdeschedulerͷΑ͏ͳந৅౓ʹͳΔͷͰɺ  ࠓճ͸ެ։͠ͳ͔ͬͨ ϥΠϒϚΠάϨʔγϣϯ͸ҙ֎ͱଳҬ৯͏ͷͰɺωοτϫʔΫΛ෼཭ͯ͠ ϨʔτϦϛοτೖΕͨ΄͏͕͍͍

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Slide 40

Slide 40 text