Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dynamic VM Scheduling
 in OpenStack

Dynamic VM Scheduling
 in OpenStack

CNDT2022で話した内容です。

Kazuhiko Yamashita

November 25, 2022
Tweet

More Decks by Kazuhiko Yamashita

Other Decks in Programming

Transcript

  1. Dynamic VM Scheduling 
 in OpenStack ʙ͜ͷલɺθΫγΟಡΜ͚ͩͲɺRFCΑΓ೉͔ͬͨ͠ฤʙ

  2. ࢁԼ࿨඙!QZBNB (.0ϖύϘٕज़ج൫νʔϜ γχΞɾϓϦϯγύϧ UFOTOBQPODPN QZBNBGVO TUOTKQ

  3. 45/4 -JOVY/444FSWFS TUOTKQ

  4. 45/4

  5. None
  6. CNDT ٱ͠ͿΓʹΦϑϥΠϯͰ 
 ࠷ߴ!!1 
 Ͱ΋͜Ε͸࿥ը͚ͩͲͶ

  7. ϖύϘͷΠϯϑϥߏ੒ CPU StealʹىҼͨ͠ɺྡਓ໰୊ εʔύʔˑϥΠϒϚΠάϨʔγϣϯ ·ͱΊͱɺ݁ࠗʹ͓͚Δେࣄͳ͜ͱ42બ ࠓ೔࿩͢͜ͱ

  8. ϖύϘͷΠϯϑϥߏ੒ 0QFO4UBDL #BSFNFUBM "84 IFSPLV  ($1 ෺ཧαʔό୆7.ن໛ ϗεςΟϯάܥͷαʔϏε΍ɺ֤छج൫αʔϏεͰར༻ &,4

    3%4 &MBTUJDYYYܥͷϚωʔδυαʔϏεΛத৺ʹར༻ (,& #JH2VFSZΛத৺ʹ"84ͷ%3ɺσʔλΤϯδχΞϦϯάͷج൫ͱͯ͠ར༻
  9. 0QFO4UBDL 0QFO4UBDLͰ؅ཧ͢Δ7.ͷ্ʹ 
 ࣗࣾ։ൃͷΤϯδϯΛར༻ͯ͠ ,VCFSOFUFTΛσϓϩΠ 0QFO4UBDL #BSFNFUBM #BSFNFUBM #BSFNFUBM 7.

    7. 7. ,VCFSOFUFT $POUBJOFS $POUBJOFS $POUBJOFS ,VCFSOFUFT
  10. CPU StealʹىҼͨ͠ɺ ྡਓ໰୊

  11. ॠؒɺ৺ɺॏͶͯ ೔ຊ࣌ؒͰαʔϏε΍ͬͯΔ͔Β ϐʔΫλΠϜ͕ॏෳ 9:00 12:00 20:00 ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ

    ϐʔΫλΠϜ ϐʔΫλΠϜ
  12. CPU Steal Ϧιʔε͕ୣ͍߹͍ʹͳΓɺCPU Steal͕ൃੜ

  13. CPU Stealͷ֓೦ Hypervisor VM1 CPU࢖͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU࢖͍͍ͨʂ 10ඵ͋͛Δ

  14. VM؀ڥʹ͓͚ΔCPU Stealͷ֓೦ Hypervisor VM1 CPU࢖͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU࢖͍͍ͨʂ 10ඵ͋͛Δ ࣮ࡍ͸8ඵ͔͋͛͠ΒΕͳ͔ͬͨ

    ࠩ෼ͷ2ඵ͕CPU Steal
  15. CPU Steal͕ى͖Δͱʁ αʔόॲཧೳྗͷ௿Լ αʔόϝτϦΫεͷѱԽ HTTPͷϨΠςϯγͷѱԽ ઀ଓ਺ͷ૿Ճ ϩʔυΞϕϨʔδ஋ͷ্ঢ

  16. εʔύʔˑϥΠϒϚΠάϨʔγϣϯ

  17. ϥΠϒϚΠάϨʔγϣϯ Hypervisor VM1 VM2 10ඵ CPU 2ඵ CPU Hypervisor VM1

    10ඵ CPU VMΛఀࢭ͢Δ͜ͱͳ͘HypervisorؒΛҠಈ
  18. ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1

    σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍৔߹
  19. ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 2

    ϝϞϦΛಉظͯ͠੩ࢭ఺Λ଴ͭ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍৔߹ Memory
  20. OpenStackͷϥΠϒϚΠάϨʔγϣϯ https://docs.openstack.org/nova/pike/admin/con fi guring-migrations.html

  21. ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk Memory STEP

    3 ੩ࢭ఺͕औΕͨΒ৽͍͠ΠϯελϯεΛ
 ༗ޮʹͯ͠ɺωοτϫʔΫܦ࿏Λ੾Γସ͑Δ ͓લ͕ࠓ೔͔ΒVM1ͩ
  22. εʔύʔˑϥΠϒϚΠάϨʔγϣϯ

  23. εʔύʔˑϥΠϒϚΠάϨʔγϣϯ 1. VMʹ͓͍ͯStealͷൃੜΛݕ஌͍ͨ͠ 2.Stealͷൃੜ͍ͯ͠ͳ͍HypervisorʹεέδϡʔϦϯά͍ͨ͠ 3.μ΢ϯλΠϜΛൃੜͤͨ͘͞ͳ͍ 4.ࣗಈͰ৸ͯΔؒʹ͢΂ͯऴΘͬͯ΄͍͠ ͱ͍͏ͷΛɺRubyͰ࣮૷ͨ͠

  24. VMͷStealൃੜݕ஌ Load5͕͖͍͠஋Ҏ্͔ͭɺSteal͕ൃੜ͍ͯ͠Δ͜ͱ Steal͕ൃੜͯ͠΋ɺॲཧ͕஗Ԇ͍ͯ͠ͳ͍ͳΒ͹໰୊ͳ͍

  25. εέδϡʔϦϯά HypervisorͷϝτϦΫεͰSteal͕͖͍͠஋ҎԼͰɺΑΓۭ͖͕͋Δ ಉ͡ϩʔϧͷVM͕ಉҰαʔόʹ͖͍͠஋Ҏ্ऩ༰͞Ε͍ͯͳ͍ ಉ͡AZͷαʔόͰ͋Δ

  26. Availability Zone LB AZ-1 VM1 AZ-2 VM2 LB AZ-1 VM1

    VM2 AZ-2 AZΛ·͍ͨͰεέδϡʔϦϯά͢Δͱ ଱ো֐ੑ͕ഁ໓!!1
  27. μ΢ϯλΠϜΛൃੜͤ͞ͳ͍ ΦʔέετϨʔλʔͰ޻෉͢Δ $ kubectl drain example.local --force --ignore-daemonsets —delete-emptydir-data #

    after migrated $ kubectl uncordon example.local $ consul maint -enable # after migrated $ consul maint -disable Kubernetes Consul
  28. kubectl drain node-1 pod-1 Scheduling Disabled node-2 pod-2 pod-1 node΁ͷPodͷׂ౰Λఀࢭͯ͠ɺ௥͍ग़͢

  29. consul node-1 
 (10.0.0.1) nginx node-2 
 (10.0.0.2) nginx node-3

    (10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 10.0.0.3 ϔϧενΣοΫͳͲΛར༻ͯ͠ɺαʔϏεσΟεΧόϦ͕Ͱ͖Δ DNSͷଞʹ΋ɺConsul TemplateΛར༻ͨ͠ ߏ੒ϑΝΠϧͷϨϯμϦϯάͳͲͰ׆༻͍ͯ͠Δ
  30. consul maint -enable node-1 
 (10.0.0.1) nginx node-2 
 (10.0.0.2)

    nginx node-3 (10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 Ϋϥελ͔Β੾Γ཭͠ consul maint -enable
  31. ॲཧܥ·ͱΊ Stealͷൃੜ͔ͭ ϩʔυΞϕϨʔδͷ্ঢ ࠷దͳ฼؋ͷબఆ Ϋϥελ͔Βͷ੾Γ཭͠ ϥΠϒϚΠάϨʔγϣϯ Ϋϥελ΁ͷ૊ΈࠐΈ ࣮ߦ؀ڥͱͯ͠͸OpenStackͷ্Ͱಈ͘ Kubernetesͷதͷpodͱͯ͠ɺ Reconciliation

    loopͬΆ͘ಈ͍͍ͯΔ
  32. ຊ౰ʹ͋ͬͨා͍࿩

  33. ʮεʔύʔˑϥΠϒϚΠάϨʔγϣϯ 
 ಈ͍ͯΔΜͰ͚͢ͲɺϨΠςϯγ 
 ߴ͍ΜͰ͢ΑͶʯ

  34. ϥΠϒϚΠάϨʔγϣϯ͕ଳҬΛ৯͍੾Δ ͓ன͔Βഁ໓!!1

  35. ϥΠϒϚΠάϨʔγϣϯ(࠶ܝ) Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1

    σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍৔߹ ʊਓਓਓਓʊ ʼɹ500Gɹʻ ʉY^Y^Y^Y^ʉ
  36. None
  37. None
  38. ·ͱΊ ྡਓ໰୊Λࣗ෼Ͱεέδϡʔϥʔॻ͘͜ͱͰղܾͨ͠ OSSʹ͢ΔͱdeschedulerͷΑ͏ͳந৅౓ʹͳΔͷͰɺ
 ࠓճ͸ެ։͠ͳ͔ͬͨ ϥΠϒϚΠάϨʔγϣϯ͸ҙ֎ͱଳҬ৯͏ͷͰɺωοτϫʔΫΛ෼཭ͯ͠ ϨʔτϦϛοτೖΕͨ΄͏͕͍͍

  39. ଟ෼ɺձ৔ʹ͍ΔͷͰɺ టʹͳΓ·͠ΐ͏ʂʂʂ

  40. None
  41. ͓͠·͍ ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ [email protected]