Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Dynamic VM Scheduling in OpenStack
Search
Kazuhiko Yamashita
November 25, 2022
Programming
1
1.1k
Dynamic VM Scheduling in OpenStack
CNDT2022で話した内容です。
Kazuhiko Yamashita
November 25, 2022
Tweet
Share
More Decks by Kazuhiko Yamashita
See All by Kazuhiko Yamashita
ペパボOpenTelemetry革命
pyama86
2
1.2k
Site Reliability Engineering for GMO
pyama86
9
1.1k
PHPアプリケーションのスケーラビリティと 信頼性を革新する nginx+ngx_mrubyとGoの融合
pyama86
2
300
ペパボOpenTelemetry革命
pyama86
0
720
ChatGPTの活用を体現し、 組織に浸透する技術
pyama86
1
920
リリースフラグと プログレッシブデリバリーを活用した 安全なWEBアプリケーションデプロイ
pyama86
2
180
PHPerのためのOpenTelemetry入門
pyama86
1
230
生成AIで仕事を もっとおもしろくする事例と詳解
pyama86
0
150
Goで実装された高速な 仮想待合室サーバの実装と詳解
pyama86
15
7.1k
Other Decks in Programming
See All in Programming
『WordPressコミュニティで学ぶ』OSS貢献の多様性
ippey
0
300
“Seeing Like a Programmer”—Resiliency, Limits, and Moral Hazards in Software Engineering (LambdaConf 2024)
chriskrycho
0
450
TypeScriptコードの漸進的改善 / Progressive Improvement of TypeScript Code
medley
1
470
From LALR to IELR: A Lrama's Next Step
junk0612
0
100
WinActorの勉強を継続する方法
tamai_63
0
130
教えて!スクラムコーチ品質とスピードのバランスはどうすりゃいいの?
pinboro
0
160
勉強会4_アップデートされたAssistantsAPIを試す
milky04
0
890
An adventure of Happy Eyeballs
coe401_
1
340
Good first issues of TypeProf
mame
4
1k
TypeScriptから始める VR生活
tamagokakeg
2
140
Runtime Objects in Rust
mitsuhiko
0
220
FoodGram
iseruuuuu
0
230
Featured
See All Featured
The Invisible Customer
myddelton
114
12k
The Pragmatic Product Professional
lauravandoore
26
5.9k
Fontdeck: Realign not Redesign
paulrobertlloyd
76
4.9k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
26
2.3k
Robots, Beer and Maslow
schacon
PRO
155
8k
Fireside Chat
paigeccino
22
2.7k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
14
8.4k
Designing for Performance
lara
601
67k
Building Applications with DynamoDB
mza
88
5.7k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
126
32k
Facilitating Awesome Meetings
lara
43
5.6k
Imperfection Machines: The Place of Print at Facebook
scottboms
261
12k
Transcript
Dynamic VM Scheduling in OpenStack ʙ͜ͷલɺθΫγΟಡΜ͚ͩͲɺRFCΑΓ͔ͬͨ͠ฤʙ
ࢁԼ!QZBNB (.0ϖύϘٕज़ج൫νʔϜ γχΞɾϓϦϯγύϧ UFOTOBQPODPN QZBNBGVO TUOTKQ
45/4 -JOVY/444FSWFS TUOTKQ
45/4
None
CNDT ٱ͠ͿΓʹΦϑϥΠϯͰ ࠷ߴ!!1 Ͱ͜Εը͚ͩͲͶ
ϖύϘͷΠϯϑϥߏ CPU StealʹىҼͨ͠ɺྡਓ εʔύʔˑϥΠϒϚΠάϨʔγϣϯ ·ͱΊͱɺ݁ࠗʹ͓͚Δେࣄͳ͜ͱ42બ ࠓ͢͜ͱ
ϖύϘͷΠϯϑϥߏ 0QFO4UBDL #BSFNFUBM "84 IFSPLV ($1 ཧαʔό7.ن ϗεςΟϯάܥͷαʔϏεɺ֤छج൫αʔϏεͰར༻ &,4
3%4 &MBTUJDYYYܥͷϚωʔδυαʔϏεΛத৺ʹར༻ (,& #JH2VFSZΛத৺ʹ"84ͷ%3ɺσʔλΤϯδχΞϦϯάͷج൫ͱͯ͠ར༻
0QFO4UBDL 0QFO4UBDLͰཧ͢Δ7.ͷ্ʹ ࣗࣾ։ൃͷΤϯδϯΛར༻ͯ͠ ,VCFSOFUFTΛσϓϩΠ 0QFO4UBDL #BSFNFUBM #BSFNFUBM #BSFNFUBM 7.
7. 7. ,VCFSOFUFT $POUBJOFS $POUBJOFS $POUBJOFS ,VCFSOFUFT
CPU StealʹىҼͨ͠ɺ ྡਓ
ॠؒɺ৺ɺॏͶͯ ຊ࣌ؒͰαʔϏεͬͯΔ͔Β ϐʔΫλΠϜ͕ॏෳ 9:00 12:00 20:00 ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ
ϐʔΫλΠϜ ϐʔΫλΠϜ
CPU Steal Ϧιʔε͕ୣ͍߹͍ʹͳΓɺCPU Steal͕ൃੜ
CPU Stealͷ֓೦ Hypervisor VM1 CPU͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU͍͍ͨʂ 10ඵ͋͛Δ
VMڥʹ͓͚ΔCPU Stealͷ֓೦ Hypervisor VM1 CPU͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU͍͍ͨʂ 10ඵ͋͛Δ ࣮ࡍ8ඵ͔͋͛͠ΒΕͳ͔ͬͨ
ࠩͷ2ඵ͕CPU Steal
CPU Steal͕ى͖Δͱʁ αʔόॲཧೳྗͷԼ αʔόϝτϦΫεͷѱԽ HTTPͷϨΠςϯγͷѱԽ ଓͷ૿Ճ ϩʔυΞϕϨʔδͷ্ঢ
εʔύʔˑϥΠϒϚΠάϨʔγϣϯ
ϥΠϒϚΠάϨʔγϣϯ Hypervisor VM1 VM2 10ඵ CPU 2ඵ CPU Hypervisor VM1
10ඵ CPU VMΛఀࢭ͢Δ͜ͱͳ͘HypervisorؒΛҠಈ
ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1
σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍߹
ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 2
ϝϞϦΛಉظͯ͠੩ࢭΛͭ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍߹ Memory
OpenStackͷϥΠϒϚΠάϨʔγϣϯ https://docs.openstack.org/nova/pike/admin/con fi guring-migrations.html
ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk Memory STEP
3 ੩ࢭ͕औΕͨΒ৽͍͠ΠϯελϯεΛ ༗ޮʹͯ͠ɺωοτϫʔΫܦ࿏ΛΓସ͑Δ ͓લ͕ࠓ͔ΒVM1ͩ
εʔύʔˑϥΠϒϚΠάϨʔγϣϯ
εʔύʔˑϥΠϒϚΠάϨʔγϣϯ 1. VMʹ͓͍ͯStealͷൃੜΛݕ͍ͨ͠ 2.Stealͷൃੜ͍ͯ͠ͳ͍HypervisorʹεέδϡʔϦϯά͍ͨ͠ 3.μϯλΠϜΛൃੜͤͨ͘͞ͳ͍ 4.ࣗಈͰ৸ͯΔؒʹͯ͢ऴΘͬͯ΄͍͠ ͱ͍͏ͷΛɺRubyͰ࣮ͨ͠
VMͷStealൃੜݕ Load5͕͖͍͠Ҏ্͔ͭɺSteal͕ൃੜ͍ͯ͠Δ͜ͱ Steal͕ൃੜͯ͠ɺॲཧ͕Ԇ͍ͯ͠ͳ͍ͳΒͳ͍
εέδϡʔϦϯά HypervisorͷϝτϦΫεͰSteal͕͖͍͠ҎԼͰɺΑΓۭ͖͕͋Δ ಉ͡ϩʔϧͷVM͕ಉҰαʔόʹ͖͍͠Ҏ্ऩ༰͞Ε͍ͯͳ͍ ಉ͡AZͷαʔόͰ͋Δ
Availability Zone LB AZ-1 VM1 AZ-2 VM2 LB AZ-1 VM1
VM2 AZ-2 AZΛ·͍ͨͰεέδϡʔϦϯά͢Δͱ োੑ͕ഁ໓!!1
μϯλΠϜΛൃੜͤ͞ͳ͍ ΦʔέετϨʔλʔͰ͢Δ $ kubectl drain example.local --force --ignore-daemonsets —delete-emptydir-data #
after migrated $ kubectl uncordon example.local $ consul maint -enable # after migrated $ consul maint -disable Kubernetes Consul
kubectl drain node-1 pod-1 Scheduling Disabled node-2 pod-2 pod-1 nodeͷPodͷׂΛఀࢭͯ͠ɺ͍ग़͢
consul node-1 (10.0.0.1) nginx node-2 (10.0.0.2) nginx node-3
(10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 10.0.0.3 ϔϧενΣοΫͳͲΛར༻ͯ͠ɺαʔϏεσΟεΧόϦ͕Ͱ͖Δ DNSͷଞʹɺConsul TemplateΛར༻ͨ͠ ߏϑΝΠϧͷϨϯμϦϯάͳͲͰ׆༻͍ͯ͠Δ
consul maint -enable node-1 (10.0.0.1) nginx node-2 (10.0.0.2)
nginx node-3 (10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 Ϋϥελ͔ΒΓ͠ consul maint -enable
ॲཧܥ·ͱΊ Stealͷൃੜ͔ͭ ϩʔυΞϕϨʔδͷ্ঢ ࠷దͳ؋ͷબఆ Ϋϥελ͔ΒͷΓ͠ ϥΠϒϚΠάϨʔγϣϯ ΫϥελͷΈࠐΈ ࣮ߦڥͱͯ͠OpenStackͷ্Ͱಈ͘ Kubernetesͷதͷpodͱͯ͠ɺ Reconciliation
loopͬΆ͘ಈ͍͍ͯΔ
ຊʹ͋ͬͨා͍
ʮεʔύʔˑϥΠϒϚΠάϨʔγϣϯ ಈ͍ͯΔΜͰ͚͢ͲɺϨΠςϯγ ߴ͍ΜͰ͢ΑͶʯ
ϥΠϒϚΠάϨʔγϣϯ͕ଳҬΛ৯͍Δ ͓ன͔Βഁ໓!!1
ϥΠϒϚΠάϨʔγϣϯ(࠶ܝ) Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1
σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍߹ ʊਓਓਓਓʊ ʼɹ500Gɹʻ ʉY^Y^Y^Y^ʉ
None
None
·ͱΊ ྡਓΛࣗͰεέδϡʔϥʔॻ͘͜ͱͰղܾͨ͠ OSSʹ͢ΔͱdeschedulerͷΑ͏ͳநʹͳΔͷͰɺ ࠓճެ։͠ͳ͔ͬͨ ϥΠϒϚΠάϨʔγϣϯҙ֎ͱଳҬ৯͏ͷͰɺωοτϫʔΫΛͯ͠ ϨʔτϦϛοτೖΕͨ΄͏͕͍͍
ଟɺձʹ͍ΔͷͰɺ టʹͳΓ·͠ΐ͏ʂʂʂ
None
͓͠·͍ ࠷৽ͷ࠾༻ใΛνΣοΫˠ !QC@SFDSVJU