Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Dynamic VM Scheduling in OpenStack
Kazuhiko Yamashita
November 25, 2022
Programming
1
580
Dynamic VM Scheduling in OpenStack
CNDT2022で話した内容です。
Kazuhiko Yamashita
November 25, 2022
Tweet
Share
More Decks by Kazuhiko Yamashita
See All by Kazuhiko Yamashita
GMOペパボにおける大規模インフラのセキュリティ管理
pyama86
1
51
Is Kubernetes On-premises Hardway?
pyama86
2
370
突然のグループ一斉在宅勤務開始!!1に おける働き方を変革する技術や仕組み
pyama86
4
990
企業に必要とされている インフラ技術とこれから
pyama86
12
8.3k
「ペパボっぽい」 エンジニアカルチャーを創る 言葉と仕組み
pyama86
20
5k
CloudNative Buildpacksで創る、CloudNativeな開発体験
pyama86
9
12k
CN Buildpacksが作る未来
pyama86
2
2.4k
クラウドネイティブを創る技術
pyama86
3
1.6k
linux-middleware-development-by-go.pdf
pyama86
1
2.4k
Other Decks in Programming
See All in Programming
tidy_rpart
bk_18
0
600
PHP でガチの電卓を作る
memory1994
PRO
1
110
WordPress(再)入門 - 基礎知識・環境編
oleindesign
1
140
Qiita Night PHP 2023
fuwasegu
0
11k
Findy - エンジニア向け会社紹介 / Findy Letter for Engineers
findyinc
2
42k
量子コンピュータ時代のプログラミングセミナー / 20221222_Amplify_seminar _route_optimization
fixstars
0
250
23年のJavaトレンドは?Quarkusで理解するコンテナネイティブJava
tatsuya1bm
1
120
Enumを自動で網羅的にテストしてみた
estie
0
1.3k
ipa-medit: Memory search and patch tool for IPA without Jailbreaking/ipa-medit-bh2022-europe
tkmru
0
130
Swift Concurrency in GoodNotes
inamiy
4
1.3k
AWSとCPUのムフフな関係
cmdemura
0
470
TSX First な Zero-Runtime SSG potato4d/dodai とその仕組み / owned static site generator #kyotojs
potato4d
0
290
Featured
See All Featured
Building Flexible Design Systems
yeseniaperezcruz
314
35k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
351
21k
Why You Should Never Use an ORM
jnunemaker
PRO
49
7.9k
Into the Great Unknown - MozCon
thekraken
2
290
We Have a Design System, Now What?
morganepeng
37
5.9k
GraphQLとの向き合い方2022年版
quramy
20
9.9k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
15
1.2k
Learning to Love Humans: Emotional Interface Design
aarron
263
38k
The Power of CSS Pseudo Elements
geoffreycrofte
52
4.3k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
270
12k
Statistics for Hackers
jakevdp
785
210k
How To Stay Up To Date on Web Technology
chriscoyier
779
250k
Transcript
Dynamic VM Scheduling in OpenStack ʙ͜ͷલɺθΫγΟಡΜ͚ͩͲɺRFCΑΓ͔ͬͨ͠ฤʙ
ࢁԼ!QZBNB (.0ϖύϘٕज़ج൫νʔϜ γχΞɾϓϦϯγύϧ UFOTOBQPODPN QZBNBGVO TUOTKQ
45/4 -JOVY/444FSWFS TUOTKQ
45/4
None
CNDT ٱ͠ͿΓʹΦϑϥΠϯͰ ࠷ߴ!!1 Ͱ͜Εը͚ͩͲͶ
ϖύϘͷΠϯϑϥߏ CPU StealʹىҼͨ͠ɺྡਓ εʔύʔˑϥΠϒϚΠάϨʔγϣϯ ·ͱΊͱɺ݁ࠗʹ͓͚Δେࣄͳ͜ͱ42બ ࠓ͢͜ͱ
ϖύϘͷΠϯϑϥߏ 0QFO4UBDL #BSFNFUBM "84 IFSPLV ($1 ཧαʔό7.ن ϗεςΟϯάܥͷαʔϏεɺ֤छج൫αʔϏεͰར༻ &,4
3%4 &MBTUJDYYYܥͷϚωʔδυαʔϏεΛத৺ʹར༻ (,& #JH2VFSZΛத৺ʹ"84ͷ%3ɺσʔλΤϯδχΞϦϯάͷج൫ͱͯ͠ར༻
0QFO4UBDL 0QFO4UBDLͰཧ͢Δ7.ͷ্ʹ ࣗࣾ։ൃͷΤϯδϯΛར༻ͯ͠ ,VCFSOFUFTΛσϓϩΠ 0QFO4UBDL #BSFNFUBM #BSFNFUBM #BSFNFUBM 7.
7. 7. ,VCFSOFUFT $POUBJOFS $POUBJOFS $POUBJOFS ,VCFSOFUFT
CPU StealʹىҼͨ͠ɺ ྡਓ
ॠؒɺ৺ɺॏͶͯ ຊ࣌ؒͰαʔϏεͬͯΔ͔Β ϐʔΫλΠϜ͕ॏෳ 9:00 12:00 20:00 ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ ϐʔΫλΠϜ
ϐʔΫλΠϜ ϐʔΫλΠϜ
CPU Steal Ϧιʔε͕ୣ͍߹͍ʹͳΓɺCPU Steal͕ൃੜ
CPU Stealͷ֓೦ Hypervisor VM1 CPU͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU͍͍ͨʂ 10ඵ͋͛Δ
VMڥʹ͓͚ΔCPU Stealͷ֓೦ Hypervisor VM1 CPU͍͍ͨʂ 10ඵ͋͛Δ VM2 CPU͍͍ͨʂ 10ඵ͋͛Δ ࣮ࡍ8ඵ͔͋͛͠ΒΕͳ͔ͬͨ
ࠩͷ2ඵ͕CPU Steal
CPU Steal͕ى͖Δͱʁ αʔόॲཧೳྗͷԼ αʔόϝτϦΫεͷѱԽ HTTPͷϨΠςϯγͷѱԽ ଓͷ૿Ճ ϩʔυΞϕϨʔδͷ্ঢ
εʔύʔˑϥΠϒϚΠάϨʔγϣϯ
ϥΠϒϚΠάϨʔγϣϯ Hypervisor VM1 VM2 10ඵ CPU 2ඵ CPU Hypervisor VM1
10ඵ CPU VMΛఀࢭ͢Δ͜ͱͳ͘HypervisorؒΛҠಈ
ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1
σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍߹
ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 2
ϝϞϦΛಉظͯ͠੩ࢭΛͭ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍߹ Memory
OpenStackͷϥΠϒϚΠάϨʔγϣϯ https://docs.openstack.org/nova/pike/admin/con fi guring-migrations.html
ϥΠϒϚΠάϨʔγϣϯ Hypervisor Hypervisor VM1 Disk Memory VM1 Disk Memory STEP
3 ੩ࢭ͕औΕͨΒ৽͍͠ΠϯελϯεΛ ༗ޮʹͯ͠ɺωοτϫʔΫܦ࿏ΛΓସ͑Δ ͓લ͕ࠓ͔ΒVM1ͩ
εʔύʔˑϥΠϒϚΠάϨʔγϣϯ
εʔύʔˑϥΠϒϚΠάϨʔγϣϯ 1. VMʹ͓͍ͯStealͷൃੜΛݕ͍ͨ͠ 2.Stealͷൃੜ͍ͯ͠ͳ͍HypervisorʹεέδϡʔϦϯά͍ͨ͠ 3.μϯλΠϜΛൃੜͤͨ͘͞ͳ͍ 4.ࣗಈͰ৸ͯΔؒʹͯ͢ऴΘͬͯ΄͍͠ ͱ͍͏ͷΛɺRubyͰ࣮ͨ͠
VMͷStealൃੜݕ Load5͕͖͍͠Ҏ্͔ͭɺSteal͕ൃੜ͍ͯ͠Δ͜ͱ Steal͕ൃੜͯ͠ɺॲཧ͕Ԇ͍ͯ͠ͳ͍ͳΒͳ͍
εέδϡʔϦϯά HypervisorͷϝτϦΫεͰSteal͕͖͍͠ҎԼͰɺΑΓۭ͖͕͋Δ ಉ͡ϩʔϧͷVM͕ಉҰαʔόʹ͖͍͠Ҏ্ऩ༰͞Ε͍ͯͳ͍ ಉ͡AZͷαʔόͰ͋Δ
Availability Zone LB AZ-1 VM1 AZ-2 VM2 LB AZ-1 VM1
VM2 AZ-2 AZΛ·͍ͨͰεέδϡʔϦϯά͢Δͱ োੑ͕ഁ໓!!1
μϯλΠϜΛൃੜͤ͞ͳ͍ ΦʔέετϨʔλʔͰ͢Δ $ kubectl drain example.local --force --ignore-daemonsets —delete-emptydir-data #
after migrated $ kubectl uncordon example.local $ consul maint -enable # after migrated $ consul maint -disable Kubernetes Consul
kubectl drain node-1 pod-1 Scheduling Disabled node-2 pod-2 pod-1 nodeͷPodͷׂΛఀࢭͯ͠ɺ͍ग़͢
consul node-1 (10.0.0.1) nginx node-2 (10.0.0.2) nginx node-3
(10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 10.0.0.3 ϔϧενΣοΫͳͲΛར༻ͯ͠ɺαʔϏεσΟεΧόϦ͕Ͱ͖Δ DNSͷଞʹɺConsul TemplateΛར༻ͨ͠ ߏϑΝΠϧͷϨϯμϦϯάͳͲͰ׆༻͍ͯ͠Δ
consul maint -enable node-1 (10.0.0.1) nginx node-2 (10.0.0.2)
nginx node-3 (10.0.0.3) nginx $ dig web.service.consul a 10.0.0.1 10.0.0.2 Ϋϥελ͔ΒΓ͠ consul maint -enable
ॲཧܥ·ͱΊ Stealͷൃੜ͔ͭ ϩʔυΞϕϨʔδͷ্ঢ ࠷దͳ؋ͷબఆ Ϋϥελ͔ΒͷΓ͠ ϥΠϒϚΠάϨʔγϣϯ ΫϥελͷΈࠐΈ ࣮ߦڥͱͯ͠OpenStackͷ্Ͱಈ͘ Kubernetesͷதͷpodͱͯ͠ɺ Reconciliation
loopͬΆ͘ಈ͍͍ͯΔ
ຊʹ͋ͬͨා͍
ʮεʔύʔˑϥΠϒϚΠάϨʔγϣϯ ಈ͍ͯΔΜͰ͚͢ͲɺϨΠςϯγ ߴ͍ΜͰ͢ΑͶʯ
ϥΠϒϚΠάϨʔγϣϯ͕ଳҬΛ৯͍Δ ͓ன͔Βഁ໓!!1
ϥΠϒϚΠάϨʔγϣϯ(࠶ܝ) Hypervisor Hypervisor VM1 Disk Memory VM1 Disk STEP 1
σΟεΫΛಉظ ※ڞ༗ετϨʔδΛར༻͍ͯ͠ͳ͍߹ ʊਓਓਓਓʊ ʼɹ500Gɹʻ ʉY^Y^Y^Y^ʉ
None
None
·ͱΊ ྡਓΛࣗͰεέδϡʔϥʔॻ͘͜ͱͰղܾͨ͠ OSSʹ͢ΔͱdeschedulerͷΑ͏ͳநʹͳΔͷͰɺ ࠓճެ։͠ͳ͔ͬͨ ϥΠϒϚΠάϨʔγϣϯҙ֎ͱଳҬ৯͏ͷͰɺωοτϫʔΫΛͯ͠ ϨʔτϦϛοτೖΕͨ΄͏͕͍͍
ଟɺձʹ͍ΔͷͰɺ టʹͳΓ·͠ΐ͏ʂʂʂ
None
͓͠·͍ ࠷৽ͷ࠾༻ใΛνΣοΫˠ
[email protected]