Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
仮想化環境での利用者公平性
Search
Takuya ASADA
November 20, 2012
Technology
0
160
仮想化環境での利用者公平性
Takuya ASADA
November 20, 2012
Tweet
Share
More Decks by Takuya ASADA
See All by Takuya ASADA
Interrupt Affinityについて
syuu1228
0
230
僕のIntel NUCが起動しないわけがない
syuu1228
3
4.3k
Introduction to bhyve
syuu1228
1
370
OSv on bhyve
syuu1228
3
380
ruby-virtualmachine
syuu1228
0
240
BHyVeでOSvを起動したい 〜BIOSがなくてもこの先生きのこるには〜
syuu1228
2
830
10GbE時代のネットワークI/O高速化
syuu1228
14
8.6k
Play with UEFI
syuu1228
1
310
/proc/irq/<IRQ>/ smp_affinity
syuu1228
0
400
Other Decks in Technology
See All in Technology
マーケットプレイス版Oracle WebCenter Content For OCI
oracle4engineer
PRO
3
540
OPENLOGI Company Profile for engineer
hr01
1
20k
JAWS FESTA 2024「バスロケ」GPS×サーバーレスの開発と運用の舞台裏/jawsfesta2024-bus-gps-serverless
ma2shita
3
340
遷移の高速化 ヤフートップの試行錯誤
narirou
6
1.9k
フォーイット_エンジニア向け会社紹介資料_Forit_Company_Profile.pdf
forit_tech
1
1.7k
完璧を捨てろ! “攻め”のQAがもたらすスピードと革新/20250306 Hiroki Hachisuka
shift_evolve
0
100
大規模アジャイルフレームワークから学ぶエンジニアマネジメントの本質
staka121
PRO
3
1.6k
データベースの負荷を紐解く/untangle-the-database-load
emiki
2
550
LayerXにおけるAI活用事例とその裏側(2025年2月) バクラクの目指す “業務の自動運転” の例 / layerx-ai-deim2025
yuya4
1
560
LINE NEWSにおけるバックエンド開発
lycorptech_jp
PRO
0
370
AIエージェント時代のエンジニアになろう #jawsug #jawsdays2025 / 20250301 Agentic AI Engineering
yoshidashingo
9
4.1k
開発組織を進化させる!AWSで実践するチームトポロジー
iwamot
2
540
Featured
See All Featured
Music & Morning Musume
bryan
46
6.4k
Testing 201, or: Great Expectations
jmmastey
42
7.2k
Optimising Largest Contentful Paint
csswizardry
34
3.1k
Faster Mobile Websites
deanohume
306
31k
Why You Should Never Use an ORM
jnunemaker
PRO
55
9.2k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
59k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.2k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
46
2.4k
Build The Right Thing And Hit Your Dates
maggiecrowley
34
2.5k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
VelocityConf: Rendering Performance Case Studies
addyosmani
328
24k
Fireside Chat
paigeccino
35
3.2k
Transcript
ԾԽڥͰͷ ར༻ऀެฏੑ ઙా @syuu1228 1211݄20Ր༵
ԾԽڥͷެฏੑʁ • KVMͷήετΛͨ͘͞Μ্ཱͪ͛ͨ࣌ɺϊʔυؒ ͷ௨৴ੑೳCPUར༻࣌ؒͷެฏੑͲͷఔอ ͨΕ͍ͯΔͷ͔ • CPUར༻࣌ؒ • ௨৴ྔ •
ϨΠςϯγ • VMลΓͷෛՙͷΒ͖ͭ 1211݄20Ր༵
ଌͬͯΈΑ͏ • ϨΠςϯγɾ௨৴ྔɾCPU༻ͳͲͷ໘Ͱ Ͳͷఔެฏͳεέδϡʔϧ͕ߦΘΕΔͷ͔ • VMΛ࣮CPUҎ্ʹ૿ͨ࣌͠ʹԿ͕ى ͖Δ͔ • ԾNICͱSR−IOVͰੑೳಛੑ͕ͲͷΑ͏ ʹҟͳΔ͔
1211݄20Ր༵
ੑೳଌఆڥ • ̎ͷLinuxػɾ10G NIC • 1ʙ64ͷKVMήετɾ128ϓϩηεͷ netperf 1VM→VM͋ͨΓ128ϑϩʔ 2VM→VM͋ͨΓ64ϑϩʔ 4VM→VM͋ͨΓ32ϑϩʔ…
64VM→VM͋ͨΓ2ϑϩʔ • TCP Request/ResponseϞʔυ 1byteͷύέοτΛϐϯϙϯ VMϗετ ήετ ςετػ 10G NIC 10G NIC ήετ netperf netserver netperf 1211݄20Ր༵
ϋʔυʗιϑτͷεϖοΫ Distribution Ubuntu Server 12.10 Linux Kernel 3.5.0-18-generic QEMU-KVM 1.2.0
Netperf 2.5.0 CPU(VMϗετ) Intel Core i7 980 (3.33GHz) Memory(VMϗετ) 24GB CPU(ςετػ) Intel Core i7 860 (2.8GHz) Memory(ςετػ) 8GB NIC Intel 82599(ixgbe) ཧ6ίΞɺཧ12ίΞ 1211݄20Ր༵
ήετϚγϯͷઃఆ • ԾCPUɿ1ͭ • ԾNICͷઃఆ • macvtapͰbridge / vhost-net •
SR-IOV • 512MBͷϝϞϦׂΓͯ 1211݄20Ր༵
͓͞Β͍ɿSR-IOV • ཧNIC͕VMʹରͯ͠ԾNICΛఏڙɹIOϋΠύʔόΠβʔΛհࡏͤͣʹߦΘΕΔ • ׂࠐΈ͚ͩԾԽग़དྷ͍ͯͳ͍ͷͰKVMΛ௨ͯ͡సૹ͞ΕΔ • ࠷ੑೳ͕ߴ͍͕ϋʔυରԠ͕ඞཁʗήετͷύέοτΛϑΟϧλʔɾվม͢Δࣄ͍͠ NIC Χʔωϧ Χʔωϧ
ήετ Ϣʔβ PF PFυϥΠό qemu kvm VF υϥΠό TCP/IP ελοΫ NIC2 ϑΥϫʔυ ׂΓࠐΈ ཧׂΓࠐΈ ׂΓࠐΈ ׂࠐΈ ϋϯυϥ VF %."ύεεϧʔ ϋΠύʔόΠβ 1211݄20Ր༵
macvtap <interface type='direct'> <mac address='52:54:00:6b:28:01'/> <source dev='eth1' mode='vepa'/> <model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </interface> 1211݄20Ր༵
SR-IOV <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x04' slot='0x10'
function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </hostdev> 1211݄20Ր༵
ൺֱରͱͯ͠ͷ ࣮ػੑೳଌఆ • ̎ͷLinuxػɾ10G NIC • KVMήετͰnetserver࣮ߦ • 128ϓϩηεͷnetperf •
TCP Request/ResponseϞʔυ 1byteͷύέοτΛϐϯϙϯ VMϗετ ςετػ 10G NIC 10G NIC netperf netserver netperf netserver 1211݄20Ր༵
VMΛ1-64·Ͱ૿Ճ vhost-netͱSR-IOVൺֱ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ 0 1375.00 2750.00 4125.00 5500.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM transaction/sec ཧίΞʹ͍ۙลΓͰ ੑೳ࠷ߴ 1211݄20Ր༵
ϨΠςϯγ 0 750.00 1500.00 2250.00 3000.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM ϨΠςϯγ ཧίΞʹ͍ۙลΓͰ ϨΠςϯγ࠷ద 1211݄20Ր༵
CPUෛՙ 0 25.00 50.00 75.00 100.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM ύʔηϯςʔδ ෛՙΛVMͷίΞʹ ͔͠ࢄͰ͖ͯͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 30% 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 30% 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 20% 1211݄20Ր༵
ൺֱ݁Ռ • ཧίΞʹ͍ۙVMͩͱੑೳ͕Ұ൪ྑ͍Α͏ʹ ݟ͑Δ • ཧίΞͷ10ഒͷVMΛ࣮ߦͯ͠ɺ͜ͷෛՙύλ ʔϯͰۃͳϨΠςϯγͷѱԽෛՙͷภΓͳͲ ݟΒΕͳ͔ͬͨ • ཧڥͱൺ͔ͯͳΓੑೳ͕ѱ͍
→ෛՙΛෳίΞࢄग़དྷ͍ͯͳ͍ࣄ͕ݪҼͷީ ิͱͯ͠ߟ͑ΒΕΔ 1211݄20Ր༵
Ծڥ্Ͱ ෳίΞωοτϫʔΫIO ෛՙࢄͤͨ࣌͞ͷੑೳ 1211݄20Ր༵
Ծڥ্ͰίΞࢄʁ • ԾCPUΛ1-16·Ͱ૿ͯ͠ΈΔ • ίΞΛ૿͚ͨͩ͠ͰωοτϫʔΫε λοΫ͕ฒྻͰΒͳ͍ • ✕ RSSɿԾNICɾVF͕ະରԠ •
◦ RPSɿήετଆLinuxͷઃఆͰ༗ޮԽ͠ ͯΈΔ 1211݄20Ր༵
RPS $ echo "f" > /sys/class/net/eth1/queues/rx-0/ rps_cpus $ echo 4096
> /sys/class/net/eth1/queues/ rx-0/rps_flow_cnt $ echo 32768 > /proc/sys/net/core/ rps_sock_flow_entries 1211݄20Ր༵
vCPU <vcpu placement='static'>16</vcpu> 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢvhost-netʣ 0 1375.00 2750.00 4125.00 5500.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢvhost-net, RPSʣ 0 1375.00 2750.00 4125.00 5500.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒRPSͰ ੑೳվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢSR-IOVʣ 0 1375.00 2750.00 4125.00 5500.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢSR-IOV, RPSʣ 0 1375.00 2750.00 4125.00 5500.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒRPSͰ ੑೳվળ 1211݄20Ր༵
ϨΠςϯγ ʢvhost-netʣ 0 750.00 1500.00 2250.00 3000.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢvhost-net, RPSʣ 0 750.00 1500.00 2250.00 3000.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢSR-IOVʣ 0 750.00 1500.00 2250.00 3000.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢSR-IOV, RPSʣ 0 750.00 1500.00 2250.00 3000.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
CPUෛՙ ʢvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VM =< 4ͳΒɺcpu = 1 ͷ࣌ΑΓෛՙΛࢄͰ ͖͍ͯΔʁ 1211݄20Ր༵
CPUෛՙ ʢvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ RPSͰΑΓෛՙΛࢄ 1211݄20Ր༵
CPUෛՙ ʢSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
CPUෛՙ ʢSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ cpu >= 8, VM >=16ͷ࣌ʹ 15%Λ͑ΔภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯ/vhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ cpu >= 8, VM >=16ͷ࣌ʹ 15%Λ͑ΔภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ͋·Γେ͖ͳภ ΓݟΒΕͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ͋·Γେ͖ͳภ ΓݟΒΕͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VMͷ্ঢͰ एׯ૿Ճ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VMͷ্ঢͰ एׯ૿Ճ 1211݄20Ր༵
ൺֱ݁Ռ • ԾϚγϯͰ࣮ػੑೳͷҎ্Λग़ͤͨͷVM͕͘͝গͳ͘ vCPU͕͔ͳΓଟ͍࣌ͷΈ → ͜ͷ݅Ͱ͔͠RPSͰੑೳ͕ग़ͳ͍ • ૯vCPUΛଟͯ͘͠ҙ֎ͱੑೳԼ͕Βͳ͍ • SR-IOVͷ߹ʹVF͕ཧίΞΛ͑ͨลΓͰ͔ͳΓੑೳʹภ
Γ͕ͰΔ • NIC্ͷΩϡʔʹରͯ͠ड৴ॲཧʹׂΓͯΔίΞ͕ෆͯ͠ ͜ͷΑ͏ͳঢ়گʹͳΔͱਪଌ • vhost-netͰͦͷΑ͏ͳݱ͕ݟΒΕͣɺ૯vCPUΛ૿ͯ͠ ൺֱతެฏʹॲཧ͞Ε͍ͯΔΑ͏ʹݟ͑Δ 1211݄20Ր༵
ϚϧνϢʔβ࣌ͷެ ฏੑ 1211݄20Ր༵
ϚϧνϢʔβ࣌ͷެฏੑ • σϑΥϧτεέδϡʔϦϯάͰɺશͯͷKVM ήετϓϩηε͕ެฏʹCPU࣌ؒΛׂΓͯΒ ΕΔ →ෳͷϢʔβ͕ෳͷVMΛىಈ͍ͯ͠Δ ߹ɺϢʔβลΓͷϦιʔεׂ͕ෆެฏʹͳΔ • ϢʔβؒͰެฏʹεέδϡʔϦϯάͯ͠ཉ͍͠ 1211݄20Ր༵
cgroup • ϓϩηεάϧʔϓʹରͯ͠Ϧιʔε ʢCPUɺϝϞϦɺσΟεΫI/Oɺωοτ ϫʔΫI/Oʣͷར༻Λ੍ݶɾִ͢Δҝ ͷLinuxͷػೳ 1211݄20Ր༵
cgroup tool install $ apt-get install cgroup-bin 1211݄20Ր༵
$ ls /sys/fs/cgroup blkioɹϒϩοΫI/Oͷ੍ݶ cpuɹCPUϦιʔεͷ੍ݶ cpuacctɹCPUͷߦ࣌ؒͳͲͷใऩू cpusetɹCPUͷׂΓ͚ʹؔ͢Δ੍ݶ devicesɹσόΠεͷΞΫηεڐՄɾڋ൱ freezerɹάϧʔϓʹଐ͢ΔϓϩηεͷҰ࣌ఀࢭɾ࠶։ memoryɹϝϞϦϦιʔεͷ੍ݶ
net_clsɹૹग़ύέοτʹtc༻ͷλάΛઃఆ nsɹωʔϜεϖʔεʢίϯςφ༻ʣ 1211݄20Ր༵
cd /sys/fs/cgroup/cpu $ mkdir grp_a $ echo ‘12254’ > grp_a/tasks
$ echo ‘2184’ > grp_a/tasks $ echo ‘512’ > grp_a/cpu.shares $ mkdir grp_b $ echo ‘9012’ > grp_b/tasks $ echo ‘1024’ > grp_b/cpu.shares άϧʔϓ࡞ άϧʔϓॴଐϓϩη εΛՃʢPIDʣ άϧʔϓͷλεΫͰ༻ग़དྷ ΔCPU࣌ؒͷ૬ରత 1211݄20Ր༵
libvirtdͱcgroup • /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>ʹ VMؔ࿈ϓϩηε͕ࣗಈొ͞ΕΔ • /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/ vcpuNʹvcpuϓϩηε͕ࣗಈొ͞ΕΔ • libvirtܦ༝ͰcgroupͷϦιʔε੍ݶઃఆ ͕ग़དྷΔ
1211݄20Ր༵
virshͱcgroup $ virsh schedinfo vm0 Scheduler : posix cpu_shares :
1024 vcpu_period : 100000 vcpu_quota : -1 $ virsh schedinfo --set cpu_shares=512 vm0 1211݄20Ր༵
̎ͭͷάϧʔϓʹެฏʹϦ ιʔεΛׂΓͯͯΈΔ • /sys/fs/cgroup/grp_a • VMΠϯελϯεɿ̒ݸ • cpu.sharesɿσϑΥϧτʢ̍̌̎̐ʣ • /sys/fs/cgroup/grp_b
• VMΠϯελϯεɿ̍̌ݸ • cpu.sharesɿσϑΥϧτʢ̍̌̎̐ʣ 1211݄20Ր༵
άϧʔϓͷ࡞Γํ $ virsh start <VM໊> $ mkdir /sys/fs/cgroup/cpu/grp_a $ cat
/sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/tasks > tmp $ cat /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/ vcpu*/tasks >> tmp $ cat tmp > /sys/fs/cgroup/cpu/grp_a/tasks 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢcgroupະ༻࣌ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢcgroup༻࣌ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ grp_a grp_b
1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢcgroupະ༻࣌ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ 1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢcgroup༻࣌ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ grp_a grp_b
1211݄20Ր༵
άϧʔϓ͝ͱͷ߹ܭൺֱ ʢcgroup༻࣌ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.10 1.14 1211݄20Ր༵
ʦࢀߟʧάϧʔϓ͝ͱͷ߹ܭൺֱ ʢcgroupະ༻࣌ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.64 1.65 1211݄20Ր༵
ൺֱ݁Ռ • άϧʔϓʹ͚Δ͚ͩͰσϑΥϧτͰάϧʔϓεέ δϡʔϦϯά͕ߦΘΕɺ࣮ଌͰάϧʔϓʹର͠ ͯެฏʹCPU͕͞Ε͍ͯΔࣄ͕֬ೝͰ͖Δ • CPUΛߜΔࣄͰ݁ՌతʹωοτϫʔΫIOΛߜΔࣄ ͋Δఔग़དྷ͍ͯΔ • Ͱ͋·Γݫີ͡Όͳ͍ʢ࣮ଌ14%͘Β͍ͷޡࠩʣ
1211݄20Ր༵
CFS bandwidth control • ͜ͷάϧʔϓcpu.cfs_period_usͷ୯Ґ࣌ ؒதʹ࠷େcpu.cfs_quota_us͚࣮ͩޮग़དྷΔ • άϧʔϓͷϓϩηεperiod୯Ґ࣌ؒ தɺಉ࣌ʹ࣮ߦ͞Ε͏ΔͷͰquotaperiod Λ͑͏Δʢ2CPUͰΊ͍͍ͬͺ͍ͬͨ
Βquota = period * 2ʣ 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢquota=200000,period=50000ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ grp_a grp_b
1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢquota=200000,period=50000ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ grp_a grp_b
1211݄20Ր༵
άϧʔϓ͝ͱͷ߹ܭൺֱ ʢquota=200000,period=50000ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.01 0.63 1211݄20Ր༵
ݕূ݁Ռ • ͔֬ʹάϥϑ্៉ྷʹͳΔͷ͕ͩɺCPU͕ͲΜͳʹ ۭ͍͍ͯͯquota·ͰߦͬͨΒઈର࣮ߦ͞Εͳ͍ • ͔ͱ͍࣮ͬͯߦ͖͠Εͳ͍ఔͷquotaΛ༩͑Δͱઃఆ͠ ͳ͍߹ͱมΘΒͳ͘ͳΔ • CPUωοτϫʔΫIOͪΐ͏ͲΑ͘ͽͬͨΓ1.0ͱ͍ ͏༁ʹ͍͔ͳ͔ͬͨ
• period͕͗͢ΔͱVM͕͍࣮ؒߦ͞Εͳͯ͘ྑ͘ͳ ͍͜ͱ͕ى͖ͦ͏ 1211݄20Ր༵
net_cls $ tc class add dev virbr0 parent 10: classid
10:1 htb rate 24Mbit $ echo 0x100001 > /sys/fs/cgroup/net_cls/ grp_a/net_cls.classid ϓϩηε͔Βૹ৴͞ΕΔύέοτʹtcͷ λάΛ͚ΔࣄʹΑΓଳҬ੍ݶ͕ग़དྷΔ 1211݄20Ր༵
·ͱΊ • KVMʹΑΔԾԽڥͷϦιʔεׂΓͯͷެฏੑʹ͍ͭͯ ݕূͨ͠ • ࠓճͷ࣮ݧͷ݅ԼͰɺ࣮CPUΛେ͖͑͘ΔVMͰ ۃͳϨΠςϯγͷѱԽෛՙͷภΓͳͲݟΒΕͳ͔ ͬͨ • ୠ͠ɺSR-IOVͷ߹ʹVF͕ཧίΞΛ͑ͨลΓͰ͔
ͳΓੑೳʹภΓ͕ൃੜͨ͠ • cgroupΛ༻͍ͯϚϧνϢʔβ࣌ʹϢʔβؒͷCPUׂΓͯΛ ެฏʹग़དྷΔࣄΛ֬ೝͨ͠ 1211݄20Ր༵