Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
仮想化環境での利用者公平性
Search
Takuya ASADA
November 20, 2012
Technology
0
140
仮想化環境での利用者公平性
Takuya ASADA
November 20, 2012
Tweet
Share
More Decks by Takuya ASADA
See All by Takuya ASADA
Interrupt Affinityについて
syuu1228
0
190
僕のIntel NUCが起動しないわけがない
syuu1228
3
4.2k
Introduction to bhyve
syuu1228
1
330
OSv on bhyve
syuu1228
3
370
ruby-virtualmachine
syuu1228
0
230
BHyVeでOSvを起動したい 〜BIOSがなくてもこの先生きのこるには〜
syuu1228
2
790
10GbE時代のネットワークI/O高速化
syuu1228
14
8.5k
Play with UEFI
syuu1228
1
280
/proc/irq/<IRQ>/ smp_affinity
syuu1228
0
360
Other Decks in Technology
See All in Technology
How CERN serves 1EB of data via FUSE
ennael
PRO
0
15k
Oracle GoldenGate 23ai 導入Tips
oracle4engineer
PRO
1
200
Valuable Software Engineering
avandeursen
0
250
小さな勉強会の始め方、広げ方、あるいは友達の作り方 / How to Start, Grow, and Build Connections with Small Study Groups
ar_tama
2
600
【shownet.conf_】多様化するネットワーク環境を柔軟に統合するルーティングテクノロジー
shownet
PRO
0
280
山手線一周のパフォーマンス改善
suzukahr
0
110
kube-vipとkube-proxy置き換えCiliumを積んだ究極のK3sクラスタを建てる
logica0419
4
190
ドメインと向き合う - 旅行予約編
hidenorigoto
4
520
Understanding and Optimising INP
akshayysharma
0
150
第45回 MLOps 勉強会 - ML Test Score を用いた機械学習システムの定量的なアセスメント
masatakashiwagi
3
170
AWSへのNIST SP800-171管理策 導入に向けての整備/20240930 Mitsutoshi Matsuo
shift_evolve
0
150
入門 バックアップ
ryuichi1208
9
3.5k
Featured
See All Featured
Design by the Numbers
sachag
278
19k
What's new in Ruby 2.0
geeforr
341
31k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Building Adaptive Systems
keathley
37
2.1k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
30
2.6k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
355
29k
Speed Design
sergeychernyshev
22
460
Producing Creativity
orderedlist
PRO
341
39k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9k
A Tale of Four Properties
chriscoyier
155
22k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
How to name files
jennybc
77
98k
Transcript
ԾԽڥͰͷ ར༻ऀެฏੑ ઙా @syuu1228 1211݄20Ր༵
ԾԽڥͷެฏੑʁ • KVMͷήετΛͨ͘͞Μ্ཱͪ͛ͨ࣌ɺϊʔυؒ ͷ௨৴ੑೳCPUར༻࣌ؒͷެฏੑͲͷఔอ ͨΕ͍ͯΔͷ͔ • CPUར༻࣌ؒ • ௨৴ྔ •
ϨΠςϯγ • VMลΓͷෛՙͷΒ͖ͭ 1211݄20Ր༵
ଌͬͯΈΑ͏ • ϨΠςϯγɾ௨৴ྔɾCPU༻ͳͲͷ໘Ͱ Ͳͷఔެฏͳεέδϡʔϧ͕ߦΘΕΔͷ͔ • VMΛ࣮CPUҎ্ʹ૿ͨ࣌͠ʹԿ͕ى ͖Δ͔ • ԾNICͱSR−IOVͰੑೳಛੑ͕ͲͷΑ͏ ʹҟͳΔ͔
1211݄20Ր༵
ੑೳଌఆڥ • ̎ͷLinuxػɾ10G NIC • 1ʙ64ͷKVMήετɾ128ϓϩηεͷ netperf 1VM→VM͋ͨΓ128ϑϩʔ 2VM→VM͋ͨΓ64ϑϩʔ 4VM→VM͋ͨΓ32ϑϩʔ…
64VM→VM͋ͨΓ2ϑϩʔ • TCP Request/ResponseϞʔυ 1byteͷύέοτΛϐϯϙϯ VMϗετ ήετ ςετػ 10G NIC 10G NIC ήετ netperf netserver netperf 1211݄20Ր༵
ϋʔυʗιϑτͷεϖοΫ Distribution Ubuntu Server 12.10 Linux Kernel 3.5.0-18-generic QEMU-KVM 1.2.0
Netperf 2.5.0 CPU(VMϗετ) Intel Core i7 980 (3.33GHz) Memory(VMϗετ) 24GB CPU(ςετػ) Intel Core i7 860 (2.8GHz) Memory(ςετػ) 8GB NIC Intel 82599(ixgbe) ཧ6ίΞɺཧ12ίΞ 1211݄20Ր༵
ήετϚγϯͷઃఆ • ԾCPUɿ1ͭ • ԾNICͷઃఆ • macvtapͰbridge / vhost-net •
SR-IOV • 512MBͷϝϞϦׂΓͯ 1211݄20Ր༵
͓͞Β͍ɿSR-IOV • ཧNIC͕VMʹରͯ͠ԾNICΛఏڙɹIOϋΠύʔόΠβʔΛհࡏͤͣʹߦΘΕΔ • ׂࠐΈ͚ͩԾԽग़དྷ͍ͯͳ͍ͷͰKVMΛ௨ͯ͡సૹ͞ΕΔ • ࠷ੑೳ͕ߴ͍͕ϋʔυରԠ͕ඞཁʗήετͷύέοτΛϑΟϧλʔɾվม͢Δࣄ͍͠ NIC Χʔωϧ Χʔωϧ
ήετ Ϣʔβ PF PFυϥΠό qemu kvm VF υϥΠό TCP/IP ελοΫ NIC2 ϑΥϫʔυ ׂΓࠐΈ ཧׂΓࠐΈ ׂΓࠐΈ ׂࠐΈ ϋϯυϥ VF %."ύεεϧʔ ϋΠύʔόΠβ 1211݄20Ր༵
macvtap <interface type='direct'> <mac address='52:54:00:6b:28:01'/> <source dev='eth1' mode='vepa'/> <model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </interface> 1211݄20Ր༵
SR-IOV <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x04' slot='0x10'
function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </hostdev> 1211݄20Ր༵
ൺֱରͱͯ͠ͷ ࣮ػੑೳଌఆ • ̎ͷLinuxػɾ10G NIC • KVMήετͰnetserver࣮ߦ • 128ϓϩηεͷnetperf •
TCP Request/ResponseϞʔυ 1byteͷύέοτΛϐϯϙϯ VMϗετ ςετػ 10G NIC 10G NIC netperf netserver netperf netserver 1211݄20Ր༵
VMΛ1-64·Ͱ૿Ճ vhost-netͱSR-IOVൺֱ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ 0 1375.00 2750.00 4125.00 5500.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM transaction/sec ཧίΞʹ͍ۙลΓͰ ੑೳ࠷ߴ 1211݄20Ր༵
ϨΠςϯγ 0 750.00 1500.00 2250.00 3000.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM ϨΠςϯγ ཧίΞʹ͍ۙลΓͰ ϨΠςϯγ࠷ద 1211݄20Ր༵
CPUෛՙ 0 25.00 50.00 75.00 100.00 1 2 4 8
16 32 64 vhost-net SR-IOV baremetal VM ύʔηϯςʔδ ෛՙΛVMͷίΞʹ ͔͠ࢄͰ͖ͯͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 30% 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 30% 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 vhost-net SR-IOV VM ύʔηϯςʔδ vhost-net < 10% SR-IOV < 20% 1211݄20Ր༵
ൺֱ݁Ռ • ཧίΞʹ͍ۙVMͩͱੑೳ͕Ұ൪ྑ͍Α͏ʹ ݟ͑Δ • ཧίΞͷ10ഒͷVMΛ࣮ߦͯ͠ɺ͜ͷෛՙύλ ʔϯͰۃͳϨΠςϯγͷѱԽෛՙͷภΓͳͲ ݟΒΕͳ͔ͬͨ • ཧڥͱൺ͔ͯͳΓੑೳ͕ѱ͍
→ෛՙΛෳίΞࢄग़དྷ͍ͯͳ͍ࣄ͕ݪҼͷީ ิͱͯ͠ߟ͑ΒΕΔ 1211݄20Ր༵
Ծڥ্Ͱ ෳίΞωοτϫʔΫIO ෛՙࢄͤͨ࣌͞ͷੑೳ 1211݄20Ր༵
Ծڥ্ͰίΞࢄʁ • ԾCPUΛ1-16·Ͱ૿ͯ͠ΈΔ • ίΞΛ૿͚ͨͩ͠ͰωοτϫʔΫε λοΫ͕ฒྻͰΒͳ͍ • ✕ RSSɿԾNICɾVF͕ະରԠ •
◦ RPSɿήετଆLinuxͷઃఆͰ༗ޮԽ͠ ͯΈΔ 1211݄20Ր༵
RPS $ echo "f" > /sys/class/net/eth1/queues/rx-0/ rps_cpus $ echo 4096
> /sys/class/net/eth1/queues/ rx-0/rps_flow_cnt $ echo 32768 > /proc/sys/net/core/ rps_sock_flow_entries 1211݄20Ր༵
vCPU <vcpu placement='static'>16</vcpu> 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢvhost-netʣ 0 1375.00 2750.00 4125.00 5500.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢvhost-net, RPSʣ 0 1375.00 2750.00 4125.00 5500.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒRPSͰ ੑೳվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢSR-IOVʣ 0 1375.00 2750.00 4125.00 5500.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒվળ 1211݄20Ր༵
ඵؒτϥϯβΫγϣϯ ʢSR-IOV, RPSʣ 0 1375.00 2750.00 4125.00 5500.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM transaction/sec VM =< 4ͳΒRPSͰ ੑೳվળ 1211݄20Ր༵
ϨΠςϯγ ʢvhost-netʣ 0 750.00 1500.00 2250.00 3000.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢvhost-net, RPSʣ 0 750.00 1500.00 2250.00 3000.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢSR-IOVʣ 0 750.00 1500.00 2250.00 3000.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
ϨΠςϯγ ʢSR-IOV, RPSʣ 0 750.00 1500.00 2250.00 3000.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 baremetal VM ϨΠςϯγ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
CPUෛՙ ʢvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2 4
8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VM =< 4ͳΒɺcpu = 1 ͷ࣌ΑΓෛՙΛࢄͰ ͖͍ͯΔʁ 1211݄20Ր༵
CPUෛՙ ʢvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ RPSͰΑΓෛՙΛࢄ 1211݄20Ր༵
CPUෛՙ ʢSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2 4
8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
CPUෛՙ ʢSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ cpu >= 8, VM >=16ͷ࣌ʹ 15%Λ͑ΔภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯ/vhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ cpu >= 8, VM >=16ͷ࣌ʹ 15%Λ͑ΔภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ඵؒτϥϯβΫγϣϯʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ tpsͱ͍͍ͩͨಉ͡ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ ϨΠςϯγʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ૯εϨου͕ཧCPUΛ ͑ͨลΓͰେ͖ͳภΓ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗvhost-netʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ͋·Γେ͖ͳภ ΓݟΒΕͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗvhost-net, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 64 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ ͋·Γେ͖ͳภ ΓݟΒΕͳ͍ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗSR-IOVʣ 0 25.00 50.00 75.00 100.00 1 2
4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VMͷ্ঢͰ एׯ૿Ճ 1211݄20Ր༵
VMؒͷภΓ ʢ CPUෛՙʗSR-IOV, RPSʣ 0 25.00 50.00 75.00 100.00 1
2 4 8 16 32 cpu1 cpu2 cpu4 cpu8 cpu16 VM ύʔηϯςʔδ VMͷ্ঢͰ एׯ૿Ճ 1211݄20Ր༵
ൺֱ݁Ռ • ԾϚγϯͰ࣮ػੑೳͷҎ্Λग़ͤͨͷVM͕͘͝গͳ͘ vCPU͕͔ͳΓଟ͍࣌ͷΈ → ͜ͷ݅Ͱ͔͠RPSͰੑೳ͕ग़ͳ͍ • ૯vCPUΛଟͯ͘͠ҙ֎ͱੑೳԼ͕Βͳ͍ • SR-IOVͷ߹ʹVF͕ཧίΞΛ͑ͨลΓͰ͔ͳΓੑೳʹภ
Γ͕ͰΔ • NIC্ͷΩϡʔʹରͯ͠ड৴ॲཧʹׂΓͯΔίΞ͕ෆͯ͠ ͜ͷΑ͏ͳঢ়گʹͳΔͱਪଌ • vhost-netͰͦͷΑ͏ͳݱ͕ݟΒΕͣɺ૯vCPUΛ૿ͯ͠ ൺֱతެฏʹॲཧ͞Ε͍ͯΔΑ͏ʹݟ͑Δ 1211݄20Ր༵
ϚϧνϢʔβ࣌ͷެ ฏੑ 1211݄20Ր༵
ϚϧνϢʔβ࣌ͷެฏੑ • σϑΥϧτεέδϡʔϦϯάͰɺશͯͷKVM ήετϓϩηε͕ެฏʹCPU࣌ؒΛׂΓͯΒ ΕΔ →ෳͷϢʔβ͕ෳͷVMΛىಈ͍ͯ͠Δ ߹ɺϢʔβลΓͷϦιʔεׂ͕ෆެฏʹͳΔ • ϢʔβؒͰެฏʹεέδϡʔϦϯάͯ͠ཉ͍͠ 1211݄20Ր༵
cgroup • ϓϩηεάϧʔϓʹରͯ͠Ϧιʔε ʢCPUɺϝϞϦɺσΟεΫI/Oɺωοτ ϫʔΫI/Oʣͷར༻Λ੍ݶɾִ͢Δҝ ͷLinuxͷػೳ 1211݄20Ր༵
cgroup tool install $ apt-get install cgroup-bin 1211݄20Ր༵
$ ls /sys/fs/cgroup blkioɹϒϩοΫI/Oͷ੍ݶ cpuɹCPUϦιʔεͷ੍ݶ cpuacctɹCPUͷߦ࣌ؒͳͲͷใऩू cpusetɹCPUͷׂΓ͚ʹؔ͢Δ੍ݶ devicesɹσόΠεͷΞΫηεڐՄɾڋ൱ freezerɹάϧʔϓʹଐ͢ΔϓϩηεͷҰ࣌ఀࢭɾ࠶։ memoryɹϝϞϦϦιʔεͷ੍ݶ
net_clsɹૹग़ύέοτʹtc༻ͷλάΛઃఆ nsɹωʔϜεϖʔεʢίϯςφ༻ʣ 1211݄20Ր༵
cd /sys/fs/cgroup/cpu $ mkdir grp_a $ echo ‘12254’ > grp_a/tasks
$ echo ‘2184’ > grp_a/tasks $ echo ‘512’ > grp_a/cpu.shares $ mkdir grp_b $ echo ‘9012’ > grp_b/tasks $ echo ‘1024’ > grp_b/cpu.shares άϧʔϓ࡞ άϧʔϓॴଐϓϩη εΛՃʢPIDʣ άϧʔϓͷλεΫͰ༻ग़དྷ ΔCPU࣌ؒͷ૬ରత 1211݄20Ր༵
libvirtdͱcgroup • /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>ʹ VMؔ࿈ϓϩηε͕ࣗಈొ͞ΕΔ • /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/ vcpuNʹvcpuϓϩηε͕ࣗಈొ͞ΕΔ • libvirtܦ༝ͰcgroupͷϦιʔε੍ݶઃఆ ͕ग़དྷΔ
1211݄20Ր༵
virshͱcgroup $ virsh schedinfo vm0 Scheduler : posix cpu_shares :
1024 vcpu_period : 100000 vcpu_quota : -1 $ virsh schedinfo --set cpu_shares=512 vm0 1211݄20Ր༵
̎ͭͷάϧʔϓʹެฏʹϦ ιʔεΛׂΓͯͯΈΔ • /sys/fs/cgroup/grp_a • VMΠϯελϯεɿ̒ݸ • cpu.sharesɿσϑΥϧτʢ̍̌̎̐ʣ • /sys/fs/cgroup/grp_b
• VMΠϯελϯεɿ̍̌ݸ • cpu.sharesɿσϑΥϧτʢ̍̌̎̐ʣ 1211݄20Ր༵
άϧʔϓͷ࡞Γํ $ virsh start <VM໊> $ mkdir /sys/fs/cgroup/cpu/grp_a $ cat
/sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/tasks > tmp $ cat /sys/fs/cgroup/cpu/libvirt/qemu/<VM໊>/ vcpu*/tasks >> tmp $ cat tmp > /sys/fs/cgroup/cpu/grp_a/tasks 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢcgroupະ༻࣌ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢcgroup༻࣌ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ grp_a grp_b
1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢcgroupະ༻࣌ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ 1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢcgroup༻࣌ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ grp_a grp_b
1211݄20Ր༵
άϧʔϓ͝ͱͷ߹ܭൺֱ ʢcgroup༻࣌ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.10 1.14 1211݄20Ր༵
ʦࢀߟʧάϧʔϓ͝ͱͷ߹ܭൺֱ ʢcgroupະ༻࣌ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.64 1.65 1211݄20Ր༵
ൺֱ݁Ռ • άϧʔϓʹ͚Δ͚ͩͰσϑΥϧτͰάϧʔϓεέ δϡʔϦϯά͕ߦΘΕɺ࣮ଌͰάϧʔϓʹର͠ ͯެฏʹCPU͕͞Ε͍ͯΔࣄ͕֬ೝͰ͖Δ • CPUΛߜΔࣄͰ݁ՌతʹωοτϫʔΫIOΛߜΔࣄ ͋Δఔग़དྷ͍ͯΔ • Ͱ͋·Γݫີ͡Όͳ͍ʢ࣮ଌ14%͘Β͍ͷޡࠩʣ
1211݄20Ր༵
CFS bandwidth control • ͜ͷάϧʔϓcpu.cfs_period_usͷ୯Ґ࣌ ؒதʹ࠷େcpu.cfs_quota_us͚࣮ͩޮग़དྷΔ • άϧʔϓͷϓϩηεperiod୯Ґ࣌ؒ தɺಉ࣌ʹ࣮ߦ͞Ε͏ΔͷͰquotaperiod Λ͑͏Δʢ2CPUͰΊ͍͍ͬͺ͍ͬͨ
Βquota = period * 2ʣ 1211݄20Ր༵
vhost-net/16VM࣌ͷCPU࣮ߦ࣌ؒ ʢquota=200000,period=50000ʣ 0 625000000 1250000000 1875000000 2500000000 ࣮ߦ࣌ؒ grp_a grp_b
1211݄20Ր༵
vhost-net/16VM࣌ͷωοτϫʔΫIO ʢquota=200000,period=50000ʣ 0 1375000 2750000 4125000 5500000 σʔλྔ grp_a grp_b
1211݄20Ր༵
άϧʔϓ͝ͱͷ߹ܭൺֱ ʢquota=200000,period=50000ʣ 0 0.50 1.00 1.50 2.00 CPU Net grp_a
grp_b 1.01 0.63 1211݄20Ր༵
ݕূ݁Ռ • ͔֬ʹάϥϑ্៉ྷʹͳΔͷ͕ͩɺCPU͕ͲΜͳʹ ۭ͍͍ͯͯquota·ͰߦͬͨΒઈର࣮ߦ͞Εͳ͍ • ͔ͱ͍࣮ͬͯߦ͖͠Εͳ͍ఔͷquotaΛ༩͑Δͱઃఆ͠ ͳ͍߹ͱมΘΒͳ͘ͳΔ • CPUωοτϫʔΫIOͪΐ͏ͲΑ͘ͽͬͨΓ1.0ͱ͍ ͏༁ʹ͍͔ͳ͔ͬͨ
• period͕͗͢ΔͱVM͕͍࣮ؒߦ͞Εͳͯ͘ྑ͘ͳ ͍͜ͱ͕ى͖ͦ͏ 1211݄20Ր༵
net_cls $ tc class add dev virbr0 parent 10: classid
10:1 htb rate 24Mbit $ echo 0x100001 > /sys/fs/cgroup/net_cls/ grp_a/net_cls.classid ϓϩηε͔Βૹ৴͞ΕΔύέοτʹtcͷ λάΛ͚ΔࣄʹΑΓଳҬ੍ݶ͕ग़དྷΔ 1211݄20Ր༵
·ͱΊ • KVMʹΑΔԾԽڥͷϦιʔεׂΓͯͷެฏੑʹ͍ͭͯ ݕূͨ͠ • ࠓճͷ࣮ݧͷ݅ԼͰɺ࣮CPUΛେ͖͑͘ΔVMͰ ۃͳϨΠςϯγͷѱԽෛՙͷภΓͳͲݟΒΕͳ͔ ͬͨ • ୠ͠ɺSR-IOVͷ߹ʹVF͕ཧίΞΛ͑ͨลΓͰ͔
ͳΓੑೳʹภΓ͕ൃੜͨ͠ • cgroupΛ༻͍ͯϚϧνϢʔβ࣌ʹϢʔβؒͷCPUׂΓͯΛ ެฏʹग़དྷΔࣄΛ֬ೝͨ͠ 1211݄20Ր༵