Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
コンテナたちを計測すること ~ ロリポップ!マネージドクラウドの裏側 ~ /wip-how-t...
Search
KONDO Uchio
September 29, 2017
Technology
4
6k
コンテナたちを計測すること ~ ロリポップ!マネージドクラウドの裏側 ~ /wip-how-to-get-container-metrics
Cookpad Tech Kitchen #12
KONDO Uchio
September 29, 2017
Tweet
Share
More Decks by KONDO Uchio
See All by KONDO Uchio
大規模レガシーテストを 倒すための CI基盤の作り方 / #CICD2023
udzura
5
2.3k
Ruby x BPF in Action / RubyKaigi 2022
udzura
0
220
Narrative of Ruby & Rust
udzura
0
190
開発者生産性指標の可視化 / pepabo-four-keys
udzura
3
1.6k
Talk of RBS
udzura
0
410
Re: みなさん最近どうですか? / FGN tech meetup in 2021
udzura
0
740
Dockerとやわらかい仮想化 - ProSec-IT/SECKUN 2021 edition -
udzura
2
700
Device access filtering in cgroup v2
udzura
1
820
"Story of Rucy" on RubyKaigi takeout 2021
udzura
0
770
Other Decks in Technology
See All in Technology
【内製開発Summit 2025】イオンスマートテクノロジーの内製化組織の作り方/In-house-development-summit-AST
aeonpeople
2
1k
Охота на косуль у древних
ashapiro
0
120
30→150人のエンジニア組織拡大に伴うアジャイル文化を醸成する役割と取り組みの変化
nagata03
0
200
OPENLOGI Company Profile for engineer
hr01
1
20k
Cracking the Coding Interview 6th Edition
gdplabs
14
28k
JAWS FESTA 2024「バスロケ」GPS×サーバーレスの開発と運用の舞台裏/jawsfesta2024-bus-gps-serverless
ma2shita
3
270
Oracle Database Technology Night #87-1 : Exadata Database Service on Exascale Infrastructure(ExaDB-XS)サービス詳細
oracle4engineer
PRO
1
200
Pwned Labsのすゝめ
ken5scal
2
500
あなたが人生で成功するための5つの普遍的法則 #jawsug #jawsdays2025 / 20250301 HEROZ
yoshidashingo
2
320
【5分でわかる】セーフィー エンジニア向け会社紹介
safie_recruit
0
19k
ExaDB-XSで利用されているExadata Exascaleについて
oracle4engineer
PRO
3
280
MIMEと文字コードの闇
hirachan
2
1.4k
Featured
See All Featured
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.1k
Speed Design
sergeychernyshev
27
810
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
49
2.3k
Intergalactic Javascript Robots from Outer Space
tanoku
270
27k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
11
1.3k
BBQ
matthewcrist
87
9.5k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
27
1.9k
StorybookのUI Testing Handbookを読んだ
zakiyama
28
5.5k
Java REST API Framework Comparison - PWX 2021
mraible
29
8.4k
Into the Great Unknown - MozCon
thekraken
35
1.6k
Building Applications with DynamoDB
mza
93
6.2k
4 Signs Your Business is Dying
shpigford
183
22k
Transcript
dϩϦϙοϓʂϚωʔδυΫϥυͷཪଆd ۙ౻Ӊஐ࿕(.01FQBCP *OD $PPLQBE5FDI,JUDIFO ίϯςφͨͪΛܭଌ͢Δ͜ͱ
ΤϯδχΞ ۙ౻Ӊஐ࿕!VE[VSB ٕज़෦ٕज़ج൫νʔϜ ͖ͳϥʔϝϯΒʔΊΜཾ (.0ϖύϘॴଐ ͬͯΔ3VCZɺNSVCZɺίϯςφ Γ͍ͨγεςϜϓϩάϥϛϯάɺ-JOVY֦ு
ࢹɾܭଌʹ͍ͭͯɺࠓ ࢼߦࡨޡ͕ଓ͍͍ͯ·͕͢ ࠓݕূͷաఔΛڞ༗͠·͢
ϩϦϙοϓʂϚωʔδυΫϥυ IUUQTNDMPMJQPQKQ ΞϧϑΝςετத
IUUQTNDMPMJQPQKQ ΞϧϑΝςετத ˞͝ڵຯͷ͋ΔํձͰΛֻ͓͚Լ͍͞
ࢹͱ
*5ΠϯϑϥͷࢹͱɺຊॻͰ࣍ͷ௨Γఆ ٛ͠·͢ɻ োൃੜͷ༗ແΛҰఆظؒͰΓͳ֬͘ೝ͢ Δ͜ͱɻ ʰ*5Πϯϑϥࢹ<࣮ફ>ೖʱΑΓ
ࢹɾϝτϦΫεͷత wӡ༻ͱͯ͠ wαʔϏεμϯʹ͙͢ʹؾ͖͍ͨɻ·ͨɺ෮چΛૉૣ͍ͨ͘͠ wαʔϏεͷػೳͱͯ͠ wίϯςφͷෛՙঢ়گɺোঢ়ଶʹͳ͍ͬͯΔ͔൱͔ͳͲɻ·ͨͦͷ ཤྺΛͪΌΜͱՄࢹԽͯ͠ఏڙͰ͖Δͱخ͍͠ wˠͦͷͨΊʹɺϝτϦΫεΛܭଌɺࢹɺෆਖ਼ͳঢ়ଶͷ࣌Ξϥʔ τͳͲΛ͋͛ΒΕΔΑ͏ʹ͠ͳ͚ΕͳΒͳ͍ɻ
ࢹͷͨΊʹ ·ͣܭଌ͠ͳ͍ͱ͍͚ͳ͍
ࠓɺࠔ͍ͬͯΔ͜ͱ wίϯςφී௨ͷ7.ͷࢹͷΑ͏ʹ͍͔ͳ͍ͱ͜Ζ͋Δ wˠϓϩηεʢ܈ʣͳͷͰ-"ͷΑ͏ͳఆ൪ͷ͕औΕͳ͔ͬͨΓͳͲ wίϯςφϗεςΟϯάͰ͋Δ͜ͱɺ·ͨϚΠΫϩαʔϏεతͰ͋Δ͜ ͱʹΑΔɺͭͷʮʯͷ͕͋Δ
ʮίϯςφʯͷ wϚωʔδυΫϥυɺίϯςφΛར༻ͨ͠ߴूੵϓϥοτϑΥʔϜ Ͱ͋Δɻ͢ͳΘͪɺίϯςφ͕ͱͯଟ͍ɻ w͔͠ɺ'BTU$POUBJOFSΛ࠾༻͍ͯ͠ΔͳͲɺίϯςφىಈ͠ଓ͚ ΔΘ͚Ͱͳ͍ɻ·ͨɺҰͭ࣌ͷυϝΠϯʹඥͮ͘ίϯςφͷม ΘΔɻ wͨ͘͞Μ͋ΔͷͰɺूΊΔͱɺ͔Γ͘͢ݟͤΔ͕͋Δ
ʮίϯϙʔωϯτʯͷ wԿ͔ର֎ൃද͍ͯ͠Δ௨ΓɺϚωʔδυΫϥυίϯϙʔωϯτ ͕ଟ͘ɺϚΠΫϩαʔϏεతʹΈ߹Θ͍ͬͯ͞Δ wʮαΠτ͕ݟ͑ͳ͍ʯͷΑ͏ͳ߹ʹɺݪҼͱߟ͑ΒΕΔՕॴͷಛఆ ͕͘͠ɺΤϥʔτϥοΩϯάʹ՝͕͋Δɻ wϘτϧωοΫͷಛఆͳͲಉ༷
গͣͭ͠ɺ֎ງΛຒΊΔ ͦΜͳઓ͍ͷաఔΛ͠·͢
ͲΜͳ͕औΕΔ͔ʁ
ܭଌͷର w$16ϝϞϦར༻ͷࢹɾܭଌ wωοτϫʔΫͷࢹɾܭଌ w*0σΟεΫͷࢹɾܭଌ wϦΫΤετͷτϨʔγϯά wίϯςφϛυϧΣΞͷϩάʢ͜ΕϩάύʔεͰૉʹूΊΔʣ
ࢀߟ wͦʔ͍ͩ͞ΜͷࢿྉΑΓɺجຊతͳγεςϜʹؔΘΔϝτϦοΫ IUUQTPVEBJIBUFOBCMPHDPNFOUSZ
DHSPVQͷTUBUͷར༻ wDQVDQVBDDUαϒγεςϜ wNFNPSZαϒγεςϜ w ଞɺCMLJP WͷJP wςΩετσʔλͰɺར༻ঢ়گ͕Θ͔Δˠύʔεͯ͠׆༻͢Δ DHSPVQࣗମʹ͍ͭͯʮDHSPVQVE[VSBʯ·ͨ
ʮDHSPVQ5FO'PSXBSEʯͰݕࡧ͢Δͱຊޠͷղઆ͕ɻɻɻ root@compute-1:~# cat /sys/fs/cgroup/cpu/$CONTAINER/cpu.stat nr_periods 140373 nr_throttled 25 throttled_time 1112602844
ωοτϫʔΫʹؔ͢ΔϝτϦΫε wݪଇɺίϯςφࣗͷ/FUXPSL/BNFTQBDFʹॴଐ͢Δ w/FUXPSL/BNFTQBDFͲ͏͠ʢࢠ͔ΒʣɺWFUIσόΠεΛܦ༝ ͯ͠௨৴͢Δ͜ͱʹͳΔ wͦͷWFUIσόΠεʹؔ͢Δز͔ͭͱΔखஈ͕͋Δ wJGUPQͷΑ͏ͳίϚϯυɺQSPDOFUEFWϑΝΠϧ wϓϩηε୯ҐͷωοτϫʔΫใQSPD1*%OFU\EFW UDQ VEQ ^
':*ίϯςφωοτϫʔΫͷτϙϩδ wCSJEHFઃఆͷྫɻ%PDLFSͷσϑΥϧτઃఆ͜Εɻ FUI EPDLFS WFUIIPTU WFUIHVFTU WFUIIPTU WFUIHVFTU WFUIIPTU WFUIHVFTU
FH FH ඞཁʹԠ͡/"5ͳͲ /FUXPSL /BNFTQBDF ͻͱͭͷ)PTU CSJEHF ͜ΕΒΛࢹ͢Δʂ
*0ΞΧϯςΟϯά wDHSPVQͷCMLJPJPɺσόΠε୯ҐͷܭଌεϩοτϧͳͷͰɺ/'4 ͷΑ͏ʹσόΠε൪߸͕ৼΒΕͳ͍ϑΝΠϧγεςϜʹ͍͠ wҰํɺϓϩηε୯ҐͷSFBEʢʣɺXSJUFʢʣͷݺͼग़͠ճɺ ಡΈࠐΈॻ͖ࠐΈʹޭͨ͠όΠτͷྦྷܭɺΧʔωϧ͕ूܭͯ͠ ͍ͯར༻Ͱ͖Δɻ wQSPD1*%JPϑΝΠϧܦ༝Ͱ؆୯ʹ֬ೝՄೳ wྫ͑͜ΕΛίϯςφʹॴଐ͢Δϓϩηε͝ͱʹूܭ͢Δ
ϦΫΤεττϥοΩϯά w͋ΔαʔόΛ௨ա͢Δ͝ͱʹಛఆͷϔομʔʹ࣌ؒΛه͍ͯ͘͠ wDG9'PSXBSEFE'PSϔομʔ w3BDLͷൃΛਅࣅͯɺϚʔΧʔΛωετ͍ͯ͘͠ͷͲ͏͔ʁ wίϯϙωϯτ͝ͱͷܦա͕࣌ؒΘ͔ΔΑ͏ʹͳΓɺϘτϧωοΫͳͲ ͕Θ͔ΔɻͦΕͦ͜(SBGBOBͳͲͰՄࢹԽͯ͠ྑ͍ wΤϥʔτϥοΩϯάʹԠ༻Ͱ͖Δʁ IUUQTQZMPOTXFCGSBNFXPSLSFBEUIFEPDTJPFOWSDDPODFQUTIUNM
DGޚࣾͷࢄτϨʔγϯάج൫ wؾʹͳΓ·͢ IUUQUFDIMJGFDPPLQBEDPNFOUSZ
σʔλΛཷΊࠐΉ
େ·͔ͳΞʔΩςΫνϟ wूΊΔˠஷΊΔˠՄࢹԽ͢Δ wσʔλͷอଘઌ1SPNFUIFVT wՄࢹԽ(SBGBOB wσʔλΤʔδΣϯτD"EWJTPSNUBJMɺࣗ࡞FYQPSUFS
1SPNFUIFVT (SBGBOB &YQPSUFS &YQPSUFS &YQPSUFS $POUBJOFS $POUBJOFS $POUBJOFS $POUBJOFS $POUBJOFS
"1*͚1SPYZ ूܭϝτϦΫεͷάϧʔϓผʹ FYQPSUFSΛ༻ҙ Ϣʔβ༷μογϡϘʔυ ʢ։ൃதʣ ӡ༻ΤϯδχΞ σΟϨΫλʔͳͲ "MFSUNBOBHFS
1SPNFUIFVTͷ࠾༻ w(PPHMFͷ#PSHNPOʹӨڹΛड͚ͨࢹπʔϧ w#PSHNPOͷհ43&ຊʹ͋Δͦ͏ wΠϯετʔϧʢجຊతʹϫϯόΠφϦΛඋ͢Ε0,ʣɺ ӡ༻ʢجຊతʹΦϯϝϞϦʴϩʔΧϧετϨʔδʣ͕༰қ wॊೈͳσʔλߏͱߴػೳͳΫΤϦ w"MFSU࿈ܞ͕༰қ w(SBGBOBͳͲՄࢹԽπʔϧ֤छFYQPSUFS͕ἧ͍ͬͯΔ
1SPNFUIFVTͱ(SBGBOBͷ࿈ܞ w(SBGBOBઆ໌ͷඞཁͳ͍΄Ͳͷఆ൪μογϡϘʔυ wެࣜʹυΩϡϝϯτ͕͋Δ wIUUQTQSPNFUIFVTJPEPDTWJTVBMJ[BUJPOHSBGBOB w1SPNFUIFVTͷΫΤϦΛͦͷ·Μ·ॻ͚ͨΓͯ͠ศརײ͕ߴ͍
Մ༻ੑͷߟ͑ํ w1SPNFUIFVT֤ϊʔυͷFYQPSUFS͔ΒσʔλΛQVMM͍ͯ͘͠ελΠ ϧΛͱ͍ͬͯΔʢDG/BHJPT /31&4BB4ܕࢹʣ wͦͷͨΊ"DUJWF"DUJWFߏΛऔΕಉ͡σʔλʹͳΓɺߏͱ ͍͏͜ͱʹͳΔɻ wSFGIUUQTHJUIVCDPNQSPNFUIFVTQSPNFUIFVTJTTVFT
4FSWJDF%JTDPWFSZ wϊʔυ͔ΒQVMM͢ΔͷͰɺ1SPNFUIFVTϊʔυ͕Ͳ͜ʹ͋Δͷ͔ͱ ͍͏ใΛ͍࣋ͬͯͳ͍ͱ͍͚ͳ͍ɻ w4FSWJDF%JTDPWFSZͷͨΊͷํ๏ࣗମ͍͔ͭ͋͘Δɻ wϑΝΠϧϕʔεɺ%/4ϕʔεɺ"840QFO4UBDLͳͲ%ZOBNJD*OGSB ͚ઃఆɺ͋Δ͍.BSBUIPO$POTVMͱ࿈ܞՄೳ
FYQPSUFSͷ֓೦ʹ͍ͭͯ wσʔλΛऔΓ͍ͨϊʔυʹɺFYQPSUFSͱݴ͏ܾ·ͬͨϑΥʔϚοτͰ σʔλΛు͖ग़͢ਓΛৗற͓ͤͯ͘͞ʢ࣮ଶIUUQ"1*ʣɻ w1SPNFUIFVTɺ4FSWJDF%JTDPWFSZΛͯ͠ɺͦͷFYQPSUFSʹΞΫη ε͠ɺఆظతʹσʔλΛಘΔ wFYQPSUFSɺطʹ࣮͞Ε͍ͯΔͷͨ͘͞Μ͋Δ͠ɺ ࣗͰ࣮͢Δ͜ͱՄೳɻ
FYQPSUFSͷྫ wOPEF@FYQPSUFSl&YQPSUFSGPSNBDIJOFNFUSJDTz w$16ɺϝϞϦɺ-"ɺωοτϫʔΫͳͲϚγϯ୯ҐͰ৭ʑͱूܭ w(SBGBOBͳͲͰ͍͍ײ͡ʹάϥϑԽͯ͠׆༻ wD"EWJTPS(PPHMF͚ࣾίϯςφࢹج൫Λϕʔεʹɺ%PDLFSͳ ͲʹରԠͨ͠ͷɻͪͳΈʹTZTUFNEͳͲͷࢹͰศར wͦͷଞɺଟ͘ͷ࣮͕͋Δɻී௨ࣗ࡞͠ͳ͍͍ͯͣ͘ IUUQTHJUIVCDPNQSPNFUIFVTEPDTCMPCNBTUFSDPOUFOUEPDTJOTUSVNFOUJOHFYQPSUFSTNE
D"EWJTPSͰ)BDPOJXBͷϝτϦΫεΛऔΔ
D"EWJTPSͰ)BDPOJXBͷϝτϦΫεΛऔΔ
NUBJMIUUQTHJUIVCDPNHPPHMFNUBJM wઌड़ͨ͠/HJOYͳͲͷϩάΛύʔε্ͨ͠ͰɺFYQPSUFSʹͳͬͯ͘ ΕΔπʔϧ w%4-ʹΑΓϧʔϧΛهड़ wԿ͔طࢹײͷ͋Δྗڧ͞ DG'MVFOUEͷJO@UBJM w/HJOYͷྫˠ IUUQTHJUIVCDPNOEJB[HOHJOYQSPNFUIFVTFYQPSUFSCMPCNBTUFSpMFTQSPHTOHJOYNUBJMɹ
ܭଌͷର w$16ϝϞϦར༻ͷࢹɾܭଌ wωοτϫʔΫͷࢹɾܭଌ w*0σΟεΫͷࢹɾܭଌ wϦΫΤετͷτϨʔγϯά wίϯςφϛυϧΣΞͷϩά (PPHMFπʔϧ܈ͷΤίγεςϜ͕ศརɻ ˡD"EWJTFSͰऔΕΔ ˡD"EWJTFSͰ͢ΕऔΕΔ ˡ·ͩཁݕ౼
ˡNUBJMͰָͰ͖Δ ˡώοʢOHY@NSVCZͳͲͰؤுΕͦ͏ʣ
FYQPSUFSࣗ࡞ wͱ͍͑ɺࣗ࡞Ұͭͷख (PͳͲͳΒ1SPNFUIFVTެࣜͷϥΠϒϥϦ͕͑Δ طଘׂ࣮͕ͱ๛ ͦͦɺຬ͖ͨ͢ϑΥʔϚοτ͕୯७ wIUUQTQSPNFUIFVTJPEPDTJOTUSVNFOUJOHXSJUJOH@FYQPSUFST ΨΠυΛݟͳ͕Βɺ͋ͱطଘ࣮ΛோΊͯॻ͘ͱྑͦ͞͏
FYQPSUFSͱ1SPNFUIFVTͷ࿈ܞ w4FSWJDF%JTDPWFSZͷதͰ$POTVMͱͷ૬ੑྑ͍ͱ͍͏ࢲݟ wFYQPSUFSଆͰɺ$POTVMͷTFSWJDFDIFDLΛՃˠಛఆͷαʔϏεΛ 1SPNFUIFVT͔ΒσΟεΧόϦɺ͕ඇৗʹ༰қʹͳ͍ͬͯΔ wྫ͑ҰʹෳFYQPSUFSͳͲ؆୯ͩ͠ɺFYQPSUFSͷՃআɺ ࢮ׆ࢹ$POTVMʹΔ͜ͱͰߟ͑ͳͯ͘ࡁΉ
͜ΜͳTFSWJDFఆٛΛॻ͚ wεοͱ+0*/͢Δ wศར { "service": { "name": "node_exporter", "tags": [
"development" ], "port": 9100, "checks": [ { "id": "ping", "tcp": "localhost:9100", "interval": "15s", "timeout": "1s" } ] } }
͜ͷষ·ͱΊ wʮσʔλͷஷΊͲ͜Ζʯʹ1SPNFUIFVTΛਅΜதʹਾ͑Δ w1SPNFUIFVTͷΤίγεςϜڧ͍ɻ w͍͟ͱͳΕཉ͍͠σʔλࣗͰऔΓʹߦ͘ w͜ΕΒͷͭͳ͗ʹ$POTVM͕ศར
ࠓޙΓ͍ͨ͜ͱ
ࠓޙΓ͍ͨ͜ͱ wूΊΔͱ͜Ζ·ͰԿͱ͔ͳΓͦ͏ wཷΊࠐΜͩσʔλͷ׆༻ʹ͍ͭͯ w"MFSUNBOBHFSͷ֓೦ʢࠓলུʣ wҟৗݕͷࣗಈԽɺՄࢹԽͳͲ
ҟৗݕͷࣗಈԽɺՄࢹԽ wίϯςφʹؔ͢ΔϝτϦοΫεɺ$.%#ܦ༝ͰूͰ͖Δঢ়ଶ wϓϩδΣΫτ͝ͱͷσʔλΛूΊͯԿ͔Ͱ͖ͳ͍͔ 1SPKFDUT EPNBJO $POUBJOFST $*% DPOUBJOFSJE $POUBJOFST $*%
DPOUBJOFSJE .FUSJDT\EBUBlBBBz ^999 .FUSJDT\EBUBlCCCz ^::: .FUSJDT\EBUBlDDDz ^;;; .FUSJDT\EBUBlEEEz ^888
ूΊͨσʔλΛͲ͏͢Δ͔ w͜Ε͕ɺࠓޙͷ՝ w୯७ʹάϥϑԽͯ͠ΈͯɺͦΕͳΓʹঢ়ଶ͕Θ͔Δؾ͢Δ͠ɺٸ ʹෛՙ͕ߴ͍ɺ௨৴Ͱ͖ͳ͘ͳͬͯΔɺͳͲࢹͰ͖ͦ͏Ͱ͋Δ w·ͣӡ༻ͯ͠Έ͍ͨͱ͜Ζ
ՄࢹԽɾࣗಈఆͷख๏ wϓϩδΣΫτʹඥ͍ͨσʔλɺجຊతʹଟ࣍ݩͷσʔλͰ͋Δͷ Ͱɺ࣍ݩѹॖΛͯ͠ɺʮ͜Εਖ਼ৗʯʮ͜Εোঢ়ଶʯͷΑ͏ʹ ྨ͕Ͱ͖ΔΜͰͳ͍͔ͱߟ͍͑ͯΔ͕ɺࠓޙͷ՝ wେྔͷίϯςφϓϩδΣΫτΛɺࣗݾ৫ԽϚοϓͳͲͰࣗಈͰ ྨͯ͠ɺͦΕΛՄࢹԽͰ͖Δͱ໘നͦ͏ʁ w͜ͷลΓʹɺʮͷʯͷղܾࡦ͕ݟ͑Δؾ͕͍ͯ͠Δ
5P#F$POUJOVFE
·ͩ·ͩख୳Γͷ ཁૉଟ͍Ͱ͕͢ Γ͕͍͕͋Γ·͢ʂ
13
ϚωʔδυΫϥυΛҰॹʹ࡞Γ·͠ΐ͏ ࠷৽ͷ࠾༻ใΛνΣοΫˠ !QC@SFDSVJU QIPUPIUUQTUFDIQFQBCPDPNFOHJOFFSUSBJOJOHWPM