Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
コンテナたちを計測すること ~ ロリポップ!マネージドクラウドの裏側 ~ /wip-how-t...
Search
KONDO Uchio
September 29, 2017
Technology
4
6.1k
コンテナたちを計測すること ~ ロリポップ!マネージドクラウドの裏側 ~ /wip-how-to-get-container-metrics
Cookpad Tech Kitchen #12
KONDO Uchio
September 29, 2017
Tweet
Share
More Decks by KONDO Uchio
See All by KONDO Uchio
大規模レガシーテストを 倒すための CI基盤の作り方 / #CICD2023
udzura
5
2.4k
Ruby x BPF in Action / RubyKaigi 2022
udzura
0
250
Narrative of Ruby & Rust
udzura
0
220
開発者生産性指標の可視化 / pepabo-four-keys
udzura
3
1.7k
Talk of RBS
udzura
0
450
Re: みなさん最近どうですか? / FGN tech meetup in 2021
udzura
0
780
Dockerとやわらかい仮想化 - ProSec-IT/SECKUN 2021 edition -
udzura
2
730
Device access filtering in cgroup v2
udzura
1
900
"Story of Rucy" on RubyKaigi takeout 2021
udzura
0
830
Other Decks in Technology
See All in Technology
Кто отправит outbox? Валентин Удальцов, автор канала Пых
lamodatech
0
340
フィンテック養成勉強会#54
finengine
0
180
標準技術と独自システムで作る「つらくない」SaaS アカウント管理 / Effortless SaaS Account Management with Standard Technologies & Custom Systems
yuyatakeyama
3
1.3k
登壇ネタの見つけ方 / How to find talk topics
pinkumohikan
5
470
Github Copilot エージェントモードで試してみた
ochtum
0
100
Welcome to the LLM Club
koic
0
170
生成AI時代 文字コードを学ぶ意義を見出せるか?
hrsued
1
420
25分で解説する「最小権限の原則」を実現するための AWS「ポリシー」大全 / 20250625-aws-summit-aws-policy
opelab
9
1.1k
Model Mondays S2E02: Model Context Protocol
nitya
0
220
Prox Industries株式会社 会社紹介資料
proxindustries
0
290
Node-REDのFunctionノードでMCPサーバーの実装を試してみた / Node-RED × MCP 勉強会 vol.1
you
PRO
0
110
~宇宙最速~2025年AWS Summit レポート
satodesu
1
1.8k
Featured
See All Featured
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
How GitHub (no longer) Works
holman
314
140k
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.3k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Scaling GitHub
holman
459
140k
It's Worth the Effort
3n
185
28k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
Site-Speed That Sticks
csswizardry
10
660
Balancing Empowerment & Direction
lara
1
370
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
331
22k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Transcript
dϩϦϙοϓʂϚωʔδυΫϥυͷཪଆd ۙ౻Ӊஐ࿕(.01FQBCP *OD $PPLQBE5FDI,JUDIFO ίϯςφͨͪΛܭଌ͢Δ͜ͱ
ΤϯδχΞ ۙ౻Ӊஐ࿕!VE[VSB ٕज़෦ٕज़ج൫νʔϜ ͖ͳϥʔϝϯΒʔΊΜཾ (.0ϖύϘॴଐ ͬͯΔ3VCZɺNSVCZɺίϯςφ Γ͍ͨγεςϜϓϩάϥϛϯάɺ-JOVY֦ு
ࢹɾܭଌʹ͍ͭͯɺࠓ ࢼߦࡨޡ͕ଓ͍͍ͯ·͕͢ ࠓݕূͷաఔΛڞ༗͠·͢
ϩϦϙοϓʂϚωʔδυΫϥυ IUUQTNDMPMJQPQKQ ΞϧϑΝςετத
IUUQTNDMPMJQPQKQ ΞϧϑΝςετத ˞͝ڵຯͷ͋ΔํձͰΛֻ͓͚Լ͍͞
ࢹͱ
*5ΠϯϑϥͷࢹͱɺຊॻͰ࣍ͷ௨Γఆ ٛ͠·͢ɻ োൃੜͷ༗ແΛҰఆظؒͰΓͳ֬͘ೝ͢ Δ͜ͱɻ ʰ*5Πϯϑϥࢹ<࣮ફ>ೖʱΑΓ
ࢹɾϝτϦΫεͷత wӡ༻ͱͯ͠ wαʔϏεμϯʹ͙͢ʹؾ͖͍ͨɻ·ͨɺ෮چΛૉૣ͍ͨ͘͠ wαʔϏεͷػೳͱͯ͠ wίϯςφͷෛՙঢ়گɺোঢ়ଶʹͳ͍ͬͯΔ͔൱͔ͳͲɻ·ͨͦͷ ཤྺΛͪΌΜͱՄࢹԽͯ͠ఏڙͰ͖Δͱخ͍͠ wˠͦͷͨΊʹɺϝτϦΫεΛܭଌɺࢹɺෆਖ਼ͳঢ়ଶͷ࣌Ξϥʔ τͳͲΛ͋͛ΒΕΔΑ͏ʹ͠ͳ͚ΕͳΒͳ͍ɻ
ࢹͷͨΊʹ ·ͣܭଌ͠ͳ͍ͱ͍͚ͳ͍
ࠓɺࠔ͍ͬͯΔ͜ͱ wίϯςφී௨ͷ7.ͷࢹͷΑ͏ʹ͍͔ͳ͍ͱ͜Ζ͋Δ wˠϓϩηεʢ܈ʣͳͷͰ-"ͷΑ͏ͳఆ൪ͷ͕औΕͳ͔ͬͨΓͳͲ wίϯςφϗεςΟϯάͰ͋Δ͜ͱɺ·ͨϚΠΫϩαʔϏεతͰ͋Δ͜ ͱʹΑΔɺͭͷʮʯͷ͕͋Δ
ʮίϯςφʯͷ wϚωʔδυΫϥυɺίϯςφΛར༻ͨ͠ߴूੵϓϥοτϑΥʔϜ Ͱ͋Δɻ͢ͳΘͪɺίϯςφ͕ͱͯଟ͍ɻ w͔͠ɺ'BTU$POUBJOFSΛ࠾༻͍ͯ͠ΔͳͲɺίϯςφىಈ͠ଓ͚ ΔΘ͚Ͱͳ͍ɻ·ͨɺҰͭ࣌ͷυϝΠϯʹඥͮ͘ίϯςφͷม ΘΔɻ wͨ͘͞Μ͋ΔͷͰɺूΊΔͱɺ͔Γ͘͢ݟͤΔ͕͋Δ
ʮίϯϙʔωϯτʯͷ wԿ͔ର֎ൃද͍ͯ͠Δ௨ΓɺϚωʔδυΫϥυίϯϙʔωϯτ ͕ଟ͘ɺϚΠΫϩαʔϏεతʹΈ߹Θ͍ͬͯ͞Δ wʮαΠτ͕ݟ͑ͳ͍ʯͷΑ͏ͳ߹ʹɺݪҼͱߟ͑ΒΕΔՕॴͷಛఆ ͕͘͠ɺΤϥʔτϥοΩϯάʹ՝͕͋Δɻ wϘτϧωοΫͷಛఆͳͲಉ༷
গͣͭ͠ɺ֎ງΛຒΊΔ ͦΜͳઓ͍ͷաఔΛ͠·͢
ͲΜͳ͕औΕΔ͔ʁ
ܭଌͷର w$16ϝϞϦར༻ͷࢹɾܭଌ wωοτϫʔΫͷࢹɾܭଌ w*0σΟεΫͷࢹɾܭଌ wϦΫΤετͷτϨʔγϯά wίϯςφϛυϧΣΞͷϩάʢ͜ΕϩάύʔεͰૉʹूΊΔʣ
ࢀߟ wͦʔ͍ͩ͞ΜͷࢿྉΑΓɺجຊతͳγεςϜʹؔΘΔϝτϦοΫ IUUQTPVEBJIBUFOBCMPHDPNFOUSZ
DHSPVQͷTUBUͷར༻ wDQVDQVBDDUαϒγεςϜ wNFNPSZαϒγεςϜ w ଞɺCMLJP WͷJP wςΩετσʔλͰɺར༻ঢ়گ͕Θ͔Δˠύʔεͯ͠׆༻͢Δ DHSPVQࣗମʹ͍ͭͯʮDHSPVQVE[VSBʯ·ͨ
ʮDHSPVQ5FO'PSXBSEʯͰݕࡧ͢Δͱຊޠͷղઆ͕ɻɻɻ root@compute-1:~# cat /sys/fs/cgroup/cpu/$CONTAINER/cpu.stat nr_periods 140373 nr_throttled 25 throttled_time 1112602844
ωοτϫʔΫʹؔ͢ΔϝτϦΫε wݪଇɺίϯςφࣗͷ/FUXPSL/BNFTQBDFʹॴଐ͢Δ w/FUXPSL/BNFTQBDFͲ͏͠ʢࢠ͔ΒʣɺWFUIσόΠεΛܦ༝ ͯ͠௨৴͢Δ͜ͱʹͳΔ wͦͷWFUIσόΠεʹؔ͢Δز͔ͭͱΔखஈ͕͋Δ wJGUPQͷΑ͏ͳίϚϯυɺQSPDOFUEFWϑΝΠϧ wϓϩηε୯ҐͷωοτϫʔΫใQSPD1*%OFU\EFW UDQ VEQ ^
':*ίϯςφωοτϫʔΫͷτϙϩδ wCSJEHFઃఆͷྫɻ%PDLFSͷσϑΥϧτઃఆ͜Εɻ FUI EPDLFS WFUIIPTU WFUIHVFTU WFUIIPTU WFUIHVFTU WFUIIPTU WFUIHVFTU
FH FH ඞཁʹԠ͡/"5ͳͲ /FUXPSL /BNFTQBDF ͻͱͭͷ)PTU CSJEHF ͜ΕΒΛࢹ͢Δʂ
*0ΞΧϯςΟϯά wDHSPVQͷCMLJPJPɺσόΠε୯ҐͷܭଌεϩοτϧͳͷͰɺ/'4 ͷΑ͏ʹσόΠε൪߸͕ৼΒΕͳ͍ϑΝΠϧγεςϜʹ͍͠ wҰํɺϓϩηε୯ҐͷSFBEʢʣɺXSJUFʢʣͷݺͼग़͠ճɺ ಡΈࠐΈॻ͖ࠐΈʹޭͨ͠όΠτͷྦྷܭɺΧʔωϧ͕ूܭͯ͠ ͍ͯར༻Ͱ͖Δɻ wQSPD1*%JPϑΝΠϧܦ༝Ͱ؆୯ʹ֬ೝՄೳ wྫ͑͜ΕΛίϯςφʹॴଐ͢Δϓϩηε͝ͱʹूܭ͢Δ
ϦΫΤεττϥοΩϯά w͋ΔαʔόΛ௨ա͢Δ͝ͱʹಛఆͷϔομʔʹ࣌ؒΛه͍ͯ͘͠ wDG9'PSXBSEFE'PSϔομʔ w3BDLͷൃΛਅࣅͯɺϚʔΧʔΛωετ͍ͯ͘͠ͷͲ͏͔ʁ wίϯϙωϯτ͝ͱͷܦա͕࣌ؒΘ͔ΔΑ͏ʹͳΓɺϘτϧωοΫͳͲ ͕Θ͔ΔɻͦΕͦ͜(SBGBOBͳͲͰՄࢹԽͯ͠ྑ͍ wΤϥʔτϥοΩϯάʹԠ༻Ͱ͖Δʁ IUUQTQZMPOTXFCGSBNFXPSLSFBEUIFEPDTJPFOWSDDPODFQUTIUNM
DGޚࣾͷࢄτϨʔγϯάج൫ wؾʹͳΓ·͢ IUUQUFDIMJGFDPPLQBEDPNFOUSZ
σʔλΛཷΊࠐΉ
େ·͔ͳΞʔΩςΫνϟ wूΊΔˠஷΊΔˠՄࢹԽ͢Δ wσʔλͷอଘઌ1SPNFUIFVT wՄࢹԽ(SBGBOB wσʔλΤʔδΣϯτD"EWJTPSNUBJMɺࣗ࡞FYQPSUFS
1SPNFUIFVT (SBGBOB &YQPSUFS &YQPSUFS &YQPSUFS $POUBJOFS $POUBJOFS $POUBJOFS $POUBJOFS $POUBJOFS
"1*͚1SPYZ ूܭϝτϦΫεͷάϧʔϓผʹ FYQPSUFSΛ༻ҙ Ϣʔβ༷μογϡϘʔυ ʢ։ൃதʣ ӡ༻ΤϯδχΞ σΟϨΫλʔͳͲ "MFSUNBOBHFS
1SPNFUIFVTͷ࠾༻ w(PPHMFͷ#PSHNPOʹӨڹΛड͚ͨࢹπʔϧ w#PSHNPOͷհ43&ຊʹ͋Δͦ͏ wΠϯετʔϧʢجຊతʹϫϯόΠφϦΛඋ͢Ε0,ʣɺ ӡ༻ʢجຊతʹΦϯϝϞϦʴϩʔΧϧετϨʔδʣ͕༰қ wॊೈͳσʔλߏͱߴػೳͳΫΤϦ w"MFSU࿈ܞ͕༰қ w(SBGBOBͳͲՄࢹԽπʔϧ֤छFYQPSUFS͕ἧ͍ͬͯΔ
1SPNFUIFVTͱ(SBGBOBͷ࿈ܞ w(SBGBOBઆ໌ͷඞཁͳ͍΄Ͳͷఆ൪μογϡϘʔυ wެࣜʹυΩϡϝϯτ͕͋Δ wIUUQTQSPNFUIFVTJPEPDTWJTVBMJ[BUJPOHSBGBOB w1SPNFUIFVTͷΫΤϦΛͦͷ·Μ·ॻ͚ͨΓͯ͠ศརײ͕ߴ͍
Մ༻ੑͷߟ͑ํ w1SPNFUIFVT֤ϊʔυͷFYQPSUFS͔ΒσʔλΛQVMM͍ͯ͘͠ελΠ ϧΛͱ͍ͬͯΔʢDG/BHJPT /31&4BB4ܕࢹʣ wͦͷͨΊ"DUJWF"DUJWFߏΛऔΕಉ͡σʔλʹͳΓɺߏͱ ͍͏͜ͱʹͳΔɻ wSFGIUUQTHJUIVCDPNQSPNFUIFVTQSPNFUIFVTJTTVFT
4FSWJDF%JTDPWFSZ wϊʔυ͔ΒQVMM͢ΔͷͰɺ1SPNFUIFVTϊʔυ͕Ͳ͜ʹ͋Δͷ͔ͱ ͍͏ใΛ͍࣋ͬͯͳ͍ͱ͍͚ͳ͍ɻ w4FSWJDF%JTDPWFSZͷͨΊͷํ๏ࣗମ͍͔ͭ͋͘Δɻ wϑΝΠϧϕʔεɺ%/4ϕʔεɺ"840QFO4UBDLͳͲ%ZOBNJD*OGSB ͚ઃఆɺ͋Δ͍.BSBUIPO$POTVMͱ࿈ܞՄೳ
FYQPSUFSͷ֓೦ʹ͍ͭͯ wσʔλΛऔΓ͍ͨϊʔυʹɺFYQPSUFSͱݴ͏ܾ·ͬͨϑΥʔϚοτͰ σʔλΛు͖ग़͢ਓΛৗற͓ͤͯ͘͞ʢ࣮ଶIUUQ"1*ʣɻ w1SPNFUIFVTɺ4FSWJDF%JTDPWFSZΛͯ͠ɺͦͷFYQPSUFSʹΞΫη ε͠ɺఆظతʹσʔλΛಘΔ wFYQPSUFSɺطʹ࣮͞Ε͍ͯΔͷͨ͘͞Μ͋Δ͠ɺ ࣗͰ࣮͢Δ͜ͱՄೳɻ
FYQPSUFSͷྫ wOPEF@FYQPSUFSl&YQPSUFSGPSNBDIJOFNFUSJDTz w$16ɺϝϞϦɺ-"ɺωοτϫʔΫͳͲϚγϯ୯ҐͰ৭ʑͱूܭ w(SBGBOBͳͲͰ͍͍ײ͡ʹάϥϑԽͯ͠׆༻ wD"EWJTPS(PPHMF͚ࣾίϯςφࢹج൫Λϕʔεʹɺ%PDLFSͳ ͲʹରԠͨ͠ͷɻͪͳΈʹTZTUFNEͳͲͷࢹͰศར wͦͷଞɺଟ͘ͷ࣮͕͋Δɻී௨ࣗ࡞͠ͳ͍͍ͯͣ͘ IUUQTHJUIVCDPNQSPNFUIFVTEPDTCMPCNBTUFSDPOUFOUEPDTJOTUSVNFOUJOHFYQPSUFSTNE
D"EWJTPSͰ)BDPOJXBͷϝτϦΫεΛऔΔ
D"EWJTPSͰ)BDPOJXBͷϝτϦΫεΛऔΔ
NUBJMIUUQTHJUIVCDPNHPPHMFNUBJM wઌड़ͨ͠/HJOYͳͲͷϩάΛύʔε্ͨ͠ͰɺFYQPSUFSʹͳͬͯ͘ ΕΔπʔϧ w%4-ʹΑΓϧʔϧΛهड़ wԿ͔طࢹײͷ͋Δྗڧ͞ DG'MVFOUEͷJO@UBJM w/HJOYͷྫˠ IUUQTHJUIVCDPNOEJB[HOHJOYQSPNFUIFVTFYQPSUFSCMPCNBTUFSpMFTQSPHTOHJOYNUBJMɹ
ܭଌͷର w$16ϝϞϦར༻ͷࢹɾܭଌ wωοτϫʔΫͷࢹɾܭଌ w*0σΟεΫͷࢹɾܭଌ wϦΫΤετͷτϨʔγϯά wίϯςφϛυϧΣΞͷϩά (PPHMFπʔϧ܈ͷΤίγεςϜ͕ศརɻ ˡD"EWJTFSͰऔΕΔ ˡD"EWJTFSͰ͢ΕऔΕΔ ˡ·ͩཁݕ౼
ˡNUBJMͰָͰ͖Δ ˡώοʢOHY@NSVCZͳͲͰؤுΕͦ͏ʣ
FYQPSUFSࣗ࡞ wͱ͍͑ɺࣗ࡞Ұͭͷख (PͳͲͳΒ1SPNFUIFVTެࣜͷϥΠϒϥϦ͕͑Δ طଘׂ࣮͕ͱ๛ ͦͦɺຬ͖ͨ͢ϑΥʔϚοτ͕୯७ wIUUQTQSPNFUIFVTJPEPDTJOTUSVNFOUJOHXSJUJOH@FYQPSUFST ΨΠυΛݟͳ͕Βɺ͋ͱطଘ࣮ΛோΊͯॻ͘ͱྑͦ͞͏
FYQPSUFSͱ1SPNFUIFVTͷ࿈ܞ w4FSWJDF%JTDPWFSZͷதͰ$POTVMͱͷ૬ੑྑ͍ͱ͍͏ࢲݟ wFYQPSUFSଆͰɺ$POTVMͷTFSWJDFDIFDLΛՃˠಛఆͷαʔϏεΛ 1SPNFUIFVT͔ΒσΟεΧόϦɺ͕ඇৗʹ༰қʹͳ͍ͬͯΔ wྫ͑ҰʹෳFYQPSUFSͳͲ؆୯ͩ͠ɺFYQPSUFSͷՃআɺ ࢮ׆ࢹ$POTVMʹΔ͜ͱͰߟ͑ͳͯ͘ࡁΉ
͜ΜͳTFSWJDFఆٛΛॻ͚ wεοͱ+0*/͢Δ wศར { "service": { "name": "node_exporter", "tags": [
"development" ], "port": 9100, "checks": [ { "id": "ping", "tcp": "localhost:9100", "interval": "15s", "timeout": "1s" } ] } }
͜ͷষ·ͱΊ wʮσʔλͷஷΊͲ͜Ζʯʹ1SPNFUIFVTΛਅΜதʹਾ͑Δ w1SPNFUIFVTͷΤίγεςϜڧ͍ɻ w͍͟ͱͳΕཉ͍͠σʔλࣗͰऔΓʹߦ͘ w͜ΕΒͷͭͳ͗ʹ$POTVM͕ศར
ࠓޙΓ͍ͨ͜ͱ
ࠓޙΓ͍ͨ͜ͱ wूΊΔͱ͜Ζ·ͰԿͱ͔ͳΓͦ͏ wཷΊࠐΜͩσʔλͷ׆༻ʹ͍ͭͯ w"MFSUNBOBHFSͷ֓೦ʢࠓলུʣ wҟৗݕͷࣗಈԽɺՄࢹԽͳͲ
ҟৗݕͷࣗಈԽɺՄࢹԽ wίϯςφʹؔ͢ΔϝτϦοΫεɺ$.%#ܦ༝ͰूͰ͖Δঢ়ଶ wϓϩδΣΫτ͝ͱͷσʔλΛूΊͯԿ͔Ͱ͖ͳ͍͔ 1SPKFDUT EPNBJO $POUBJOFST $*% DPOUBJOFSJE $POUBJOFST $*%
DPOUBJOFSJE .FUSJDT\EBUBlBBBz ^999 .FUSJDT\EBUBlCCCz ^::: .FUSJDT\EBUBlDDDz ^;;; .FUSJDT\EBUBlEEEz ^888
ूΊͨσʔλΛͲ͏͢Δ͔ w͜Ε͕ɺࠓޙͷ՝ w୯७ʹάϥϑԽͯ͠ΈͯɺͦΕͳΓʹঢ়ଶ͕Θ͔Δؾ͢Δ͠ɺٸ ʹෛՙ͕ߴ͍ɺ௨৴Ͱ͖ͳ͘ͳͬͯΔɺͳͲࢹͰ͖ͦ͏Ͱ͋Δ w·ͣӡ༻ͯ͠Έ͍ͨͱ͜Ζ
ՄࢹԽɾࣗಈఆͷख๏ wϓϩδΣΫτʹඥ͍ͨσʔλɺجຊతʹଟ࣍ݩͷσʔλͰ͋Δͷ Ͱɺ࣍ݩѹॖΛͯ͠ɺʮ͜Εਖ਼ৗʯʮ͜Εোঢ়ଶʯͷΑ͏ʹ ྨ͕Ͱ͖ΔΜͰͳ͍͔ͱߟ͍͑ͯΔ͕ɺࠓޙͷ՝ wେྔͷίϯςφϓϩδΣΫτΛɺࣗݾ৫ԽϚοϓͳͲͰࣗಈͰ ྨͯ͠ɺͦΕΛՄࢹԽͰ͖Δͱ໘നͦ͏ʁ w͜ͷลΓʹɺʮͷʯͷղܾࡦ͕ݟ͑Δؾ͕͍ͯ͠Δ
5P#F$POUJOVFE
·ͩ·ͩख୳Γͷ ཁૉଟ͍Ͱ͕͢ Γ͕͍͕͋Γ·͢ʂ
13
ϚωʔδυΫϥυΛҰॹʹ࡞Γ·͠ΐ͏ ࠷৽ͷ࠾༻ใΛνΣοΫˠ !QC@SFDSVJU QIPUPIUUQTUFDIQFQBCPDPNFOHJOFFSUSBJOJOHWPM