Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
コンテナたちを計測すること ~ ロリポップ!マネージドクラウドの裏側 ~ /wip-how-t...
Search
KONDO Uchio
September 29, 2017
Technology
4
6.1k
コンテナたちを計測すること ~ ロリポップ!マネージドクラウドの裏側 ~ /wip-how-to-get-container-metrics
Cookpad Tech Kitchen #12
KONDO Uchio
September 29, 2017
Tweet
Share
More Decks by KONDO Uchio
See All by KONDO Uchio
大規模レガシーテストを 倒すための CI基盤の作り方 / #CICD2023
udzura
5
2.4k
Ruby x BPF in Action / RubyKaigi 2022
udzura
0
250
Narrative of Ruby & Rust
udzura
0
220
開発者生産性指標の可視化 / pepabo-four-keys
udzura
3
1.7k
Talk of RBS
udzura
0
450
Re: みなさん最近どうですか? / FGN tech meetup in 2021
udzura
0
780
Dockerとやわらかい仮想化 - ProSec-IT/SECKUN 2021 edition -
udzura
2
730
Device access filtering in cgroup v2
udzura
1
920
"Story of Rucy" on RubyKaigi takeout 2021
udzura
0
840
Other Decks in Technology
See All in Technology
今だから言えるセキュリティLT_Wordpress5.7.2未満を一斉アップデートせよ
cuebic9bic
2
170
TLSから見るSREの未来
atpons
2
310
AIでテストプロセス自動化に挑戦する
sakatakazunori
1
530
SREのためのeBPF活用ステップアップガイド
egmc
2
1.3k
助けて! XからWaylandに移行しないと新しいGNOMEが使えなくなっちゃう 2025-07-12
nobutomurata
2
200
AIエージェントが書くのなら直接CloudFormationを書かせればいいじゃないですか何故AWS CDKを使う必要があるのさ
watany
18
7.6k
サービスを止めるな! DDoS攻撃へのスマートな備えと最前線の事例
coconala_engineer
1
180
Introduction to Sansan for Engineers / エンジニア向け会社紹介
sansan33
PRO
5
39k
Transformerを用いたアイテム間の 相互影響を考慮したレコメンドリスト生成
recruitengineers
PRO
2
440
【あのMCPって、どんな処理してるの?】 AWS CDKでの開発で便利なAWS MCP Servers特集
yoshimi0227
6
950
Snowflake Intelligenceという名のAI Agentが切り開くデータ活用の未来とその実現に必要なこと@SnowVillage『Data Management #1 Summit 2025 Recap!!』
ryo_suzuki
1
160
Rethinking Incident Response: Context-Aware AI in Practice
rrreeeyyy
2
940
Featured
See All Featured
A better future with KSS
kneath
238
17k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
The Pragmatic Product Professional
lauravandoore
35
6.7k
Why Our Code Smells
bkeepers
PRO
337
57k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Making Projects Easy
brettharned
116
6.3k
Bash Introduction
62gerente
613
210k
The Cost Of JavaScript in 2023
addyosmani
51
8.6k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
What's in a price? How to price your products and services
michaelherold
246
12k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Transcript
dϩϦϙοϓʂϚωʔδυΫϥυͷཪଆd ۙ౻Ӊஐ࿕(.01FQBCP *OD $PPLQBE5FDI,JUDIFO ίϯςφͨͪΛܭଌ͢Δ͜ͱ
ΤϯδχΞ ۙ౻Ӊஐ࿕!VE[VSB ٕज़෦ٕज़ج൫νʔϜ ͖ͳϥʔϝϯΒʔΊΜཾ (.0ϖύϘॴଐ ͬͯΔ3VCZɺNSVCZɺίϯςφ Γ͍ͨγεςϜϓϩάϥϛϯάɺ-JOVY֦ு
ࢹɾܭଌʹ͍ͭͯɺࠓ ࢼߦࡨޡ͕ଓ͍͍ͯ·͕͢ ࠓݕূͷաఔΛڞ༗͠·͢
ϩϦϙοϓʂϚωʔδυΫϥυ IUUQTNDMPMJQPQKQ ΞϧϑΝςετத
IUUQTNDMPMJQPQKQ ΞϧϑΝςετத ˞͝ڵຯͷ͋ΔํձͰΛֻ͓͚Լ͍͞
ࢹͱ
*5ΠϯϑϥͷࢹͱɺຊॻͰ࣍ͷ௨Γఆ ٛ͠·͢ɻ োൃੜͷ༗ແΛҰఆظؒͰΓͳ֬͘ೝ͢ Δ͜ͱɻ ʰ*5Πϯϑϥࢹ<࣮ફ>ೖʱΑΓ
ࢹɾϝτϦΫεͷత wӡ༻ͱͯ͠ wαʔϏεμϯʹ͙͢ʹؾ͖͍ͨɻ·ͨɺ෮چΛૉૣ͍ͨ͘͠ wαʔϏεͷػೳͱͯ͠ wίϯςφͷෛՙঢ়گɺোঢ়ଶʹͳ͍ͬͯΔ͔൱͔ͳͲɻ·ͨͦͷ ཤྺΛͪΌΜͱՄࢹԽͯ͠ఏڙͰ͖Δͱخ͍͠ wˠͦͷͨΊʹɺϝτϦΫεΛܭଌɺࢹɺෆਖ਼ͳঢ়ଶͷ࣌Ξϥʔ τͳͲΛ͋͛ΒΕΔΑ͏ʹ͠ͳ͚ΕͳΒͳ͍ɻ
ࢹͷͨΊʹ ·ͣܭଌ͠ͳ͍ͱ͍͚ͳ͍
ࠓɺࠔ͍ͬͯΔ͜ͱ wίϯςφී௨ͷ7.ͷࢹͷΑ͏ʹ͍͔ͳ͍ͱ͜Ζ͋Δ wˠϓϩηεʢ܈ʣͳͷͰ-"ͷΑ͏ͳఆ൪ͷ͕औΕͳ͔ͬͨΓͳͲ wίϯςφϗεςΟϯάͰ͋Δ͜ͱɺ·ͨϚΠΫϩαʔϏεతͰ͋Δ͜ ͱʹΑΔɺͭͷʮʯͷ͕͋Δ
ʮίϯςφʯͷ wϚωʔδυΫϥυɺίϯςφΛར༻ͨ͠ߴूੵϓϥοτϑΥʔϜ Ͱ͋Δɻ͢ͳΘͪɺίϯςφ͕ͱͯଟ͍ɻ w͔͠ɺ'BTU$POUBJOFSΛ࠾༻͍ͯ͠ΔͳͲɺίϯςφىಈ͠ଓ͚ ΔΘ͚Ͱͳ͍ɻ·ͨɺҰͭ࣌ͷυϝΠϯʹඥͮ͘ίϯςφͷม ΘΔɻ wͨ͘͞Μ͋ΔͷͰɺूΊΔͱɺ͔Γ͘͢ݟͤΔ͕͋Δ
ʮίϯϙʔωϯτʯͷ wԿ͔ର֎ൃද͍ͯ͠Δ௨ΓɺϚωʔδυΫϥυίϯϙʔωϯτ ͕ଟ͘ɺϚΠΫϩαʔϏεతʹΈ߹Θ͍ͬͯ͞Δ wʮαΠτ͕ݟ͑ͳ͍ʯͷΑ͏ͳ߹ʹɺݪҼͱߟ͑ΒΕΔՕॴͷಛఆ ͕͘͠ɺΤϥʔτϥοΩϯάʹ՝͕͋Δɻ wϘτϧωοΫͷಛఆͳͲಉ༷
গͣͭ͠ɺ֎ງΛຒΊΔ ͦΜͳઓ͍ͷաఔΛ͠·͢
ͲΜͳ͕औΕΔ͔ʁ
ܭଌͷର w$16ϝϞϦར༻ͷࢹɾܭଌ wωοτϫʔΫͷࢹɾܭଌ w*0σΟεΫͷࢹɾܭଌ wϦΫΤετͷτϨʔγϯά wίϯςφϛυϧΣΞͷϩάʢ͜ΕϩάύʔεͰૉʹूΊΔʣ
ࢀߟ wͦʔ͍ͩ͞ΜͷࢿྉΑΓɺجຊతͳγεςϜʹؔΘΔϝτϦοΫ IUUQTPVEBJIBUFOBCMPHDPNFOUSZ
DHSPVQͷTUBUͷར༻ wDQVDQVBDDUαϒγεςϜ wNFNPSZαϒγεςϜ w ଞɺCMLJP WͷJP wςΩετσʔλͰɺར༻ঢ়گ͕Θ͔Δˠύʔεͯ͠׆༻͢Δ DHSPVQࣗମʹ͍ͭͯʮDHSPVQVE[VSBʯ·ͨ
ʮDHSPVQ5FO'PSXBSEʯͰݕࡧ͢Δͱຊޠͷղઆ͕ɻɻɻ root@compute-1:~# cat /sys/fs/cgroup/cpu/$CONTAINER/cpu.stat nr_periods 140373 nr_throttled 25 throttled_time 1112602844
ωοτϫʔΫʹؔ͢ΔϝτϦΫε wݪଇɺίϯςφࣗͷ/FUXPSL/BNFTQBDFʹॴଐ͢Δ w/FUXPSL/BNFTQBDFͲ͏͠ʢࢠ͔ΒʣɺWFUIσόΠεΛܦ༝ ͯ͠௨৴͢Δ͜ͱʹͳΔ wͦͷWFUIσόΠεʹؔ͢Δز͔ͭͱΔखஈ͕͋Δ wJGUPQͷΑ͏ͳίϚϯυɺQSPDOFUEFWϑΝΠϧ wϓϩηε୯ҐͷωοτϫʔΫใQSPD1*%OFU\EFW UDQ VEQ ^
':*ίϯςφωοτϫʔΫͷτϙϩδ wCSJEHFઃఆͷྫɻ%PDLFSͷσϑΥϧτઃఆ͜Εɻ FUI EPDLFS WFUIIPTU WFUIHVFTU WFUIIPTU WFUIHVFTU WFUIIPTU WFUIHVFTU
FH FH ඞཁʹԠ͡/"5ͳͲ /FUXPSL /BNFTQBDF ͻͱͭͷ)PTU CSJEHF ͜ΕΒΛࢹ͢Δʂ
*0ΞΧϯςΟϯά wDHSPVQͷCMLJPJPɺσόΠε୯ҐͷܭଌεϩοτϧͳͷͰɺ/'4 ͷΑ͏ʹσόΠε൪߸͕ৼΒΕͳ͍ϑΝΠϧγεςϜʹ͍͠ wҰํɺϓϩηε୯ҐͷSFBEʢʣɺXSJUFʢʣͷݺͼग़͠ճɺ ಡΈࠐΈॻ͖ࠐΈʹޭͨ͠όΠτͷྦྷܭɺΧʔωϧ͕ूܭͯ͠ ͍ͯར༻Ͱ͖Δɻ wQSPD1*%JPϑΝΠϧܦ༝Ͱ؆୯ʹ֬ೝՄೳ wྫ͑͜ΕΛίϯςφʹॴଐ͢Δϓϩηε͝ͱʹूܭ͢Δ
ϦΫΤεττϥοΩϯά w͋ΔαʔόΛ௨ա͢Δ͝ͱʹಛఆͷϔομʔʹ࣌ؒΛه͍ͯ͘͠ wDG9'PSXBSEFE'PSϔομʔ w3BDLͷൃΛਅࣅͯɺϚʔΧʔΛωετ͍ͯ͘͠ͷͲ͏͔ʁ wίϯϙωϯτ͝ͱͷܦա͕࣌ؒΘ͔ΔΑ͏ʹͳΓɺϘτϧωοΫͳͲ ͕Θ͔ΔɻͦΕͦ͜(SBGBOBͳͲͰՄࢹԽͯ͠ྑ͍ wΤϥʔτϥοΩϯάʹԠ༻Ͱ͖Δʁ IUUQTQZMPOTXFCGSBNFXPSLSFBEUIFEPDTJPFOWSDDPODFQUTIUNM
DGޚࣾͷࢄτϨʔγϯάج൫ wؾʹͳΓ·͢ IUUQUFDIMJGFDPPLQBEDPNFOUSZ
σʔλΛཷΊࠐΉ
େ·͔ͳΞʔΩςΫνϟ wूΊΔˠஷΊΔˠՄࢹԽ͢Δ wσʔλͷอଘઌ1SPNFUIFVT wՄࢹԽ(SBGBOB wσʔλΤʔδΣϯτD"EWJTPSNUBJMɺࣗ࡞FYQPSUFS
1SPNFUIFVT (SBGBOB &YQPSUFS &YQPSUFS &YQPSUFS $POUBJOFS $POUBJOFS $POUBJOFS $POUBJOFS $POUBJOFS
"1*͚1SPYZ ूܭϝτϦΫεͷάϧʔϓผʹ FYQPSUFSΛ༻ҙ Ϣʔβ༷μογϡϘʔυ ʢ։ൃதʣ ӡ༻ΤϯδχΞ σΟϨΫλʔͳͲ "MFSUNBOBHFS
1SPNFUIFVTͷ࠾༻ w(PPHMFͷ#PSHNPOʹӨڹΛड͚ͨࢹπʔϧ w#PSHNPOͷհ43&ຊʹ͋Δͦ͏ wΠϯετʔϧʢجຊతʹϫϯόΠφϦΛඋ͢Ε0,ʣɺ ӡ༻ʢجຊతʹΦϯϝϞϦʴϩʔΧϧετϨʔδʣ͕༰қ wॊೈͳσʔλߏͱߴػೳͳΫΤϦ w"MFSU࿈ܞ͕༰қ w(SBGBOBͳͲՄࢹԽπʔϧ֤छFYQPSUFS͕ἧ͍ͬͯΔ
1SPNFUIFVTͱ(SBGBOBͷ࿈ܞ w(SBGBOBઆ໌ͷඞཁͳ͍΄Ͳͷఆ൪μογϡϘʔυ wެࣜʹυΩϡϝϯτ͕͋Δ wIUUQTQSPNFUIFVTJPEPDTWJTVBMJ[BUJPOHSBGBOB w1SPNFUIFVTͷΫΤϦΛͦͷ·Μ·ॻ͚ͨΓͯ͠ศརײ͕ߴ͍
Մ༻ੑͷߟ͑ํ w1SPNFUIFVT֤ϊʔυͷFYQPSUFS͔ΒσʔλΛQVMM͍ͯ͘͠ελΠ ϧΛͱ͍ͬͯΔʢDG/BHJPT /31&4BB4ܕࢹʣ wͦͷͨΊ"DUJWF"DUJWFߏΛऔΕಉ͡σʔλʹͳΓɺߏͱ ͍͏͜ͱʹͳΔɻ wSFGIUUQTHJUIVCDPNQSPNFUIFVTQSPNFUIFVTJTTVFT
4FSWJDF%JTDPWFSZ wϊʔυ͔ΒQVMM͢ΔͷͰɺ1SPNFUIFVTϊʔυ͕Ͳ͜ʹ͋Δͷ͔ͱ ͍͏ใΛ͍࣋ͬͯͳ͍ͱ͍͚ͳ͍ɻ w4FSWJDF%JTDPWFSZͷͨΊͷํ๏ࣗମ͍͔ͭ͋͘Δɻ wϑΝΠϧϕʔεɺ%/4ϕʔεɺ"840QFO4UBDLͳͲ%ZOBNJD*OGSB ͚ઃఆɺ͋Δ͍.BSBUIPO$POTVMͱ࿈ܞՄೳ
FYQPSUFSͷ֓೦ʹ͍ͭͯ wσʔλΛऔΓ͍ͨϊʔυʹɺFYQPSUFSͱݴ͏ܾ·ͬͨϑΥʔϚοτͰ σʔλΛు͖ग़͢ਓΛৗற͓ͤͯ͘͞ʢ࣮ଶIUUQ"1*ʣɻ w1SPNFUIFVTɺ4FSWJDF%JTDPWFSZΛͯ͠ɺͦͷFYQPSUFSʹΞΫη ε͠ɺఆظతʹσʔλΛಘΔ wFYQPSUFSɺطʹ࣮͞Ε͍ͯΔͷͨ͘͞Μ͋Δ͠ɺ ࣗͰ࣮͢Δ͜ͱՄೳɻ
FYQPSUFSͷྫ wOPEF@FYQPSUFSl&YQPSUFSGPSNBDIJOFNFUSJDTz w$16ɺϝϞϦɺ-"ɺωοτϫʔΫͳͲϚγϯ୯ҐͰ৭ʑͱूܭ w(SBGBOBͳͲͰ͍͍ײ͡ʹάϥϑԽͯ͠׆༻ wD"EWJTPS(PPHMF͚ࣾίϯςφࢹج൫Λϕʔεʹɺ%PDLFSͳ ͲʹରԠͨ͠ͷɻͪͳΈʹTZTUFNEͳͲͷࢹͰศར wͦͷଞɺଟ͘ͷ࣮͕͋Δɻී௨ࣗ࡞͠ͳ͍͍ͯͣ͘ IUUQTHJUIVCDPNQSPNFUIFVTEPDTCMPCNBTUFSDPOUFOUEPDTJOTUSVNFOUJOHFYQPSUFSTNE
D"EWJTPSͰ)BDPOJXBͷϝτϦΫεΛऔΔ
D"EWJTPSͰ)BDPOJXBͷϝτϦΫεΛऔΔ
NUBJMIUUQTHJUIVCDPNHPPHMFNUBJM wઌड़ͨ͠/HJOYͳͲͷϩάΛύʔε্ͨ͠ͰɺFYQPSUFSʹͳͬͯ͘ ΕΔπʔϧ w%4-ʹΑΓϧʔϧΛهड़ wԿ͔طࢹײͷ͋Δྗڧ͞ DG'MVFOUEͷJO@UBJM w/HJOYͷྫˠ IUUQTHJUIVCDPNOEJB[HOHJOYQSPNFUIFVTFYQPSUFSCMPCNBTUFSpMFTQSPHTOHJOYNUBJMɹ
ܭଌͷର w$16ϝϞϦར༻ͷࢹɾܭଌ wωοτϫʔΫͷࢹɾܭଌ w*0σΟεΫͷࢹɾܭଌ wϦΫΤετͷτϨʔγϯά wίϯςφϛυϧΣΞͷϩά (PPHMFπʔϧ܈ͷΤίγεςϜ͕ศརɻ ˡD"EWJTFSͰऔΕΔ ˡD"EWJTFSͰ͢ΕऔΕΔ ˡ·ͩཁݕ౼
ˡNUBJMͰָͰ͖Δ ˡώοʢOHY@NSVCZͳͲͰؤுΕͦ͏ʣ
FYQPSUFSࣗ࡞ wͱ͍͑ɺࣗ࡞Ұͭͷख (PͳͲͳΒ1SPNFUIFVTެࣜͷϥΠϒϥϦ͕͑Δ طଘׂ࣮͕ͱ๛ ͦͦɺຬ͖ͨ͢ϑΥʔϚοτ͕୯७ wIUUQTQSPNFUIFVTJPEPDTJOTUSVNFOUJOHXSJUJOH@FYQPSUFST ΨΠυΛݟͳ͕Βɺ͋ͱطଘ࣮ΛோΊͯॻ͘ͱྑͦ͞͏
FYQPSUFSͱ1SPNFUIFVTͷ࿈ܞ w4FSWJDF%JTDPWFSZͷதͰ$POTVMͱͷ૬ੑྑ͍ͱ͍͏ࢲݟ wFYQPSUFSଆͰɺ$POTVMͷTFSWJDFDIFDLΛՃˠಛఆͷαʔϏεΛ 1SPNFUIFVT͔ΒσΟεΧόϦɺ͕ඇৗʹ༰қʹͳ͍ͬͯΔ wྫ͑ҰʹෳFYQPSUFSͳͲ؆୯ͩ͠ɺFYQPSUFSͷՃআɺ ࢮ׆ࢹ$POTVMʹΔ͜ͱͰߟ͑ͳͯ͘ࡁΉ
͜ΜͳTFSWJDFఆٛΛॻ͚ wεοͱ+0*/͢Δ wศར { "service": { "name": "node_exporter", "tags": [
"development" ], "port": 9100, "checks": [ { "id": "ping", "tcp": "localhost:9100", "interval": "15s", "timeout": "1s" } ] } }
͜ͷষ·ͱΊ wʮσʔλͷஷΊͲ͜Ζʯʹ1SPNFUIFVTΛਅΜதʹਾ͑Δ w1SPNFUIFVTͷΤίγεςϜڧ͍ɻ w͍͟ͱͳΕཉ͍͠σʔλࣗͰऔΓʹߦ͘ w͜ΕΒͷͭͳ͗ʹ$POTVM͕ศར
ࠓޙΓ͍ͨ͜ͱ
ࠓޙΓ͍ͨ͜ͱ wूΊΔͱ͜Ζ·ͰԿͱ͔ͳΓͦ͏ wཷΊࠐΜͩσʔλͷ׆༻ʹ͍ͭͯ w"MFSUNBOBHFSͷ֓೦ʢࠓলུʣ wҟৗݕͷࣗಈԽɺՄࢹԽͳͲ
ҟৗݕͷࣗಈԽɺՄࢹԽ wίϯςφʹؔ͢ΔϝτϦοΫεɺ$.%#ܦ༝ͰूͰ͖Δঢ়ଶ wϓϩδΣΫτ͝ͱͷσʔλΛूΊͯԿ͔Ͱ͖ͳ͍͔ 1SPKFDUT EPNBJO $POUBJOFST $*% DPOUBJOFSJE $POUBJOFST $*%
DPOUBJOFSJE .FUSJDT\EBUBlBBBz ^999 .FUSJDT\EBUBlCCCz ^::: .FUSJDT\EBUBlDDDz ^;;; .FUSJDT\EBUBlEEEz ^888
ूΊͨσʔλΛͲ͏͢Δ͔ w͜Ε͕ɺࠓޙͷ՝ w୯७ʹάϥϑԽͯ͠ΈͯɺͦΕͳΓʹঢ়ଶ͕Θ͔Δؾ͢Δ͠ɺٸ ʹෛՙ͕ߴ͍ɺ௨৴Ͱ͖ͳ͘ͳͬͯΔɺͳͲࢹͰ͖ͦ͏Ͱ͋Δ w·ͣӡ༻ͯ͠Έ͍ͨͱ͜Ζ
ՄࢹԽɾࣗಈఆͷख๏ wϓϩδΣΫτʹඥ͍ͨσʔλɺجຊతʹଟ࣍ݩͷσʔλͰ͋Δͷ Ͱɺ࣍ݩѹॖΛͯ͠ɺʮ͜Εਖ਼ৗʯʮ͜Εোঢ়ଶʯͷΑ͏ʹ ྨ͕Ͱ͖ΔΜͰͳ͍͔ͱߟ͍͑ͯΔ͕ɺࠓޙͷ՝ wେྔͷίϯςφϓϩδΣΫτΛɺࣗݾ৫ԽϚοϓͳͲͰࣗಈͰ ྨͯ͠ɺͦΕΛՄࢹԽͰ͖Δͱ໘നͦ͏ʁ w͜ͷลΓʹɺʮͷʯͷղܾࡦ͕ݟ͑Δؾ͕͍ͯ͠Δ
5P#F$POUJOVFE
·ͩ·ͩख୳Γͷ ཁૉଟ͍Ͱ͕͢ Γ͕͍͕͋Γ·͢ʂ
13
ϚωʔδυΫϥυΛҰॹʹ࡞Γ·͠ΐ͏ ࠷৽ͷ࠾༻ใΛνΣοΫˠ !QC@SFDSVJU QIPUPIUUQTUFDIQFQBCPDPNFOHJOFFSUSBJOJOHWPM