マネージドクラウドでのPrometheusなどの活用 /managed-cloud-with-prometheus

Geeks Who Drink in Fukuoka @Nulab
https://nulab.connpass.com/event/68737/

Transcript

1. ۙ౻͏͓ͪ
   
   (.0
   1FQBCP
   *OD
   
   (FFLT
   8IP
   %SJOL ίϯςφ؂ࢹͷجຊ
2. ۙ౻͏͓ͪ
   
   (.0
   1FQBCP
   *OD
   
   (FFLT
   8IP
   %SJOL ίϯςφ؂ࢹͷجຊ ❌
3. ϚωʔδυΫϥ΢υݱ৔ใࠂ 1ZUIBHPSB
   4XJUDIFTä
   JO
   .POJUPSJOH ۙ౻͏͓ͪ
   
   (.0
   1FQBCP
   *OD
   
   (FFLT
   8IP
   %SJOL
4. ΤϯδχΞ ۙ౻͏͓ͪ
   !VE[VSB ίϯςφ΁ͷࣥண͸·ΔͰ൥೰Ͱ͋Γɺ։ൃऀҰಉ͸ᛉ૝ͨ͠ํ͕͍͍ͱࢥ͍·͢ IUUQIBDPOJXBNSVCZPSH
5. ΞδΣϯμ wϩϦϙοϓʂϚωʔδυΫϥ΢υͷίϯςφ؂ࢹ֓ཁ
   wίϯςφ؂ࢹͷίπ
   w࢖͍ͬͯΔϛυϧ
   wϐλΰϥεΠονä
   ʹΑΓίϯςφͷෛՙΛ͍͍ײ͡ʹϢʔβ༷ʹݟ ͤΔ༷ࢠ
6. ΞδΣϯμ wϩϦϙοϓʂϚωʔδυΫϥ΢υͷίϯςφ؂ࢹ֓ཁ
   wίϯςφ؂ࢹͷίπ
   w࢖͍ͬͯΔϛυϧ
   wϐλΰϥεΠονä
   ʹΑΓίϯςφͷෛՙΛ͍͍ײ͡ʹϢʔβ༷ʹݟ ͤΔ༷ࢠ ࡶʹߦ͖·͢ʂ
   ࠷ۙ΍ͬͯΔ͜ͱΛී௨ʹ࿩͠·͢ʂ
7. ϩϦϙοϓʂ
   ϚωʔδυΫϥ΢υ
8. ϩϦϙοϓʂϚωʔδυΫϥ΢υ wίϯςφʢ੿࡞)BDPOJXBʣϕʔεͷϗεςΟϯάαʔϏε
   wΊͪΌ೉͍͠ɺͱࢥΘΕ͕ͪͳΫϥ΢υΒ͍͠ػೳΛ؆୯ʹ
   wΦʔτεέʔϧɺηΩϡϦςΟ΍όʔδϣϯΞοϓରԠͳͲ
   w1)1ͷ΄͔ɺ3VCZ
   PO
   3BJMTͷରԠ΋։࢝
   wϕʔλʹ޲͚Ӷҙ᱕ਐத
9. ίϯςφϕʔε
10. ϚωΫϥͷίϯςφ؂ࢹ
11. ͜Μͳײ͡ wલఏ
 ӡ༻ͷ؂ࢹͰ͸ͳ͍ɻϢʔβ༷ʹ؂ࢹঢ়گΛఏڙ͢ΔͨΊͷ࢓૊Έɻ $PNQVUF
    ίϯςφ͕৐Δ 8FC
    QSPYZ 1SPNFUIFVT $PSF"1*
    શମ؅ཧ"1* Ϣʔβ༷
    μογϡϘʔυ 1PTUHSF42- NUBJM D"EWJTPS ϝτϦοΫͷྲྀΕ ίϯςφ΁ͷϦΫΤετ "MFSUNBOBHFS /HJOYͷϩά DHSPVQ
    QSPD
12. ίϯςφΛ؂ࢹ͢Δ
13. ͜ͷลΛऔΔ w$16ɺϝϞϦˠDHSPVQ
    wωοτϫʔΫˠ
    QSPDOFUEFW
    w8FCϦΫΤετʢεϧʔϓοτɺεςʔλείʔυɺ࣮ࡍͷσʔλస ૹྔʣˠ
    /HJOY
    MPH
    
    NUBJM
14. ݱ৔ͷ
    DHSPVQ
    ͷ༷ࢠ wྫ͑͹
    DQVBDDUTUBU root@compute-1:~# cat /sys/fs/cgroup/cpu/$CID/cpu.stat nr_periods 45609 nr_throttled 99 throttled_time 4583736837 w͜Ε͸
    UISPUUMFE
    ͱݴ͏TUBUͷྫ
    wଞʹ΋ɺϢʔβ࣌ؒɺγεςϜ࣌ؒͳͲ͕Θ͔Δ
15. ݱ৔ͷ
    QSPDOFUEFW
    ͷ༷ࢠ w3Y5Y
    ͦΕͧΕΘ͔ͬͯศར
    wͲͷίϯςφͷ௨৴ͳͷ͔ύοͱΘ͔Βͳ͍ͷͰɺඥ෇͚ඞཁ
16. ':*
    %PDLFSͳͲͰΑ͋͘ΔΞϨ FUI EPDLFS WFUIIPTU WFUIHVFTU WFUIIPTU WFUIHVFTU WFUIIPTU WFUIHVFTU FH
     FH
        ඞཁʹԠ͡/"5ͳͲ /FUXPSL
    /BNFTQBDFͰִ཭͢Δ͕ɺ
    ҰํωοτϫʔΫ͸͔ΒͳͣϗετଆͷσόΠεΛ௨Δ ͻͱͭͷ)PTU CSJEHF
17. ࢖͍ͬͯΔϛυϧ
18. D"EWJTPS wલड़ͨ͠DHSPVQ΍ɺQSPDͷԼ͔Βίϯςφ͝ͱͷ஋ΛऔಘͰ͖Δ
    w(PPHMFπʔϧ
    w%PDLFS͚ͩͰ͸ͳ͘ʢͱ͸͍͑ɺ%PDLFSͷରԠ΋ਐΜͰ͍Δʣɺ -JOVYతʹ΋͏͢͜͠௿͍ػೳ͔Βऔͬͯ͘ΕΔͷͰศརʹ࢖ͬͯ
 ͍Δɻ࣮૷΋ࢀߟʹͳΔɻ
19. NUBJM wߏ੒తʹɺ/HJOY͕શͯͷίϯςφͷલஈʹ഑ஔ͞ΕΔͷͰɺͦ͜ͷ ϩάΛύʔε͢Δͱ༗ӹͳ৘ใ͕ͨ͘͞ΜऔΕΔ
    w%4-ϕʔεʹϩάύʔε͢Δ܅͕NUBJM
    w͜Ε΋(PPHMF
    wਖ਼نදݱͳͲɺےྗΛײ͡Δ
20. ͦͯ͠
    1SPNFUIFVT
21. "MFSUNBOBHFS w1SPNFUIFVTͱ࿈ܞͯ͠ΞϥʔτΛ֎෦8FCIPPLʹૹͬͯ͘ΕΔ
    wΞϥʔτͷ৘ใࣗମ͸1SPNFUIFVTͷϧʔϧͰཷΊࠐ·Ε͍ͯ͘
    wͲ͏͍͏ϧʔϧ͕͋Δ͔1SPNFUIFVTͷμογϡϘʔυͰݟΒΕΔͷ Ͱศར
22. "MFSUNBOBHFS
    XFCIPPL wXFCIPPLʹΑΓɺ೚ҙͷαʔϏεʹΞϥʔτͷ৘ใΛ౉ͤΔ
    wΞϥʔτൃੜղܾɺؔ܎͢ΔϨʔϕϧͳͲ͸ͦͷ··Ξϥʔτͷ৘ ใʹܧঝ͞ΕΔͷͰɺՃ޻͍͍ͯ͠ײ͡ʹ
    wͳ͓ɺ
    XFCIPPL
    Ͱड͚औΓʹ࢖͏ߏ଄ମ͸BMFSUNBOBHFSͷ΋ͷΛ ͦͷ··ྲྀ༻͢Δͱศརͩͬͨʢ(PͳΒʣ
23. ෛՙ্͕͕ͬͨ͜ͱ͕
    Ϣʔβ༷ͷμογϡϘʔυʹ
    ࡌΔ·Ͱͷྫ
24. ֓ཁ $PNQVUF
    ίϯςφ͕৐Δ 8FC
    QSPYZ 1SPNFUIFVT $PSF"1*
    શମ؅ཧ"1* Ϣʔβ༷
    μογϡϘʔυ 1PTUHSF42- NUBJM D"EWJTPS "MFSUNBOBHFS
25. ίϯςφผ$16ར༻࣌ؒΛܭࢉ͢Δ $PNQVUF
    ίϯςφ͕৐Δ 8FC
    QSPYZ 1SPNFUIFVT $PSF"1*
    શମ؅ཧ"1* Ϣʔβ༷
    μογϡϘʔυ 1PTUHSF42- NUBJM D"EWJTPS "MFSUNBOBHFS ίϯςφͷ
    ੜͷ$16౷ܭΛ࢖͏ 1SPNͷSVMFͰ
    ʢΞϥʔτͰҾֻ͚ͬ
 ΍͍͢Α͏ʣ࠶ूܭ
26. Ξϥʔτ্͕͕Δ $PNQVUF
    ίϯςφ͕৐Δ 8FC
    QSPYZ 1SPNFUIFVT $PSF"1*
    શମ؅ཧ"1* Ϣʔβ༷
    μογϡϘʔυ 1PTUHSF42- NUBJM D"EWJTPS "MFSUNBOBHFS ूܭޙͷ஋Λϕʔεʹ
    Ҿֻ͚ͬΔ
27. 8FCIPPLΛୟ͘ $PNQVUF
    ίϯςφ͕৐Δ 8FC
    QSPYZ 1SPNFUIFVT $PSF"1*
    શମ؅ཧ"1* Ϣʔβ༷
    μογϡϘʔυ 1PTUHSF42- NUBJM D"EWJTPS "MFSUNBOBHFS ಺෦"1*ʹ
    ઐ༻ͷ8FCIPPLΛ
    ࣮૷͢Δ
28. $.%#ʹೖ͍ͬͯ͘ $PNQVUF
    ίϯςφ͕৐Δ 8FC
    QSPYZ 1SPNFUIFVT $PSF"1*
    શମ؅ཧ"1* Ϣʔβ༷
    μογϡϘʔυ 1PTUHSF42- NUBJM D"EWJTPS "MFSUNBOBHFS +40/ܕ͕
    ΊͪΌศར
29. ͋ͱ͸"1*Ͱࢀরͯ͠ग़͢ $PNQVUF
    ίϯςφ͕৐Δ 8FC
    QSPYZ 1SPNFUIFVT $PSF"1*
    શମ؅ཧ"1* Ϣʔβ༷
    μογϡϘʔυ 1PTUHSF42- NUBJM D"EWJTPS "MFSUNBOBHFS ϑϩϯτͰ
    Θ͔Γ΍͍͢Α͏Ճ޻ $PSF"1*Ͱ
    ͋Δఔ౓Ωϟογϡ
30. ͳΜͱ͍͏
    ϐλΰϥεΠονä
31. 1SPNFUIFVT
    BT
    B
    ϐλΰϥεΠονä
    1MBUGPSN
32. 1BB11
34. ઈࢍ։ൃதʂ
35. ͕࣌ؒ༨Δͱࢥ͏ͷͰ
    ϑϦʔτʔΫ
36. ฉ͖͍ͨͳ͋ʙ w
    1SPNFUIFVT
    ࢖ͬͯΔʁ
    ͍ͬͯ͏͔Ͳ͏͍͏ҹ৅ʁ
    w
    1SPN2-
    ׂͱେม͡Όͳ͍͔ͬ͢ʁ
    w
    "MFSUNBOBHFS
    ͍Ζ͍ΖͰ͖ͦ͏Ͱ͚͢Ͳɺ
 ɹ
    ͲΜͳײ͡Ͱ࢖ͬͯ·͢ʁ
    wओʹ
    !NBUTVNBOB
    ͞Μʹฉ͖͍ͨͰ͢ʂʂʢ໊ࢦ͠ʣ
37. པΉʂʂҰॹʹαʔϏεΛ࡞ͬͯ͘Εʂʂ̍ ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU