Կނ1SPNFUIFVT w ,VCFSOFUFTͱੑ͕ߴ͍ w ੑೳ͕ྑ͍ w 1SPN2-͕͍͍͢ w &YQPSUFSΛॻ͖͍͢ w "MFSUNBOBHFS͕ศར ˔ 1SPNFUIFVT BQJTFSWFS 1PE 1PE 1PE TFSWJDFEJTDPWFSZ TDSBQFNFUSJDT .BTUFSOPEF 8PSLFSOPEF
Կނ1SPNFUIFVT 1SPNFUIFVT BQJTFSWFS 1PE 1PE 1PE TFSWJDFEJTDPWFSZ TDSBQFNFUSJDT .BTUFSOPEF 8PSLFSOPEF w ,VCFSOFUFTͱੑ͕ߴ͍ w ੑೳ͕ྑ͍ w 1SPN2-͕͍͍͢ w &YQPSUFSΛॻ͖͍͢ w "MFSUNBOBHFS͕ศར ˔
Կނ1SPNFUIFVT 1SPNFUIFVT BQJTFSWFS 1PE 1PE TFSWJDFEJTDPWFSZ TDSBQFNFUSJDT .BTUFSOPEF 8PSLFSOPEF 1PE w ,VCFSOFUFTͱੑ͕ߴ͍ w ੑೳ͕ྑ͍ w 1SPN2-͕͍͍͢ w &YQPSUFSΛॻ͖͍͢ w "MFSUNBOBHFS͕ศར ˔
Կނ1SPNFUIFVT 1SPNFUIFVT BQJTFSWFS 1PE 1PE TFSWJDFEJTDPWFSZ TDSBQFNFUSJDT .BTUFSOPEF 8PSLFSOPEF 1PE 1PE w ,VCFSOFUFTͱੑ͕ߴ͍ w ੑೳ͕ྑ͍ w 1SPN2-͕͍͍͢ w &YQPSUFSΛॻ͖͍͢ w "MFSUNBOBHFS͕ศར ˔
Կނ1SPNFUIFVT 1SPNFUIFVT BQJTFSWFS 1PE 1PE TFSWJDFEJTDPWFSZ TDSBQFNFUSJDT .BTUFSOPEF 8PSLFSOPEF 1PE 1PE w ,VCFSOFUFTͱੑ͕ߴ͍ w ੑೳ͕ྑ͍ w 1SPN2-͕͍͍͢ w &YQPSUFSΛॻ͖͍͢ w "MFSUNBOBHFS͕ศར ˔
Կނ1SPNFUIFVT σʔλྔͷ໘ NJMMJPOBDUJWFUJNFTFSJFT TTDSBQFJOUFSWBM NPOUIPGSFUFOUJPO ѹॖͳ͠ CZUFUJNFTUBNQCZUFWBMVF5# 1SPNFUIFVTͷWBSCJUΤϯίʔυ CZUFTBNQMF5# w ,VCFSOFUFTͱੑ͕ߴ͍ w ੑೳ͕ྑ͍ w 1SPN2-͕͍͍͢ w &YQPSUFSΛॻ͖͍͢ w "MFSUNBOBHFS͕ศར ˔
Կނ1SPNFUIFVT w ,VCFSOFUFTͱੑ͕ߴ͍ w ੑೳ͕ྑ͍ w 1SPN2-͕͍͍͢ w &YQPSUFSΛॻ͖͍͢ w "MFSUNBOBHFS͕ศར ˔ w QSPNFUIFVTDMJFOU@HPMBOH w $PVOUFS w (BVHF w 4VNNBSZ w )JTUPHSBN w طଘͷ&YQPSUFSଟ͍ w OPEFFYQPSUFS w NPOHPEC
SFEJT
NZTRM
ʜ
Կނ1SPNFUIFVT w ,VCFSOFUFTͱੑ͕ߴ͍ w ੑೳ͕ྑ͍ w 1SPN2-͕͍͍͢ w &YQPSUFSΛॻ͖͍͢ w "MFSUNBOBHFS͕ศར ˔ w 1SPNFUIFVT͕ൃੜͨ͠Ξϥʔτͷཧ w ॏෳ͍ͯ͠ΔΞϥʔτͷղআ w ΞϥʔτͷάϧʔϓԽ w ௨ͷҰ࣌తࢭͳఀࢭ w ৭ʑͳ௨৴ઌͷରԠ w 4MBDL
&NBJM
)5518FCIPPL
4MPXRVFSZ w (SBGBOBଆͰෳࡶͳ2VFSZΛॻ͘ͱɺ ͕͍ͪ࣌ؒ w 1SPNFUIFVTͰ3FDPSEJOH3VMFTΛՃ w ࣄલܭࢉΛߦ͍ɺ ৽͍͠UJNFTFSJFTͱͯ͠อଘ 100 * sum by (service) ( rate(abema_http_requests_total{status=~"^5..$"}[2m]) ) / sum by (service) ( rate(abema_http_requests_total[2m]) ) (SBGBOB service:abema_http_request_5xx_percent:rate2m = 100 * sum by (service) ( rate(abema_http_requests_total{status=~"^5..$"}[2m]) ) / sum by (service) ( rate(abema_http_requests_total[2m]) ) 1SPNFUIFVT service:abema_http_request_5xx_percent:rate2m (SBGBOB
4UBDLESJWFSͷϝτϦΫε w (PPHMFͷNPOJUPSJOHBQJΛ༻͍ͯಠࣗͷFYQPSUFSΛ࡞ w -PBE#BMBODJOH w 4UPSBHF w 1VC4VC w #JH2VFSZ w %BUBqPX TUBDLESJWFSFYQPSUFS HPPHMFNPOJUPSJOHBQJ 1SPNFUIFVT
1SPNFUIFVTTIBSEJOH w ϝτϦΫελΠϓ w ϝτϦΫεछྨͰ1SPNFUIFVTΛ͚Δ w λʔήοτ w औಘରͰ1SPNFUIFVTΛ͚Δ ˔ QSPNFUIFVT TFSWJDF" TFSWJDF# NFUSJDT NFUSJDT OPEF FYQPSUFS NFUSJDT
1SPNFUIFVTTIBSEJOH w ϝτϦΫελΠϓ w ϝτϦΫεछྨͰ1SPNFUIFVTΛ͚Δ w λʔήοτ w औಘରͰ1SPNFUIFVTΛ͚Δ ˔ OPEF FYQPSUFS OPEFNFUSJDT TFSWJDF" HPNFUSJDT IUUQNFUSJDT TFSWJDF# HPNFUSJDT HSQDNFUSJDT QSPNFUIFVT
1SPNFUIFVTTIBSEJOH w ϝτϦΫελΠϓ w ϝτϦΫεछྨͰ1SPNFUIFVTΛ͚Δ w λʔήοτ w औಘରͰ1SPNFUIFVTΛ͚Δ ˔ QSPNFUIFVT TFSWJDF" NFUSJDT TFSWJDF# NFUSJDT TFSWJDF$ NFUSJDT TFSWJDF% NFUSJDT
1SPNFUIFVTTIBSEJOH TFSWJDF" NFUSJDT TFSWJDF# NFUSJDT TFSWJDF$ NFUSJDT TFSWJDF% NFUSJDT QSPNFUIFVT QSPNFUIFVT w ϝτϦΫελΠϓ w ϝτϦΫεछྨͰ1SPNFUIFVTΛ͚Δ w λʔήοτ w औಘରͰ1SPNFUIFVTΛ͚Δ ˔
)"1SPNFUIFVT w ܦҢ w 1SPNFUIFVTͷϝτϦΫε͔ΒNJDSPTFSWJDFTΛࣗಈεέʔϦϯά w ಉ͡$POpHɺಉ͡λʔήοτϦετͰෳ1SPNFUIFVTΛߏங w NFNPSZEJTLͷσʔλ΄΅ಉ͡ w σʔλͷࠩͷٵऩ w Ξϥʔτ w ීஈͷΞϥʔτҰఆͷ࣌ؒͰఆ w (SBGBOB w εςΟοΩʔηογϣϯ TFSWJDF" NFUSJDT TFSWJDF# NFUSJDT QSPNFUIFVT QSPNFUIFVT QSPNFUIFVT
1SPNFUIFVT0QFSBUPS w $POpHϑΝΠϧΛࣗಈੜ w DPOpHͷΘΓʹɺ TFSWJDFNPOJUPSΛهड़ w ϗοτϦϩʔυ w 4JEFDBSίϯςφʔͰDPOpHɺ EBTICPBSEΛࢹɺϦϩʔυ SFMPBEFS QSPNFUIFVTIUUQ 6QEBUF 3FMPBE 8BUDI 1SPNFUIFVT1PE 3FMPBE $POpH QSPNFUIFVTPQFSBUPS
"MFSUNBOHFS w /PEF@)JHI$166TBHF w /PEF@)JHI.FNPSZ6TBHF w 1PE@)JHI.FNPSZ6TBHF w )UUQ3FRVFTU@)JHI&SSPS3BUF w )UUQ3FRVFTU@)JHI-BUFODZ w (31$@)JHI&SSPS3BUF w 3FEJT@)JHI&SSP3BUF w ʜ "-&35(31$@)JHI&SSPS3BUF *'NFUIPEBCFNB@HSQD@SFRVFTU@JOUFSOBM@FSSPS@QFSDFOUSBUFN '03N -"#&-4\ TFWFSJUZXBSOJOH ^ "//05"5*0/4\ EFTDSJQUJPO\\MBCFMTTFSWJDF^^@\\MBCFMTNFUIPE^^@IBTBOFSSPSSBUF PG\\QSJOUGaGaWBMVF^^GPSN
^ HSQDSVMFT TMBDL௨
)FMN w ,VCFSOFUFTQBDLBHFNBOBHFS w )FMN$IBSUΛ࡞ w ඞཁͳϑΝΠϧͷςϯϓϨʔτͷఆٛ w )FMNͰΠϯετϧ w WBMVFTZBNMϑΝΠϧͰมΛઃఆ helm install --name prometheus-grpc -f dev-values.yaml ./charts/prometheus
1SPNWJ[ w 1SPNWJ[ w 1SPNFUIFVT͔ΒͷσʔλΛऔಘɺ άϥϑσʔλΛੜ w (SBQI"1*Λఏڙ w 1SPNWJ[GSPOU w /FUqJYͷWJ[DFSBMΛج͍ͮͯɺ τϥϑΟοΫͷϨϯμϦϯά promviz Retrieval API Cache Storage HDD/SSD promviz-front Prometheus Prometheus Prometheus