$30 off During Our Annual Pro Sale. View Details »

Prometheus setup with long term storage

Prometheus setup with long term storage

Ryotaro Tamura

July 22, 2019
Tweet

More Decks by Ryotaro Tamura

Other Decks in Programming

Transcript

  1. ాଜཾଠ࿠(.01FQBCP *OD
    $MPVE/BUJWF%BZT5PLZP
    1SPNFUIFVTTFUVQ
    XJUIMPOHUFSNTUPSBHF

    View Slide

  2. γχΞΤϯδχΞ
    ాଜཾଠ࿠!@UPLJCJ
    ϗεςΟϯάࣄۀ෦ϗεςΟϯάάϧʔϓϚωʔδυΫϥ΢υνʔϜ

    View Slide

  3. ໨࣍
    wϝτϦΫεͷ௕ظอଘʹऔΓ૊Μͩഎܠ
    w1SPNFUIFVTͷ࣌ܥྻσʔλϕʔεʹؔ͢Δղઆ
    w௕ظอଘͷखஈͱݕ౼
    w 3FNPUF4UPSBHF
    w 5IBOPT
    w5IBOPTӡ༻ධՁ

    View Slide

  4. എܠ

    View Slide

  5. View Slide

  6. -0-*101ϚωʔδυΫϥ΢υ
    wίϯςφϕʔεͷϗεςΟϯάαʔϏε
    w!NBUTVNPUPSZͷߟҊͨ͠'BTU$POUBJOFSΞʔΩςΫνϟΛݩʹ։ൃ
    wϦΫΤετܖػͰίϯςφ͕ىಈ
    wϦΫΤετͷདྷͳ͍ίϯςφ͸ҰఆظؒͰఀࢭ͢Δ
    wߴूੵ

    View Slide

  7. ϝτϦΫεͷඞཁੑ
    wΦʔτεέʔϧػೳͷఏڙ
    wϢʔβ؀ڥͷϦιʔεར༻ঢ়گ͸αʔϏεଆͰ؂ࢹ
    wඞཁʹԠͯ͡ίϯςφΛ૿΍ͯ͠ෛՙ෼ࢄΛࢼΈΔ
    wϢʔβ͕ઃఆ͢Δͷ͸ΦʔτεέʔϧͷՄ൱ͷΈ
    w՝ۚࠜڌͱͳΔ਺ࣈͷఏڙ
    wϢʔβ΁ͷ੥ٻֹۚ͸ίϯςφͷىಈճ਺Λݩʹܾఆ͢Δ
    wεέʔϧͨ࣌͠ࠁ΍ཧ༝ͷఏࣔ

    View Slide

  8. View Slide

  9. 1SPNFUIFVT
    wαʔϏε؀ڥϢʔβ؀ڥͷ྆ํͰ؂ࢹʹ1SPNFUIFVTΛ࢖༻
    wOPEF@FYQPSUFS DBEWJTPS NUBJM FUD
    wϝτϦΫεͷऩूࣗମ͸ಛʹ໰୊ͳ͘ӡ༻த
    wҰํͰ
    wΫϥελϦϯά΍ϨϓϦέʔγϣϯͷػೳ͸ఏڙ͍ͯ͠ͳ͍
    wσΟεΫ΍ϊʔυো֐΁ͷ଱ٱੑ͸جຊతʹείʔϓ֎

    View Slide

  10. ࠔͬͨ͜ͱ
    wσΟεΫ΍ϊʔυͷো֐΁ͷ଱ٱੑ
    w৑௕ߏ੒Λऔ͍ͬͯͯ΋ɺͦΕͧΕͷ1SPNFUIFVT͸ݸผʹಈ࡞͢Δ
    wෳ਺ϊʔυͰ໰୊͕ൃੜ͢Δͱ׬શͳσʔλΛ࣋ͭϊʔυ͕ଘࡏ͠ͳ͘ͳΔ

    View Slide

  11. ࠔͬͨ͜ͱ
    wΦʔτεέʔϧͷਫ਼౓Λ্͍͛ͨ
    wΑΓଟ͘ͷϝτϦοΫΛɺΑΓ୹ִ͍ؒͰऩू͍ͨ͠
    wϝϞϦͷར༻ྔ͕ͲΜͲΜ૿͍͑ͯ͘
    w͋Δఔ౓·Ͱ͸εέʔϧΞοϓՄೳ
    w͍ͣΕΫϥελ΍ϩʔϧผʹ1SPNFUIFVTΛ෼ࢄ͍ͨ͠

    View Slide

  12. ඞཁͳ΋ͷ
    wෳ਺ͷ1SPNFUIFVTͷ্ʹߏங͞ΕͨάϩʔόϧϏϡʔ
    w௕ظؒʹ౉ͬͯϝτϦοΫΛอଘ͢ΔετΞ
    wߴՄ༻
    wߴ଱ٱ

    View Slide

  13. 1SPNFUIFVTͷ
    ࣌ܥྻσʔλϕʔεʹ͍ͭͯ

    View Slide

  14. 5JNF4FSJFT

    View Slide

  15. 5JNF4FSJFT
    w5JNF
    w࣌ؒͷܦաʹ൐ͬͯFYQPSUFS͔Βͷऩू͕࣮ࢪ͞ΕΔ౓ʹ૿Ճ
    wऩूִؒ͸ϝτϦοΫʹΑͬͯҟͳΔ
    w4FSJFT
    wऩू͢ΔϝτϦοΫʹ෇༩͞Εͨϥϕϧͷ૊Έ߹Θ͕ͤҰக͢Δ΋ͷ
    wFYQPSUFSͷ૿ݮͷଞʹ΋ɺΠϯελϯεͷ૿ݮ౳ʹΑͬͯมಈ
    w4BNQMF
    w5JNFͱ4FSJFTͷަࠩ఺ ϝτϦοΫͷ࠷খ୯Ґ

    View Slide

  16. 5JNF4FSJFT
    http_requests_total{instance="webproxy-1", code="200", method="get", path="/"}
    http_requests_total{instance=“webproxy-1", code="200", method="get", path=“/status"}
    http_requests_total{instance="webproxy-3", code="200", method="get", path="/status"}
    {__name__="http_requests_total", instance="webproxy-1", code="200", method="get", path="/"}
    {__name__="http_requests_total", instance="webproxy-1", code="200", method="get", path=“/status"}
    {__name__="http_requests_total", instance="webproxy-3", code="200", method="get", path="/status"}
    ϝτϦοΫͷ໊લ΋಺෦తʹ͸ϥϕϧ
    ϥϕϧ͸ऩूͨ͠ϝτϦοΫʹ෇༩͞Εͨ৘ใ

    View Slide

  17. #MPDL

    View Slide

  18. #MPDL
    wऩूͨ͠σʔλ͸ҰఆִؒͰ#MPDLͱݺ͹ΕΔ࿦ཧతͳմʹ·ͱΊΒΕΔ

    View Slide

  19. #MPDL
    w#MPDLʹ͸)FBE#MPDLͱͦΕҎ֎ͷछྨ͕ଘࡏ͢Δ

    View Slide

  20. #MPDL
    w)FBE#MPDL
    wϝϞϦ্ʹଘࡏ͢Δ
    w།Ұ௥هՄೳͳ#MPDL
    w#MPDL
    wҰఆظ͕ؒܦաͯ͠σΟεΫ্ʹӬଓԽ͞Εͨ΋ͷ
    w௥هෆՄ
    wมߋ͕Ճ͑ΒΕΔͷ͸$PNQBDUJPOͱ࡟আ࣌ͷΈ ޙड़

    View Slide

  21. ӬଓԽ
    w8"- 8SJUF"IFBE-PH

    w1SPNFUIFVT͸ऩूͨ͠σʔλΛॳΊʹ)FBE#MPDLʹॻ͖ࠐΉ
    wϝϞϦ্ʹ͋ΔͷͰҟৗऴྃ౳Ͱ༧ظͤͣشൃ͢ΔՄೳੑ͕͋Δ
    wσʔλͷ૕ࣦΛ๷͙ͨΊʹ8"-ϑΝΠϧ΁ஞ࣍ॻ͖ࠐΈ
    wѹॖ౳͸ߦΘͣ3BXσʔλͷ··ͳͷͰαΠζ͸େ͖Ί
    w࠶ىಈ࣌ʹ8"-ϑΝΠϧͷ಺༰ΛϝϞϦʹల։͢Ε͹ϓϩηε͕ऴྃͨ͠
    ࣌ͷঢ়ଶʹ໭Δ

    View Slide

  22. ӬଓԽ
    w#MPDL
    wҰఆظؒ σϑΥϧτͰ࣌ؒ
    Ͱ)FBE#MPDLΛσΟεΫ্ʹӬଓԽ
    wҎԼͷσʔλΛؚΉ
    wNFUBKTPO#MPDLͷϝλ৘ใ
    wJOEFYϥϕϧͱ஋͔ΒDIVOLͷϙδγϣϯΛಛఆ͢ΔసஔΠϯσοΫε
    wDIVOL֤4FSJFTͷαϯϓϧΛ·ͱΊͨ΋ͷ
    wUPNCTUPOFT࡟আ͞ΕͨϝτϦΫεͷσʔλ
    wͦΕͧΕͷ#MPDL͸׬શʹಠཱͨ࣌͠ܥྻσʔλϕʔεͱͯ͠ৼΔ෣͏

    View Slide

  23. ӬଓԽ
    #MPDLͷߏ଄ NFUBKTPOͷ಺༰

    View Slide

  24. *OEFY
    w֤#MPDLͷதͰ͸4FSJFTʹݻ༗ͷ*%ׂ͕ΓৼΒΕ͍ͯΔ
    http_requests_total{instance="webproxy-1", code="200", method="get", path="/"}
    http_requests_total{instance=“webproxy-1", code="200", method="get", path=“/status"}
    http_requests_total{instance="webproxy-3", code="200", method="get", path="/status"}
    ID: 4
    ID: 5
    ID: 10
    w*OEFY͸ϥϕϧͱ஋ͷશ૊Έ߹ΘͤʹରԠ͢ΔιʔτࡁΈͷ*%ϦετΛ࣋ͭ
    code=“200”: [ 3, 4, 5, 8, 10 ... ]
    path=“/“: [ 2, 4, 100, 901 ... ]
    path=“/status”: [ 5, 10, 11, 125, 12099 ... ]
    ...

    View Slide

  25. code=“200”: [ 3, 4, 5, 8, 10 ... ]
    path=“/“: [ 2, 4, 100, 901 ... ]
    path=“/status”: [ 5, 10, 11, 125, 12099 ... ]
    *OEFY
    w*OEFY͸ϥϕϧͱ஋ͷશ૊Έ߹ΘͤʹରԠ͢ΔιʔτࡁΈϦετΛ࣋ͭ
    w1SPN2-Ͱࢦఆ͞ΕͨϥϕϧͷϦετΛऔಘͯ͠ڞ௨͢Δ*%Λಛఆ
    PromQL: http_requests_total{code=“200”, path=“/status“}
    code=“200”: [ 3, 4, 5, 8, 10 ... ]
    path=“/status”: [ 5, 10, 11, 125, 12099 ... ]
    intersect: [ 5, 10 ]

    View Slide

  26. 1SPN2-࣮ߦ࣌ͷେ·͔ͳྲྀΕ
    ࢦఆظؒͷσʔλΛؚΉ#MPDLΛNFUBKTPO
    Λࢀরͯ͠ಛఆ
    *OEFY͔Β৚݅ʹҰக͢Δ4FSJFTͷ*%Λಛఆ
    4FSJFTͷ*%͔Β$IVOLͷϙδγϣϯΛಛఆ͠
    ͯαϯϓϧΛऔಘ
    ର৅ͷશ#MPDL͔Βಘͨ݁ՌΛϚʔδ
    #MPDL୯ମͰΫΤϦͷ࣮ߦʹ
    ඞཁͳ৘ใ͕શͯἧ͍ͬͯΔ

    View Slide

  27. $PNQBDUJPO
    wӬଓԽ͞Εͨ#MPDLΛఆظతʹ·ͱΊΔ࡞ۀ
    w໨త͸ΫΤϦͷޮ཰Խ
    wΫΤϦ࣮ߦ࣌͸ෳ਺ͷ#MPDL͔ΒσʔλΛऔಘͯ͠Ϛʔδ͢Δ
    w௕ظؒʹ౉ΔΫΤϦΫΤϦͷ৔߹ɺ͜ͷ࡞ۀΛେྔͷ#MPDLʹରͯ͠ߦ͏ͷ
    Ͱޮ཰͕ѱ͍

    View Slide

  28. $PNQBDUJPO

    View Slide

  29. $PNQBDUJPO

    View Slide

  30. 3FUFOUJPO
    wอ࣋ظؒΛ௒͑ͨ#MPDL͸࡟আ͞ΕΔ

    View Slide

  31. ༨ஊNNBQ

    w1SPNFUIFVTͷԾ૝ϝϞϦ
    ෺ཧϝϞϦ(#
    Ծ૝ϝϞϦ(#

    View Slide

  32. ༨ஊNNBQ

    wશ#MPDLͷJOEFYͱDIVOL͕Ծ૝ϝϞϦۭؒʹϚοϐϯά͞Ε͍ͯΔ

    View Slide

  33. ༨ஊNNBQ

    wΫΤϦ࣮ߦͰ#MPDLΛࢀরͨ࣌͠ʹ04͕σΟεΫ͔ΒϖʔδΛಡΈࠐΉ
    wଞͷ༻్͕෺ཧϝϞϦΛѹഭͨ࣌͠΋04ͷ൑அͰϖʔδ͕௥͍ग़͞ΕΔ
    w04ʹϝϞϦͷ؅ཧΛҕͶ͍ͯΔ

    View Slide

  34. ௕ظอଘͷखஈͱݕ౼

    View Slide

  35. 3FNPUF4UPSBHF

    View Slide

  36. 3FNPUF4UPSBHF͸ޙड़ͷཧ༝͔Β
    ࢖༻͠ͳ͔ͬͨͨΊϦϑΝϨϯεతͳ಺༰Ͱ͢

    View Slide

  37. 3FNPUF4UPSBHF
    wαʔυύʔςΟͷ֎෦ετϨʔδʹऩूͨ͠αϯϓϧΛॻ͖ࠐΈಡΈࠐΈ
    wετϨʔδ͕3FNPUF8SJUF3FBEͷ)551ϦΫΤετʹωΠςΟϒͰରԠ͠
    ͍ͯΔ৔߹͸ΤϯυϙΠϯτΛࢦఆ͢Δ͚ͩ
    wରԠ͍ͯ͠ͳ͍৔߹͸อଘՄೳͳܗࣜʹม׵͢ΔΞμϓλʔΛ࢖༻
    IUUQTQSPNFUIFVTJPEPDTQSPNFUIFVTMBUFTUTUPSBHFSFNPUFTUPSBHFJOUFHSBUJPOT

    View Slide

  38. 3FNPUF4UPSBHF
    wετϨʔδʹΑͬͯ͸XSJUFͷΈՄೳ
    ͳ΋ͷ΋
    wXSJUFͷΈͷετϨʔδ͸1SPN2-Λ
    αϙʔτ͍ͯ͠ͳ͍ͷͰɺετϨʔ
    δ͕ରԠ͢ΔΫΤϦͰࢀর͢Δ
    IUUQTQSPNFUIFVTJPEPDTPQFSBUJOHJOUFHSBUJPOTSFNPUFFOEQPJOUTBOETUPSBHF

    View Slide

  39. 3FNPUF4UPSBHFΛஅ೦ͨ͠ཧ༝
    wW·Ͱͷ࿩
    wऩू͞ΕͨαϯϓϧΛ3FNPUF8SJUFͷΩϡʔʹ௚઀όοϑΝ͍ͯͨ͠
    wΩϡʔʹαϯϓϧ͕ཷ·ΔͱϝϞϦΛফඅ͍ͯ͘͠
    wΩϡʔ͔ΒᷓΕͨσʔλ͸ૹ৴͞ΕͣʹࣦΘΕΔ

    View Slide

  40. 3FNPUF4UPSBHFΛஅ೦ͨ͠ཧ༝
    w3FNPUF4UPSBHFͷԠ౴஗Ԇ΍μ΢ϯൃੜ࣌ͷϝϞϦফඅྔ
    wࢲୡͷ؀ڥͰ͸࣌ؒ಺ʹऩू͢Δαϯϓϧ਺͸ίϯςφ਺ 4FSJFT਺
    ͷ૿ݮ
    ʹΑͬͯมಈ͢ΔͨΊ༧ଌ͕೉͍͠
    wΩϡʔ͕େ͖͗͢Δͱ࠷େϝϞϦফඅྔ͕૿͑ͯ00.Λى͜͢Մೳੑ͕͋Δ
    wΩϡʔ͕খ͗͢͞Δͱαϯϓϧ͕ᷓΕͯσʔλ͕ࣦΘΕΔ
    w৑௕Խ͞Εͨ1SPNFUIFVT͸ಉִؒ͡Ͱಉ͡ϝτϦΫεΛऩू͢Δ
    wಉ͡ઃఆͰಈ͘1SPNFUIFVTͰಉ࣌ʹ00.͕ൃੜ͢ΔՄೳੑ͕ߴͦ͏
    wσʔλ͕શ͘ऩूͰ͖ͳ͍λΠϛϯά͕ൃੜ͢Δ͔΋

    View Slide

  41. 8"-CBTFESFNPUFXSJUF
    wW͔Βಋೖ
    wWҎલऩूͨ͠αϯϓϧΛ௚઀ΤϯΩϡʔ
    wWҎ߱8"-͔Βஞ࣍ಡΈऔͬͯΤϯΩϡʔ
    w8"-΁ͷϙΠϯλΛอ࣋͠ɺॻ͖ࠐΈͷ੒ޭʹԊͬͯϙΠϯλΛલʹਐΊΔ
    w1SPNFUIFVTͷϝϞϦΛେ͖͘ফඅ͢Δ༻్͕ݮͬͨ
    w)FBE#MPDL
    wΫΤϦ࣮ߦ࣌ʹσΟεΫ͔Βϩʔυ͞ΕΔϖʔδ
    w3FNPUF8SJUF2VFVF

    View Slide

  42. 8"-CBTFESFNPUFXSJUF
    WҎલ
    WҎ߱

    View Slide

  43. 8"-CBTFESFNPUFXSJUF

    View Slide

  44. $PSUFYͷ঺հ

    View Slide

  45. $PSUFY
    w1SPNFUIFVTͷͨΊͷετϨʔδ
    w)PSJ[POUBMMZTDBMBCMF
    w)JHIMZ"WBJMBCMF
    w.VMUJUFOBOU
    w-POHUFSNTUPSBHF
    IUUQTHJUIVCDPNDPSUFYQSPKFDUDPSUFYCMPCNBTUFSJNHTMPHPQOH

    View Slide

  46. IUUQTHJUIVCDPNDPSUFYQSPKFDUDPSUFYCMPCNBTUFSEPDTBSDIJUFDUVSFQOH

    View Slide

  47. $PSUFY
    wओཁͳίϯϙʔωϯτͷ঺հ
    w%JTUSJCVUPS
    w*OHFTUFS
    w2VFSZ'SPOUFOE
    w2VFSJFS

    View Slide

  48. IUUQTHJUIVCDPNDPSUFYQSPKFDUDPSUFYCMPCNBTUFSEPDTBSDIJUFDUVSFQOH

    View Slide

  49. %JTUSJCVUPS
    w3FNPUF8SJUF࣌ͷ࠷લஈʹҐஔ͢ΔαʔϏε
    w෼ࢄϋογϡςʔϒϧΛ༻͍ͯ*OHFTUFSʹϦΫΤετΛৼΓ෼͚Δ
    wϋογϡϦϯά͸$POTVMʹอଘ
    wϋογϡؔ਺ʹ౉͞ΕΔཁૉ͸ͲͪΒ͔ࢦఆՄೳ
    wϝτϦοΫ໊ςφϯτ*%
    w4FSJFTͷશϥϕϧςφϯτ*%
    wޙऀͷํ͕෼ࢄ͠΍͍͕͢ɺΫΤϦ࣮ߦ࣌ʹෳ਺ͷ*OHFTUFSʹ໰߹͕ͤඞཁ
    wεςʔτϨεͰਫฏʹεέʔϧ͢Δ

    View Slide

  50. IUUQTHJUIVCDPNDPSUFYQSPKFDUDPSUFYCMPCNBTUFSEPDTBSDIJUFDUVSFQOH

    View Slide

  51. *OHFTUFS
    w%JTUSJCVUPS͔Βड͚औͬͨαϯϓϧΛ$IVOLTUPSFʹอଘ͢Δ
    w*OEFY%ZOBNP%# #JHUBCMF $BTTBOESB
    w$IVOL%ZOBNP%# #JHUBCMF $BTTBOESB ($4 4
    wաڈ࣌ؒ෼ͷαϯϓϧΛอ࣋͢ΔηϛεςʔτϑϧͳαʔϏε
    w௚ۙͷσʔλಡΈࠐΈͷߴ଎Խ
    w$IVOLTUPSF΁ͷॻ͖ࠐΈ૿෯Λճආ 8SJUFEFBNQMJpDBUJPO

    wϊʔυ͸جຊతʹಠཱͯ͠ಈ࡞͢Δ͕ɺϩʔϦϯάΞοϓσʔτ࣌ͷΈྫ֎త
    ʹάϧʔϓ͔Βग़Δϊʔυ͕ࢀՃ͢ΔϊʔυʹΠϯϝϞϦͷσʔλΛసૹ͢Δ

    View Slide

  52. ॻ͖ࠐΈ૿෯
    wྫ%ZOBNP%#ͷ8$6 8SJUF$BQBDJUZ6OJU

    w8$6͋ͨΓ,#·Ͱͷॻ͖ࠐΈ͕ඵؒճՄೳ
    wςʔϒϧʹରͯ͠8$6ͷ਺Λࢦఆ͢Δ ར༻ྉۚʹӨڹ

    w,#ͷॻ͖ࠐΈΛճͰߦ͏৔߹͸8$6Λফඅ͢Δ
    wॻ͖ࠐΈαΠζͷ୺਺͸੾Γ্͛ΒΕΔ ,#ˠ,#

    wখ͍͞αΠζͷσʔλΛࡉ͔͘ॻ͖ࠐΜͰ͍Δͱ8$6ΛΑΓଟ͘ফඅ͢Δ
    w,#Λճॻ͖ࠐΉͱ8$6ফඅ਺͸
    w,#ʹ·ͱΊͯճͰॻ͖ࠐΉͱ8$6ফඅ਺͸

    View Slide

  53. IUUQTHJUIVCDPNDPSUFYQSPKFDUDPSUFYCMPCNBTUFSEPDTBSDIJUFDUVSFQOH

    View Slide

  54. 2VFSJFS
    w1SPN2-Λ࣮ࡍʹධՁͯ͠αϯϓϧΛऔಘ͢ΔαʔϏε
    w௚ۙͷσʔλ͸*OHFTUFS͔Βऔಘ
    wݹ͍σʔλ͸$IVOLTUPSF͔Βऔಘ
    wެࣜͷυΩϡϝϯτʹ΋ߦ͙Β͍͔͠આ໌͕ॻ͔Ε͍ͯͳ͍

    View Slide

  55. IUUQTHJUIVCDPNDPSUFYQSPKFDUDPSUFYCMPCNBTUFSEPDTBSDIJUFDUVSFQOH

    View Slide

  56. 2VFSZ'SPOUFOE
    w0QUJPOBMͳαʔϏεͰɺΫΤϦ࣮ߦ࣌ͷޮ཰ԽΛਤΔ
    w2VFVFJOHࣦഊͨ͠ΫΤϦͷϦτϥΠ 2VFSJFSͷෛՙ෼ࢄ
    w4QMJUUJOH௕ظؒͷΫΤϦΛ೔୯Ґʹ෼ղ֤ͯ͠2VFSJFSʹྲྀ͢
    w୯Ұͷ2VFSJFS΁ͷภΓΛͳͯ͘͠00.ͷൃੜΛ཈ࢭ
    w֤2VFSJFS͔Βड͚औͬͨ݁Ռ͸Ϛʔδ͞ΕΔ
    w$BDIJOHΫΤϦ݁ՌͷΩϟογϡ
    wಉ༷ͷΫΤϦΛ࠶౓࣮ߦ͢Δࡍ͸લճ࣮ߦ͍ͯ͠ͳ͍࣌ؒͷΈऔಘͯ͠
    ΩϟογϡͱϚʔδ

    View Slide

  57. 5IBOPT

    View Slide

  58. 5IBOPT
    w1SPNFUIFVTͷ#MPDL͸ಠཱͨ͠
    σʔλϕʔεͱͯ͠ৼΔ෣͏
    wͰ͸ϩʔΧϧετϨʔδʹແͯ͘΋
    ྑ͍ͷͰ͸ʁ
    w#MPDLΛΦϒδΣΫτετϨʔδʹ
    ίϐʔ͢Δ
    w1SPNFUIFVTͱΦϒδΣΫτετ
    Ϩʔδ্ͷσʔλʹରͯ͠ಁաతʹ
    ΫΤϦΛ࣮ߦ͢Δ؀ڥΛߏங
    IUUQTHJUIVCDPNUIBOPTJPUIBOPTCMPCNBTUFSEPDTJNH
    5IBOPTMPHP@GVMMNFEJVNQOH

    View Slide

  59. IUUQTHJUIVCDPNJNQSPCBCMFFOHUIBOPTCMPCNBTUFSEPDTJNHBSDIKQH

    View Slide

  60. 5IBOPT
    wओཁίϯϙʔωϯτͷղઆ
    w4JEFDBS
    w$PNQBDUPS
    w4UPSF(BUFXBZ
    w2VFSJFS
    w3VMFS

    View Slide

  61. IUUQTHJUIVCDPNJNQSPCBCMFFOHUIBOPTCMPCNBTUFSEPDTJNHBSDIKQH

    View Slide

  62. 4JEFDBS
    w1SPNFUIFVTͷ4JEFDBSͱͯ͠ಈ͘ϓϩηε
    w4IJQQFS
    w#MPDLΛΦϒδΣΫτετϨʔδʹΞοϓϩʔυ
    wJOEFY DIVOL͸ͦͷ··
    wNFUBKTPOʹ͸5IBOPTಛ༗ͷ৘ใΛ௥Ճ
    w4UPSF"1*
    w2VFSJFS͔ΒͷϦΫΤετʹԠͯ͡ϩʔΧϧͷ1SPNFUIFVTʹΫΤϦΛ࣮ߦ

    View Slide

  63. IUUQTHJUIVCDPNJNQSPCBCMFFOHUIBOPTCMPCNBTUFSEPDTJNHBSDIKQH

    View Slide

  64. 4JEFDBS
    ϩʔΧϧͷNFUBKTPO ΦϒδΣΫτετϨʔδͷNFUBKTPO

    View Slide

  65. IUUQTHJUIVCDPNJNQSPCBCMFFOHUIBOPTCMPCNBTUFSEPDTJNHBSDIKQH

    View Slide

  66. $PNQBDUPS
    w$PNQBDUJPO
    w4JEFDBS͸࡞੒͞Εͨ#MPDLΛग़དྷΔ͚ͩૣ͘Ξοϓϩʔυ͢Δ
    wϩʔΧϧͰߦΘΕΔ$PNQBDUJPOͷ࢓૊Έ͔Β͸֎ΕΔ
    wΦϒδΣΫτετϨʔδͷ#MPDLʹಉ༷ͷॲཧΛ࣮ࢪ͢Δϓϩηε͕ඞཁ
    w%PXOTBNQMJOH
    w෼ ࣌ؒͷ࿮ ݻఆ
    Ͱ#MPDLͷμ΢ϯαϯϓϦϯάΛߦ͏
    w࣌ؒ࿮಺ͷ֤ूܭ஋ $PVOU 4VN .BY .JO $PVOUFS
    Λอଘ

    View Slide

  67. IUUQTHJUIVCDPNJNQSPCBCMFFOHUIBOPTCMPCNBTUFSEPDTJNHBSDIKQH

    View Slide

  68. 4UPSF(BUFXBZ
    w4UPSF"1*
    wΦϒδΣΫτετϨʔδͷσʔλΛࢀর͢ΔΤϯυϙΠϯτ
    w4JEFDBSͷ4UPSF"1*ͱಉ͡ΠϯλʔϑΣʔε

    View Slide

  69. IUUQTHJUIVCDPNJNQSPCBCMFFOHUIBOPTCMPCNBTUFSEPDTJNHBSDIKQH

    View Slide

  70. 2VFSJFS
    w2VFSZ"1*
    wάϩʔόϧϏϡʔΛఏڙ͢Δίϯϙʔωϯτ
    w1SPNFUIFVTYʹޓ׵ੑͷ͋Δ2VFSZ"1*Λެ։
    w֤4UPSF"1*ͷσʔλΛݩʹΫΤϦΛධՁ͢Δ
    w4UPSF͸ىಈ࣌ʹࢦఆ͢Δ͔ɺ%/4 'JMFϕʔεͰαʔϏεσΟεΧόϦ
    w%FEVQMJDBUJPO
    w֤4UPSF"1*͔Βऔಘͨ͠σʔλΛϨϓϦΧϥϕϧΛݩʹॏෳഉআ͢Δ

    View Slide

  71. 2VFSJFS
    ௚ۙͷσʔλ աڈͷσʔλ
    ଞͷ5IBOPT
    Ϋϥελͷσʔλ

    View Slide

  72. 2VFSJFS
    w2VFSJFSͷࢀর͢Δ4UPSF͕4UPSF(BUFXBZ͚ͩͷঢ়ଶͰಈ͔ͯ͠ΈΔͱ

    View Slide

  73. 2VFSJFS
    w2VFSJFSͷࢀর͢Δ4UPSF͕4UPSF(BUFXBZ͚ͩͷঢ়ଶͰಈ͔ͯ͠ΈΔͱ
    #MPDLʹͳ͍ͬͯͳ͍
    ௚ۙͷσʔλ͸ݟ͑ͳ͍
    4JEFDBSͷࢀরΛ௥Ճ͢
    Δͱ׬શͳάϥϑʹͳΔ

    View Slide

  74. %FEVQMJDBUJPO
    w1SPNFUIFVTͷઃఆʹFYUFSOBM@MBCFMTΛ௥Ճ
    w5IBOPT2VFSJFSىಈίϚϯυͰॏෳഉআ༻ͷϥϕϧΛࢦఆ

    View Slide

  75. %FEVQMJDBUJPO
    up{job="prometheus",env="2",cluster="1",replica="A"} 1
    up{job="prometheus",env="2",cluster="1",replica="B"} 1
    up{job="prometheus",env="2",cluster="2",replica="A"} 1
    up{job="prometheus",env="2",cluster="1"} 1
    up{job="prometheus",env="2",cluster="2"} 1
    ϨϓϦΧϥϕϧҎ֎ͷϥϕϧ͕શͯ
    ಉ͡αϯϓϧ͸͚ͭͩΛ࢒͢

    View Slide

  76. IUUQTHJUIVCDPNJNQSPCBCMFFOHUIBOPTCMPCNBTUFSEPDTJNHBSDIKQH

    View Slide

  77. 3VMFS
    w5IBOPTΫϥελ಺ͷσʔλʹରͯ͠SFDPSEJOH΍BMFSUͷϧʔϧΛద༻͍ͨ͠
    ৔߹ʹ࢖༻
    w2VFSJFSܦ༝Ͱఆظతʹ1SPN2-ΛධՁ͢Δίϯϙʔωϯτ
    w1SPNFUIFVTͷϩʔΧϧʹσʔλ͸࢒ΔͷͰطଘͷϧʔϧ΋࢖͑Δ
    wϩʔΧϧͷσʔλ͚ͩͰ͸ෆे෼ͳ৔߹ʹ3VMFSΛ࢖༻
    wͭͷΫϥελͷ಺ͭͰҟৗ͕ൃੜͨ͠ͱ͖ʹΞϥʔτΛൃՐ͍ͤͨ͞
    wϩʔΧϧετϨʔδͷอ࣋ظؒΛ௒͑ΔൣғͷσʔλΛର৅ʹ͍ͨ͠
    wFUD

    View Slide

  78. 5IBOPTͷӡ༻ධՁ

    View Slide

  79. ӡ༻ίετ
    w5IBOPTͷίϯϙʔωϯτ͸΄ͱΜͲεςʔτΛ࣋ͨͳ͍
    w$PNQBDUPSҎ֎͸؆୯ʹ৑௕ԽՄೳ
    wϑϧϚωʔδυͷΦϒδΣΫτετϨʔδΛ࢖༻͢Δ৔߹͸ӡ༻ίετ͕ඇৗ
    ʹ௿͍
    w࣌఺Ͱ͸($4 4 "[VSF4UPSBHF͕4UBCMF
    wIUUQTHJUIVCDPNUIBOPTJPUIBOPTCMPCNBTUFSEPDTTUPSBHFNE

    View Slide

  80. ΫΤϦͷ࣮ߦ଎౓
    wΦϒδΣΫτετϨʔδʹ#MPDLΛஔ͘ͱɺϑΝΠϧͷμ΢ϯϩʔυ͕ඞཁͰ
    ΫΤϦͷ࣮ߦ଎౓͕஗͘ͳΔͷͰ͸
    wର৅ظ͕ؒ೔ఔ౓ͳΒ΄ͱΜͲ஗Ԇ͸ײ͡ΒΕͳ͍ ମײ஋

    w4UPSF(BUFXBZ͸#MPDLͷ*OEFYΛϩʔΧϧʹΩϟογϡ͢Δ
    wΦϒδΣΫτετϨʔδ͔Βຖճऔಘ͠ͳ͍ͷͰ$IVOLͷಛఆ͸ߴ଎
    w#MPDLͷσʔλ͸$PNQBDUJPOͷ࣌Ҏ֎͸ෆมͳͷͰΩϟογϡ͠΍͍͢
    w$IVOLͷऔಘ࣌΋ϑΝΠϧશମΛμ΢ϯϩʔυ͢Δඞཁ͸ͳ͍
    wόΠτ୯ҐͰൣғࢦఆͯ͠μ΢ϯϩʔυՄೳ

    View Slide

  81. খ࢝͘͞ΊΔ
    w͍͖ͳΓશͯͷίϯϙʔωϯτΛ࢖͍࢝ΊΔඞཁ͸ͳ͍
    w΋͠৑௕Խ͞Εͨ1SPNFUIFVT͕ࠓͷ؀ڥʹ͋ΔͳΒ
    wͱΓ͋͑ͣ#MPDLΞοϓϩʔυͷઃఆ͸ೖΕͣʹ4JEFDBSΛಈ͔ͯ͠ΈΔ
    w2VFSJFSܦ༝Ͱࢀর͢Ε͹྆ऀͷҧ͍Λҙࣝ͢Δඞཁ͕ͳ͘ͳΔ

    View Slide

  82. খ࢝͘͞ΊΔ
    w௕ظอଘ͕ඞཁͳΒΦϒδΣΫτετϨʔδʹ#MPDLͷΞοϓϩʔυΛ։࢝

    View Slide

  83. খ࢝͘͞ΊΔ
    w$PNQBDUPSΛ௥Ճͯ͠ఆظతʹ$PNQBDUJPO%PXOTBNQMJOH3FUFOUJPO

    View Slide

  84. খ࢝͘͞ΊΔ
    w4UPSF(BUFXBZΛ௥Ճͯ͠2VFSJFS͔Βࢀর͢Ε͹άϩʔόϧϏϡʔͷ׬੒

    View Slide

  85. ஫ҙ఺
    w1SPNFUIFVTΛ׬શʹεςʔτϨεʹͰ͖ΔΘ͚Ͱ͸ͳ͍
    w4JEFDBS͕ΞοϓϩʔυͰ͖Δͷ͸ӬଓԽ͞Εͨ#MPDLͷΈ
    w௚ۙ࣌ؒఔͷσʔλ͸1SPNFUIFVTͷϩʔΧϧετϨʔδʹ͔͠ଘࡏ͠ͳ͍
    wΦϓγϣϯͰ#MPDLԽͷλΠϛϯάΛม͑Δ͜ͱ͸Ͱ͖Δ
    wTUPSBHFUTECNJOCMPDLEVSBUJPOI
    wTUPSBHFUTECNBYCMPDLEVSBUJPOI
    w5IBOPTW͔Β৽͘͠3FDFJWFSίϯϙʔωϯτ͕௥Ճ
    wFYQFSJNFOUBM

    View Slide

  86. IUUQTHJUIVCDPNUIBOPTJPUIBOPTCMPCNBTUFSEPDTQSPQPTBMTBQQSPWFE@UIBOPTSFNPUF
    SFDFJWFNEUFDIOJDBMTVNNBSZPGUIBOPTSFDFJWFSDPNQPOFOU

    View Slide

  87. 3VMFSͷϦεΫ
    w3VMFS͕1SPN2-ΛධՁ͢Δܦ࿏͸෼ࢄ͍ͯ͠Δ
    w3VMFS2VFSJFS4UPSF"1*
    wΫΤϦ࣮ߦͷ৴པੑ͕௿͍ͷͰΫϦςΟΧϧͳΞϥʔτͷධՁʹ͸࢖Θͳ͍
    wඞཁʹഭΒΕͳ͚Ε͹1SPNFUIFVTͷϩʔΧϧϧʔϧΛ͓͢͢Ί͢Δ

    View Slide

  88. ·ͱΊ

    View Slide

  89. 5IBOPT
    w·ͣ͸4JEFDBSͱ2VFSJFSΛೖΕΔ͚ͩͰ΋ϝϦοτ͸͋Δ
    wΦϒδΣΫτετϨʔδؔ࿈ͷઃఆ͸ඞཁʹͳͬͨλΠϛϯάͰ௥Ճ
    wϑϧϚωʔδυͳΦϒδΣΫτετϨʔδΛ࢖͑͹ӡ༻ίετ͸௿͍

    View Slide

  90. 3FNPUF4UPSBHF
    w3FNPUF8SJUF͕8"-CBTFEʹͳͬͯඇৗʹ࢖͍΍͘͢ͳͬͨ
    w1SPNFUIFVTଆͷઃఆ͸ѹ౗తʹ3FNPUF8SJUFͷํָ͕
    wઃఆϑΝΠϧʹΤϯυϙΠϯτΛ௥ه͢Δ͚ͩ
    wϦΞϧλΠϜੑ΋ߴ͍

    View Slide

  91. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
    ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU

    View Slide