ELKstackとAthenaの素敵な関係

2017/02/10 JAWS-UG#9で発表した資料になります

Transcript

1. & - , T U B D L ͱ " U I F O B ͷ
   ૉ ఢ ͳ ؔ ܎
   JAWS-UGԣ඿ #9 - Operational Excellence
   2017/02/10 ੢ଜ ༡
   

   View Slide

2. ˞ ຊ εϥ Π υ ͸
   & - , T U B D L ͱ " N B [ P O " U I F O B Λ
   ٕ ज़ త ʹ ࿈ ܞ ͞ ͤ ͨ ࿩ Ͱ ͸ ͋ Γ · ͤ Μ
   ͢ ͍ · ͤ Μ ʂ
   ͱ ͍ ͏ ͔ ٕ ज़ త ͳ ࿩ ͋ Γ · ͤ Μ
   

   View Slide

3. & - , T U B D L Ͱ ͷ ӡ ༻ Ͱ ײ ͡ ͨ ෆ ศ Λ
   " N B [ P O " U I F O B ͕
   औ Γ আ ͍ͯ ͘ Ε ͨ ͷ Ͱ
   ײ ಈ Λ ڞ ༗ ͠ ͨ ͍ ͱ ͍ ͏ ࿩
   

   View Slide

4. ͔ ͍ ͠ Ό ͍ Μ
   ͤ ͍ ΂ ͭ ɿ ͓ ͱ ͜
   Ϩ ϕ ϧ ɿ
   ) 1
   . 1
   ࣗ ݾ ঺ հ
   ࣾ ձ ਓ ྺ
   d ɹ % $ Ͱ ؂ ࢹ Φϖ Ϩ ʔ λ ʔ
   d ɹ ๭ ι ʔ γ ϟϧ ήʔϜ ܥ ձ ࣾ
   d / 0 8 ɹ ɹ ג ࣜ ձ ࣾ " 1 1 - * # 05
   "8 4 ྺ ໿ ೥
   ͳ · ͑ ɹ ɹ ɿ ੢ ଜ ɹ ༡
   ͭ ͍ ͬ ͨ ʔ ɿ
   !HBDIBSJPO
   

   View Slide

5. View Slide

6. View Slide

7. &-,TUBDL "NB[PO"UIFOB
   FMBTUJDTFBSDILJCBOBͰͷϩάࢀর
   ٹੈओ"UIFOB
   ӡ༻5JQTతͳ
   ҰԠ৮Ε͓ͯ͘΂͖͜ͱ
   ·ͱΊ
   ໨ ࣍
   

   View Slide

8. & - , T U B D L
   • elastic search + log stash + kibana = elk stack
   • L ͷ෦෼ʢσʔλ౤ೖʣ͸fluentdͩͬͨΓ͢Δͱ
   EFKͱͳΔΒ͍͠ɻ
   • elasticsearchʹೖΕͯkibanaͰݟͯΔͷͰ

   εϥΠυ಺Ͱ͸૯শͱͯ͠࢖ͬͯ·͢ɻ
   

   View Slide

9. " N B [ P O " U I F O B
   • 2016/12/01 ެ։
   • US East (Northern Virginia)ͱUS West (Oregon)

   Ͱར༻Մೳ(2017/02/09 ݱࡏ)
   • S3্ͷσʔλʹϑΟʔϧυΛఆٛͯ͠

   ΫΤϦΛ౤͛Δͱɺ਺ඵͰ݁Ռ͕ฦͬͯ͘Δɻ
   

   View Slide

10. " N B [ P O " U I F O B
    • 2016/12/01 ެ։
    • US East (Northern Virginia)ͱUS West (Oregon)

    Ͱར༻Մೳ(2017/02/09 ݱࡏ)
    • S3্ͷσʔλʹϑΟʔϧυΛఆٛͯ͠

    ΫΤϦΛ౤͛Δͱɺ਺ඵͰ݁Ռ͕ฦͬͯ͘Δɻ
    ˢϚδͰ ਆ͡Όͳ͍͔͢
    

    View Slide

11. FMBTUJDTFBSDILJCBOB
    Ͱͷϩάࢀর
    

    View Slide

12. Ͷ Β ͍
    • ͱΓ͋͑ͣௐࠪʹ࢖͏ϩά͸elasticsearch
    • ֤αʔόͷϩάΛfluentdͰू໿ɺ౤ೖ
    • ݸʑͷαʔόͷΈʹ͔͠࢒Βͳ͍ϩάΛͳ͘͢

    ໨త΋͋Γ
    • ͨ·ʹlogstash
    • ΫΤϦʹΑΔΞϥʔτઃఆʢࣗલεΫϦϓτʣ
    

    View Slide

13. $ V T U P N - 0 (
    

    View Slide

14. & - #
    

    View Slide

15. 3 % 4
    

    View Slide

16. F M B T U J D T F B S D I ӡ ༻ ϧ ʔ ϧ త ͳ
    • ೖΕΔσʔλ͸ϑΝΠϧͱͯ͠΋ॻ͖ग़͓ͯ͘͠ʢJSONʣ
    • dailyͰgzipͯ͠S3΁backup
    • ྔͷଟ͍ϩάʢ1೔10GBҎ্ʣஷ·Δϩά͸3೔Ͱindexຖ࡟আ
    • Ҏલɺ௕ظͰݟΕΔΑ͏ʹ͍͕ͯͨ͠ɺkibanaද͕͔ࣔͳΓ஗͘ͳͬͯ͠·ͬͨ
    • ͦ΋ͦ΋ͦΜͳʹաڈͷΛݟͳ͔ͬͨ
    • ετϨʔδ΋༰ྔΛऔΒͳͯ͘͸͍͚ͳ͍ͨΊ௕ظؒอ࣋ͨ͘͠ͳ͔ͬͨ
    • εφοϓγϣοτ͸ͱͬͨΓͱͬͯͳ͔ͬͨΓ(ũųųƅƁ
    • ͱ͍ͬͯͨ΍ͭ΋Ұिؒ΄ͲͰফ͍ͯͨ͠ɻ
    • ࠓ͸rundeck͞ΜͰຖ೔ࣗಈ࣮ߦ؅ཧɻS3ʹอଘɻ
    

    View Slide

17. ໰୊఺
    

    View Slide

18. ໰ ୊ ఺
    • index࡟আޙͷௐࠪ
    • ݟ͍ͨϩά͸΄ΜͷҰ෦෼
    • snapshotͷϦετΞͰ͖ͳ͍৔߹
    • ਺ेGBͷϩάΛDLͯ͠ղౚͯ͠grepͯ͠ɾɾɾ
    

    View Slide

19. ໰ ୊ ఺
    • elasticsearchʹೖΕΔͷ΋ͦΕͳΓʹ༻ҙ͕ඞཁ
    • ύʔαʔ୳ͨ͠Γɺॻ͍ͨΓ
    • template࡞ͬͨΓ(dynamic templateͰେମରԠ)
    • ٸͳґཔʹରԠ͖͠Εͳ͍
    

    View Slide

20. ٹ ੈ ओ
    

    View Slide

21. ࣄ ྫ
    ໰ ͍ ߹ Θ ͤ ௐ ࠪ ͷ ҝ ɺ
    ա ڈ ͷ " 1 * ϩ ά ͕ Έ ͨ ͍
    

    View Slide

22. ͦ Ε ͳ Β L J C B O B Ͱ ݟ Ε · ͬ ͤ ʂ
    

    View Slide

23. ͦ Ε ͳ Β L J C B O B Ͱ ݟ Ε · ͬ ͤ ʂ
    Ұिؒ΄ͲલͳΜͰ͕͢ʜ
    

    View Slide

24. T O B Q T I P U ΋ ͏ ͳ ͍ Θ ʜ
    Ұिؒ΄ͲલͳΜͰ͕͢ʜ
    

    View Slide

25. 4͔Βϩά (#௒
    Λ%-ͯ͠ղౚͯ͠HSFQ
    ͭΒ͍
    

    View Slide

26. ٹ ੈ ओ
    

    View Slide

27. \
    EBUFz
    UISFBElŠl
    MFWFM*/'0
    IBOEMFSlŠl
    64&3@*%lŠl
    WFSTJPOlŠl
    4&44*0/@*%lŠl
    OB[P
    NFTTBHFl63-IUUQ1BSBNFUFS\dུd^
    IPTUOBNFlBQJTEGTGPN
    ^
    ݩϩά +40/ܗࣜ
    
    

    View Slide

28. $3&"5&&95&3/"-5"#-&*'/05&9*454BQJTFSWFSBQJMPH@@
    AEBUFAUJNFTUBNQ
    AUISFBEATUSJOH
    AMFWFMATUSJOH
    AIBOEMFSATUSJOH
    A64&3@*%ATUSJOH
    AWFSTJPOATUSJOH
    A4&44*0/@*%ATUSJOH
    AOB[PATUSJOH
    ANFTTBHFATUSJOH
    AIPTUOBNFATUSJOH
    
    
    308'03."54&3%&PSHPQFOYEBUBKTPOTFSEF+TPO4FS%F
    8*5)4&3%&1301&35*&4
    TFSJBMJ[BUJPOGPSNBU
    
    -0$"5*0/TMPHCVDLFUBQJTFSWFS
    %#5BCMF࡞੒
    

    View Slide

29. 4&-&$5
    '30.BQJMPH@@
    8)&3&VTFS@JE
    ΫΤϦ࣮ߦ
    

    View Slide

30. ݁Ռ
    DTWͰͷ%-΋Մೳ
    

    View Slide

31. Β͘Β͘ڞ༗
    

    View Slide

32. ࣄ ྫ
    4 ͷ ΞΫ η ε ϩ ά ͕ ݟ ͨ ͍
    

    View Slide

33. %FW 0QT
    -BNCEBͰը૾ॲཧͯ͠Δ͠
    "1*͔Β΋ૢ࡞͍ͯ͠ΔͷͰɺ
    4ΞΫηεϩάΈ͍ͨͰ͢ʂ
    XFCϖʔδೖͬͯΔΘ͚Ͱ΋ͳ͍͔
    Β0/ʹͯ͠ͳ͔ͬͨΘʜ
    
    ઃఆ͠·͢
    

    View Slide

34. MPHHJOH0/
    0QT
    ϩάग़Δ·Ͱʹ਺͔͔࣌ؒΔͱॻ͍ͯ͋Δ
    ˞IUUQEPDTBXTBNB[PODPN
    ϩάLJCBOBͰݟΕ·͔͢Ͷʁ
    %&-&5&Λߦͳ͍ͬͯΔϦΫΤετ͕ݟ͍ͨ
    ɹ;Ή
    ɹFMBTUJDTFBSDIʹೖΕͯ
    ɹݕࡧͰ͖ΔΑ͏ʹ͠Α͏
    %FW
    

    View Slide

35. ͯ͞FMBTUJDTFBSDIʹೖΕΔ४උ͠ͱ͔͘ɻ
    ͲΜͳײ͡ͷ΍͚ͭͩͬʁ
    0QT
    φχίϨ
    IUUQEPDTBXTBNB[PODPN
    KB@KQ"NB[PO4MBUFTUEFW
    4ΞΫηεϩάϩάܗࣜαϯϓϧ
    BEGCFEBFGCBDFEGEFEFBDGGEFDEFGCFNZCVDLFU'FC
    >
    BEGCFEBFGCBDFEGEFEFBDGGEFDEFGCF&'&9".1-&
    3&45(&57&34*0/*/((&5NZCVDLFU WFSTJPOJOH)5514$POTPMF
    

    View Slide

36. MPHTUBTIͰύʔεͰ͖Δ༷ʹͯ͠ʜ
    FMBTUJDTFBSDIʹೖΕͯʜ
    LJCBOBͰEBTICPBSE࡞ͬͯʜ
    

    View Slide

37. %&-&5&ͷϩάΛݟ͍͚ͨͩͳͷʹʜ
    ࣌ؒʜ͔͔Δʜͳ͊
    

    View Slide

38. ٹ ੈ ओ
    

    View Slide

39. $3&"5&&95&3/"-5"#-&*'/05&9*454T@BDDFTTMPHSFTPVSTF@CVDLFU
    #VDLFU0XOFSTUSJOH
    #VDLFUTUSJOH
    3FRVFTU%BUF5JNFTUSJOH
    3FNPUF*1TUSJOH
    3FRVFTUFSTUSJOH
    3FRVFTU*%TUSJOH
    0QFSBUJPOTUSJOH
    ,FZTUSJOH
    3FRVFTU63*@PQFSBUJPOTUSJOH
    3FRVFTU63*@LFZTUSJOH
    3FRVFTU63*@IUUQ1SPUPWFSTJPOTUSJOH
    )551TUBUVTTUSJOH
    &SSPS$PEFTUSJOH
    #ZUFT4FOUTUSJOH
    0CKFDU4J[FTUSJOH
    5PUBM5JNFTUSJOH
    5VSO"SPVOE5JNFTUSJOH
    3FGFSSFSTUSJOH
    6TFS"HFOUTUSJOH
    7FSTJPO*ETUSJOH
    
    
    308'03."54&3%&PSHBQBDIFIBEPPQIJWFTFSEF3FHFY4FS%F
    8*5)4&3%&1301&35*&4
    TFSJBMJ[BUJPOGPSNBU JOQVUSFHFY >
    >
    aa<
    aa> >
    >
    >
    
    >
    >
    aaa >
    >
    c>
    aaa c>
    >
    >
    >
    >
    >
    >
    
    a>a
    >
    
    
    -0$"5*0/bTBXTMPH4SFTPVSTF@CVDLFU
    %#5BCMF
    ࡞੒
    

    View Slide

40. 4&-&$5SFNPUFJQ
    SFRVFTUFS
    DPVOU
    DPVOU
    '30.SFTPVSTF@CVDLFU
    8)&3&SFRVFTUVSJ@PQFSBUJPO%&-&5&
    HSPVQCZSFNPUFJQ SFRVFTUFS
    ΫΤϦ࣮ߦ
    

    View Slide

41. SFNPUFJQSFRVFTUFS DPVOU
    BSOBXTJBNˎˎˎˎˎˎˎˎVTFSEFWFMPQFS
    BSOBXTJBNˎˎˎˎˎˎˎˎVTFSEFWFMPQFS
    BSOBXTJBNˎˎˎˎˎˎˎˎVTFSEFWFMPQFS
    dུd
    BSOBXTJBNˎˎˎˎˎˎˎˎVTFSEFWFMPQFS
    ˎˎˎˎˎˎˎˎ BSOBXTJBNˎˎˎˎˎˎˎˎVTFSEFWFMPQFS
    ˎˎˎˎˎˎˎˎ BSOBXTJBNˎˎˎˎˎˎˎˎVTFSEFWFMPQFS
    ˎˎˎˎˎˎˎˎ BSOBXTJBNˎˎˎˎˎˎˎˎVTFSEFWFMPQFS
    ݁Ռ
    

    View Slide

42. ·ͱΊ
    

    View Slide

43. · ͱ Ί
    • Amazon AthenaͰS3্ͷϩά͕ଈղੳՄೳঢ়ଶʹ

    AthenaͰݟΔͨΊʹ~ΛS3ʹ͸͘Α͏ʹͯ͠…ͱ͔Ͱ͸ͳ͘ɺ

    ͢ͰʹόοΫΞοϓͱ্͍ͯ͛ͯͨ͠ϩάશ͕ͯࣗಈతʹର৅ͱݴ͑Δɻ

    AWSͷαʔϏε(S3 / CloudFront / ELB / CloudTrail)͔Βॻ͖ग़͢ϩά΋ɺ
    ௐ͚ࠪͩͳΒ͜ΕͰे෼ɻ
    • elasticsearch͔Β࡟আͯ͠΋S3ʹϩάϑΝΠϧ͋Ε͹ௐࠪ͸OK

    ௕ظؒݟΕΔ༷ʹ͢ΔͨΊʹ

    αʔόεϖοΫΛ্͓͛ͯ͘ඞཁ͸ແ͘ͳͬͨɻ

    ˠ ຊ౰ʹඞཁͳظؒͷΈͰΑ͘ͳͬͨɻ

    ɹ→ snapshot͸ͪΌΜͱऔΖ͏+͋Δఔ౓ظؒ࣋ͬͯͯ΋Α͔ͬͨ(൓ল)
    

    View Slide

44. · ͱ Ί
    • ϦΞϧλΠϜͷϩάղੳ → elasticsearch + kibana
    • S3ʹ͓͍ͨϩάௐࠪ → Amazon Athena
    

    View Slide

45. & - , T U B D L Ͱ ͷ ӡ ༻ Ͱ ײ ͡ ͨ ෆ ศ Λ
    " N B [ P O " U I F O B ͕
    औ Γ আ ͍ͯ ͘ Ε ͨ ͷ Ͱ
    ײ ಈ Λ ڞ ༗ ͠ ͨ ͍ ͱ ͍ ͏ ࿩ Ͱ ͠ ͨ
    

    View Slide

46. ޚ ਗ਼ ௌ
    ͋ Γ ͕ ͱ ͏ ͝ ͟ ͍ · ͠ ͨ
    

    View Slide