AWSをフル活用した大規模ログ収集基盤 / log collecting foundation of big data by AWS

Transcript

1. 1.

   "84Λϑϧ׆༻ͨ͠
   େن໛ϩάऩूج൫ 
   4PGUCBOL
   $PSQ

2. 2.

   ᖛాɹ༎ ιϑτόϯΫגࣜձࣾ
   ݉ "HPPQ
   ޷͖ͳ"84αʔϏε "UIFOB
   
   (MVF όοΫΤϯυશൠ

3. 3.

   Ґஔ৘ใऩू ม׵ ׆༻

4. 4.

   Ґஔ৘ใऩू ม׵ ׆༻

5. 5.

   
    εϚϗΞϓϦͷاը։ൃ اը͔Β։ൃ·Ͱશͯ಺੡Խ ϩάΛूΊΔػೳΛΞϓϦʹ௥Ճ ϏοάσʔλͷऩूɾՁ஋ͷఏڙ

6. 6.

   None
7. 7.

   None
8. 8.

   Ґஔ৘ใऩू ม׵ ׆༻

9. 9.

   None
10. 10.

    Title Text Main Point -! - 10 ϐϯϙΠϯτਓྲྀσʔλͷՄࢹԽ

11. 11.

    ڵຯ͕͋Δํ͸ฐࣾ)1·Ͱ IUUQTXXXBHPPQDPKQ

12. 12.

    Ґஔ৘ใऩू ม׵ ਓͷྲྀΕ

13. 13.

    ϩάσʔλͷ֓ཁ
    εϚʔτϑΥϯΞϓϦ͔Βऩू
    த਎͸௨৴ঢ়گͳͲʹؔ͢Δϩά
    +40/ܗࣜ
    ೔Ͱऩू͢ΔσʔλϘϦϡʔϜ
    ݅਺
    ਺ԯ݅
    σʔλαΠζ
    5#௒ γεςϜ͕ऩू͢Δϩά

14. 14.

    γεςϜΞʔΩςΫνϟ "VUP4DBMJOH
    &$ ,JOFTJT "VUP4DBMJOH
    &$ 4 4 3FETIJGU "UIFOB

    1SFTUP
    PO
    &.3 4 4QBSL
    PO
    &.3 (MVF %FWFMPQFST
    %BUB
    "OBMZTUT σʔλϑϩʔ ࢀরͳͲ 3%4
    )JWF
    .FUBTUPSF
15. 15.

    γεςϜΞʔΩςΫνϟ "VUP4DBMJOH
    &$ ,JOFTJT "VUP4DBMJOH
    &$ 4 4 3FETIJGU "UIFOB

    1SFTUP
    PO
    &.3 4 4QBSL
    PO
    &.3 (MVF +FOLJOT
    PO
    &$ -BNCEB %FWFMPQFST
    %BUB
    "OBMZTUT σʔλϑϩʔ ࢀরͳͲ 3%4
    )JWF
    .FUBTUPSF ऩूɾ஝ੵ
16. 16.

    ऩूɾ஝ੵ +BWB൛ͷ,JOFTJT
    1SPEVDFS
    -JCSBSZͱ,JOFTJT
    $POTVNFS
    -JCSBSZΛར༻
    ,JOFTJTϓϩσϡʔαʔ͸ड৴ͨ͠+40/Λ΄΅ͦͷ··,JOFTJTʹ౤ೖ
    ,JOFTJTʹೖΕ͓͚ͯ͹࣌ؒ
    ࠷௕೔ؒ
    σʔλϩετͷ৺഑͕ແ͍ͨΊ
    ,JOFTJTίϯγϡʔϚʔ͸ҎԼͷॲཧΛ࣮ࢪ
    +40/Λύʔεͯ͠ޙଓͷ෼ੳͰ࢖༻͢Δ৘ใΛ෇Ճ
    &$಺ͰՔಇ͢ΔqVFOUEʹόοϑΝͯ͠ҰఆִؒͰ4ʹ([JQܗࣜͰग़ྗ
    <1PJOU>
    ,JOFTJTಋೖͰ҆ఆੑ͕͔ͳΓ޲্ˠσʔλऩूܥʹ͸͔ͳΓΦεεϝ

    "VUP4DBMF
    &$ ,JOFTJT "VUP4DBMF
    &$ 4
17. 17.

    γεςϜΞʔΩςΫνϟ "VUP4DBMJOH
    &$ ,JOFTJT "VUP4DBMJOH
    &$ 4 4 3FETIJGU "UIFOB

    1SFTUP
    PO
    &.3 4 4QBSL
    PO
    &.3 (MVF %FWFMPQFST
    %BUB
    "OBMZTUT σʔλϑϩʔ ࢀরͳͲ 3%4
    )JWF
    .FUBTUPSF &5-
18. 18.

    &5-
    3FETIJGUɾ)JWFϩʔυ <1PJOU>
    )JWFϝλετΞ͸3%4ΑΓ΋(MVF
    %BUB
    $BUBMPH͕Φεεϝ
    3%4͕410'
    ୯Ұো֐఺
    ʹͳΒͳ͍Α͏ʹؾΛݣ͏ඞཁ͕͋Δ
    3%4͸ಉ࣌઀ଓ਺ͷ੍ݶ͕͋ΔͨΊ&.3Ϋϥελʔͷಉ࣌ىಈ਺ʹ੍ݶ͕ੜ͡Δ
    ಉ࣌઀ଓ਺͸ϝϞϦґଘͷύϥϝʔλʔ
    ಉ࣌઀ଓ਺ͷ্ݶΛ্͛ΔͨΊʹ͸ΠϯελϯελΠϓΛ্͛Δඞཁ͕͋Δ
    ϝλσʔλ͔͠؅ཧ͠ͳ͍ͷʹϦονͳεϖοΫ͸໪ମແ͍
    

    (MVF
    %BUB
    $BUBMPH͸ϑϧϚωʔδυ͔ͭಉ࣌઀ଓ਺ͷ੍ݶͳ͠ 4QBSL
    PO
    &.3 4
    ([JQ 4
    1BSRVFU 3FETIJGU 3%4
    )JWF
    .FUBTUPSF ऩूɾ஝ੵ 4
19. 19.

    &5-
    3FETIJGUɾ)JWFϩʔυ 4QBSL
    PO
    &.3Ͱ͸࣌ؒ෼ͷϩάΛೖྗͱͯ͠ҎԼεςοϓΛ࣮ߦ
    4%JTU$QͰ໿ ݸ
    ,#ڃ
    ͷϑΝΠϧΛ໿ݸ
    .#d(#ڃ
    ͷϑΝΠϧʹू໿
    <1PJOU>
    4QBSL
    

    )BEPPQ
    ͸େྔͷখ͞ͳϑΝΠϧΛѻ͏ͷ͸ෆಘҙ
    4QBSLͰ([JQQFE
    +40/Λग़ྗ
    
    )JWFςʔϒϧʹ*/4&35
    07&383*5&
    εςοϓऴྃޙɺ&.3Ϋϥελʔ͸ࣗಈ࡟আ
    ࣌ؒ෼ͷॲཧΛಠཱͨ͠ΫϥελʔͰ࣮ߦ͢ΔͷͰϦΧόϦʔͳͲ͕΍Γ΍͍͢
    4্ͷ([JQϑΝΠϧΛ3FETIJGUͷ$01:ίϚϯυͰϩʔυ 4QBSL
    PO
    &.3 4
    ([JQ 4
    1BSRVFU 3FETIJGU 3%4
    )JWF
    .FUBTUPSF ऩूɾ஝ੵ 4
20. 20.

    γεςϜΞʔΩςΫνϟ "VUP4DBMJOH
    &$ ,JOFTJT "VUP4DBMJOH
    &$ 4 4 3FETIJGU "UIFOB

    1SFTUP
    PO
    &.3 4 4QBSL
    PO
    &.3 (MVF +FOLJOT
    PO
    &$ -BNCEB %FWFMPQFST
    %BUB
    "OBMZTUT σʔλϑϩʔ ࢀরͳͲ 3%4
    )JWF
    .FUBTUPSF ෼ੳ
21. 21.

    ෼ੳπʔϧ "UIFOB 1SFTUP
    PO
    &.3 (MVF %FWFMPQFST
    %BUB
    "OBMZTUT 3FETIJGU "UIFOB
    1SFTUP
    PO
    &.3
    

    ೥ࠒΑΓར༻։࢝
    ɹ
    3FETIJGU
    ϩάऩू։࢝౰ॳΑΓར༻
    ࠓޙͷϩά૿ՃΛݟࠐΈɺ
    &.3
    "UIFOB΁Ҡߦத
    4 "UIFOBͷར༻ྉۚߴಅ
    
22. 22.

    "UIFOBͷར༻ྉۚߴಅͷݪҼ 3FEBTIͷΫΤϦεέδϡʔϧػೳ
    3FEBTIʹอଘͨ͠ΫΤϦΛࢦఆͨ͠ස౓Ͱ࣮ߦͰ͖Δ "UIFOBͷ࢖͍উखͷྑ͞
    ͋Δఔ౓ɺແ஡ͳΫΤϦ΋ݱ࣮తͳ࣌ؒͰॲཧͰ͖Δ ਺5#ΛεΩϟϯ͢ΔΫΤϦ܈͕සൟʹ࣮ߦ
    τʔλϧεΩϟϯαΠζ͕1#ΛΦʔόʔ

23. 23.

    "UIFOBར༻ͷڭ܇ ίετҙࣝ
    ͜ͷΫΤϦΛ࣮ߦ͢Δ͜ͱͰԿ(#εΩϟϯ͢Δͷ͔ɺͦͷ݁Ռྉۚ͸͍͘Βֻ ͔Δ͔Λҙࣝ͢Δ͜ͱ͸ඇৗʹॏཁ
    "84؅ཧऀ޲͚
    σʔλΞφϦετʹྉۚΛೝࣝͤ͞Δ࢓૊Έ͕ඞཁ
    %3:ݪଇ
    %PO`U
    3FQFBU
    :PVSTFMG
    ϔϏʔͳΫΤϦΛԿճ΋࣮ߦ͠ͳ͍ɾͤ͞ͳ͍
    

    Α͋͘ΔΫΤϦ͸ճ͚࣮ͩߦͯ݁͠ՌΛอଘ͢Δ
    ͓࡞๏ͷపఈ
    εΩϟϯྔ௿ݮͷͨΊύʔςΟγϣϯΛඞͣࢦఆ͢Δ
    ඞཁ࠷௿ݶͷΧϥϜΛ4&-&$5͢Δ
    4&-&$5
    
    ͸μϝ
    FUD "UIFOBഁ࢈͠ͳ͍ͨΊʹϢʔβʔ͕࣋ͭ΂͖ϚΠϯυηοτ
24. 24.

    ࠓޙͷల๬ SFJOWFOUͰൃද͞Εͨ৽αʔϏεΛੵۃతʹಋೖ༧ఆ 5JNFTUSFBN
    $POUPM
    5PXFS
    -BLF
    'PSNBUJPO
    5SBOTJU
    (BUFXBZ

25. 25.

    &0'