Upgrade to Pro — share decks privately, control downloads, hide ads and more …

リブセンスのデータ分析基盤の全貌

 リブセンスのデータ分析基盤の全貌

データ分析基盤Night #1
yusaku omasa

Livesense Inc.
PRO

February 28, 2017
Tweet

More Decks by Livesense Inc.

Other Decks in Technology

Transcript

  1. Ϧϒηϯεͷ
    σʔλ෼ੳج൫ͷશ๴
    σʔλ෼ੳج൫/JHIU

    View Slide

  2. ג ࣜ ձ ࣾ Ϧ ϒη ϯε
    w ೥ ݄ " O B M Z U J D T άϧ ʔϓ 

    σ ʔ λ Τ ϯ δχΞ 

    ػ ց ֶ श Τ ϯ δχΞ ݟ श ͍
    w ೥ ݄ σ ʔ λ Τ ϯ δχΞ Ϧ ϯ ά ν ʔϜ 

    σ ʔ λ Τ ϯ δχΞ 

    ( J U I V C U B J T F 5X J U U F S ! @ F V S L
    :VTBLV0NBTB
    %BUB&OHJOFFS

    View Slide

  3. View Slide

  4. w σʔλऩू
    w ϩάσʔλ
    w σʔλಉظ ࣄۀͷ8FCαʔϏε

    w ֎෦σʔλ ޿ࠂɾ4&0

    w σʔλ෼ੳɾ׆༻
    w ΞυϗοΫ෼ੳ
    w ,1*νΣοΫ
    w μογϡϘʔυ
    w σʔλɾιʔε
    w Ϩίϝϯυ
    w ޿ࠂΞτϦϏϡʔγϣϯ෼ੳ
    -JWFTFOTF"OBMZUJDT σʔλ෼ੳج൫

    View Slide

  5. -JWFTFOTF"OBMZUJDT σʔλ෼ੳج൫

    ς ʔϒϧ ਺
    Ϩί ʔ υ ਺ ԯ Ϩί ʔ υ
    ར ༻ ༰ ྔ 5 #
    ΞΫ ςΟϒϢʔ β ໊
    ΞυϗοΫɾΫΤϦ EBZ
    # * δ ϣϒ ɾ Ϋ Τ Ϧ E B Z
    Ϧ Ϧ ʔε ೥ ݄

    View Slide

  6. View Slide

  7. σʔλઃܭ

    View Slide

  8. σʔλઃܭ
    ΞΫηεϩά
    Πϕϯτϩά
    ΠϯϓϨογϣϯϩά
    ΠϯϓϨογϣϯϩά
    ϝʔϧ഑৴ϩά
    ϝʔϧ։෧ϩά
    8FCαʔϏεɾσʔλ
    ωΠςΟϒΞϓϦϩά
    ޿ࠂɾ4&0σʔλ

    View Slide

  9. σʔλઃܭ
    σʔλ໊ औಘ୯Ґ ߲໨ྫ ߲໨಺༰
    ΞΫηεϩά ϖʔδϏϡʔ
    [email protected]
    [email protected]
    [email protected]@JE
    [email protected]@JE
    ϒϥ΢β͝ͱͷ66*%
    ๚໰͝ͱͷ66*%
    αʔϏεͷσʔλʹώϞ෇͘*%

    Πϕϯτϩά ೚ҙ PO$MJDLͳͲ
    [email protected] ೚ҙͷऔಘ͍ͨ͠஋
    ωΠςΟϒΞϓϦϩά εΫϦʔϯϏϡʔ [email protected] ΞϓϦىಈ͝ͱͷ66*%
    ϝʔϧ։෧ϩά ϝʔϧ։෧ [email protected] ϝʔϧ͝ͱͷ66*%
    ΠϯϓϨογϣϯϩά දࣔ͞ΕͨΞΠςϜ [email protected] දࣔͨ͠ύʔπ໊
    ϝʔϧ഑৴ϩά ഑৴͞ΕͨΞΠςϜ [email protected] ϝʔϧ͝ͱͷ66*%
    ˞ΞΠςϜŋŋŋ঎඼΍ٻਓͳͲɺର৅αʔϏεͷத৺తͳ඼໨

    View Slide

  10. γεςϜઃܭ
    ϩάɾετϦʔϜॲཧฤ

    View Slide

  11. ϩάɾετϦʔϜॲཧ
    w ΞΫηεϩά
    w Πϕϯτϩά
    w ΠϯϓϨογϣϯϩά
    "1αʔό

    4JOBUSB
    3JO

    3FETIJGUEBUB*NQPSUFS
    3FETIJGU
    424
    4
    ϩάऩू+4
    ε τ Ϧ ʔϜ Ͱ
    σ ʔ λ Ճ ޻
    V T F S @ J E ΍ 

    ๚ ໰ ৘ ใ Λ อ ଘ

    View Slide

  12. w ϩάऩू+4 ϒϥ΢β

    w औಘσʔλ
    w ΞΫηεϩάɺΠϕϯτϩάͳͲ
    w [email protected]΍๚໰৘ใΛ$PPLJFʹอଘ
    w ϏʔίϯʹσʔλΛ٧Ίͯ"1αʔόʹඈ͹͢
    w "1αʔό
    w σʔλՃ޻ 63- 6TFSBHFOUͳͲ

    w qVFOUEͰόοϑΝϦϯάͯ͠ϩάΛ4ʹอଘ
    w 3JO 3FETIJGUEBUB*NQPSUFS

    w 4ͷ1VUΠϕϯτ͕424ʹϓογϡ͞ΕΔ
    w 3JO͕424ϝοηʔδΛऔΓग़͠3FETIJGUʹϩάΛอଘ
    γεςϜઃܭϩάɾετϦʔϜॲཧ

    View Slide

  13. ΞʔΩςΫνϟ࠶ߟ

    View Slide

  14. ΞʔΩςΫνϟ طଘ

    3JO 3FETIJGU
    424
    4
    %BUB4PVSDF
    "1αʔό
    ϩάऩू+4
    ๚ ໰ σ ʔ λ Λ 

    ม ߋ ͠ ͨ ͍
    %BUB4USFBN
    %BUB-BLF
    %BUB.BSU

    View Slide

  15. w ར఺
    w +4ΛಡΈࠐΉ͚ͩͰࣗಈͰϩάऩूͰ͖Δ
    w ετϨʔδίϯϐϡʔςΟϯάΛ

    ֤ΫϥΠΞϯτʹ෼ࢄͰ͖Δ

    w ՝୊
    w σʔλͷఆٛมߋ͕ࠔ೉
    w $PPLJFʹσʔλΛ΋ͬͯ͠·͍ͬͯΔ
    w ྫ͑͹ηογϣϯ൑ఆΛ్தͰม͍͑ͨ࣌ʹ

    σʔλͷҰ؏ੑ͕อূͰ͖ͳ͍
    ΞʔΩςΫνϟ࠶ߟ

    View Slide

  16. w σʔλͷఆٛΛมߋ༰қʹ͢Δ
    w ్தͰఆٛมߋ͢ΔͱσʔλͷҰ؏ੑ͕ࣦΘΕΔ

    աڈʹḪͬͯશͯमਖ਼͍ͨ͠

    w ఆٛมߋͷ͋Δσʔλͱແ͍σʔλ
    w ఆٛมߋͳ͠Ϛελʔσʔλ
    w 63-
    w ΞΫηε࣌ؒ
    w ఆٛมߋ͋Γ
    w ௚લ63-ɺΞΫηε͔࣌ؒΒ๚໰ ηογϣϯ
    ൑ఆ

    Ϛελʔσʔλ͔Β࠶ੜ੒͢Δ
    ΞʔΩςΫνϟ࠶ߟ

    View Slide

  17. ΞʔΩςΫνϟ ਐߦܗ

    %BUB4USFBN
    3FETIJGU
    4
    %BUB.BSU
    "1αʔό

    4JOBUSB
    ϩάऩू+4
    Ӭ ଓ Խ ͠ ͨ ͍ 

    σ ʔ λ ͷ Έ
    &5-
    4QBSLPO&.3
    ๚ ໰ σ ʔ λ ͷ 

    ࠶ ੜ ੒
    ఆ ٛ ม ߋ ͷ ͳ ͍ 

    Ϛ ε λ ʔ σ ʔ λ
    %BUB-BLF
    %BUB4PVSDF

    View Slide

  18. ΞʔΩςΫνϟ কདྷ

    3FETIJGU
    4
    "1αʔό
    ϩάऩू+4
    όον૚
    4QBSLPO&.3
    3JO
    424
    ଎౓૚
    ϥϜμɾΞʔΩςΫνϟԽ΋ݟ͍͑ͯΔʁ

    View Slide

  19. ͱ͜ΖͰɺ
    σʔλ෼ੳج൫

    ͪΌΜͱ࢖ΘΕͯ·͔͢ʁ

    View Slide

  20. Α͘ฉ͘࿩
    w αʔϏε։ൃͰ๩ͯ͘͠ɺ෼ੳج൫͸ϝϯς͞Εͳ͍
    w ࢖͓͏ͱͨ͠Βɺσʔλ͕Ԛͯ͘੔ཧ͔Β࢝·Δ
    w χʔζ͕ෆ໌ͳ··࡞ͬͨ
    w ϩά͸ͨ·͍ͬͯΔ͚ͲɺԿʹ࢖͑Δ͔Θ͔Βͳ͍
    w ཉ͍͠σʔλ͕෼ੳج൫ʹͳ͍
    w (6*͕͋Δ͚Ͳɺ஌Γ͍ͨσʔλʹ౸ୡͰ͖ͳ͍
    w σʔλͷఆ͕ٛෆ໌ͰɺԿ͕ਖ਼͍͠ͷ͔Θ͔Βͳ͍
    ࢖ΘΕͳ͍෼ੳج൫

    View Slide

  21. ͳͥ
    -JWFTFOTF"OBMZUJDT͸
    ࡞ΒΕ͔ͨ

    View Slide

  22. ϏδωεϞσϧʹ͓͚Δඞཁੑ
    w Ϧϒηϯε͸੒Ռใुܕࣄۀ
    w αΠτ಺ߦಈ ΞΫηεϩά
    ͱ

    $3. 8FCαʔϏεɾσʔλ
    ͷಥ͖߹Θ͕ͤඞਢ
    w ಛʹ޿ࠂ
    w ʮ͋Ε͹Կ͔Ͱ͖Δ͔΋ʯͰ͸ͳ͘ɺ

    ʮͳ͍ͱ·͍ͣʯͱ͍͏໌Β͔ͳχʔζ͕͋ͬͨ
    ͳͥ࡞ΒΕ͔ͨ

    View Slide

  23. ʮ࢖ΘΕΔʯ

    ෼ੳج൫ʹ͢ΔͨΊʹ

    View Slide

  24. w ૊৫Λ౿·͑ͨઃܭɾମ੍
    w اۀจԽΛཧղ͠ࢧ͑Δ
    w ػೳ௥Ճͷ༏ઌॱҐ
    ʮ࢖ΘΕΔʯ෼ੳج൫ʹ͢ΔͨΊʹ

    View Slide

  25. αʔϏεల։

    View Slide

  26. w ΤϯδχΞ͸αʔϏε։ൃͰ๩͍͠
    w σʔλ෼ੳج൫Λ։ൃ͢ΔΤϯδχΞΛઐ೚Ͱ͓͘
    w ϊ΢ϋ΢ͷ஝ੵɾઐ໳ੑ޲্

    w ෳ਺ͷαʔϏεΛఏڙ
    w ػೳͷڞ௨෦෼ͱΧελϚΠζͷόϥϯε
    w ؆୯ʹϩάऩू͕Ͱ͖Δઃܭʹ͢Δ

    w ΤϯδχΞ͸σʔλઃܭͷϊ΢ϋ΢͕ෆे෼
    w ΞφϦετʹσʔλઃܭϨϏϡʔΛͯ͠΋Β͏
    w ू·ͬͨσʔλ͕෼ੳͰ͖Δ΋ͷʹͳΔΑ͏ʹ
    ૊৫Λ౿·͑ͨઃܭɾମ੍

    View Slide

  27. w จԽ
    w σΟϨΫλʔ͔ΒӦۀ·Ͱ42-͕ॻ͚Δ

    ࢀߟӦۀ͞Μ·Ͱɺࣾһશһ͕42-Λ࢖͏ʮӽڥܕ૊৫ʯ͕Ͱ͖Δ·ͰͷͷϙΠϯτ

    w ࣗ༝ʹ෼ੳͰ͖Δঢ়ଶ͕๬·͍͠

    (6*͸੍໿Λ༩͑ΔͷͰ࡞Βͳ͍

    w ࢧ͑Δ
    w ٙ໰ΛղফͰ͖Δαϙʔτνϟοτ
    w σʔλͷఆٛ΍42-ͷॻ͖ํɺෆ۩߹ใࠂ·Ͱ
    w ΤϯδχΞ͕ډͳ͍νʔϜͷσʔλॲཧͷ͓ख఻͍
    w ʮσʔλɾΤϯδχΞϦϯάʯ͸૊৫ԣஅͰߦ͏
    จԽΛཧղ͠ࢧ͑Δ

    View Slide

  28. w ϩά͸աڈʹḪͬͯऩू͢Δ͜ͱ͸Ͱ͖ͳ͍
    w औಘ։࢝೔ΛͰ͖Δ͚ͩૣ͘͢Δ
    w ʮ΄͍͠ʯͱࢥͬͨͱ͖ʹ෼ੳͰ͖ΔΑ͏ʹ͢Δ

    w ϩά͚͕ͩ͋ͬͯ΋࢖ΘΕͳ͍
    w ࣄۀͷσʔλ͕Ұॹʹ͋ͬͯॳΊͯ࢖ΘΕΔ
    w ϩάͱࣄۀσʔλ͕ώϞ෇͘Α͏ʹͳ͔ͬͯΒɺ

    αʔϏε෼ੳ͕Ͱ͖ΔΑ͏ʹͳΔ
    ػೳ௥Ճͷ༏ઌ౓

    View Slide

  29. w σʔλઃܭ
    w ࣄۀͷσʔλͱώϞ෇͘ϩάऩू

    w γεςϜઃܭ
    w σʔλͷఆٛͷมߋ͸ආ͚ΒΕͳ͍
    w ͦΕʹ଱͑͏ΔΞʔΩςΫνϟΛ࡞Δ
    w ʮ࢖ΘΕΔʯ෼ੳج൫ʹ͢ΔͨΊʹ
    w χʔζ͔Β࢝ΊΔ
    w ૊৫Λ౿·͑ͨઃܭɾମ੍
    w จԽΛཧղ͠ࢧ͑Δ
    w ػೳ௥Ճͷ༏ઌॱҐ

    ·ͱΊ

    View Slide

  30. w σʔλΤϯδχΞ
    w ΞφϦετ
    w ΠϯϑϥΤϯδχΞ
    8F`SFIJSJOH

    View Slide