Data Analysis Platform Development @M3, inc.

Transcript

1. Data Analysis Platform Development merpay × M3 ػցֶश NIGHT Hirohito Sasakawa data engineer @ M3, inc.
2. ࠓ೔࿩͢͜ͱ w .ͷσʔλ෼ੳج൫ʹ͍ͭͯ
   w σʔλ࿈ܞδϣϒͷ։ൃ
   w ·ͱΊ
3. M3ͷσʔλ෼ੳج൫ʹ͍ͭͯ
4. M3ͷσʔλ෼ੳج൫ͷݱঢ় 2VFSZ 2VFSZ 2VFSZ 1SPEVDUJPO … 3FQPSU … 4ZOD 4ZOD 2VFSZ #JH2VFSZ
5. M3ͷσʔλ෼ੳج൫ͷݱঢ় 2VFSZ 2VFSZ 2VFSZ 1SPEVDUJPO … 3FQPSU … 4ZOD 4ZOD 2VFSZ #JH2VFSZ ҆Ձ͔ͭߴ଎ͳ
   #JH2VFSZΛ
   ׆༻͍͖͍ͯͨ͠
6. σʔλ෼ੳج൫ͷ໰୊఺ Ͳ͜ʹɺͲΜͳ
   σʔλ͕͋Δͷ͔
   Θ͔Βͳ͍ #JH2VFSZʹ
   ࿈ܞ͞Ε͍ͯͳ͍
   σʔλ͕͋Δ  XJLJΛݟΔ
   ͳ͍৔߹΋
    ਓʹฉ͘
    ෼ੳʹ͔͔Δ࣌ؒͷ
 େ෦෼͸͜͜
   ݸਓͷײ૝  ࿈ܞδϣϒͷ࣮૷଴ͪ
    ͳΔ΂͘؆୯ʹ࣮૷Ͱ͖Δ
 ͱخ͍͠
7. σʔλ෼ੳج൫ͷ໰୊఺ Ͳ͜ʹɺͲΜͳ
   σʔλ͕͋Δͷ͔
   Θ͔Βͳ͍ #JH2VFSZʹ
   ࿈ܞ͞Ε͍ͯͳ͍
   σʔλ͕͋Δ  XJLJΛݟΔ
   ͳ͍৔߹΋
    ਓʹฉ͘
    ෼ੳʹ͔͔Δ࣌ؒͷ
 େ෦෼͸͜͜
   ݸਓͷײ૝  ࿈ܞδϣϒͷ࣮૷଴ͪ
    ͳΔ΂͘؆୯ʹ࣮૷Ͱ͖Δ
 ͱخ͍͠
8. ϝλσʔλͷݕࡧγεςϜ #JH2VFSZͷ
   wσʔληοτ໊ɺςʔϒϧ໊ɺΧϥϜ໊
   wσʔληοτɺςʔϒϧɺΧϥϜͷEFTDSJQUJPO
   ͰΩʔϫʔυݕࡧͰ͖ΔγεςϜΛߏங
9. ϝλσʔλͷݕࡧγεςϜ #JH2VFSZͷ
   wσʔληοτ໊ɺςʔϒϧ໊ɺΧϥϜ໊
   wσʔληοτɺςʔϒϧɺΧϥϜͷEFTDSJQUJPO
   ͰΩʔϫʔυݕࡧͰ͖ΔγεςϜΛߏங Ωʔϫʔυݕࡧ
10. ϝλσʔλͷݕࡧγεςϜ #JH2VFSZͷ
    wσʔληοτ໊ɺςʔϒϧ໊ɺΧϥϜ໊
    wσʔληοτɺςʔϒϧɺΧϥϜͷEFTDSJQUJPO
    ͰΩʔϫʔυݕࡧͰ͖ΔγεςϜΛߏங Ωʔϫʔυݕࡧ ࠷ۙӾཡͨ͠ਓ
    ࠷΋Α͘Ӿཡͨ͠ਓ
    Λදࣔ͠໰͍߹ΘͤͨΓͰ͖ΔΑ͏ʹ
11. σʔλ෼ੳج൫ͷ໰୊఺ Ͳ͜ʹɺͲΜͳ
    σʔλ͕͋Δͷ͔
    Θ͔Βͳ͍ #JH2VFSZʹ
    ࿈ܞ͞Ε͍ͯͳ͍
    σʔλ͕͋Δ  XJLJΛݟΔ
    ͳ͍৔߹΋
     ਓʹฉ͘
     ෼ੳʹ͔͔Δ࣌ؒͷ
 େ෦෼͸͜͜
    ݸਓͷײ૝  ࿈ܞδϣϒͷ࣮૷଴ͪ
     ͳΔ΂͘؆୯ʹ࣮૷Ͱ͖Δ
 ͱخ͍͠
12. σʔλ෼ੳج൫ͷ໰୊఺ Ͳ͜ʹɺͲΜͳ
    σʔλ͕͋Δͷ͔
    Θ͔Βͳ͍ #JH2VFSZʹ
    ࿈ܞ͞Ε͍ͯͳ͍
    σʔλ͕͋Δ  XJLJΛݟΔ
    ͳ͍৔߹΋
     ਓʹฉ͘
     ෼ੳʹ͔͔Δ࣌ؒͷ
 େ෦෼͸͜͜
    ݸਓͷײ૝  ࿈ܞδϣϒͷ࣮૷଴ͪ
     ͳΔ΂͘؆୯ʹ࣮૷Ͱ͖Δ
 ͱخ͍͠
13. σʔλ࿈ܞδϣϒͷ࣮૷
14. ڊਓͷݞʹ৐Δ If I have seen further it is by standing on the shoulders of giants. - Bernardus Carnotensis ԕ͘·Ͱݟ౉ͤΔͷ͸ɺͻͱ͑ʹڊਓͷݞʹ৐͍ͬͯΔ͔Β
    
    γϟϧτϧͷϕϧφʔϧ
15. ৽͍͠σʔλ࿈ܞδϣϒͷٕज़ελοΫ
16. ৽͍͠σʔλ࿈ܞδϣϒͷٕज़ελοΫ σʔλͷόϧΫϩʔυπʔϧ
    ༷ʑͳιʔε
    λʔήοτʹ
    ରԠ
    ϓϥάΠϯػߏ
     3%#.4
     #JH2VFSZ
     4
     $47
    +TPO
    pMF
    :BNMϕʔεͷ؆୯ͳهड़Ͱ
    ઃఆΛॻ͚Δ
     ςʔϒϧ౰ͨΓΧϥϜ਺
    
    ߦ
 ͘Β͍ͷهड़ྔ
     ΤϯδχΞͳΒ෼͋Ε͹
 ॻ͚ΔΑ͏ʹ
17. ৽͍͠σʔλ࿈ܞδϣϒͷٕज़ελοΫ
18. ৽͍͠σʔλ࿈ܞδϣϒͷٕज़ελοΫ ϫʔΫϑϩʔ؅ཧπʔϧ
    ॲཧͷ࿈࠯
    ϫʔΫϑϩʔ
    Λ
 ίʔυͰ؅ཧ͢Δπʔϧ
     ΠέͯΔ$SPO
     δϣϒͷґଘؔ܎Λ:BNM
    ϕʔε
    Ͱهड़
     ಠཱͳॲཧ͸ฒྻ࣮ߦ΋Մೳ
     ஗Ԇ؂ࢹػೳ
     ֤छ௨஌
    ਖ਼ৗɺҟৗऴྃ࣌
     ϦτϥΠػೳ
    ਓؒ͸ॲཧͷલޙͰႈ౳ʹͳΔ Α͏ʹॲཧΛॻ͚ͩ͘ͰΑ͍
     ॲཧಉ࢜ͷؔ܎͸%JHEBH͕؅ཧɺ
 ࣮ߦͯ͘͠ΕΔ
19. ϫʔΫϑϩʔઃܭͷϕετϓϥΫςΟε (WIP) w EJHEBHͷXPSLqPXઃఆϑΝΠϧ͸ɺͦͷϑΝΠϧͷ αϒσΟϨΫτϦͷϑΝΠϧ͔͠ॲཧର৅ʹͰ͖ͳ͍
20. ϫʔΫϑϩʔઃܭͷϕετϓϥΫςΟε (WIP) w EJHEBHͷXPSLqPXઃఆϑΝΠϧ͸ɺͦͷϑΝΠϧͷ αϒσΟϨΫτϦͷϑΝΠϧ͔͠ॲཧର৅ʹͰ͖ͳ͍ EC@XPSLqPXEJH͸ɺ
 EC@MPBEҎԼͷFNCVMLͷ
 ϩʔυઃఆϑΝΠϧ΍ɺ
    DPOGҎԼͷڞ௨ઃఆΛ
 ࢀরͰ͖ͳ͍
21. ϫʔΫϑϩʔઃܭͷϕετϓϥΫςΟε (WIP) w σΟϨΫτϦߏ੒ΛΉ΍Έʹਂͨ͘͘͠ͳ͍
    w ιϑτϦϯΫͰରԠ
22. ϫʔΫϑϩʔઃܭͰࠔ͍ͬͯΔ͜ͱ w FNCVMLͷQMVHJOΛCVOEMFͰ؅ཧ͢Δ৔߹ͷઃܭ
    w ݱঢ়͍͍ํ๏͕ݟ͔ͭΒͣɺXPSLqPXͷઌ಄Ͱ
    FNCVML
    CVOEMF
    JOTUBMM
    ͢ΔλεΫΛىಈ
    w ͍͍ํ๏͕͋Ε͹ڭ͍͑ͯͩ͘͞
23. ·ͱΊ
24. ·ͱΊ w .ͷσʔλ෼ੳج൫ʹ͍ͭͯ
    w #JH2VFSZͷར༻Λਪਐ͍͖͍ͯͨ͠
    w ෼ੳج൫ͷ՝୊
    w Ͳ͜ʹɺͳΜͷσʔλ͕͋Δʁ
    w #JH2VFSZʹ͞Ε͍ͯͳ͍σʔλ͕͋Δ
    w σʔλ࿈ܞδϣϒͷ࣮૷
    w σʔλ࿈ܞδϣϒΛ؆୯ʹॻ͚ΔEJHEBH
    FNCVML͸ศར
    w ਓؒ͸ຊདྷ͢Δ΂͖࢓ࣄʹूத