etl-in-the-media-company

 etl-in-the-media-company

ETLの実際
- 分析基盤刷新PoC
- ETL実例紹介
- ETLでぶつかった壁

53c93eb80b5958590b18205dc5fd3634?s=128

Masatoshi Shimada

September 19, 2017
Tweet

Transcript

  1. ಈըܥϝσΟΞاۀͰ ߦΘΕ͍ͯΔ&5-ͷ࣮ࡍ ౡాխ೥!TNENUT

  2. ࣗݾ঺հ w ౡాխ೥!TNENUT w ιϑτ΢ΣΞΤϯδχΞ w  4DBMBc1ZUIPOc+BWBc(PMBOHc%%%  w

    גࣜձࣾΤϒϦʔॴଐ w %&-*4),*5$)&/σʔλ෼ੳܥશൠ୲౰
  3. ΞϓϦ঺հ

  4. ໨࣍ w ෼ੳج൫࡮৽1P$ w &5-࣮ྫ঺հ w &5-ͰͿ͔ͭͬͨน

  5. ෼ੳج൫࡮৽1P$

  6. 4JOHMFTPVSDFPGUSVUI The state of your whole application is stored in

    an object tree within a single store. by redux. w ΞϓϦέʔγϣϯͷঢ়ଶ͸Ұݩ؅ཧ͞ΕΔ΂͖ Ͱ͋Δ w ϏοάσʔλͰ΋Ұͭͷ৔ॴʹू໿ͯ͠ɺ42- ͰݕࡧՄೳʹ͓ͯ͘͠΂͖Ͱ͋Δ w ݱ࣮తʹ͸اۀ಺ͷσʔλ͸༷ʑͳܗͰଘࡏ͠ ͓ͯΓ݁߹͢Δ͜ͱ͸೉͍͠
  7. γεςϜߏ੒ਤʢ౰ॳʣ "1*αʔό ޿ࠂޮՌଌఆ Πϕϯτܭଌ "UIFOB #JH2VFSZ ݁߹͕ग़དྷͳ͍ͨΊɺ 4405͕ຬͨͤͳ͍ ՄࢹԽ %8)

  8. ౰ॳͷ෼ੳج൫ʹ͓͚Δ՝୊ w εΩϟϯྔʢΫΤϦ՝ۚʣ͕೔ʑ૿େ͍ͯͨ͠ w ΫΤϦɺ5#ͷεΩϟϯ͕ଟ਺ w #2ͷσʔλߏ଄͕ෳࡶͰ42-͕೉ಡঢ়ଶ w Ϩίʔυܕͷ෼ղ͕ओͳݪҼ w

    "UIFOBͱ#JH2VFSZͷ+0*/͕ग़དྷͳ͍ w SFEBTIͳΒ͹݁߹Մೳ w ͔͠͠σʔλ࿈܎Λॏࢹͨͨ͠Ίߟྀ֎ͱͨ͠ w ޿ࠂޮՌଌఆͷϩά͕࿈ܞ͞Ε͍ͯͳ͍ "UIFOB #JH2VFSZ
  9. w 4405͕࣮ݱ͞Ε͍ͯΔ͜ͱ w εΩϟϯྔΛؾʹ͢Δ͜ͱແ͍෼ੳج൫Ͱ͋Δ͜ͱ w ෼ੳ༻42-͸ɺͳΔ΂͘؆қͳ42-ͱ͍ͨ͠ w ಛघͳ8JOEPXؔ਺΍+0*/ͷআڈΛ૝ఆ w ඇΤϯδχΞ޲͚ʹ෼ੳͷෑډΛԼ͍͛ͨ

    w Ϩίʔυͷߏ଄͸ՄมͰεΩʔϚϨεʹ͍ͨ͠ w શͯϑϧϚωʔδυͰ࣮ݱ͞ΕΔ͜ͱ ৽෼ੳج൫ʹ޲͚ͨ1P$
  10. ৽෼ੳج൫ʹ޲͚ͨ1P$ w Ϧιʔε՝ۚͰ͋Δ͜ͱ w ϑϧϚωʔδυɾδϣϒεέδϡʔϥ͕ଘࡏ͢Δ͜ͱ w ϑϧϚωʔδυɾόϧΫΠϯϙʔτ͕ଘࡏ͢Δ͜ͱ w εΩʔϚϨε%#ͰεΩʔϚࣗಈߋ৽͕ଘࡏ͢Δ͜ͱ w

    3&45Ͱσʔλ౤ೖՄೳͳίʔϧόοΫ༻ͷΤϯυϙ Πϯτ͕ଘࡏ͢Δ͜ͱ w σʔλ࿈ܞʢ*NQPSU&YQPSU ͕༰қͰ͋Δ͜ͱ
  11. "1*αʔό ޿ࠂޮՌଌఆ Πϕϯτܭଌ %BUB-BLF %8) 4405 1P$ ୡ੒ σʔλ͕Ұݩ؅ཧ͞ΕΔͨΊ ͲͷΑ͏ͳσʔλͰ΋݁߹Մೳ

    #JH2VFSZ ՄࢹԽ 3&45"1* $PODMVTJPO
  12. &5-࣮ྫ঺հ

  13. &5-࣮ྫ঺հ "1*αʔό ޿ࠂޮՌଌఆ Πϕϯτܭଌ %BUB-BLF %8) #JH2VFSZ ՄࢹԽ 3&45"1* ͜ͷ෦෼ͷ࿩

  14. 'JSFCBTF"OBMZUJDT w #BB4πʔϧ w ओʹ"#ςετɺϓογϡ௨஌ɺΠϕϯτϩ άܭଌͰར༻͍ͯ͠Δ w #JH2VFSZʹΠϕϯτܭଌϩά͕࿈ܞՄೳ w #JH2VFSZͰੜϩάΛ෼ੳʹճ͍͕ͯͨ͠ɺɹ

    εΩϟϯྔ૿େ໰୊ͱ42-೉ಡԽ໰୊͕͋ͬͨ
  15. 'JSFCBTF 4%, #JH2VFSZ 5SFBTVSF %BUB ($4 4 &.3 5SFBTVSF8PSLqPX 'JSFCBTF

    "OBMZUJDT &5-"SDIJUFDUVSF .3FDPSE%BZ
  16. 'JSFCBTF 4%, &5-"SDIJUFDUVSF 'JSFCBTF "OBMZUJDT #JH2VFSZ 5SFBTVSF %BUB ($4 4

    &.3 ϑϧϚωʔδυ .3FDPSE%BZ 5SFBTVSF8PSLqPX
  17. 5SFBTVSF8PSLqPX w %JHEBHͷϑϧϚʔωδυδϣϒεέδϡʔϥ w 3VCZ1ZUIPO4IFMM͕࢖͑ͳ͍ w Πϯϑϥૢ࡞͸"84-BNCEBΛར༻͢Δ w "84΍($1͸$-*Λ-BNCEBͰىಈ͢Δ w

    &.3ΦϖϨʔλΑΓ΋ॊೈʹ੍ޚ͔ͨͬͨ͠ w ॲཧܥ͸ϑϧϚωʔδυͳαʔόϨεߏ੒
  18. %JHEBH8PSLqPX CR@FYUSBDU #JH2VFSZ ($4 TUFQ IUUQ "1*(BUFXBZ -BNCEB &.3 ($4

    4 5% TUFQ 45&1 #JH2VFSZ͔Β($4ʹμϯϓ 45&1 ($4͔Β&.3Ͱ5%ʹ࿈ܞ
  19. &NCVMLPO&.3 &.3 w .BQ3FEVDF&YFDVUPSΛར༻͢Ε͹ɺɹ )BEPPQΫϥελʔ্Ͱ෼ࢄ࣮ߦͰ͖Δ IUUQTHJTUHJUIVCDPNTNENUTDGEFCFCEEDBCCCE IUUQRJJUBDPNKPLFSJUFNTECBFCGBBF w Լهͷ̎εςοϓͰར༻ՄೳʢUIBOLT!KPLFS 

    w MPH༻ͷ+"3Λ&.3֤ϊʔυʹࣄલʹϩʔυ w ϚελʔϊʔυͰ.BQ3FEVDF&YFDVUPSΛ ར༻ͯ͠&NCVMLΛىಈ͢Δ
  20. ($4 4 4 5% 4UFQ"730ʢ#JH2VFSZʣ+40/ʢ3FDPSE&WFOUʣ 4UFQ+40/ʢ3FDPSE&WFOUʣ1MB[NB%# 5% "730ʢ3FDPSE.BOZ&WFOUʣΛɺ +40/ʢ3FDPSE&WFOUʣʹม׵͍ͯ͠Δ %BUB$POOFDUPSͰ+40/಺ͷ+40/ܕΛ෼ղ͢ΔͨΊɺ

    5%ΦϓγϣϯͷFYQMPEF@KTPO@DPMVNOTΛར༻͍ͯ͠Δ FYQMPEF@KTPO@DPMVNOT͸%BUB$POOFDUPSઐ༻Ͱ&.3͔Βར༻Ͱ͖ͳ͍ &.3Ͱ࣮ߦͯ͠΋ͬͱߴ଎Խ͍ͨ͠ͷͰ044൛Ͱ͜ͷΦϓγϣϯΛެ։ͯ͠ཉ͍͠ʢཁ๬ʣ &NCVML࣮ߦܭը
  21. Ϩίʔυͷঢ়ଶભҠ $ % " # ' $ % " #

    & ' Ћ Ћ Ћ Ћ Ћ Ћ Ћ Ϩίʔυ ஋ ܕ" ຌྫ ෼ׂ & " # $ % & ' Ћ Ћ Ћ " # $ % & ' ൃࢄ ܕ# ܕ$ 4UFQ"730+40/ʢ3FDPSE&WFOUʣ 4UFQ+40/ʢ3FDPSE&WFOUʣ1MB[NB%# 5%
  22. $PODMVTJPO w .BQ3FEVDF&YFDVUPSΛར༻ͯ͠&.3Ͱ෼ࢄ࣮ ߦ͢Δ͜ͱͰେྔσʔλͷऔΓࠐΈΛՄೳͱͨ͠ w ϨίʔυΠϕϯτʹ෼ׂ͢Δ͜ͱͰ෼ੳ༻42- ͷ೉қ౓ΛԼ͛ΔࣄΛ࣮ݱͨ͠ w ྻࢦ޲%#Λੜ͔ͯ͠+40/Λൃࢄͤ͞Δ͜ͱͰɺε ΩʔϚ؅ཧίετͷ࡟ݮΛ࣮ݱͨ͠

    w ϑϧϚωʔδυͰ&5-Λ࣮ݱ͢Δ͜ͱͰ؅ཧίετ ࡟ݮΛ࣮ݱͨ͠
  23. &5-ͰͿ͔ͭͬͨน

  24. 'JSFCBTFऔΓࠐΈ༻ &NCVMLϓϥάΠϯ͕ͳ͍

  25. &NCVMLϓϥάΠϯ͕ͳ͍ 'JSFCBTFऔΓࠐΈ༻ ແ͔ͬͨΒ࡞Δ

  26. &NCVMLϓϥάΠϯ͕ͳ͍ 'JSFCBTFऔΓࠐΈ༻ ແ͔ͬͨΒ࡞Δ ͦͯ͠ɺ૊Έ߹ΘͤΔ

  27. ੒Ռ෺Ұཡ w FNCVMLpMUFSLFZ@JO@SFEJT w FNCVMLpMUFSKTPO@LFZ@KPJOFS w FNCVMLGPSNBUUFSGBTU@KTPOM w FNCVMLQBSTFSpSFCBTF@BWSP w

    FNCVMLEFDPEFSSFNPWF@OPOTUBOEBSE@VUG@CZUFT w FNCVMLPVUQVUqVFOUEʢFYQFSJNFOUBMBTXBJUJOHGPS+BWB  w FNCVMLPVUQVULFZ@UP@SFEJT ͱͷ֨ಆͷ຤ɺੜ੒͞ΕͨϓϥάΠϯ
  28. &NCVMLϓϥάΠϯ૊Έ߹Θͤ঺հ &NCVMLΛར༻ͨ͠ େྔϨίʔυࠩ෼࿈ܞͷ঺հ

  29. &NCVMLϓϥάΠϯ૊Έ߹Θͤ঺հ w ຊ೔Ϩίʔυ͸#2ͷຊ೔ςʔϒϧ JOUSBEBZ ʹ࿈ܞ͞ΕΔ w ఆࠁʢே࣌ࠒʣʹຊ೔ςʔϒϧ͔Βࡢ೔ςʔϒϧ͕࡞੒͞ Εͯຊ೔Ϩίʔυ͕ࡢ೔ςʔϒϧʹҠಈ͢Δ w ࠷େͰաڈ೔લ·ͰͷΠϕϯτϩά͕೔෇ςʔϒϧʹఆࠁ

    ʢே࣌ࠒʣͰ࿈ܞ͞ΕΔ w ࡢ೔͕࡞੒ɺ̎೔લɾ̏೔લ͕௥هͷ̏छྨ w ͲͷϨίʔυ͕௥ه͞Ε͔ͨ͸42-Ͱ͸෼͔Βͳ͍ #JH2VFSZ
  30. &NCVMLϓϥάΠϯ૊Έ߹Θͤ঺հ ຊ೔ ࡢ೔ ೔લ ೔લ աڈϩά ੵΈ্͛ ఆࠁʢே࣌ࠒͷڍಈʣ ࡞੒ ௥ه

    ௥ه JOUSBEBZ #JH2VFSZ
  31. &NCVMLϓϥάΠϯ૊Έ߹Θͤ঺հ Ұࡢ೔ ຊ೔࿈ܞ͍ͨ͠σʔλ ࡢ೔࿈ܞࡁΈ w ຊ೔࿈ܞ͍ͨ͠σʔλΛநग़͢Δʹ͸ɺશྔσʔλͱࡢ೔࿈ ܞࡁΈσʔλͱ%J⒎ΛऔΔඞཁ͕͋Δ ɹʢ3FETIJGUʢ42-ʣͰ΋࣮ݱՄೳ͕ͩ&NCVMLͰ࣮ݱ͔ͨͬͨ͠ʣ શྔ ͷϨίʔυʹ͍ͭͯ෼ղ͢ΔͱԼهͷΑ͏ʹͳΔ

    ʹ
  32. &NCVMLϓϥάΠϯ૊Έ߹Θͤ঺հ ࿈ܞࡁΈσʔλ 4UFQ FNCVMLPVUQVULFZ@UP@SFEJT 4UFQ શྔσʔλʢ.%ʣ — ʹ ຊ೔࿈ܞ෼Ϩίʔυ FNCVMLpMUFSLFZ@JO@SFEJT

    #JH2VFSZ 4UFQ FNCVMLQBSTFSpSFCBTF@BWSP ࿈ܞࡁΈσʔλʢ.%ʣ ࿈ܞࡁΈσʔλʢ.%ʣ શྔσʔλ
  33. &NCVMLϓϥάΠϯ૊Έ߹Θͤ঺հ w ૊Έ߹Θͤͯ࢖͏&NCVMLϓϥάΠϯ w େྔσʔλͷࠩ෼Λൈͨ͘Ίʹ͸ɺϩʔυࡁΈͷঢ়ଶ͕ඞཁͱ ͳΓ3FEJTͷ4FUΛར༻ͨ͠ w BLLBΛར༻্ͯ͠ྲྀ͔ΒྲྀΕΔϨίʔυ 1BHF ΛόοϑΝϦ

    ϯάͯ͠ɺҰఆྔஷ·͔ͬͯΒϑϥογϡ͢Δ͜ͱͰɺ3FEJT ΁ͷ໰͍߹Θͤޮ཰޲্Λૂͬͨ FNCVMLPVUQVULFZ@UP@SFEJT FNCVMLpMUFSLFZ@JO@SFEJT
  34. $PODMVTJPO w &NCVMLϓϥάΠϯΛ૊Έ߹Θ࣮ͤͯߦܭըΛߟྀ͢ Ε͹ɺଟछଟ༷ͳ&5-͕࣮ݱՄೳͱͳΔ w ྑ͍&NCVMLϓϥάΠϯ͕ແ͔ͬͨΒ13͢Δ͔ɹ ࡞ͬͯ044Ͱެ։͠Α͏ ࠔͬͨΒ5XJUUFSͰͭͿ΍͘ͱ৭ʑͱڭ͑ͯ͘ΕΔʢUIBOLT!IJSPZTBUP  w

    &NCVMLϓϥάΠϯ։ൃʹ4DBMB͸ڧྗͳϒΩʹͳΔ BLLBͱ͔DJSDFͱ͔DBUTͱ͔TDBMB[ͱ͔TIBQFMFTTͱ͔
  35. ձࣾ֓ཁ FOOD FAMILY BEAUTY NEWS ಈըΛ௨ͯ͡  ੈքΛ΋ͬͱָ͘͠
 ΋ͬͱॆ࣮ͨ͠ຖ೔ʹ

  36. https://corp.every.tv/recruits We are hiring! Chief Dog Officer EVE

  37. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠