Why Cookpad Is Choosing Redshift as Our Data Platform

Data Engineering Study #7「Redshift最新アップデートと活用事例」での発表資料です。
https://forkwell.connpass.com/event/203403/

Transcript

1. .JOFSP"PLJ
   ͳͥΫοΫύου͸
   3FETIJGUΛσʔλج൫ʹ
   બͼଓ͚Δͷ͔
   ʙσʔλج൫͕Ռͨ͢΂͖໾ׂʙ
   

   View Slide

2. ࣗݾ঺հ
   ‣ ੨໦ๆ࿠ʢ͓͖͋ΈͶΖ͏ʣ
   ‣ ݩ3VCZίϛολʔ
   ‣ લ৬ฒྻ3%#.4ϕϯμʔ
   ‣ ฒྻσʔλϕʔεͱग़ձ͏
   ‣ ݱ৬ΫοΫύου
   ‣ ෼ੳαʔϏεʮͨ΂ΈΔʯͷ։ൃ
   ‣ σʔλ׆༻ج൫ͷߏங
   ‣ ϨγϐαʔϏεͷϦΞʔΩςΫνϟ
   

   View Slide

3. ΫοΫύουͷ
   σʔλج൫ͷ֓ཁ
   

   View Slide

4. ݕࡧαʔϏεʮͨ΂ΈΔʯʹ3FETIJGUಋೖ
   શࣾσʔλج൫Λ3FETIJGU΁Ҡߦ
   4QFDUSVNಋೖ
   $PODVSSFODZ4DBMJOHɺ6TBHF-JNJUಋೖ
   'FEFSBUFE2VFSZɺ"VUP8-.ಋೖ
   3"ಋೖ
   3FETIJGUར༻ͷྺ࢙
   

   View Slide

5. ΫοΫύουͷσʔλج൫ͷ֓ཁ
   3FETIJGUSBYMBSHFº
   5# OPEFTMJDF
   
   όϧΫΤΫεϙʔτ
   12 +%#$
   .Z42-
   1PTUHSF42-
   %ZOBNP%#
   qVFOUE
   Ϛελʔ
   ϩά
   +40/
   
   4
   ଞγεςϜ
   42-DMJFOU
   CEBTI
   5BCMFBV
   ؅ཧը໘
   8PSLEBZ
   4BMFTGPSDF
   ;FOEFTL
   τϥϯβ
   Ϋγϣϯ
   ςʔϒϧఔ౓
   ຊऑ
   ຊऑ
   ϩά
   αϚϦʔ
   αϚϦʔ
   42-όον໿ຊ
   12 +%#$
   ఆܕΫΤϦʔ
   2VFVFSZ
   ඇఆܕΫΤϦʔ
   

   View Slide

6. 3FETIJGU
   4
   w 1SJTNʢ಺੡ʣʹΑΔϩάͷετϦʔϛϯάϩʔυ
   w 1BSRVFUͷϚʔδͱɺ೚ҙλΠϜκʔϯͰͷύʔςΟγϣχϯά͕Մೳ
   w 1SJTNৄࡉIUUQTUFDIMJGFDPPLQBEDPNFOUSZ
   1SJTNʹΑΔϩάϩʔυ
   ֎෦
   ςʔϒϧ
   +40/
   +40/
   +40/
   1SJTN4USFBN
   1BSRVFU
   1BSRVFU
   MJWFPCKFDUT
   1BSRVFU
   1SJTN
   .FSHF
   1BSRVFU
   NFSHFEPCKFDUT
   ౸ண͍ͩ͠ॲཧ
   ໿࣌ؒ
   ͝ͱ
   (MVFΧλϩά
   Ͱ੾Γସ͑
   

   View Slide

7. ϚελʔςʔϒϧͷऔΓࠐΈ
   %#λΠϓ ςʔϒϧ਺ γεςϜ ִؒ ํࣜ
   .Z42- 1JQFMJOFE.JHSBUPSʢ಺੡ʣ ೔࣍ શߦચ͍ସ͑
   1PTUHSF42-
   3FETIJGU'FEFSBUFE2VFSZ
   "84%BUBCBTF.JHSBUJPO4FSWJDF
   ೔࣍ શߦચ͍ସ͑
   %ZOBNP%# %ZOBNP%#4USFBNTˠ1SJTN ৗ࣌ ࠩ෼ߋ৽
   

   View Slide

8. σʔλج൫͸ԿΛ͢΂͖͔
   

   View Slide

9. σʔλج൫ͭͷ࢖͍ಓ
   ఆܕΫΤϦʔ
   ,1*௥੻΍ޮՌଌఆͷμογϡϘʔυ
   ඇఆܕʢΞυϗοΫʣΫΤϦʔ
   ঢ়گʹԠͯ͡ਓ͕ؒ౎౓ߟ࣮͑ͯߦ͢ΔΫΤϦʔ
   όϧΫΤΫεϙʔτ
   ଞγεςϜ΁ͷσʔλసૹ
   ͜Ε͕Ұ൪Ձ஋͕ߴ͍ʂ
   ʜʜ͠ɺ੍໿΋େ͖͍
   

   View Slide

10. σʔλج൫͸ΞϓϦέʔγϣϯͷͨΊʹ
    αʔϏε
    σʔλج൫
    αʔϏε αʔϏε
    Ϛελʔɺ
    ϩά
    վળ
    ϑΟʔυ
    όοΫ
    ,1*μογϡϘʔυ΋ΞυϗοΫΫΤϦʔ΋σʔλΤΫεϙʔτ΋
    ݁ہͷͱ͜ΖαʔϏεͷύϑΥʔϚϯεΛվળ͢ΔͨΊʹ͋Δɻ
    લऀͭ͸ਓؒܦ༝ͰϑΟʔυόοΫ͞Εɺޙऀ͸ϓϩάϥϜ͔Β௚઀ϑΟʔυόοΫ͞ΕΔɻ
    

    View Slide

11. σʔλ׆༻ࣄྫ
    

    View Slide

12. ࣄྫͨ΂ΈΔʢݕࡧස౓෼ੳʣ
    ‣ ಛఆͷޠ͕ݕࡧ͞Εͨස౓ͷมԽΛάϥϑͰݟΔ͜ͱ͕Ͱ͖Δ
    ‣ ͍·੝Γ্͕͍ͬͯΔݕࡧ΍ɺ૊Έ߹ΘͤΒΕ΍͍͢ޠ͕ݟΒΕΔ
    ৯΂͍ͨ
    ࡞Γ͍ͨ
    ʢݕࡧʣ
    

    View Slide

13. ͨ΂ΈΔͷΞʔΩςΫνϟ
    ‣ ݕࡧϩάͱϢʔβʔϚελʔΛδϣΠϯͯ͠೔෇ͱޠͰूܭ͢Δ
    ‣ ࡞੒ͨ͠αϚϦʔ͸ΞϓϦ༻ͷ3FETIJGUΫϥελʔ΁సૹͯ͠ར༻
    3FETIJGUʢ%8)Ϋϥελʔʣ 3FETIJGUʢͨ΂ΈΔΫϥελʔʣ
    ݕࡧϩά
    Ϣʔβʔ
    Ϛελʔ
    αϚϦʔ
    ೔෇ɺ୯ޠ
    *%ɺ౷ܭ஋
    αϚϦʔ
    DPPLQBE
    42-
    όον
    

    View Slide

14. ࣄྫਓؾॱϨγϐݕࡧ
    ‣ ݕࡧ݁ՌΛʮਓؾʯͷ͋ΔॱʹݟΒΕΔ
    ‣ ༗ྉձһϝχϡʔͰ࠷΋࢖ΘΕ͍ͯΔػೳ
    

    View Slide

15. ਓؾॱݕࡧͷΞʔΩςΫνϟ
    ‣ γϯϓϧͳ42-όονͰਓؾॱϥϯΫΛܭࢉ͠ɺ"UIFOBͰ࠷ऴυ
    ΩϡϝϯτΛੜ੒ͯ͠4PMSʹಥͬࠐΉ
    ‣ ͢΂ͯ3VCZͰॻ͔Ε͍ͯͨόονΛҠ২ͨͨ͠Ί·ͩແବ͕ଟ͍
    3FETIJGU 4
    17ϩά
    Ϛελʔ
    ਓؾॱ
    ϥϯΫ
    DPPLQBE
    ͦͷଞͷ
    Πϯσο
    Ϋε
    ਓؾॱ
    ϥϯΫ
    ͦͷଞͷ
    Πϯσο
    Ϋε
    4PMS
    υΩϡ
    ϝϯτ
    "UIFOBͰ݁߹
    6/-0"%
    42-
    όον
    4PMS΁
    

    View Slide

16. ࣄྫλʔήςΟϯά޿ࠂ
    ‣ ಉ͡ଐੑΛ࣋ͭϢʔβʔͷηάϝϯτΛ࡞੒͠ɺΑΓޮՌ͕͋Δͱ
    ࢥΘΕΔ޿ࠂΛબ୒తʹ഑৴͢Δ
    ‣ ֦ு৘ใ͸42-ͱ1ZUIPOͰੜ੒͍ͯ͠Δ
    3FETIJGU %ZOBNP%#
    17ϩά
    Ϛελʔ
    Ϣʔβʔ
    ηάϝϯτ
    DPPLQBE
    ֦ு
    Ϣʔβʔ
    ηάϝϯτ
    ηάϝϯτ
    42-
    όον
    ΦʔσΟΤϯε
    ֦ு
    ֦ு৘ใ
    

    View Slide

17. ͳͥ3FETIJGUͳͷ͔ʁ
    

    View Slide

18. 3FETIJGUΛબͿཧ༝
    ΞϓϦέʔγϣϯ͕"84ʹ͋Δ͔Β
    σʔλج൫͸੒௕͢Δ΄ͲΞϓϦέʔγϣϯͱͷσʔλͷ΍ΓͱΓ͕૿͍͑ͯ͘ɻ͕ͨͬͯ͠Ξ
    ϓϦέʔγϣϯͱ࿈ܞ͠΍͍͢ɺ͍ۙ৔ॴʹ͋Δ΂͖ɻ
    ଞγεςϜͱͷ࿈ܞ͕༏Ε͍ͯΔ͔Β
    'FEFSBUFE2VFSZ΍4QFDUSVNɺ%BUB"1*ͷΑ͏ͳɺଞγεςϜͱͷ࿈ܞͷͨΊͷػೳ͕༏
    लɻ·ͨ*".Ͱ͢΂ͯͷ࿈ܞΛ౷Ұతʹ؅ཧͰ͖Δ఺͸஍ຯ͕ͩඇৗʹॏཁɻ
    ඞཁͳػೳͳΒ௥Ճ͞ΕΔͱ͍͏৴པײ͕͋Δ
    4QFDUSVN΍3"ɺ1BSUJ2-ͷΑ͏ͳ௒ڊେͳػೳͰ΋ɺඞཁʹͳͬͨ͜Ζʹண࣮ʹ௥Ճ͞Εͯ
    ͍͘ͷͰɺ͍·ػೳ͕ෆ଍͍ͯͯ͠΋͍ۙকདྷʹղܾ͢ΔͩΖ͏ͱָ؍Ͱ͖Δɻ
    

    View Slide

19. ·ͱΊ
    

    View Slide

20. ·ͱΊ
    σʔλج൫ʹ͸ओʹ௨Γͷ࢖͍ಓ͕͋Δ
    όϧΫΤΫεϙʔτʢ௚઀తͳσʔλ׆༻ʣ͕࠷
    ΋Ձ஋͕ߴ੍͘໿΋ڧ͍ͷͰɺΞʔΩςΫνϟ͸
    ͦΕʹ࠷దԽ͢΂͖
    ΫοΫύουͷγνϡΤʔγϣϯͰ͸3FETIJGU͕
    ্هͷ৚݅ʹরΒͯ͠࠷దͰ͋ͬͨ
    

    View Slide

21. ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
    

    View Slide