Why Cookpad Is Choosing Redshift as Our Data Platform

Data Engineering Study #7「Redshift最新アップデートと活用事例」での発表資料です。
https://forkwell.connpass.com/event/203403/

Transcript

1. .JOFSP
   "PLJ
    ͳͥΫοΫύου͸ 3FETIJGUΛσʔλج൫ʹ
   બͼଓ͚Δͷ͔ ʙσʔλج൫͕Ռͨ͢΂͖໾ׂʙ
2. ࣗݾ঺հ ‣ ੨໦ๆ࿠ʢ͓͖͋
   ΈͶΖ͏ʣ
   ‣ ݩ3VCZίϛολʔ
   ‣ લ৬
   ฒྻ3%#.4ϕϯμʔ
   ‣ ฒྻσʔλϕʔεͱग़ձ͏
   ‣ ݱ৬
   ΫοΫύου
   ‣ ෼ੳαʔϏεʮͨ΂ΈΔʯͷ։ൃ
   ‣ σʔλ׆༻ج൫ͷߏங
   ‣ ϨγϐαʔϏεͷϦΞʔΩςΫνϟ
3. ΫοΫύουͷ
   σʔλج൫ͷ֓ཁ
4. 
   ݕࡧαʔϏεʮͨ΂ΈΔʯʹ3FETIJGUಋೖ
   
   શࣾσʔλج൫Λ3FETIJGU΁Ҡߦ
   
   4QFDUSVNಋೖ
   
   $PODVSSFODZ
   4DBMJOHɺ6TBHF
   -JNJUಋೖ
   
   'FEFSBUFE
   2VFSZɺ"VUP8-.ಋೖ
   
   3"ಋೖ 3FETIJGUར༻ͷྺ࢙
5. ΫοΫύουͷσʔλج൫ͷ֓ཁ 3FETIJGU
   SBYMBSHF
   º
   
   5#
   
   OPEF
   TMJDF όϧΫΤΫεϙʔτ 12
   +%#$ .Z42-
   1PTUHSF42-
   %ZOBNP%# qVFOUE Ϛελʔ ϩά
   +40/ 4 ଞγεςϜ 42-
   DMJFOU
   CEBTI 5BCMFBV
   ؅ཧը໘ 8PSLEBZ
   4BMFTGPSDF
   ;FOEFTL τϥϯβ
   Ϋγϣϯ ςʔϒϧఔ౓ ຊऑ ຊऑ ϩά αϚϦʔ αϚϦʔ 42-όον
   ໿ຊ 12
   +%#$ ఆܕΫΤϦʔ 2VFVFSZ ඇఆܕΫΤϦʔ
6. 3FETIJGU 4 w 1SJTNʢ಺੡ʣʹΑΔϩάͷετϦʔϛϯάϩʔυ
   w 1BSRVFUͷϚʔδͱɺ೚ҙλΠϜκʔϯͰͷύʔςΟγϣχϯά͕Մೳ
   w 1SJTNৄࡉ
   IUUQTUFDIMJGFDPPLQBEDPNFOUSZ 1SJTNʹΑΔϩάϩʔυ ֎෦
   ςʔϒϧ +40/ +40/ +40/ 1SJTN
   4USFBN 1BSRVFU 1BSRVFU MJWF
   PCKFDUT 1BSRVFU 1SJTN
   .FSHF 1BSRVFU NFSHFE
   PCKFDUT ౸ண͍ͩ͠ॲཧ ໿࣌ؒ
   ͝ͱ (MVFΧλϩά
   Ͱ੾Γସ͑
7. ϚελʔςʔϒϧͷऔΓࠐΈ %#λΠϓ ςʔϒϧ਺ γεςϜ ִؒ ํࣜ .Z42-  1JQFMJOFE
   .JHSBUPSʢ಺੡ʣ ೔࣍ શߦચ͍ସ͑ 1PTUHSF42-  3FETIJGU
   'FEFSBUFE
   2VFSZ
   "84
   %BUBCBTF
   .JHSBUJPO
   4FSWJDF ೔࣍ શߦચ͍ସ͑ %ZOBNP%#  %ZOBNP%#
   4USFBNT
   ˠ
   1SJTN ৗ࣌ ࠩ෼ߋ৽
8. σʔλج൫͸ԿΛ͢΂͖͔
9. σʔλج൫ͭͷ࢖͍ಓ  ఆܕΫΤϦʔ
   ,1*௥੻΍ޮՌଌఆͷμογϡϘʔυ
    ඇఆܕʢΞυϗοΫʣΫΤϦʔ
   ঢ়گʹԠͯ͡ਓ͕ؒ౎౓ߟ࣮͑ͯߦ͢ΔΫΤϦʔ
    όϧΫΤΫεϙʔτ
   ଞγεςϜ΁ͷσʔλసૹ ͜Ε͕Ұ൪Ձ஋͕ߴ͍ʂ
   ʜʜ͠ɺ੍໿΋େ͖͍
10. σʔλج൫͸ΞϓϦέʔγϣϯͷͨΊʹ αʔϏε σʔλج൫ αʔϏε αʔϏε Ϛελʔɺ
    ϩά վળ
    ϑΟʔυ
    όοΫ ,1*μογϡϘʔυ΋ΞυϗοΫΫΤϦʔ΋σʔλΤΫεϙʔτ΋
    ݁ہͷͱ͜ΖαʔϏεͷύϑΥʔϚϯεΛվળ͢ΔͨΊʹ͋Δɻ
    લऀͭ͸ਓؒܦ༝ͰϑΟʔυόοΫ͞Εɺޙऀ͸ϓϩάϥϜ͔Β௚઀ϑΟʔυόοΫ͞ΕΔɻ
11. σʔλ׆༻ࣄྫ
12. ࣄྫ
    ͨ΂ΈΔʢݕࡧස౓෼ੳʣ ‣ ಛఆͷޠ͕ݕࡧ͞Εͨස౓ͷมԽΛάϥϑͰݟΔ͜ͱ͕Ͱ͖Δ
    ‣ ͍·੝Γ্͕͍ͬͯΔݕࡧ΍ɺ૊Έ߹ΘͤΒΕ΍͍͢ޠ͕ݟΒΕΔ ৯΂͍ͨ
    ࡞Γ͍ͨ
    ʢݕࡧʣ
13. ͨ΂ΈΔͷΞʔΩςΫνϟ ‣ ݕࡧϩάͱϢʔβʔϚελʔΛδϣΠϯͯ͠೔෇ͱޠͰूܭ͢Δ
    ‣ ࡞੒ͨ͠αϚϦʔ͸ΞϓϦ༻ͷ3FETIJGUΫϥελʔ΁సૹͯ͠ར༻ 3FETIJGUʢ%8)Ϋϥελʔʣ 3FETIJGUʢͨ΂ΈΔΫϥελʔʣ ݕࡧϩά Ϣʔβʔ
    Ϛελʔ αϚϦʔ ೔෇ɺ୯ޠ *%ɺ౷ܭ஋ αϚϦʔ DPPLQBE 42-
    όον
14. ࣄྫ
    ਓؾॱϨγϐݕࡧ ‣ ݕࡧ݁ՌΛʮਓؾʯͷ͋ΔॱʹݟΒΕΔ
    ‣ ༗ྉձһϝχϡʔͰ࠷΋࢖ΘΕ͍ͯΔػೳ
15. ਓؾॱݕࡧͷΞʔΩςΫνϟ ‣ γϯϓϧͳ42-όονͰਓؾॱϥϯΫΛܭࢉ͠ɺ"UIFOBͰ࠷ऴυ ΩϡϝϯτΛੜ੒ͯ͠4PMSʹಥͬࠐΉ
    ‣ ͢΂ͯ3VCZͰॻ͔Ε͍ͯͨόονΛҠ২ͨͨ͠Ί·ͩແବ͕ଟ͍ 3FETIJGU 4 17ϩά Ϛελʔ ਓؾॱ
    ϥϯΫ DPPLQBE ͦͷଞͷ
    Πϯσο
    Ϋε ਓؾॱ
    ϥϯΫ ͦͷଞͷ
    Πϯσο
    Ϋε 4PMS
    υΩϡ
    ϝϯτ "UIFOBͰ݁߹ 6/-0"% 42-
    όον 4PMS΁
16. ࣄྫ
    λʔήςΟϯά޿ࠂ ‣ ಉ͡ଐੑΛ࣋ͭϢʔβʔͷηάϝϯτΛ࡞੒͠ɺΑΓޮՌ͕͋Δͱ ࢥΘΕΔ޿ࠂΛબ୒తʹ഑৴͢Δ
    ‣ ֦ு৘ใ͸42-ͱ1ZUIPOͰੜ੒͍ͯ͠Δ 3FETIJGU %ZOBNP%# 17ϩά Ϛελʔ Ϣʔβʔ
    ηάϝϯτ DPPLQBE ֦ு
    Ϣʔβʔ
    ηάϝϯτ ηάϝϯτ 42-
    όον ΦʔσΟΤϯε
    ֦ு ֦ு৘ใ
17. ͳͥ3FETIJGUͳͷ͔ʁ
18. 3FETIJGUΛબͿཧ༝  ΞϓϦέʔγϣϯ͕"84ʹ͋Δ͔Β
    σʔλج൫͸੒௕͢Δ΄ͲΞϓϦέʔγϣϯͱͷσʔλͷ΍ΓͱΓ͕૿͍͑ͯ͘ɻ͕ͨͬͯ͠Ξ ϓϦέʔγϣϯͱ࿈ܞ͠΍͍͢ɺ͍ۙ৔ॴʹ͋Δ΂͖ɻ
     ଞγεςϜͱͷ࿈ܞ͕༏Ε͍ͯΔ͔Β
    'FEFSBUFE
    2VFSZ΍4QFDUSVNɺ%BUB
    "1*ͷΑ͏ͳɺଞγεςϜͱͷ࿈ܞͷͨΊͷػೳ͕༏ लɻ·ͨ*".Ͱ͢΂ͯͷ࿈ܞΛ౷Ұతʹ؅ཧͰ͖Δ఺͸஍ຯ͕ͩඇৗʹॏཁɻ
     ඞཁͳػೳͳΒ௥Ճ͞ΕΔͱ͍͏৴པײ͕͋Δ
    4QFDUSVN΍3"ɺ1BSUJ2-ͷΑ͏ͳ௒ڊେͳػೳͰ΋ɺඞཁʹͳͬͨ͜Ζʹண࣮ʹ௥Ճ͞Εͯ ͍͘ͷͰɺ͍·ػೳ͕ෆ଍͍ͯͯ͠΋͍ۙকདྷʹղܾ͢ΔͩΖ͏ͱָ؍Ͱ͖Δɻ
19. ·ͱΊ
20. ·ͱΊ  σʔλج൫ʹ͸ओʹ௨Γͷ࢖͍ಓ͕͋Δ
     όϧΫΤΫεϙʔτʢ௚઀తͳσʔλ׆༻ʣ͕࠷ ΋Ձ஋͕ߴ੍͘໿΋ڧ͍ͷͰɺΞʔΩςΫνϟ͸ ͦΕʹ࠷దԽ͢΂͖
     ΫοΫύουͷγνϡΤʔγϣϯͰ͸3FETIJGU͕ ্هͷ৚݅ʹরΒͯ͠࠷దͰ͋ͬͨ
21. ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠