Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Why Cookpad Is Choosing Redshift as Our Data Platform

Why Cookpad Is Choosing Redshift as Our Data Platform

Data Engineering Study #7「Redshift最新アップデートと活用事例」での発表資料です。
https://forkwell.connpass.com/event/203403/

0d9e60b12d0a40841b1a746b112ad36b?s=128

Minero Aoki

April 06, 2021
Tweet

Transcript

  1. .JOFSP"PLJ  ͳͥΫοΫύου͸ 3FETIJGUΛσʔλج൫ʹ બͼଓ͚Δͷ͔ ʙσʔλج൫͕Ռͨ͢΂͖໾ׂʙ

  2. ࣗݾ঺հ ‣ ੨໦ๆ࿠ʢ͓͖͋ΈͶΖ͏ʣ ‣ ݩ3VCZίϛολʔ ‣ લ৬ฒྻ3%#.4ϕϯμʔ ‣ ฒྻσʔλϕʔεͱग़ձ͏ ‣

    ݱ৬ΫοΫύου ‣ ෼ੳαʔϏεʮͨ΂ΈΔʯͷ։ൃ ‣ σʔλ׆༻ج൫ͷߏங ‣ ϨγϐαʔϏεͷϦΞʔΩςΫνϟ
  3. ΫοΫύουͷ σʔλج൫ͷ֓ཁ

  4. ݕࡧαʔϏεʮͨ΂ΈΔʯʹ3FETIJGUಋೖ શࣾσʔλج൫Λ3FETIJGU΁Ҡߦ 4QFDUSVNಋೖ $PODVSSFODZ4DBMJOHɺ6TBHF-JNJUಋೖ 'FEFSBUFE2VFSZɺ"VUP8-.ಋೖ 3"ಋೖ 3FETIJGUར༻ͷྺ࢙

  5. ΫοΫύουͷσʔλج൫ͷ֓ཁ 3FETIJGUSBYMBSHFº 5# OPEFTMJDF όϧΫΤΫεϙʔτ 12 +%#$ .Z42- 1PTUHSF42- %ZOBNP%#

    qVFOUE Ϛελʔ ϩά +40/ 4 ଞγεςϜ 42-DMJFOU CEBTI 5BCMFBV ؅ཧը໘ 8PSLEBZ 4BMFTGPSDF ;FOEFTL τϥϯβ Ϋγϣϯ ςʔϒϧఔ౓ ຊऑ ຊऑ ϩά αϚϦʔ αϚϦʔ 42-όον໿ຊ 12 +%#$ ఆܕΫΤϦʔ 2VFVFSZ ඇఆܕΫΤϦʔ
  6. 3FETIJGU 4 w 1SJTNʢ಺੡ʣʹΑΔϩάͷετϦʔϛϯάϩʔυ w 1BSRVFUͷϚʔδͱɺ೚ҙλΠϜκʔϯͰͷύʔςΟγϣχϯά͕Մೳ w 1SJTNৄࡉIUUQTUFDIMJGFDPPLQBEDPNFOUSZ 1SJTNʹΑΔϩάϩʔυ ֎෦

    ςʔϒϧ +40/ +40/ +40/ 1SJTN4USFBN 1BSRVFU 1BSRVFU MJWFPCKFDUT 1BSRVFU 1SJTN .FSHF 1BSRVFU NFSHFEPCKFDUT ౸ண͍ͩ͠ॲཧ ໿࣌ؒ ͝ͱ (MVFΧλϩά Ͱ੾Γସ͑
  7. ϚελʔςʔϒϧͷऔΓࠐΈ %#λΠϓ ςʔϒϧ਺ γεςϜ ִؒ ํࣜ .Z42-  1JQFMJOFE.JHSBUPSʢ಺੡ʣ ೔࣍

    શߦચ͍ସ͑ 1PTUHSF42-  3FETIJGU'FEFSBUFE2VFSZ "84%BUBCBTF.JHSBUJPO4FSWJDF ೔࣍ શߦચ͍ସ͑ %ZOBNP%#  %ZOBNP%#4USFBNTˠ1SJTN ৗ࣌ ࠩ෼ߋ৽
  8. σʔλج൫͸ԿΛ͢΂͖͔

  9. σʔλج൫ͭͷ࢖͍ಓ  ఆܕΫΤϦʔ ,1*௥੻΍ޮՌଌఆͷμογϡϘʔυ  ඇఆܕʢΞυϗοΫʣΫΤϦʔ ঢ়گʹԠͯ͡ਓ͕ؒ౎౓ߟ࣮͑ͯߦ͢ΔΫΤϦʔ  όϧΫΤΫεϙʔτ ଞγεςϜ΁ͷσʔλసૹ

    ͜Ε͕Ұ൪Ձ஋͕ߴ͍ʂ ʜʜ͠ɺ੍໿΋େ͖͍
  10. σʔλج൫͸ΞϓϦέʔγϣϯͷͨΊʹ αʔϏε σʔλج൫ αʔϏε αʔϏε Ϛελʔɺ ϩά վળ ϑΟʔυ όοΫ

    ,1*μογϡϘʔυ΋ΞυϗοΫΫΤϦʔ΋σʔλΤΫεϙʔτ΋ ݁ہͷͱ͜ΖαʔϏεͷύϑΥʔϚϯεΛվળ͢ΔͨΊʹ͋Δɻ લऀͭ͸ਓؒܦ༝ͰϑΟʔυόοΫ͞Εɺޙऀ͸ϓϩάϥϜ͔Β௚઀ϑΟʔυόοΫ͞ΕΔɻ
  11. σʔλ׆༻ࣄྫ

  12. ࣄྫͨ΂ΈΔʢݕࡧස౓෼ੳʣ ‣ ಛఆͷޠ͕ݕࡧ͞Εͨස౓ͷมԽΛάϥϑͰݟΔ͜ͱ͕Ͱ͖Δ ‣ ͍·੝Γ্͕͍ͬͯΔݕࡧ΍ɺ૊Έ߹ΘͤΒΕ΍͍͢ޠ͕ݟΒΕΔ ৯΂͍ͨ ࡞Γ͍ͨ ʢݕࡧʣ

  13. ͨ΂ΈΔͷΞʔΩςΫνϟ ‣ ݕࡧϩάͱϢʔβʔϚελʔΛδϣΠϯͯ͠೔෇ͱޠͰूܭ͢Δ ‣ ࡞੒ͨ͠αϚϦʔ͸ΞϓϦ༻ͷ3FETIJGUΫϥελʔ΁సૹͯ͠ར༻ 3FETIJGUʢ%8)Ϋϥελʔʣ 3FETIJGUʢͨ΂ΈΔΫϥελʔʣ ݕࡧϩά Ϣʔβʔ Ϛελʔ

    αϚϦʔ ೔෇ɺ୯ޠ *%ɺ౷ܭ஋ αϚϦʔ DPPLQBE 42- όον
  14. ࣄྫਓؾॱϨγϐݕࡧ ‣ ݕࡧ݁ՌΛʮਓؾʯͷ͋ΔॱʹݟΒΕΔ ‣ ༗ྉձһϝχϡʔͰ࠷΋࢖ΘΕ͍ͯΔػೳ

  15. ਓؾॱݕࡧͷΞʔΩςΫνϟ ‣ γϯϓϧͳ42-όονͰਓؾॱϥϯΫΛܭࢉ͠ɺ"UIFOBͰ࠷ऴυ ΩϡϝϯτΛੜ੒ͯ͠4PMSʹಥͬࠐΉ ‣ ͢΂ͯ3VCZͰॻ͔Ε͍ͯͨόονΛҠ২ͨͨ͠Ί·ͩແବ͕ଟ͍ 3FETIJGU 4 17ϩά Ϛελʔ

    ਓؾॱ ϥϯΫ DPPLQBE ͦͷଞͷ Πϯσο Ϋε ਓؾॱ ϥϯΫ ͦͷଞͷ Πϯσο Ϋε 4PMS υΩϡ ϝϯτ "UIFOBͰ݁߹ 6/-0"% 42- όον 4PMS΁
  16. ࣄྫλʔήςΟϯά޿ࠂ ‣ ಉ͡ଐੑΛ࣋ͭϢʔβʔͷηάϝϯτΛ࡞੒͠ɺΑΓޮՌ͕͋Δͱ ࢥΘΕΔ޿ࠂΛબ୒తʹ഑৴͢Δ ‣ ֦ு৘ใ͸42-ͱ1ZUIPOͰੜ੒͍ͯ͠Δ 3FETIJGU %ZOBNP%# 17ϩά Ϛελʔ

    Ϣʔβʔ ηάϝϯτ DPPLQBE ֦ு Ϣʔβʔ ηάϝϯτ ηάϝϯτ 42- όον ΦʔσΟΤϯε ֦ு ֦ு৘ใ
  17. ͳͥ3FETIJGUͳͷ͔ʁ

  18. 3FETIJGUΛબͿཧ༝  ΞϓϦέʔγϣϯ͕"84ʹ͋Δ͔Β σʔλج൫͸੒௕͢Δ΄ͲΞϓϦέʔγϣϯͱͷσʔλͷ΍ΓͱΓ͕૿͍͑ͯ͘ɻ͕ͨͬͯ͠Ξ ϓϦέʔγϣϯͱ࿈ܞ͠΍͍͢ɺ͍ۙ৔ॴʹ͋Δ΂͖ɻ  ଞγεςϜͱͷ࿈ܞ͕༏Ε͍ͯΔ͔Β 'FEFSBUFE2VFSZ΍4QFDUSVNɺ%BUB"1*ͷΑ͏ͳɺଞγεςϜͱͷ࿈ܞͷͨΊͷػೳ͕༏ लɻ·ͨ*".Ͱ͢΂ͯͷ࿈ܞΛ౷Ұతʹ؅ཧͰ͖Δ఺͸஍ຯ͕ͩඇৗʹॏཁɻ 

    ඞཁͳػೳͳΒ௥Ճ͞ΕΔͱ͍͏৴པײ͕͋Δ 4QFDUSVN΍3"ɺ1BSUJ2-ͷΑ͏ͳ௒ڊେͳػೳͰ΋ɺඞཁʹͳͬͨ͜Ζʹண࣮ʹ௥Ճ͞Εͯ ͍͘ͷͰɺ͍·ػೳ͕ෆ଍͍ͯͯ͠΋͍ۙকདྷʹղܾ͢ΔͩΖ͏ͱָ؍Ͱ͖Δɻ
  19. ·ͱΊ

  20. ·ͱΊ  σʔλج൫ʹ͸ओʹ௨Γͷ࢖͍ಓ͕͋Δ  όϧΫΤΫεϙʔτʢ௚઀తͳσʔλ׆༻ʣ͕࠷ ΋Ձ஋͕ߴ੍͘໿΋ڧ͍ͷͰɺΞʔΩςΫνϟ͸ ͦΕʹ࠷దԽ͢΂͖  ΫοΫύουͷγνϡΤʔγϣϯͰ͸3FETIJGU͕ ্هͷ৚݅ʹরΒͯ͠࠷దͰ͋ͬͨ

  21. ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠