Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Why Cookpad Is Choosing Redshift as Our Data Platform

Why Cookpad Is Choosing Redshift as Our Data Platform

Data Engineering Study #7「Redshift最新アップデートと活用事例」での発表資料です。
https://forkwell.connpass.com/event/203403/

Minero Aoki

April 06, 2021
Tweet

More Decks by Minero Aoki

Other Decks in Technology

Transcript

  1. .JOFSP"PLJ
    ͳͥΫοΫύου͸
    3FETIJGUΛσʔλج൫ʹ
    બͼଓ͚Δͷ͔
    ʙσʔλج൫͕Ռͨ͢΂͖໾ׂʙ

    View Slide

  2. ࣗݾ঺հ
    ‣ ੨໦ๆ࿠ʢ͓͖͋ΈͶΖ͏ʣ
    ‣ ݩ3VCZίϛολʔ
    ‣ લ৬ฒྻ3%#.4ϕϯμʔ
    ‣ ฒྻσʔλϕʔεͱग़ձ͏
    ‣ ݱ৬ΫοΫύου
    ‣ ෼ੳαʔϏεʮͨ΂ΈΔʯͷ։ൃ
    ‣ σʔλ׆༻ج൫ͷߏங
    ‣ ϨγϐαʔϏεͷϦΞʔΩςΫνϟ

    View Slide

  3. ΫοΫύουͷ
    σʔλج൫ͷ֓ཁ

    View Slide

  4. ݕࡧαʔϏεʮͨ΂ΈΔʯʹ3FETIJGUಋೖ
    શࣾσʔλج൫Λ3FETIJGU΁Ҡߦ
    4QFDUSVNಋೖ
    $PODVSSFODZ4DBMJOHɺ6TBHF-JNJUಋೖ
    'FEFSBUFE2VFSZɺ"VUP8-.ಋೖ
    3"ಋೖ
    3FETIJGUར༻ͷྺ࢙

    View Slide

  5. ΫοΫύουͷσʔλج൫ͷ֓ཁ
    3FETIJGUSBYMBSHFº
    5# OPEFTMJDF

    όϧΫΤΫεϙʔτ
    12 +%#$
    .Z42-
    1PTUHSF42-
    %ZOBNP%#
    qVFOUE
    Ϛελʔ
    ϩά
    +40/

    4
    ଞγεςϜ
    42-DMJFOU
    CEBTI
    5BCMFBV
    ؅ཧը໘
    8PSLEBZ
    4BMFTGPSDF
    ;FOEFTL
    τϥϯβ
    Ϋγϣϯ
    ςʔϒϧఔ౓
    ຊऑ
    ຊऑ
    ϩά
    αϚϦʔ
    αϚϦʔ
    42-όον໿ຊ
    12 +%#$
    ఆܕΫΤϦʔ
    2VFVFSZ
    ඇఆܕΫΤϦʔ

    View Slide

  6. 3FETIJGU
    4
    w 1SJTNʢ಺੡ʣʹΑΔϩάͷετϦʔϛϯάϩʔυ
    w 1BSRVFUͷϚʔδͱɺ೚ҙλΠϜκʔϯͰͷύʔςΟγϣχϯά͕Մೳ
    w 1SJTNৄࡉIUUQTUFDIMJGFDPPLQBEDPNFOUSZ
    1SJTNʹΑΔϩάϩʔυ
    ֎෦
    ςʔϒϧ
    +40/
    +40/
    +40/
    1SJTN4USFBN
    1BSRVFU
    1BSRVFU
    MJWFPCKFDUT
    1BSRVFU
    1SJTN
    .FSHF
    1BSRVFU
    NFSHFEPCKFDUT
    ౸ண͍ͩ͠ॲཧ
    ໿࣌ؒ
    ͝ͱ
    (MVFΧλϩά
    Ͱ੾Γସ͑

    View Slide

  7. ϚελʔςʔϒϧͷऔΓࠐΈ
    %#λΠϓ ςʔϒϧ਺ γεςϜ ִؒ ํࣜ
    .Z42- 1JQFMJOFE.JHSBUPSʢ಺੡ʣ ೔࣍ શߦચ͍ସ͑
    1PTUHSF42-
    3FETIJGU'FEFSBUFE2VFSZ
    "84%BUBCBTF.JHSBUJPO4FSWJDF
    ೔࣍ શߦચ͍ସ͑
    %ZOBNP%# %ZOBNP%#4USFBNTˠ1SJTN ৗ࣌ ࠩ෼ߋ৽

    View Slide

  8. σʔλج൫͸ԿΛ͢΂͖͔

    View Slide

  9. σʔλج൫ͭͷ࢖͍ಓ
    ఆܕΫΤϦʔ
    ,1*௥੻΍ޮՌଌఆͷμογϡϘʔυ
    ඇఆܕʢΞυϗοΫʣΫΤϦʔ
    ঢ়گʹԠͯ͡ਓ͕ؒ౎౓ߟ࣮͑ͯߦ͢ΔΫΤϦʔ
    όϧΫΤΫεϙʔτ
    ଞγεςϜ΁ͷσʔλసૹ
    ͜Ε͕Ұ൪Ձ஋͕ߴ͍ʂ
    ʜʜ͠ɺ੍໿΋େ͖͍

    View Slide

  10. σʔλج൫͸ΞϓϦέʔγϣϯͷͨΊʹ
    αʔϏε
    σʔλج൫
    αʔϏε αʔϏε
    Ϛελʔɺ
    ϩά
    վળ
    ϑΟʔυ
    όοΫ
    ,1*μογϡϘʔυ΋ΞυϗοΫΫΤϦʔ΋σʔλΤΫεϙʔτ΋
    ݁ہͷͱ͜ΖαʔϏεͷύϑΥʔϚϯεΛվળ͢ΔͨΊʹ͋Δɻ
    લऀͭ͸ਓؒܦ༝ͰϑΟʔυόοΫ͞Εɺޙऀ͸ϓϩάϥϜ͔Β௚઀ϑΟʔυόοΫ͞ΕΔɻ

    View Slide

  11. σʔλ׆༻ࣄྫ

    View Slide

  12. ࣄྫͨ΂ΈΔʢݕࡧස౓෼ੳʣ
    ‣ ಛఆͷޠ͕ݕࡧ͞Εͨස౓ͷมԽΛάϥϑͰݟΔ͜ͱ͕Ͱ͖Δ
    ‣ ͍·੝Γ্͕͍ͬͯΔݕࡧ΍ɺ૊Έ߹ΘͤΒΕ΍͍͢ޠ͕ݟΒΕΔ
    ৯΂͍ͨ
    ࡞Γ͍ͨ
    ʢݕࡧʣ

    View Slide

  13. ͨ΂ΈΔͷΞʔΩςΫνϟ
    ‣ ݕࡧϩάͱϢʔβʔϚελʔΛδϣΠϯͯ͠೔෇ͱޠͰूܭ͢Δ
    ‣ ࡞੒ͨ͠αϚϦʔ͸ΞϓϦ༻ͷ3FETIJGUΫϥελʔ΁సૹͯ͠ར༻
    3FETIJGUʢ%8)Ϋϥελʔʣ 3FETIJGUʢͨ΂ΈΔΫϥελʔʣ
    ݕࡧϩά
    Ϣʔβʔ
    Ϛελʔ
    αϚϦʔ
    ೔෇ɺ୯ޠ
    *%ɺ౷ܭ஋
    αϚϦʔ
    DPPLQBE
    42-
    όον

    View Slide

  14. ࣄྫਓؾॱϨγϐݕࡧ
    ‣ ݕࡧ݁ՌΛʮਓؾʯͷ͋ΔॱʹݟΒΕΔ
    ‣ ༗ྉձһϝχϡʔͰ࠷΋࢖ΘΕ͍ͯΔػೳ

    View Slide

  15. ਓؾॱݕࡧͷΞʔΩςΫνϟ
    ‣ γϯϓϧͳ42-όονͰਓؾॱϥϯΫΛܭࢉ͠ɺ"UIFOBͰ࠷ऴυ
    ΩϡϝϯτΛੜ੒ͯ͠4PMSʹಥͬࠐΉ
    ‣ ͢΂ͯ3VCZͰॻ͔Ε͍ͯͨόονΛҠ২ͨͨ͠Ί·ͩແବ͕ଟ͍
    3FETIJGU 4
    17ϩά
    Ϛελʔ
    ਓؾॱ
    ϥϯΫ
    DPPLQBE
    ͦͷଞͷ
    Πϯσο
    Ϋε
    ਓؾॱ
    ϥϯΫ
    ͦͷଞͷ
    Πϯσο
    Ϋε
    4PMS
    υΩϡ
    ϝϯτ
    "UIFOBͰ݁߹
    6/-0"%
    42-
    όον
    4PMS΁

    View Slide

  16. ࣄྫλʔήςΟϯά޿ࠂ
    ‣ ಉ͡ଐੑΛ࣋ͭϢʔβʔͷηάϝϯτΛ࡞੒͠ɺΑΓޮՌ͕͋Δͱ
    ࢥΘΕΔ޿ࠂΛબ୒తʹ഑৴͢Δ
    ‣ ֦ு৘ใ͸42-ͱ1ZUIPOͰੜ੒͍ͯ͠Δ
    3FETIJGU %ZOBNP%#
    17ϩά
    Ϛελʔ
    Ϣʔβʔ
    ηάϝϯτ
    DPPLQBE
    ֦ு
    Ϣʔβʔ
    ηάϝϯτ
    ηάϝϯτ
    42-
    όον
    ΦʔσΟΤϯε
    ֦ு
    ֦ு৘ใ

    View Slide

  17. ͳͥ3FETIJGUͳͷ͔ʁ

    View Slide

  18. 3FETIJGUΛબͿཧ༝
    ΞϓϦέʔγϣϯ͕"84ʹ͋Δ͔Β
    σʔλج൫͸੒௕͢Δ΄ͲΞϓϦέʔγϣϯͱͷσʔλͷ΍ΓͱΓ͕૿͍͑ͯ͘ɻ͕ͨͬͯ͠Ξ
    ϓϦέʔγϣϯͱ࿈ܞ͠΍͍͢ɺ͍ۙ৔ॴʹ͋Δ΂͖ɻ
    ଞγεςϜͱͷ࿈ܞ͕༏Ε͍ͯΔ͔Β
    'FEFSBUFE2VFSZ΍4QFDUSVNɺ%BUB"1*ͷΑ͏ͳɺଞγεςϜͱͷ࿈ܞͷͨΊͷػೳ͕༏
    लɻ·ͨ*".Ͱ͢΂ͯͷ࿈ܞΛ౷Ұతʹ؅ཧͰ͖Δ఺͸஍ຯ͕ͩඇৗʹॏཁɻ
    ඞཁͳػೳͳΒ௥Ճ͞ΕΔͱ͍͏৴པײ͕͋Δ
    4QFDUSVN΍3"ɺ1BSUJ2-ͷΑ͏ͳ௒ڊେͳػೳͰ΋ɺඞཁʹͳͬͨ͜Ζʹண࣮ʹ௥Ճ͞Εͯ
    ͍͘ͷͰɺ͍·ػೳ͕ෆ଍͍ͯͯ͠΋͍ۙকདྷʹղܾ͢ΔͩΖ͏ͱָ؍Ͱ͖Δɻ

    View Slide

  19. ·ͱΊ

    View Slide

  20. ·ͱΊ
    σʔλج൫ʹ͸ओʹ௨Γͷ࢖͍ಓ͕͋Δ
    όϧΫΤΫεϙʔτʢ௚઀తͳσʔλ׆༻ʣ͕࠷
    ΋Ձ஋͕ߴ੍͘໿΋ڧ͍ͷͰɺΞʔΩςΫνϟ͸
    ͦΕʹ࠷దԽ͢΂͖
    ΫοΫύουͷγνϡΤʔγϣϯͰ͸3FETIJGU͕
    ্هͷ৚݅ʹরΒͯ͠࠷దͰ͋ͬͨ

    View Slide

  21. ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠

    View Slide