Ruby で作るデータ分析基盤

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

©2018 Wantedly, Inc. ⾣ ϢʔβʔͷߦಈΛݩʹΞϓϦέʔγϣϯ ͷಈ͖Λม͍͑ͨ w FHॏཁͳίϯόʔδϣϯɾεςοϓΛ௨ա͍ͯ͠Δ͔൱͔ w FHίϯόʔδϣϯʹࢸΔ·Ͱͷࡉ͔͍εςοϓ͝ͱͷέΞ w ʮ☓☓Λ/ճݟ͍ͯͨΒʯ w ػցֶशʹΑΔ࠷దԽ ⾣ Ϣʔβʔʹ෼ੳػೳΛఏڙ͍ͨ͠ w FHʮࡢ೔ͷ17ͱɺ͜Ε·Ͱͷ17ͷ߹ܭʯ w FHʮଞͱൺֱ͢ΔͨΊͷ17ϥϯΩϯάʯ ͳͥσʔλ෼ੳΛ͍͔ͨ͠

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

©2018 Wantedly, Inc. ⾣ ॏཁͳίϯόʔδϣϯɾεςοϓΛ௨ա͍ͯ͠Δ͔൱͔  ˠ౰વϦϨʔγϣφϧσʔλϕʔεʹอଘ͞Ε͍ͯΔ ⾣ ʮϢʔβʔ:͕ΞΠςϜΛ/ճݟ͍ͯͨΒʯ  ˠࣄ৅͕ى͜Δ౓ʹΧ΢ϯτ͓ͯ͘͠PSϩά͔Βܭࢉ͢Δ ⾣ ʮϢʔβʔ:͕ΞΠςϜ9Λݟ͍ͯͨΒʯ  ˠϦϨʔγϣφϧσʔλϕʔεʹϩάΛอଘͯ͠ࢀর͢Δ ϢʔβʔͷߦಈΛݩʹΞϓϦέʔγϣϯͷಈ͖Λม͍͑ͨ

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

©2018 Wantedly, Inc. ⾣ lϦϨʔγϣφϧσʔλϕʔεʹϩάΛอଘz w %#༰ྔΛѹഭɺू໿ܭࢉ΋͠ʹ͘͘த్൒୺ͳଘࡏʹ ⾣ lதؒσʔλશମΛूܭ͢ΔΫΤϦΛॻ͖ɺ3BJMTΩϟογϡz w ࣌ؒͷܦաʜதؒσʔλͩͱࢥ͍ͬͯͨ΋ͷࣗମ͕ॏ͘ͳΔ w Ϣʔβʔ਺ͷ૿Ճʜಉ࣌ΞΫηε਺͕૿Ճ͠ɺ3BJMTΩϟογϡ ͕ඈΜͩλΠϛϯάͰͷ࠶ܭࢉ͕ಉ࣌ฒߦͰ૸Δ w ্هͷֻ͚߹ΘͤͰ%#͕ϝϞϦΛ࢖͍Ռͨ͢ࣄ݅΋ ੒௕͖ͯͨ͜͠ͱͰى͖ͨ໰୊

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

©2018 Wantedly, Inc. ⾣ δϣϒϑΝΠϧΛ௥Ճ͠ɺQVMMSFRVFTUʹ͢Δ ⾣ Ϛʔδ͢Δͱɺδϣϒͱͯ͠ొ࿥͞Εఆظ࣮ߦ͞ΕΔ ⾣ $-*͔Β೚ҙͷ೔ͷδϣϒΛ࣮ߦՄೳ ⾣ ೚ҙͷ3VCZίʔυΛ࣮ߦ͢ΔSVOOFS ࢖͍ํ $ ./bin/job run daily_page_view -s '2018-07-14' run :proc, -> (scheduled_time) { # … return [[:foo, 1], [:bar, 2]] }

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

©2018 Wantedly, Inc. ⾣ ʮࡢ೔ΠϯϑϥϨϕϧͰෆ۩߹͕͋ͬͨͷͰʓʓؔ܎ͷδϣϒ Λػցతʹ࠶࣮ߦ͢Δʯ͕Մೳ͔ʁ ⾣ ΠϯλʔϑΣΠεɾϨϕϧͰ࠶࣮ߦՄೳੑΛଅਐ w FYQPSUͷࡍ͸VQEBUF͕σϑΥϧτ w SVO͸֎෦͔Β࣮ߦ࣌ؒΛҾ਺Λ༩͑Δ w TDIFEVMJOH͸EBJMZͷΑ͏ͳେ·͔ͳࢦఆ͕σϑΥϧτ ⾣ δϣϒϑΝΠϧ͸3VCZΦϒδΣΫτͱͯ͠ϩʔυ͞ΕΔͷͰɺ ϓϩάϥϚϒϧʹ΋ॲཧͰ͖Δ δϣϒͷ඼࣭ jobs = Analytics::Backend::JobLoader.load_all; daily_jobs = jobs.select {|job| job.schedule&.frequency == :daily } daily_jobs.each do |job| job.execute(scheduled_time: Time.new(2018,1,10), export: true) end

Slide 23

Slide 23 text

©2018 Wantedly, Inc. ⾣ ΫϥελϦϯά͞Εͨෳ਺ͷϚγϯ্Ͱ࣮ߦ͞ΕΔ  ʢޮ཰Խɾ৑௕Խʣ ⾣ ,VCFSOFUFTͷίϚϯυͰొ࿥δϣϒͷҰཡ΍࣮ߦεςʔλ ε͕͋Δఔ౓෼͔Δ δϣϒ࣮ߦج൫ͱͯ͠ͷ,VCFSOFUFTͷ׆༻ export do table :daily_page_views columns [:day, :pv] mode :update, [:day] end schedule do frequency :daily end run :bq, <