Digdag@表参道.rb #14

by Takuya Arita

Slide 1

Slide 1 text

Digdag@දࢀಓ.rb #14 @ariarijp

Slide 2

Slide 2 text

ࣗݾ঺հ • @ariarijp (Takuya Arita) • දࢀಓ.rbࢀՃ͸3ճ໨ɺൃද͸2ճ໨ • ιϑτ΢ΣΞΤϯδχΞ@ίίϥϒϧ

Slide 3

Slide 3 text

࢓ࣄͷ8ׂ͸όονॲཧ

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

όονॲཧͷͭΒΈ • ஍ຯ • -> ผʹݏ͡Όͳ͍ • CSV • -> ͭΒ͍͚Ͳࠓ೔͸࿩୊ʹ͠ͳ͍ • ґଘؔ܎ͷߟྀ • -> ͭΒ͍

Slide 6

Slide 6 text

Α͋͘Δґଘؔ܎ • όονA, B, C, D, E, F, G͕͋Δ • A, B, C͸ฒྻ࣮ߦͰ͖Δ • D͸A, B, C͕͢΂ͯ׬͔ྃͯ͠Β࣮ߦ • E, F͸D͕׬͔ྃͯ͠Β࣮ߦɺฒྻ࣮ߦͰ͖Δ • G͸E, F͕͢΂ͯ׬͔ྃͯ͠Β࣮ߦ

Slide 7

Slide 7 text

ґଘؔ܎Λ͕Μ͹ͬͯղܾ͢Δ • ௚ྻʹ࣮ߦ • -> ॱ൪͸कΒΕΔ͠ɺΘ͔Γ΍͍͚͢Ͳ஗͍ • εςʔλεΛԿΒ͔ͷܗͰ؅ཧͯ͠ฒྻ࣮ߦΛఘΊͳ͍ • -> ҟৗऴྃͨ࣌͠ͷεςʔλεΫϦΞͳͲͷߟྀ͕໘౗ • ࣮ߦִؒΛௐ੔ͭͭ͠ɺͳʹ͔ى͖ͨΒӡ༻ͰΧόʔ͢ΔલఏͰ crontab΍γΣϧεΫϦϓτΛ࢖ͬͯฒྻ࣮ߦ • ->

Slide 8

Slide 8 text

ͦ͜Ͱ Digdag

Slide 9

Slide 9 text

Digdag • Fluentd, Embulkʹଓ͘Treasure DataϓϩμΫτ • JavaͳͷͰ͍͍ͩͨͲ͜Ͱ΋ಈ͘ɺΠϯετʔϧ΋ָ • YamlͰϫʔΫϑϩʔΛఆٛ͠ɺdigdag runͰ࣮ߦ • όονΛฒྻ࣮ߦɺόοΫάϥ΢ϯυ࣮ߦ͢Δ͜ͱ͕ Ͱ͖ɺґଘؔ܎·ΘΓͷࠔΓࣄ͸Ұ௨ΓϑΥϩʔ͞Ε ͍ͯΔҹ৅

Slide 10

Slide 10 text

Digdagʹ͍ͭͯͷޡղ • TD੡ͰEmbulk΍TD, BigQueryͱ߹Θͤͯ঺ հ͞ΕΔ͜ͱ͕ଟ͍ͨΊɺେྔͷσʔλΛѻ ͏ͨΊͷπʔϧͱࢥΘΕ͕ͪͳͷͰ͸ʁ • Digdag͸ϫʔΫϑϩʔΛ؅ཧͯ͘͠ΕΔ͚ͩ ͳͷͰɺόονॲཧશൠʹར༻Ͱ͖Δ

Slide 11

Slide 11 text

Digdag࢖ͬͯΈͨ • PHPͰॻ͔Ε͍͔ͨͭ͘ͷόονΛ࣮ߦ͢ΔϫʔΫϑϩʔΛ ఆٛ • sh> ΦϖϨʔλʔΛ࢖͑Ε͹PHPͰ΋ͳΜͱ͔ͳΔ • ఆٛͨ͠ϦετΛ࢖ͬͯ܁Γฦ࣮͠ߦͰ͖Δfor_each> ΋ศར • ฒྻԽΛఘΊ͍ͯͨόον΍ɺࠞಱͱͨ͠crontabʹฏ࿨ ͕΋ͨΒ͞Εͨ

Slide 12

Slide 12 text

ࠓ೔͸දࢀಓ.rbͳͷʹ ·ͩRubyͷ࿩ͯ͠ͳ͍ɾɾɾ

Slide 13

Slide 13 text

Ruby API

Slide 14

Slide 14 text

Ruby APIͷ࢖͍ํ • ϫʔΫϑϩʔͷ֤εςοϓͷॲཧΛϝιουͱͯ࣋͠ ͭΫϥεΛॻ͍ͨRubyεΫϦϓτΛಡΈࠐΉ • rb> ΦϖϨʔλʔͰͲͷϝιουΛ࣮ߦ͢Δ͔Λఆٛ ͢Δ • RubyεΫϦϓτ͔ΒϫʔΫϑϩʔ಺Ͱ࢖༻͢Δม਺ Λఆٛͨ͠ΓɺࢠλεΫΛ࡞੒ɾ࣮ߦ͢Δ͜ͱ΋Ͱ͖ Δ

Slide 15

Slide 15 text

Ruby APIͷ஫ҙ఺ • rubyͷύεΛࢦఆͰ͖ͳͦ͞͏ͳͷͰɺrbenvͳͲͰόʔδϣϯΛ ੾Γସ͑Δ • υΩϡϝϯτʹهࡌ͞Ε͍ͯΔαϯϓϧ͸ɺσϑΥϧτ஋ͳ͠ͷ ΩʔϫʔυҾ਺͕͋ͬͨͷͰ2.1Ҏ߱લఏͬΆ͍ • εΫϦϓτ͸ผϓϩηεͰಈ͘ͷͰɺ࣮ߦ݁ՌͳͲΛڞ༗͍ͨ͠৔ ߹͸Digdagͷม਺΍DB, ϑΝΠϧͳͲͰڞ༗͢Δ • Ruby API͔Βઃఆͨ͠ม਺͕഑ྻͩͬͨ৔߹ɺͦͷม਺Λ࢖ͬͯ for_each> Ͱ͙Δ͙Δճ͢ͷ͸Ͱ͖ͳͦ͞͏

Slide 16

Slide 16 text

͕࣌ؒ͋Ε͹ DigdagͱRuby APIͷσϞ

Slide 17

Slide 17 text

·ͱΊ • ग़ੈ͠Α͏ͱ͠ͳ͘ͱ΋όονॲཧ͸ͦΕͳ Γʹॻ͘͜ͱʹͳΔ • Digdag͸σʔλΤϯδχΞϦϯάͷ༻్Ҏ֎ Ͱ΋ϫʔΫϑϩʔΤϯδϯͱͯ͠࢖͑Δ • ґଘؔ܎ͷ͋Δͪΐͬͱෳࡶͳόονॲཧ͸ Digdagͷग़൪͔΋

Slide 18

Slide 18 text

ࢀߟࢿྉ • υΩϡϝϯτ • http://www.digdag.io/ • ݹڮః೭͞ΜΠϯλϏϡʔ - Tokyo RubyKaigi 11 #tkrk11 • http://regional.rubykaigi.org/tokyo11/interview/frsyuki/ • DigdagͰruby/pythonؒͰม਺Λ౉͢ • http://qiita.com/toru-takahashi/items/ 418eb5a4dda4e8555a00