Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ETLツールを1から作り直した話
Search
katsuyan
September 02, 2017
Technology
1
320
ETLツールを1から作り直した話
ETLツールを1から作り直した話(embulkを利用したETLツール)
katsuyan
September 02, 2017
Tweet
Share
More Decks by katsuyan
See All by katsuyan
realtime_marketing_system
katsuyan
0
2.5k
digdag-3tips
katsuyan
2
4.5k
ファッションチェックランキングRubyKaigiの裏側 / Fashion check ranking app for RubyKaigi2019
katsuyan
1
5.8k
embulkを利用した開発でちょっと困ったこと
katsuyan
1
490
Other Decks in Technology
See All in Technology
How Do I Contact HP Printer Support? [Full 2025 Guide for U.S. Businesses]
harrry1211
0
130
ClaudeCodeにキレない技術
gtnao
0
560
DatabricksにOLTPデータベース『Lakebase』がやってきた!
inoutk
0
150
第64回コンピュータビジョン勉強会「The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition」
x_ttyszk
0
170
ロールが細分化された組織でSREは何をするか?
tgidgd
1
200
Delta airlines®️ USA Contact Numbers: Complete 2025 Support Guide
airtravelguide
0
350
ABEMAの本番環境負荷試験への挑戦
mk2taiga
5
820
united airlines ™®️ USA Contact Numbers: Complete 2025 Support Guide
flyunitedhelp
1
470
IPA&AWSダブル全冠が明かす、人生を変えた勉強法のすべて
iwamot
PRO
2
220
QuickSight SPICE の効果的な運用戦略~S3 + Athena 構成での実践ノウハウ~/quicksight-spice-s3-athena-best-practices
emiki
0
260
AIの全社活用を推進するための安全なレールを敷いた話
shoheimitani
2
640
伴走から自律へ: 形式知へと導くSREイネーブリングによる プロダクトチームの信頼性オーナーシップ向上 / SRE NEXT 2025
visional_engineering_and_design
3
230
Featured
See All Featured
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Writing Fast Ruby
sferik
628
62k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
10
970
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
5.9k
YesSQL, Process and Tooling at Scale
rocio
173
14k
Six Lessons from altMBA
skipperchong
28
3.9k
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.4k
How to Think Like a Performance Engineer
csswizardry
25
1.7k
Side Projects
sachag
455
42k
Transcript
© 2017 VASILY,Inc. &5-πʔϧΛ͔Β࡞Γͨ͠ ୈճֶੜΤϯδχΞݶఆ-5େձʂʂʂ ాౡࠀ࠸
© 2017 VASILY,Inc. ࣗݾհ {:name “Katsuya Tajima” :twitter“@katsuyan121” :school {:name“芝浦工大”
:year 4} :intern “VASILY,Inc.” :langs [:Clojure :Go :Ruby] :likes [:Aikido :Shibainu]}
© 2017 VASILY,Inc. ໊ࣾגࣜձࣾ7"4*-: ϰΝγϦʔ 7"4*-: *OD ઃཱ݄ ॴࡏ౦ژ۠ޒాδχΞεϏϧ' ैۀһ໊
ࢿຊۚԯ දऔకۚࢁ༟थ औకࠓଜխઍ༿େี ओཁגओ άϩʔϏεϕϯνϟʔΩϟϐλϧ ҏ౻ςΫϊϩδʔϕϯνϟʔζ (.0ϕϯνϟʔύʔτφʔζ ,%%*גࣜձࣾ גࣜձࣾߨஊࣾ
© 2017 VASILY,Inc. Ҏ্ͷϑΝογϣϯ&$αΠτ͔ΒͷສΛ͑ΔΛܝࡌ ݄ؒສਓҎ্͕ར༻͢Δຊ࠷େڃͷϑΝογϣϯαΠτ
© 2017 VASILY,Inc. Google Play ϕετΞϓϦ J04ΞϓϦ ɾ"QQ4UPSF#&45 ɾ&TTFOUJBMೝఆ "OESPJEΞϓϦ
ɾ(PPHMF1MBZϕετΞϓϦ ɾ(PPHMF1MBZϕετΞϓϦ ɾ(PPHMF1MBZϕετΞϓϦ ɹɹϕετΠϊϕʔςΟϒΞϓϦେ ɾ(PPHMF͔Βτοϓσϕϩούʔೝఆ ࿈ଓϕετΞϓϦड ੈքͰ7"4*-:͚ͩ
© 2017 VASILY,Inc. 7"4*-:ΠϯλʔϯͰ͖ͬͯͨ͜ͱ ‣ &5-πʔϧͷ࡞Γ͠ ‣ %#Ҡߦ .POHP%#ͷ໓
‣ $JSDMF$*ͷΞοϓσʔτ ‣ ΫϩʔϥʔͷҰ෦ίϯςφԽ
© 2017 VASILY,Inc. 7"4*-:ΠϯλʔϯͰ͖ͬͯͨ͜ͱ ‣ &5-πʔϧͷ࡞Γ͠ ‣ %#Ҡߦ .POHP%#ͷ໓
‣ $JSDMF$*ͷΞοϓσʔτ ‣ ΫϩʔϥʔͷҰ෦ίϯςφԽ
© 2017 VASILY,Inc. &5-ͱ ‣ &YUSBDU5SBOTGPSN-PBE ‣ &YUSBDUใݯ͔ΒσʔλΛநग़ ‣ 5SBOTGPSNσʔλΛมɾՃ
‣ -PBE֨ೲઌʹσʔλΛϩʔυ
© 2017 VASILY,Inc. ࠓճΓ͔ͨͬͨ͜ͱ 3%4 .Z42- ͔Β#JH2VFSZͷσʔλಉظ
© 2017 VASILY,Inc. چ&5-πʔϧ σʔλಉظͷྲྀΕ .Z42-͔ΒεΩʔϚऔಘ #JH2VFSZʹσʔλಉظ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF
© 2017 VASILY,Inc. چ&5-πʔϧ σʔλಉظͷྲྀΕ .Z42-͔ΒεΩʔϚऔಘ #JH2VFSZʹσʔλಉظ σʔλͷ૿Ճʹ͑ΒΕ ͳ͘ͳͬͨ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF
© 2017 VASILY,Inc. ৽&5-πʔϧ σʔλಉظͷྲྀΕ .Z42-͔ΒεΩʔϚऔಘ ઃఆϑΝΠϧΛ࡞ :".- &NCVMLΛ࣮ߦͯ͠σʔλಉظ
&NCVMLΛར༻ʂʂ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#:
© 2017 VASILY,Inc. ᶃ&NCVMLʹΑΔσʔλಉظ ͜ͷ෦⾪ :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#:
© 2017 VASILY,Inc. &NCVMLͱ ‣ σʔλసૹϑϨʔϜϫʔΫ ‣ 'MVFOUEͷόον൛ͷΑ͏ͳπʔϧ ‣ ϓϥάΠϯʹΑΓෳͷೖྗɾग़ྗʹରԠ
‣ ೖྗɾग़ྗ࣌ͷΤϥʔॲཧͳͲΛߦͬͯ͘ΕΔ
© 2017 VASILY,Inc. &NCVMLͷ͍ํ ‣ :".-ܗࣜͰઃఆϑΝΠϧهड़ ‣ &NCMVLΛ࣮ߦ ‣ FNCVMLSVOʙZNM
:.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#:
© 2017 VASILY,Inc. &NCVMLઃఆϑΝΠϧͷྫ in: type: mysql host: ϗετ໊ user:
Ϣʔβʔ໊ password: ύεϫʔυ database: ରͷσʔλϕʔε table: ରͷςʔϒϧ select: "*" out: type: bigquery mode: replace auth_method: json_key json_keyfile: json_keyfileͷύε project: ϓϩδΣΫτ໊ dataset: σʔληοτ໊ table: ςʔϒϧ໊ gcs_bucket: GCSͷόέοτ໊ auto_create_gcs_bucket: false compression: GZIP source_format: NEWLINE_DELIMITED_JSON .Z42-ˠ#JH2VFSZͷ߹ FNCVMLJOQVUNZTRM FNCVMLPVUQVUCJHRVFSZ
© 2017 VASILY,Inc. ᶄ(Pϥούʔ ͜ͷ෦⾪ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#:
© 2017 VASILY,Inc. &NCVML͚ͩͰ࣮ݱͰ͖ͳ͍͜ͱ ‣ ෳςʔϒϧͷಉظ ‣ ςʔϒϧʹରͯͭ͠ઃఆϑΝΠϧ͕ඞཁ ‣ ॊೈͳϩάऩू
‣ Τϥʔൃੜ࣌ͷ௨ॲཧ
© 2017 VASILY,Inc. (Pϥούʔͷׂ ‣ ઃఆϑΝΠϧͷੜ ‣ &NCVMLͷ࣮ߦ ‣ ϩάऩू
‣ Τϥʔൃੜ࣌ͷॲཧ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#: 4BMBEJDPO*DPOTNBEFCZ.BEFCZPMJWFSGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF
© 2017 VASILY,Inc. ࠷ऴߏ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#: 4BMBEJDPO*DPOTNBEFCZ.BEFCZPMJWFSGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF
© 2017 VASILY,Inc. ͬͱศརʹ IUUQUFDIWBTJMZKQFOUSZEJHEBH@BOE@FNCVML
© 2017 VASILY,Inc. Πϯλʔϯืूத IUUQTXXXXBOUFEMZDPNQSPKFDUT