Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ETLツールを1から作り直した話
Search
katsuyan
September 02, 2017
Technology
1
320
ETLツールを1から作り直した話
ETLツールを1から作り直した話(embulkを利用したETLツール)
katsuyan
September 02, 2017
Tweet
Share
More Decks by katsuyan
See All by katsuyan
realtime_marketing_system
katsuyan
0
2.6k
digdag-3tips
katsuyan
2
4.6k
ファッションチェックランキングRubyKaigiの裏側 / Fashion check ranking app for RubyKaigi2019
katsuyan
1
5.9k
embulkを利用した開発でちょっと困ったこと
katsuyan
1
490
Other Decks in Technology
See All in Technology
Android Studio の 新しいAI機能を試してみよう / Try out the new AI features in Android Studio
yanzm
0
260
R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localization
takmin
0
430
実践アプリケーション設計 ③ドメイン駆動設計
recruitengineers
PRO
1
150
Postman MCP 関連機能アップデート / Postman MCP feature updates
yokawasa
0
140
ソフトウェア エンジニアとしての 姿勢と心構え
recruitengineers
PRO
2
600
制約理論(ToC)入門
recruitengineers
PRO
2
250
Backboneとしてのtimm2025
yu4u
4
1.4k
攻撃と防御で実践するプロダクトセキュリティ演習~導入パート~
recruitengineers
PRO
1
140
LLM時代の検索とコンテキストエンジニアリング
shibuiwilliam
2
1.1k
Amazon Bedrock AgentCore でプロモーション用動画生成エージェントを開発する
nasuvitz
6
420
人と組織に偏重したEMへのアンチテーゼ──なぜ、EMに設計力が必要なのか/An antithesis to the overemphasis of people and organizations in EM
dskst
5
600
mruby(PicoRuby)で ファミコン音楽を奏でる
kishima
1
220
Featured
See All Featured
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.8k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
570
How to Ace a Technical Interview
jacobian
279
23k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
The Straight Up "How To Draw Better" Workshop
denniskardys
236
140k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1.1k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
Music & Morning Musume
bryan
46
6.7k
How GitHub (no longer) Works
holman
315
140k
Transcript
© 2017 VASILY,Inc. &5-πʔϧΛ͔Β࡞Γͨ͠ ୈճֶੜΤϯδχΞݶఆ-5େձʂʂʂ ాౡࠀ࠸
© 2017 VASILY,Inc. ࣗݾհ {:name “Katsuya Tajima” :twitter“@katsuyan121” :school {:name“芝浦工大”
:year 4} :intern “VASILY,Inc.” :langs [:Clojure :Go :Ruby] :likes [:Aikido :Shibainu]}
© 2017 VASILY,Inc. ໊ࣾגࣜձࣾ7"4*-: ϰΝγϦʔ 7"4*-: *OD ઃཱ݄ ॴࡏ౦ژ۠ޒాδχΞεϏϧ' ैۀһ໊
ࢿຊۚԯ දऔకۚࢁ༟थ औకࠓଜխઍ༿େี ओཁגओ άϩʔϏεϕϯνϟʔΩϟϐλϧ ҏ౻ςΫϊϩδʔϕϯνϟʔζ (.0ϕϯνϟʔύʔτφʔζ ,%%*גࣜձࣾ גࣜձࣾߨஊࣾ
© 2017 VASILY,Inc. Ҏ্ͷϑΝογϣϯ&$αΠτ͔ΒͷສΛ͑ΔΛܝࡌ ݄ؒສਓҎ্͕ར༻͢Δຊ࠷େڃͷϑΝογϣϯαΠτ
© 2017 VASILY,Inc. Google Play ϕετΞϓϦ J04ΞϓϦ ɾ"QQ4UPSF#&45 ɾ&TTFOUJBMೝఆ "OESPJEΞϓϦ
ɾ(PPHMF1MBZϕετΞϓϦ ɾ(PPHMF1MBZϕετΞϓϦ ɾ(PPHMF1MBZϕετΞϓϦ ɹɹϕετΠϊϕʔςΟϒΞϓϦେ ɾ(PPHMF͔Βτοϓσϕϩούʔೝఆ ࿈ଓϕετΞϓϦड ੈքͰ7"4*-:͚ͩ
© 2017 VASILY,Inc. 7"4*-:ΠϯλʔϯͰ͖ͬͯͨ͜ͱ ‣ &5-πʔϧͷ࡞Γ͠ ‣ %#Ҡߦ .POHP%#ͷ໓
‣ $JSDMF$*ͷΞοϓσʔτ ‣ ΫϩʔϥʔͷҰ෦ίϯςφԽ
© 2017 VASILY,Inc. 7"4*-:ΠϯλʔϯͰ͖ͬͯͨ͜ͱ ‣ &5-πʔϧͷ࡞Γ͠ ‣ %#Ҡߦ .POHP%#ͷ໓
‣ $JSDMF$*ͷΞοϓσʔτ ‣ ΫϩʔϥʔͷҰ෦ίϯςφԽ
© 2017 VASILY,Inc. &5-ͱ ‣ &YUSBDU5SBOTGPSN-PBE ‣ &YUSBDUใݯ͔ΒσʔλΛநग़ ‣ 5SBOTGPSNσʔλΛมɾՃ
‣ -PBE֨ೲઌʹσʔλΛϩʔυ
© 2017 VASILY,Inc. ࠓճΓ͔ͨͬͨ͜ͱ 3%4 .Z42- ͔Β#JH2VFSZͷσʔλಉظ
© 2017 VASILY,Inc. چ&5-πʔϧ σʔλಉظͷྲྀΕ .Z42-͔ΒεΩʔϚऔಘ #JH2VFSZʹσʔλಉظ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF
© 2017 VASILY,Inc. چ&5-πʔϧ σʔλಉظͷྲྀΕ .Z42-͔ΒεΩʔϚऔಘ #JH2VFSZʹσʔλಉظ σʔλͷ૿Ճʹ͑ΒΕ ͳ͘ͳͬͨ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF
© 2017 VASILY,Inc. ৽&5-πʔϧ σʔλಉظͷྲྀΕ .Z42-͔ΒεΩʔϚऔಘ ઃఆϑΝΠϧΛ࡞ :".- &NCVMLΛ࣮ߦͯ͠σʔλಉظ
&NCVMLΛར༻ʂʂ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#:
© 2017 VASILY,Inc. ᶃ&NCVMLʹΑΔσʔλಉظ ͜ͷ෦⾪ :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#:
© 2017 VASILY,Inc. &NCVMLͱ ‣ σʔλసૹϑϨʔϜϫʔΫ ‣ 'MVFOUEͷόον൛ͷΑ͏ͳπʔϧ ‣ ϓϥάΠϯʹΑΓෳͷೖྗɾग़ྗʹରԠ
‣ ೖྗɾग़ྗ࣌ͷΤϥʔॲཧͳͲΛߦͬͯ͘ΕΔ
© 2017 VASILY,Inc. &NCVMLͷ͍ํ ‣ :".-ܗࣜͰઃఆϑΝΠϧهड़ ‣ &NCMVLΛ࣮ߦ ‣ FNCVMLSVOʙZNM
:.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#:
© 2017 VASILY,Inc. &NCVMLઃఆϑΝΠϧͷྫ in: type: mysql host: ϗετ໊ user:
Ϣʔβʔ໊ password: ύεϫʔυ database: ରͷσʔλϕʔε table: ରͷςʔϒϧ select: "*" out: type: bigquery mode: replace auth_method: json_key json_keyfile: json_keyfileͷύε project: ϓϩδΣΫτ໊ dataset: σʔληοτ໊ table: ςʔϒϧ໊ gcs_bucket: GCSͷόέοτ໊ auto_create_gcs_bucket: false compression: GZIP source_format: NEWLINE_DELIMITED_JSON .Z42-ˠ#JH2VFSZͷ߹ FNCVMLJOQVUNZTRM FNCVMLPVUQVUCJHRVFSZ
© 2017 VASILY,Inc. ᶄ(Pϥούʔ ͜ͷ෦⾪ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#:
© 2017 VASILY,Inc. &NCVML͚ͩͰ࣮ݱͰ͖ͳ͍͜ͱ ‣ ෳςʔϒϧͷಉظ ‣ ςʔϒϧʹରͯͭ͠ઃఆϑΝΠϧ͕ඞཁ ‣ ॊೈͳϩάऩू
‣ Τϥʔൃੜ࣌ͷ௨ॲཧ
© 2017 VASILY,Inc. (Pϥούʔͷׂ ‣ ઃఆϑΝΠϧͷੜ ‣ &NCVMLͷ࣮ߦ ‣ ϩάऩू
‣ Τϥʔൃੜ࣌ͷॲཧ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#: 4BMBEJDPO*DPOTNBEFCZ.BEFCZPMJWFSGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF
© 2017 VASILY,Inc. ࠷ऴߏ 'JMFJDPO*DPOTNBEFCZ'SFFQJLGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF :.-JDPO*DPOTNBEFCZ:BOOJDLGSPNXXXqBUJDPODPNJTMJDFOTFECZ$$#: 4BMBEJDPO*DPOTNBEFCZ.BEFCZPMJWFSGSPNXXXqBUJDPODPNJTMJDFOTFECZ'MBUJDPO#BTJD-JDFOTF
© 2017 VASILY,Inc. ͬͱศརʹ IUUQUFDIWBTJMZKQFOUSZEJHEBH@BOE@FNCVML
© 2017 VASILY,Inc. Πϯλʔϯืूத IUUQTXXXXBOUFEMZDPNQSPKFDUT