Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyconjp / 20170909

Jupyter + BigQuery ʹΑΔ 
σʔλ෼ੳج൫ ͷ DevˍOps
2017-09-09 (Sat) 
4:15 p.m. – 4:45 p.m. in Room 201 
PyCon JP 2017 
presented by @yuzutas0 
https://www.pexels.com/photo/close-up-of-computer-keyboard-257949/ɹhttps://www.pexels.com/photo/technology-computer-lines-board-50711/ɹhttps://www.pexels.com/photo/black-and-white-business-chart-computer-241544/

View Slide

ຊ೔ͷࢿྉ͸WEBʹެ։͠·͢ 
#pyconjp #pyconjp_201 

ࡱӨɾه࿥ͷඞཁ͸͋Γ·ͤΜ 
ͭ͘Ζ͍Ͱௌ͍͍͚ͯͨͩΕ͹ͱࢥ͍·͢

View Slide

εϥΠυ 150+ / 30min 
ʢ஌ݟΛ͕ͬͭΓڞ༗͠·͢ʣ
 
͕Μ͕Μߦ͖·͢ʂ

View Slide

ɹSho Yokoyamaɹ@yuzutas0
ɹɹ
 
 

● Recruit Technologies Co., Ltd.  
ɹɹάϧʔϓͷITࢪࡦશൠΛ୲͏ byʰϦΫϧʔτɺਐԽΛࢭΊͳ͍ITݱ৔ྗʱ
● Site Reliability Engineering 
ɹɹج൫γεςϜͷӡ༻ɾվળ
● Finance & Econometrics (Before) 
ɹɹRʹΑΔ౷ܭॲཧ - ͋ͷͱ͖Jupyter͕͋Ε͹Α͔ͬͨͷʹʂ

View Slide

https://github.com/yuzutas0

෼ੳج൫
ɹޱ΋ख΋ಈ͔͢೿

View Slide

ϓϩμΫτ։ൃ ʹ͓͚Δ 
σʔλ׆༻ ͷ ࢀߟࣄྫ ͷڞ༗

ɹຊ೔ͷΰʔϧ

View Slide

Python Λ׆༻ͯ͠ 
։ൃݱ৔ ʹ σʔλจԽ Λਁಁ͍ͤͨ͞ 
ιϑτ΢ΣΞΤϯδχΞ

ɹ૝ఆ͢Δର৅

View Slide

ɹɹɹ
ɹPythonͷ࿩
ɹɹɹ
"ɹPythonΛۀ຿Ͱ࢖͏࿩

ɹPythonͷConferenceͰ͕͢
த൫·Ͱग़͖ͯ·ͤΜʢ׼ʣ

View Slide

1. ϓϩμΫτ / νʔϜ
 
2. ෼ੳج൫ͷDev&Ops
ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε
 
3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

ɹΞδΣϯμ

View Slide

ࠗ׆ ࿀׆

View Slide

ֹ݄՝ۚϞσϧ 
※࿀݁ͼঁੑϢʔβʔ͸ຊਓ֬ೝྉͷΈ

ɹ͝ར༻ͷྲྀΕ

View Slide

※αϒγεςϜͷٕज़ཁૉ: Golang, Groovy, Python, Ruby, AWS, GCP
ɹɹɹɹɹɹɹɹɹɹɹɹΦϯϓϨ

ɹ֤छσόΠεରԠ - γεςϜུ֓ (AP)

View Slide

※ถࠃͰ͸݁ࠗͨ͠Χοϓϧͷ1/3͕ΦϯϥΠϯͰͷग़ձ͍

ɹσʔςΟϯάࢢ৔ͷ֦େ

View Slide

ɹࣄۀ੒௕ → εςʔΫϗϧμʔͷଟ༷Խ
σβΠϯ
ηΩϡϦςΟ
๏຿
Πϯϑϥ
SRE
ΞϓϦج൫
ू٬
ΧελϚʔ 
αϙʔτ
σʔλ 
αΠΤϯε
ػցֶश 
ΤϯδχΞ
։ൃӡ༻ 
νʔϜA
։ൃӡ༻ 
νʔϜB
։ൃӡ༻ 
νʔϜC
PO PO PO
σΟϨΫγϣϯ
޿ใ

View Slide

 
 

ɹΞδΣϯμ

View Slide

ɹ֤෦ॺ΁ͷσʔλఏڙ
σʔλج൫

View Slide

 
 

ɹΞδΣϯμ

View Slide

ɹͲͷΑ͏ʹ׆༻͍ͯ͠Δ͔
σʔλج൫

View Slide

ɹσʔλ׆༻ࣄྫ
ɹ1. KPIϞχλϦϯάɹɹ 
ɹ2. ϢʔβʔߦಈͷՄࢹԽ 
ɹ3. ࢪࡦޮՌͷ༧ଌɾܭଌ
Ϗδωε
4. ػցֶशʹΑΔϨίϝϯυ 
5. ۀऀɾεύϜͷࣗಈ൑ఆ
ϓϩμΫτ
ɹ6. ໰͍߹Θͤͷ܏޲෼ੳ 
ɹ7. ޿ࠂ഑৴ͷࣗಈ࠷దԽ 
ɹ8. ϓϨεϦϦʔε
ΧελϚʔνϟωϧ
ɹ9. γεςϜϞχλϦϯά 
ɹ10. ো֐ൃੜ࣌ͷӨڹௐࠪ
γεςϜ
ɹ11. νʔϜͷΩϟύγςΟ
νʔϜ

View Slide

Ϗδωεࢦඪʢച্΍DAUʣͷਪҠ 
 
ຖேͷSlack௨஌ɹɹɹɹSpreadsheetɹɹɹɹμογϡϘʔυ

ɹɹɹKPIϞχλϦϯά
1

View Slide

ओཁಋઢʢձһొ࿥ˠϝοηʔδʣͷར༻ɾ཭୤
ߦಈϑΝϯωϧɹɹɹɹɹɹδϟʔχʔϚοϓ

ɹɹɹϢʔβʔߦಈͷՄࢹԽ
2

View Slide

Split (AB) ςετ݁Ռɹɹ৽ػೳͷར༻౓߹͍

ɹɹɹࢪࡦͷޮՌ༧ଌˍܭଌ
3

View Slide

ɹɹɹػցֶशʹΑΔϨίϝϯυ
4

View Slide

ɹɹɹۀऀɾεύϜͷࣗಈ൑ఆ
5

View Slide

ɹɹɹ໰͍߹Θͤͷ܏޲෼ੳ
6

View Slide

ɹɹɹ޿ࠂ഑৴ͷࣗಈ࠷దԽ
7

View Slide

σʔλ෼ੳ݁Ռͷެ։ → ࢢ৔ͷ׆ੑԽʹߩݙ

ɹɹɹϓϨεϦϦʔε
8
[email protected]

View Slide

Ϋϥογϡ཰΍ϨεϙϯελΠϜ

ɹɹɹγεςϜϞχλϦϯά
9
IUUQZV[VUBTIBUFOBCMPHDPNFOUSZ

View Slide

ɹɹɹো֐ൃੜ࣌ͷӨڹௐࠪ
10

View Slide

εΫϥϜνʔϜͷϕϩγςΟ ΍ νέοτফԽͷϝτϦΫε

ɹɹɹνʔϜͷΩϟύγςΟ
11

View Slide

σβΠϯ
ηΩϡϦςΟ
๏຿
Πϯϑϥ
SRE
ΞϓϦج൫
ू٬
ΧελϚʔ 
αϙʔτ
σʔλ 
αΠΤϯε
ػցֶश 
ΤϯδχΞ
։ൃӡ༻ 
νʔϜA
։ൃӡ༻ 
νʔϜB
։ൃӡ༻ 
νʔϜC
PO PO PO
σΟϨΫγϣϯ
޿ใ
։ൃӡ༻ 
νʔϜB
PO

ɹ͜ΕΒͷࢪࡦ : ֤෦ॺ͕ϘτϜΞοϓͰਪਐ

View Slide

σʔλ׆༻Ͱ࣮ݱͰ͖Δ͜ͱ

ɹސ٬Ձ஋ͷ௥ٻ → σʔλ׆༻͸ඞવ
1. ࣗಈԽ 2. ՄࢹԽ
ސ٬Ձ஋ͷఏڙ

View Slide

“ΑΓૣ͘ɾ҆ఆͨ͠” αʔϏεͷఏڙ

ɹ1. ࣗಈԽ
ػցֶशͰީิऀϦετΛࣄલநग़
໨ࢹ͚ͩͰۀऀɾεύϜΛ൑ఆ
[email protected] [email protected]

View Slide

“ΑΓޮՌతͳ” ΤϯδχΞϦϯάͷ࣮ݱ

ɹ2. ՄࢹԽ
͜ͷσβΠϯ͸Ϣʔβʔͷ 
ຊԻʹ౴͍͑ͯΔͷ͔ʁ
͜ͷػೳ௥Ճ͸ޮՌ͕͋Δͷ͔ʁ 
࣮ࡍʹޮՌ͸͋ͬͨͷ͔ʁ
IUUQHBUBHOFUGSFFJMMVTUSBJUJPO [email protected]

View Slide

σβΠϯ
ηΩϡϦςΟ
๏຿
Πϯϑϥ
SRE
ΞϓϦج൫
ू٬
ΧελϚʔ 
αϙʔτ
σʔλ 
αΠΤϯε
ػցֶश 
ΤϯδχΞ
։ൃӡ༻ 
νʔϜA
։ൃӡ༻ 
νʔϜB
։ൃӡ༻ 
νʔϜC
PO PO PO
σΟϨΫγϣϯ
޿ใ
։ൃӡ༻ 
νʔϜB
PO

ɹ֤෦ॺʹΑΔੵۃతͳσʔλ׆༻

View Slide

 
 

ɹΞδΣϯμ

View Slide

ɹͳͥج൫͕ඞཁʹͳ͔ͬͨ
σʔλج൫

View Slide

DBσʔλΛ࢖͍͍ͨͷͰ 
೔࣍ͰS3ʹCSVग़ྗ͍ͯͩ͘͠͞ 

ɹ࠷ॳ͸খ͞ͳσʔλૄ௨ґཔ
෦ॺA

View Slide

ɹεέʔϧ → ֤෦ॺ͔Βґཔ
෦ॺB
෦ॺC
෦ॺD

View Slide

ͬͪ͜Ͱαʔόͱૄ௨πʔϧΛ༻ҙ͠·͢ 
ϨϏϡʔˍຊ൪؀ڥʹஔ͍ͯ΋Β͑·͔͢

ɹͪΐͬͱֻ͕͔࣌ؒΓͦ͏
ੜσʔλΛ౉ͯ͠΋Β͑Ε͹ 
ͬͪ͜Ͱ࢖͑ΔΑ͏ʹՃ޻͢ΔͷͰ
෦ॺE

View Slide

ɹ֤෦ॺ͝ͱʹσʔλૄ௨
෦ॺA 
πʔϧ
෦ॺB 
πʔϧ
෦ॺA 
αʔό
෦ॺB 
αʔό

View Slide

ɹ֤෦ॺ͝ͱʹσʔλՃ޻
෦ॺA 
நग़σʔλ
෦ॺB 
நग़σʔλ
෦ॺA 
Ճ޻ϩδοΫ
෦ॺB 
Ճ޻ϩδοΫ

View Slide

ɹମ੍ͷ··ʹγεςϜෳࡶԽ - ίϯ΢ΣΠͷ๏ଇ
ݩσʔλ
֤෦ॺ޲͚ʹσʔλࢀরγεςϜΛܧ͗଍͠

View Slide

 
http://jp.techcrunch.com/2017/06/25/20170623ﬁve-building-blocks-of-a-data-driven-culture/
ελοϑ͕ͨͪҟͳΔγεςϜ͔ΒࣅͨΑ͏ͳϝτϦοΫΛ 
Ҿ͖ग़͢͜ͱ͕Ͱ͖ͯ͠·͏ͳΒɺඞવతʹͦΕΒͷγεςϜ͔Β͸ 
ҟͳΔ਺ࣈ͕ੜ੒͞ΕΔɻ…ΑΓ·͠ͳ৘ใݯΛ࢖͑Δഺͩͬͨͷʹɺ 
ݹͯ͘඼࣭ͷ௿͍ɺ͋Δ͍͸ؒҧͬͨσʔλ΍ࢦඪΛ஌Βͣʹ 
࢖ͬͯ͠·͍ɺѱ͍ܾఆΛԼ͢νʔϜ΋ग़ͯ͠·͏͔΋஌Εͳ͍ɻ

ɹ෦ॺ͝ͱʹʮച্ʯ͕ζϨΔ໰୊

View Slide

ɹΤϯδχΞʹௐࠪґཔᶃ - σʔλ࢓༷͸೔ʑมΘΔ
͜ͷϨίʔυ͕ 
௥Ճ͞ΕΔλΠϛϯάɺ 
࠷ۙมߋ͞Ε·ͨ͠ʁ
ͪΐͬͱมߋཤྺΛ 
௥ͬͯΈ·͢Ͷ
෦ॺF ΤϯδχΞ

View Slide

ɹΤϯδχΞʹௐࠪґཔᶄ - Өڹൣғ͸೔ʑ޿͕Δ
Re:dashͷ਺ࣈ͕ 
ζϨΔΜͰ͚͢Ͳ
ଞ෦ॺ͕࡞ͬͨγεςϜΛ 
ௐ΂Δͱ͜Ζ͔Β……
෦ॺG ΤϯδχΞ

View Slide

ɹݸผରԠͷѱ॥؀
ಠࣗσʔλૄ௨ͷґཔ 
ʢ͜ΕͳΒ͙͢Ͱ͖·͢ΑͶʣ
ΤϯδχΞνʔϜͷෛՙˢ
γεςϜෳࡶԽˢ
ௐࠪґཔ

View Slide

ɹ୭͔ͷ൑அ͕ѱ͔ͬͨΘ͚Ͱ͸ͳ͍
εέʔϧ΍ຊ֨Քಇ࣌ʹ 
໰୊͕ݦࡏԽ
ɹɹ·ͣ͸খ͘͞ࢼ͢ 
ɹ ● ࠷খݶͷσʔλૄ௨ 
ɹ ● ࣗલͰσʔλՃ޻
෦ॺ͝ͱʹ 
৽πʔϧಋೖ΍ࢪࡦணख
[email protected]
[email protected] [email protected]

View Slide

 
http://jp.techcrunch.com/2017/06/25/20170623ﬁve-building-blocks-of-a-data-driven-culture/

ɹମ੍֦େ → σʔλ΢ΣΞϋ΢εͷඞཁੑ
ࣄ࣮ͷ୯Ұ৘ใݯΛ͍࣋ͬͯΔ৔߹ʹ͸ɺΞφϦετ΍ 
ଞͷҙࢥܾఆऀͱ͍ͬͨΤϯυϢʔβʔͨͪʹɺ༏ΕͨՁ஋Λ 
ఏڙ͢Δ͜ͱ͕Ͱ͖Δɻ൴Β͸૊৫಺ͰσʔλΛ୳͕࣌ؒ͢ 
গͳͯ͘ࡁΉΑ͏ʹͳΓɺσʔλͷར༻ʹΑΓଟ͘ͷ࣌ؒΛ 
ׂ͘͜ͱ͕Ͱ͖ΔΑ͏ʹͳΔ͔Βͩɻ

View Slide

 
 

ɹΞδΣϯμ

View Slide

ɹͲͷΑ͏ͳγεςϜΛߏங͔ͨ͠
σʔλج൫

View Slide

1. ModelͱViewΛ෼͚Δ΂͠ 
2. ͳΔ΂͘ϥΫͯ͠࡞Δ΂͠

ɹઃܭϙϦγʔ

View Slide

ɹModelͱViewΛ෼͚Δ΂͠
ɹɹɹɹɹɹɹɹɹɹɹɹɹDWH
੹຿ʹԠͨ͡γεςϜͷ෼཭ɾ݁߹ 
Model (஝ੵ/Ճ޻) ͱView (ࢀর)

View Slide

ɹ஫ҙɿ෦ॺ͝ͱʹ࠷దͳView͸ҟͳΔ
Excel
ؾܰʹ਺ࣈΛม͑ͯ 
γϛϡϨʔγϣϯ
Ϗδωε෦໳
Tableau
ߴՁ֨ɾߴػೳ 
෼ੳཁٻʹରԠ
Re:dash
SQL͕ॻ͚Δͻͱ 
͓खܰར༻
σΟϨΫλʔ
Jupyter 
GitͰίʔυ؅ཧ 
ϓϩάϥϜͷԸܙ
ΤϯδχΞ
෼ੳ෦໳

View Slide


ɹઃܭϙϦγʔ

View Slide

ɹจ໌ͷརث
σʔλૢ࡞ͳΒPythonҰ୒
ΤίγεςϜʹ৐Δ
σʔλอଘͳΒBigQueryҰ୒ 
GoogleʹཔΔ
ΞϓϦ૚ Πϯϑϥ

View Slide

ɹɹɹ● ʢRͱൺ΂ͯʣॻ͖΍͘͢ಡΈ΍࣮͘͢ߦ͠΍͍͢ 
ɹɹɹ● ൚༻తͳϓϩάϥϜॲཧΛॻ͚ΔεΫϦϓτݴޠ

ɹPythonͷ࠾୒ཧ༝ - ൚༻LL

View Slide

 
ɹɹɹɹ● ʢଞͷ൚༻ݴޠʹൺ΂ͯʣσʔλͷՃ޻΍ඳը͕ಘҙ 
ɹɹɹɹ● DB㱻Dataframeม׵ͳͲଞγεςϜ࿈ܞ΋༰қ

ɹPythonͷ࠾୒ཧ༝ - σʔλૢ࡞޲͚ϥΠϒϥϦ

View Slide

 
ɹɹɹ● σόοά΍σʔλͷՄࢹԽ͕ίϚϯυ1ͭͰՄೳ 
ɹɹɹ● ಺෦తʹ͸json → GitͰࠩ෼؅ཧ → ࠶ݱՄೳͳه࿥Λ࢒ͤΔ

ɹPythonͷ࠾୒ཧ༝ - ࠷ڧͷ࣮ߦ؀ڥ

View Slide

ػցֶशࢪࡦεΫϦϓτ
 
Ҡ২Մೳ → γεςϜͷ෼཭ɾ݁߹͕༰қ → རศੑɾอकੑ

ɹPythonͷ࠾୒ཧ༝ - طଘεΫϦϓτͱͷ૬ੑ

View Slide

ɹจ໌ͷརث
ΤίγεςϜʹ৐Δ
GoogleʹཔΔ
Πϯϑϥ
ΞϓϦ૚

View Slide

ɹBQͷ࠾୒ཧ༝ɿ͍҆
IUUQZBQDBTJBPSHUBMLTIPXDFBGBDFDBCEDB
● Googleͷσʔληϯλʔ 
ɹεέʔϧϝϦοτʹΑΔ 
ɹίετ༏Ґ
● ϢʔβʔͷҙݟΛ΋ͱʹ 
ɹྉۚϓϥϯΛݟ௚ͨ͠ܦҢ

View Slide

ɹBQͷ࠾୒ཧ༝ɿૣ͍
IUUQZBQDBTJBPSHUBMLTIPXDFBGBDFDBCEDB
● ΧϥϜϕʔεઃܭʹΑΔ 
ɹΞΫηεߴ଎Խ
● େن໛σʔλ׆༻࣌ʹ 
ɹϘτϧωοΫͱͳΔ 
ɹDisk I/Oͷฒྻॲཧ

View Slide

ɹBQͷ࠾୒ཧ༝ɿ҆৺
GCPɾGsuiteͰݖݶ؅ཧ - ΈΜͳ͕࢖͑Δ
SQLΠϯλʔϑΣʔε - ಺෦ॲཧɾอकӡ༻Λؾʹͤͣʹ࢖͑Δ
ඪ४SQL 
ΫΤϦͷҠ২ੑ͕ߴ͍ 
ֶशίετ͕খ͍͞ 
ςετ͠΍͍͢
पลπʔϧ 
ओཁBIπʔϧͳΒେ఍ܨ͕Δ 
pandas.io.gbq → Jupyterσόοά͕༰қ

View Slide

ɹจ໌ͷརث
ΤίγεςϜʹ৐Δ
GoogleʹཔΔ
Πϯϑϥ
ΞϓϦ૚

View Slide


ɹઃܭϙϦγʔ

View Slide

 
 

ɹΞδΣϯμ

View Slide

ɹσʔλϑϩʔͷߏங
ऩू Ճ޻ ׆༻
஝ੵ
ύΠϓϥΠϯ؅ཧ
Model View

View Slide

ऩू Ճ޻ ׆༻
஝ੵ
Model View
ऩू
ύΠϓϥΠϯ؅ཧ

View Slide

ɹσʔλऩू
ݸਓ৘ใ 
ϚεΫࡁΈView
Recruit Ads
Console
Recruit Clash 
Report
όον
ετϦʔϛϯά
WebAPI
εΫϨΠϐϯά
requests +
beautiful soup

View Slide

ओཁͳDBɾϩά͔Βͷసૹ → ࠷খઃఆͰ࣮ݱՄೳ

ɹσʔλసૹΛࢧ͑Δٕज़

View Slide

Python (requests + beautiful soup) ʹΑΔ 
WebAPIίʔϧɾը໘εΫϨΠϐϯά

ɹJupyterͰૉૣ͘ࢼ͢
·ͣ͸ϩʔΧϧ؀ڥͰࡁ·ͤΔ ܁Γฦ͠࢖͏ͳΒεΫϦϓτԽ
IUUQTXXXTIBSFJDPOOFUEPDVNFOUpMFQZ

View Slide

ΤϯδχΞ͕෼ੳʹܞΘ͍ͬͯͳ͍ͱEventσʔλ͕ϩάཁ͔݅Βൈ͚࿙Ε͕ͪ 
ෆཁσʔλͷա৒஝ੵʹΑΔετϨʔδѹഭΛݒ೦ʢͦ͜ͰBQͰ͢Αʣ

ɹσʔλੑ࣭ʹԠͨ͡ऩूํ๏
4UBUFঢ়ଶσʔλ &WFOUཤྺσʔλ
ྫ Ϣʔβʔͷ࠷ऴϩάΠϯ೔࣌ʢ্ॻ͖͞ΕΔʣ աڈͷϩάΠϯཤྺʢ஝ੵ͞ΕΔʣ
ओͳ༻్
ը໘දࣔ
 
ʮ͜ͷਓ͸೔લʹϩάΠϯ͠·ͨ͠Αʯ
෼ੳ
 
Ͳͷ͘Β͍ͷස౓ͰϩάΠϯ͢Δ͔
ڧΈɾಛ௃
ϓϩμΫτຊମͷύϑΥʔϚϯε޲্ 
͍͍ͪͪཤྺςʔϒϧͱ+PJOͯ͠ 
࠷৽ϨίʔυΛ୳͢ͷ͸%#ෛՙ
σʔλ෼ੳͷ࠶ݱੑΛ୲อ 
 
ຖճϩάΠϯ೔࣌Λ্ॻ͖͞ΕͨΒ 
෼ੳͰ͖ͳ͘ͳͬͯ͠·͏
෼ੳج൫Ͱͷ 
σʔλऩू
೔࣍ͰҰׅஔ͖׵͑
 
%#ϩάˠࠩ෼ه࿥ˠ׆༻͸ͻͱ޻෉ඞཁ
ࠩ෼൓ө
 
લճҎ߱ͷϨίʔυ͚ͩΛநग़

View Slide

ऩू Ճ޻ ׆༻
஝ੵ
Model View
஝ੵ
ύΠϓϥΠϯ؅ཧ

View Slide

3૚ߏ଄ 
ࢀߟʰ10೥ઓ͑Δσʔλ෼ੳೖ໳ - SQLΛ෢ثʹσʔλ׆༻࣌୅Λੜ͖ൈ͘ʱ

ɹσʔλ஝ੵ
ɹɹɹɹɹɹBigQuery - Google Cloud Platform
Source 
ݩσʔλͷίϐʔ
Warehouse 
ओཁࢦඪɾසग़σʔλ 
தؒςʔϒϧ
App 
֤πʔϧ͔Β௚઀ࢀর 
ඇٕज़ऀ޲͚IF
஝ੵ

View Slide

ྫɿ enmusubi__source__db
ΞϯμʔείΞ2ͭͰཁૉΛ۠੾Δ 
BEMʢCSSͷઃܭख๏ʣϥΠΫͳنଇ

ɹσʔληοτͷ໋໊نଇ
ϓϩμΫτ໊ source
warehouse
app
ϝΠϯDB
αʔόϩά
ΞΫηεղੳπʔϧ

View Slide

ɹσʔλࣙॻ
IUUQKQUFDIDSVODIDPNpWFCVJMEJOHCMPDLTPGBEBUBESJWFODVMUVSF
ར༻ऀ͸σʔλϑΟʔϧυͱͦͷ஋ʢϝτϦΫεʣͷҙຯΛ஌Δඞཁ͕͋Δɻ*
ݩσʔλ৘ใ΁ͷϦϯΫूͷΈఏڙ 
γεςϜʢʹσʔλ࢓༷ʣ͸೔ʑมԽ͢ΔͨΊ 
refs. ϩθολετʔϯ
֤σʔλιʔεଆͰ࢓༷؅ཧ 
e.g. ຊମDB → MySQL Workbench 
ɹERߏ੒/ίϝϯτ͸DDLͰৗʹ࠷৽Խ

View Slide

ൃల్্ͷϓϩμΫτɹ → ɹػೳ௥ՃͰසൟͳERมߋɹ → ɹBQ΋௥ਵ͍ͨ͠

ɹ஝ੵσʔλʹϚΠάϨʔγϣϯ͕ඞཁ

View Slide

ɹJupyter for Migration
1. ϩʔΧϧͷJupyterͰ 
ɹϚΠάϨʔγϣϯεΫϦϓτΛॻ͘
2. ͦͷ··JupyterͰ 
ɹ1೔෼ͷσʔλΛର৅ʹಈ࡞֬ೝ
3. PythonϑΝΠϧΛग़ྗ 
ɹ → αʔό্ͰશσʔλΛҠߦ
IUUQTXXXTIBSFJDPOOFUEPDVNFOUpMFQZ

View Slide

※ͨͩ͠Python2.7੍໿͕͋ΔͷͰҰ෦ॻ͖௚͕͠ඞཁ → ύλʔϯ͕ݟ͖͑ͯͨΒม׵εΫϦϓτ
● Φʔτεέʔϧ → Dataﬂow 
ɹGKE͸෼ࢄ޲͚ͷઃܭ͕ඞཁ 
● σʔλྔ͕গͳ͚Ε͹Datalab 
ɹJupyterͷίʔυ͕··࢖͑Δ
ɹGCPͷαʔόΛར༻
ɹ ● Googleઐ༻ઢͰBQΞΫηε 
ɹ ● ಉ͡ถࠃϦʔδϣϯʹཱͯΔ

ɹBQͰ100ԯϨίʔυΛҠߦ͢Δ
ϘτϧωοΫᶃ N/W ϘτϧωοΫᶄ ϝϞϦ

View Slide

ऩू Ճ޻ ׆༻
஝ੵ
Model View
Ճ޻
ύΠϓϥΠϯ؅ཧ

View Slide

3૚ߏ଄ 
ࢀߟʰ10೥ઓ͑Δσʔλ෼ੳೖ໳ - SQLΛ෢ثʹσʔλ׆༻࣌୅Λੜ͖ൈ͘ʱ

ɹσʔλՃ޻
ɹɹɹɹɹɹBigQuery - Google Cloud Platform
Source 
ݩσʔλͷίϐʔ
Warehouse 
ओཁࢦඪɾසग़σʔλ 
தؒςʔϒϧ
App 
֤πʔϧ͔Β௚઀ࢀর 
ඇٕज़ऀ޲͚IF
Ճ޻ Ճ޻

View Slide

IUUQTHSPXUIIBDLKPVSOBMDPNLQJUSFFGPSBQQ
ɹओཁࢦඪΛதؒςʔϒϧͰ؅ཧ

View Slide

ɹPythonεΫϦϓτʹΑΔՃ޻

View Slide

ɹsh → exec.py
● γΣϧҾ਺Λ΋ͱʹॲཧΛಈతৼΓ෼͚ 
● ݸʑͷΤϯυϙΠϯτ͸ࡉ෼Խ 
ɹˠ ΦʔτεέʔϧɺϦτϥΠɺಈ࡞֬ೝ͠΍͘͢

View Slide

ɹController
● ॲཧͷશମ૾Λهड़ 
● Facadeͱͯ͠ଞॲཧΛݺͼग़͢

View Slide

ɹModel
● Ϗδωεࢦඪͱ1ର1ͰରԠ͢ΔΫϥε 
● Create/ReadॲཧͷΈ → ஋͸BQ؅ཧ 
ɹ → ϓϩάϥϜ্͸εςʔτϨεʹอͭ

View Slide

ɹQuery
● BigQueryͱͷR/WΛѻ͏SQL 
● ςΩετॲཧͷࢧԉػೳΛ׆༻ 
ɹ → f-strings (3.6~)΍TypeHints (3.5~) 
● ϝιουͱͯ͠੾Γग़͠ 
ɹ → ஋ͷҾ͖౉͠ʹΑΔείʔϓ੍ޚ

View Slide

ɹEnum
● Enum(3.4~)Ͱσʔλݻ༗ͷίʔυ஋Λ؅ཧ 
● ੡඼ຊମ (Java) ͷEnumΛҠ২ → Dictܕʹม׵ 
● ܧঝݩͷbaseEnumΛ༻ҙ → ڞ௨ͷσʔλऔಘॲཧΛ࡞੒

View Slide

ɹResource
● ֎෦γεςϜͱͷ࿈ܞॲཧ 
● retryॲཧͷҰݩԽ → σʔλྔ͕૿͑ΔͱBQ͸Αࣦ͘ഊ͢ΔͷͰ 
● dotenvͰ؀ڥม਺ΛಡΈࠐΉ → ຊ൪ɾ։ൃͰSlackνϟϯωϧΛ෼͚Δ

View Slide

ऩू Ճ޻ ׆༻
஝ੵ
Model View
׆༻
ύΠϓϥΠϯ؅ཧ

View Slide

ɹσʔλ׆༻ - Pushܕ
IUUQGSFFJMMVTUSBUJPOTHBUBHOFUIUNM
ػցֶशࢪࡦ

View Slide

ΞυϗοΫ
ࣗಈԽ

ɹσʔλ׆༻ - Pullܕ

View Slide

ɹ୭ʹεΫϦϓτΛ༻ҙ͢Δ͔
Yes No
ͦͷ࡞ۀ͸ 
܁Γฦ͢ʁ
ϞχλϦϯά 
ఆظతͳॲཧ 
ʲPushܕʳʢҰ෦ྫ֎͋Γʣ
εφοϓγϣοτ 
ΞυϗοΫͳ෼ੳɾௐࠪ 
ʲPullܕʳ
SQLΘ͔Δ 
ͻͱɾ෦ॺʁ
ELTɿૄ௨ˠՃ޻ 
ʢྫʣػցֶशΤϯδχΞ 
ʲPullܕʳ
ETLɿՃ޻ˠૄ௨ 
SpreadsheetͰI/F 
ʲPushܕʳ
※Ճ޻ࡁΈσʔλʹରͯ͠γϯϓϧͳSELECTจΛൃߦ͢Δ͚ͩͰࡁΉέʔε͸ଟ͍

View Slide

ɹDecorator
● ֤View޲͚ͷσʔλՃ޻ʢग़ྗઌʹԠͨ͡ม׵ॲཧʹಛԽʣ 
● ྫɿຖேSlackʹ౤͛ΔάϥϑΛ matplotlib Ͱඳը

View Slide

ऩू Ճ޻ ׆༻
஝ੵ
Model View
ύΠϓϥΠϯ؅ཧ

View Slide

؅ཧπʔϧʹظ଴͢Δཁ݅ʢཧ૝ʣ
 
 

ɹσʔλύΠϓϥΠϯͷ؅ཧ
1. ίʔυͰόʔδϣϯ؅ཧ 
2. εέδϡʔϧࣗಈىಈ 
3. ηοτΞοϓ΍ϝϯςφϯεͷָ͞(DB಺แͳͲ) 
4. GUI/CUI྆ํͰϩάɾύϑΥʔϚϯεͷ؅ཧ 
5. GUI/CUI྆ํͰม਺ࢦఆͷ࠶࣮ߦ
6. ࣗಈϦτϥΠॲཧɾεΩοϓػೳ 
7. εΫϦϓτͷฒྻ࣮ߦ 
8. αʔόͷΦʔτεέʔϧ 
9. ༗޲ඇ८ճάϥϑͷࣗಈ࡞੒ 
10. Python3.6~ରԠʢςΩετॲཧ͕֨ஈʹָʣ

View Slide

࿩୊ͷπʔϧΛܰ͘৮ͬͯࢼͯ͠ΈͨΓ 
ɹɹɹɹɹɹ
 
ɹɹɹɹɹͲΕ΋ັྗత͚ͩͲҰேҰ୹ → ͔Ώ͍ͱ͜Ζʹख͕ಧ͔ͳ͍ 
ɹɹɹɹɹɹ ● ίϯτϦϏϡʔτͷνϟϯε͔΋ʁ 
ɹɹɹɹɹɹ ● ೔ਐ݄าͰັྗతͳͷͰ͍ͣΕҠ২͍ͨ͠ؾ࣋ͪ͸͋Δ

ɹWorkﬂow Engine

View Slide

ͳΜ͔ͩΜͩͰ࠷ॳʹ͓खܰߏஙͨ͠ jenkins Λ࢖͍ଓ͚͍ͯΔ 
 
ɹɹɹɹɹɹ● 2ܥ + BlueOcean + Pipeline of jenkinsﬁles 
ɹɹɹɹɹɹ● ϦτϥΠػߏɾॲཧ࠷దԽ͸ଞʹྼΔ 
ɹɹɹɹɹɹ● jenkinsδϣϒ͔ΒγΣϧܦ༝Ͱ֤छॲཧΛίʔϧ

ɹ҆ఆͷJenkins

View Slide

ɹσʔλϑϩʔͷߏஙʢ׬ʣ
ऩू Ճ޻ ׆༻
஝ੵ
Model View
ύΠϓϥΠϯ؅ཧ

View Slide

 
 

ɹΞδΣϯμ

View Slide

ɹͲͷΑ͏ͳϓϩηεΛ࠾༻͔ͨ͠
σʔλج൫

View Slide

ʮσʔλج൫Λ࡞Δͧʯܭը͸
ࣦഊ͔Β࢝·ͬͨ

View Slide

μογϡϘʔυΛ 
෦԰ͷϞχλʔʹө͢
ɹ࠷ॳͷҰา

View Slide

1. ຖ೔ͷKPIΛ͢Β͢Βݴ͑ΔΑ͏ʹͳΔ͔΋ 
 
ɹ2. ҟৗ஋ʹؾ͍ͮͯਝ଎ʹಈ͚ΔΑ͏ʹͳΔ͔΋
ɹߴ·Δظ଴

View Slide

1िؒͰ୭΋ݟͳ͘ͳͬͨ
ɹݱ࣮͸ඇ৘Ͱ͋Δ

View Slide

View Slide

ɹɹɹ׬ᘳͳاըॻʁ
ɹɹɹѹ౗తͳ༧ࢉʁ
ɹɹɹ1೥ؒͷ։ൃϓϩδΣΫτʁ

ɹʮ࠷ڧͷσʔλج൫ʯΛࢧ͑Δϓϩηεʁ

View Slide

ɹɹɹ׬ᘳͳاըॻʁ
ɹɹɹѹ౗తͳ༧ࢉʁ
ɹɹɹ1೥ؒͷ։ൃϓϩδΣΫτʁ

ɹʮ࠷ڧͷσʔλج൫ʯΛࢧ͑Δϓϩηεʁ
❌

View Slide

ɹɹ● ࢥ͍ͬͯͨͷͱҧͬͨ
ɹɹ● ྑͦ͞͏͚ͩͲ݁ہ࢖Θͳ͔ͬͨ

ɹ1೥ޙʹ଴ͭ΋ͷ

View Slide

ɹʮԶͷߟ͑ͨ࠷ڧͷσʔλج൫ʯ͸࢖ΘΕͳ͍
IUUQXXXQSPKFDUDBSUPPODPN

View Slide

→ ͜ΕͳΒ࣮֬ʹ࢖ͬͯ΋Β͑Δ

ɹগͣͭ͠ஔ͖׵͑Α͏
 
࣮͸͏ͪͷ෦ॺͰຖேSlackʹάϥϑΛ 
खಈͰྲྀ͍ͯ͠ΔΜ͚ͩͲʂ
 
͜ͷػʹࣗಈԽ͠·͔͢ʂ

View Slide

ɹɹɹQ.ɹ࠷ॳʹཁ݅Λશ෦ચ͍ग़ͤ͹OKʁ 
 
ɹɹɹA.ɹNGʂʮ΍Γ͍ͨ͜ͱʯ͸มΘΔʂ 
ɹɹɹɹɹɹɹ ● ଞࣾϒϩάΛݟͯʮ͏ͪ΋΍Γ͍ͨʂʯ 
ɹɹɹɹɹɹɹ ● ৽͍͠BIπʔϧͷఏҊʹʮ͜Ε͍͍ʂʯ 
ɹɹɹɹɹɹɹ ● ࣮ࡍʹը໘Λ࡞ͬͨΒʮͳΜ͔ҧ͏ʂʯ
 
ɹɹ ɹɹ → มԽʹॊೈʹରԠ͢Δʂ

ɹΠςϨʔγϣϯΛճ͢͜ͱ͕େࣄ

View Slide

TodoΛνέοτ؅ཧ → ॊೈʹ༏ઌॱҐΛೖΕସ͑Δ

ɹνέοτۦಈ։ൃ

View Slide

ɹλεΫͷ༏ઌॱҐ
1 σʔλ͕·͕͍ͪͬͯΔ 
ٙ࿭ͷௐࠪ
ɹ1ͭؒҧ͍ͬͯΔͱશ෦৴༻Ͱ͖ͳ͘ͳΔͷͰ࠷༏ઌରԠɻ 
ɹ͙͢ௐࠪ͢Ε͹ؔ܎ऀͷ৴༻࢒ߴΛ૿΍ͤΔɻ
2 Modelվम
ɹσʔλ߲໨௥ՃͳͲɻ 
ɹଟগݟʹͯ͘͘΋࢖͑Δঢ়ଶʹ͢Δ͜ͱ͕༏ઌɻ 
ɹ࠷ѱͷ৔߹Ͱ΋Ͳ͏ʹ͔࢖͏ଆͰ޻෉Ͱ͖ΔͷͰɻ
3 Viewվम
ɹάϥϑͷݟӫ͑ͳͲɻ 
ɹҰ൪ʮมԽ͍ͯ͠ΔʯΠϯύΫτ → ؔ܎ߏஙʹ͸د༩ɻ 
ɹ࢖͍खʹͱͬͯ͸ॏཁͳͷͰՄೳͳൣғͰαϙʔτɻ
4 γεςϜͷอकੑͱ 
ύϑΥʔϚϯε
ɹϘτϧωοΫʹͳͬͨΒରॲ͢Δɻ 
ɹ࠷ॳ͔ΒڽΔͱʮૣ͗͢Δ࠷దԽʯʹؕΓ͕ͪɻ 
ɹ࡞ۀνέοτʹ͢Δ·Ͱ΋ͳ͍ͭ͘Ͱʹ௚͢श׳Λ͚ͭΔɻ

View Slide

ɹTest
● unittest ʹΑΔࣗಈςετΛ׆༻ 
● Excel಺༰Λظ଴஋ʹ൓ө → GreenʹͳΕ͹࠶ݱ׬ྃ - ·͞ʹ “TDD” 
● SQLνϡʔχϯά΍ϦϑΝΫλ࣌ʹσάϨΛଈ࣌ݕ஌Ͱ͖ΔͷͰ૬ੑྑ͍

View Slide

ɹॏް௕େͳExcelΛ࠶ݱ͢Δˠ·ͣղಡ
ϚΠϯυϚοϓʹΑΔ੔ཧ 
εϥΠυʹೖΔΑ͏ʹʢೖΓ͖ͬͯͳ͍͚ͲʣॖΊͨΒը૾͕௵Εͯ͠΋͏ͨ……

View Slide

ɹɹ1. Jupyter Notebook Ͱௐࠪ಺༰ͱ 
ɹɹɹσʔλΛ࠶ݱՄೳͳܗͰอଘ͢Δ
 
 
ɹɹ2. Github Preview Λͦͷ·· 
ɹɹɹઆ໌ࢿྉͱͯ͠ར༻͢Δ

ɹطଘͷܭࢉϛε΍σʔλෆ੔߹Λൃݟ

View Slide

ɹχʔζ։୓ͷ޷॥؀
֤෦ॺͷϢʔεέʔεΛ 
ώΞϦϯά͠΍͘͢ͳΔ
ʮσʔλͷ͜ͱ͸͍ͭ͜ʹฉ͚ʯ 
ʹ ࣭໰΍૬ஊ͕དྷΔΑ͏ʹͳΔ
طଘσʔλͷϛε΍໰୊ɺ 
͞ΒͳΔհࡏ༨஍ʹؾ෇͚Δ
খ͞ͳ࣮੷ 
طଘσʔλͷܭࢉϛε΍ෆ੔߹Λൃݟ 
→ ใࠂͯ͠௚͢

View Slide

ͱʹ͔͘਺ࣈ͕߹Θͳ͍

ɹ޷॥؀Λࢧ͑Δ೜଱
γεςϜ࿈ܞͷ్தͰ࢛ࣺޒೖ͕ੵΈॏͳΔ
Excel͕طʹ 4 x 1990 = 7959 ͱͳ͍ͬͯΔ
࣮ߦλΠϛϯάͰूܭ݁Ռ͕มΘΔ 
→ طଘͷ਺஋͸୭΋࠶ݱͰ͖ͳ͍

View Slide

࣮ࡍʹඅ΍ͨ࣌ؒ͠ͷ80% ʹ ͻͨ͢Β஍ಓͳ࡞ۀ
 
·͊ɺେมͰͨ͠ɻ

ɹ׼ͱྦͷτϥϒϧγϡʔςΟϯά

View Slide

ɹ֤෦ॺ΁ͷσʔλఏڙΛ࣮ݱ
σʔλج൫

View Slide

 
 

ɹΞδΣϯμ

View Slide

ɹ͜Ε·ͰɿϏδωε෦໳ͷϝΠϯExcel
● σʔλͷ஝ੵ → ॏ͍ϑΝΠϧ 
 
● ؔ਺/ηϧͷଟॏࢀর → ௐࠪ/ղऍ͕ࠔ೉ 
 
● macOSͰ։͘ͱͳΜ͔յΕΔ
։͘ͷʹ3෼ɻಈ͔ͦ͏ͱ͢Δͱ·ͨࢭ·Δɻ
ΤϯδχΞ͕ؾܰʹݟΒΕͳ͍ঢ়گ

View Slide

ɹɹɹɹɹ ʴ
ɹ͜Ε͔ΒɿTech + Culture = Value

ςΫϊϩδʔ 
จԽɾϓϩηε
σʔλΛ׆༻ͨ͠ 
੡඼։ൃ
ੈͷதʹ
ސ٬Ձ஋Λఏڙ
Done

View Slide

ɹɹɹɹɹ ʴ
ɹ͜Ε͔ΒɿTech + Culture = Value

ςΫϊϩδʔ 
จԽɾϓϩηε
σʔλΛ׆༻ͨ͠ 
੡඼։ൃ
ੈͷதʹ
ސ٬Ձ஋Λఏڙ
Done Todo

View Slide

ɹϓϩμΫτ։ൃͱԾઆݕূ
Build
Measure
Learn

View Slide

Build
Measure
Learn
Ͱ͖͍ͯΔ
΍ͬͯΈΔ

View Slide

ɹɹͲΜͳϢʔβʔ͕৽ػೳΛ࢖ͬͨͷ͔ 
ɹɹϓϩμΫτʹͲΜͳظ଴Λ΋͍ͬͯΔͷ͔
ɹɹϏδωεKPI͸Ͳͷఔ౓޲্ͨ͠ͷ͔ 
ɹɹ෭࡞༻͸͔͋ͬͨʢ͋ΔػೳΛԡ͠ग़͢ → ผͷػೳ͕࢖ΘΕͳ͘ͳΔՄೳੑʣ

ɹ࣮ࢪͨ͠ࢪࡦͷ݁ՌΛ෼ੳ
IUUQHBUBHOFUGSFFJMMVTUSBJUJPO

View Slide

ɹ͜Μͳײ͡Ͱ΍Γ·ͨ͠
1. ϗϫΠτϘʔυʹ෼ੳཁ݅ 
ɹɹΧελϚʔߦಈભҠΛϕʔεͱͨ͠ࢦඪπϦʔ 
ɹɹϩάΠϯ཰͕޲্ → ΞΫγϣϯ͕… → ച্׵ࢉ
 
ɹɹɹɹ2. JupyterͰ෼ੳ 
ɹɹɹɹɹίʔυ͸શͯίϛοτཤྺͰ؅ཧ 
 
ɹɹɹɹɹɹɹɹ3. ෼ੳ݁ՌΛؔ܎ऀʹڞ༗ 
ɹɹɹɹɹɹɹɹɹGithubͷϓϨϏϡʔը໘Λ׆༻

View Slide

ɹKeepɹϏδωεࢹ఺Ͱ։ൃΛݟΔ
ࣗ෼ͷ੒ՌΛࣗ෼Ͱݴ͑ΔΑ͏ʹͳͬͨ 
ʢྫɿ1ϲֻ݄͚ͨҊ݅Ͱച্xxԁ/೥ʣ 

View Slide

ɹProblemɹνʔϜͷվળ༨஍͕ݦࡏԽ
ɹɹϝϯόʔ͕ࣗ෼ͷཧղෆ଍ʹڻ͘ 
 
ɹɹɹ ● ੡඼࢓༷ - ͜ͷέʔεͩͱσʔλͷத਎͸Ͳ͏ͳΔʁ 
ɹɹɹ ● ϑϩϯτدΓϝϯόʔ͕ෳࡶͳSQLʹखؒऔΔ 
ɹɹɹ ● ͦ΋ͦ΋ϏδωεKPIΛ೺Ѳ͍ͯ͠Δ͔ 
ɹɹɹ ● ͖ͪΜͱ෼ੳ͢ΔͨΊʹຊ౰͸ඞཁͩͬͨϩάཁ݅ͷ࿙Ε 

View Slide

ɹTryɹϓϩμΫτࢤ޲νʔϜ΁ͷୈҰา
ݴΘΕͨ΋ͷΛ࡞Δ͚ͩʢBuildॏࢹʣࢥߟ͔Βͷ୤٫

ʮ͞Βʹ਺ࣈΛ৳͹͢ʹ͸Ͳ͏͢Δʁʯ
ࢪࡦΛىҊ͢ΔΤϯδχΞ͕ग़࢝ΊΔ
Ϗδωε෦໳ͱนଧͪ 㲗 ࢼߦࡨޡ
 
αϯϓϧ࣮૷ɹษڧձͰڝ߹ͷΤϯδχΞΛั·͑ͯ࿩Λฉ͘ 
ࣾ಺ؔ܎ऀʹ໰୊ҙࣝΛώΞϦϯάɹ਺ࣈΛ΋ͱʹޮՌݟཱͯ
νʔϑϓϩσϡʔαʔ͔ΒGoαΠϯ

View Slide

Build
Measure
Learn
Ͱ͖͍ͯΔ
΍ͬͯΈΔ

View Slide

ɹ͍͔ʹਁಁͤ͞Δ͔
Build
Measure
Learn
Ͱ͖͍ͯΔ
΍ͬͯΈΔ ਁಁͤ͞Δ

View Slide

͋ͷख͜ͷखͰʮσʔλ෼ੳ΍Ζ͏ʯͷۭؾΛৢ੒
ɹ ● σʔλ෼ੳϋοΧιϯͷ։࠵
ɹ ● ே׆ x ࣗ༝ݚڀʢϏοτίΠϯͷ஋্͕ΓཁҼΛ౷ܭղੳ͢Δۚͷ๢ऀ͕େྔൃੜʣ
ɹ ● JupyterHubΛ֦ுͨ͠PythonֶशαΠτ΍ 
ɹɹ֎ʹஔ͚ͳ͍σʔλ޲͚ͷnbviewerΛࣾ಺ఏڙ

ɹϝοηʔδϯά

View Slide

1ͭͷϞχλʔΛғΜͰશһͰ࡞ۀ͢Δ
ɹɹ ● पғ͕ௐ΂ͨΓΞυόΠε͠ͳ͕ΒਐΊΔ 
ɹɹɹ → ϋϚΒͳ͍ / ࠳ંΛ๷͙ɺTips΍ίπΛڞ༗͋͠͏ 
ɹɹ ● օ͕΍ΔͳΒࣗ෼΋΍Δ͔ʂͷޙԡ͠ 
ɹɹɹ → ʮ΍ͬͯΈͨΒࢥͬͨҎ্ʹྑ͔ͬͨʯͷମݧ

ɹϞϒ ϓϩάϥϛϯά σʔλ෼ੳ

View Slide

 
σʔλ࢓༷ʹৄ͍͠ʢগͳ͘ͱ΋ௐࠪ͢ΔεΩϧ͸΋͍ͬͯΔʣΤϯδχΞ͕ 
୲౰ྖҬΛ޿͛Δ͜ͱͰલޙ޻ఔͷϦʔυλΠϜΛ୹ॖ 

ɹϓϩηε૷ண - ։ൃ޻ఔʹ૊ΈࠐΉ
੡଄
ઃܭ ࢼݧ ϦϦʔε
ޮՌ 
ଌఆ
ཁ݅ 
ఆٛ
ཁٻ 
੔ཧ
੡଄
ઃܭ ࢼݧ ϦϦʔε
ޮՌ 
ଌఆ
ཁ݅ 
ఆٛ
ཁٻ 
੔ཧ
طʹ΍͍ͬͯΔ
෼ੳཁٻ ϩάཁ݅ ܭଌ
ΞΠσΞ

View Slide

ΤϯδχΞ ʹ ͦͷ৔Ͱσʔλ࢓༷Λௐ΂Δ͜ͱ͕Ͱ͖Δ
 
σʔλαΠΤϯε෦໳͕࢓༷ώΞϦϯάΛॏͶͯ 
1ि͔͚͍ؒͯͨ෼ੳ͕1࣌ؒͰ׬ྃʢϑϩʔޮ཰Խʣ

ɹ࣮੷ɿΞδϦςΟͷ޲্
ϝΠϯϞχλʔ͸ 
Jupyter Notebook
αΠυϞχλʔ͸ 
ϓϩμΫτͷιʔείʔυ
ʴ

View Slide

Build
Measure
Learn
Ͱ͖͍ͯΔ
΍ͬͯΈΔ Ͱ͖͍ͯΔ

View Slide

ɹLearn
ࠓޙͷ՝୊ 
·ͩࢼߦࡨޡதͰ͢

View Slide

ɹؾ෇͖ - “Ծઆ” Λ஌Βͳ͍ͱ “ݕূ” Ͱ͖ͳ͍
IUUQXXXOJLLFJCQDPKQBUDMQVCNLUCPPL1
Ϗδϣϯ
ઓུ
ઓज़

View Slide

“Ծઆ” Λνʔϑϓϩσϡʔαʔʹ࣭໰
↓ 
 
ߟ͑Λฉ͍ͨΤϯδχΞʮ͍΍ɺԶ͸͜͏ࢥ͏ʂʯ
↓ 
ϒϥϯυ࠶ݕ౼ϫʔΫγϣοϓͷ։࠵ 
ͲΜͳސ٬ʹͲΜͳՁ஋Λఏڙ͢Δͷ͔ 
෦ॺΛԣஅ͓ͯ͠ޓ͍ͷߟ͑ΛͿ͚ͭ߹͏

ɹର࿩ͷ׆ੑԽ

View Slide

֤ࣗͷҙݟ͕ʮ͋ͳͨͷײ૝Ͱ͢ΑͶʯʮͳΜ͔σʔλͱ͔͋ΔΜͰ͔͢ʯঢ়ଶ

ɹҙݟͷͿ͚ͭ߹͍ → ݟ͖͑ͯͨ՝୊
ࣗ෼ͨͪ͸ސ٬ͷ͜ͱΛ 
ຊ౰ʹཧղ͍ͯ͠Δͷ͔ʁ
উखͳࢥ͍ࠐΈͰ 
ϓϩμΫτΛ 
։ൃ͍ͯ͠ͳ͍͔ʁ

View Slide

ɹϦΫϧʔτͷྺ࢙
Ӧۀ͕௚઀ళฮʹग़޲͍ͯ 
σʔλΛऩू͍ͯͨ͠
σʔλʢࣄ࣮ʣΛ࣋ͪدͬͯ 
෦ॺؒͰҙݟΛͿ͚ͭ߹͏
ࢢ৔ͷ࠶ఆٛ 
৽͍͠ϏδωεՁ஋ͷߏங 
ྫɿঁੑ޲͚ब৬ϝσΟΞ → ࣾձਐग़ͷࢧԉ 
ྫɿΫʔϙϯࡶࢽ → ༧໿αΠτ ΁ͷస׵
[email protected] [email protected] IUUQXXXJSBTVUPZBDPNOFXQPQIUNM

View Slide

4&$*Ϟσϧ

ɹσʔλ͕ࢧ͑Δ஌ࣝ૑଄
ސ٬Ձ஋ উͪےɾUVP
ΤϯδχΞ ू٬
σβΠϯ ηΩϡϦςΟ
๏຿
ΧελϚʔαϙʔτ ޿ใ
ࢧԉऀɾ։୓ऀͱͯ͠ͷ 
σʔλαΠΤϯςΟετ
σʔλ

View Slide

શͯ͸ ސ٬ཧղ ͔Β࢝·Δ
σʔλ͸ͦͷͨΊͷ൑அࡐྉ

View Slide

Ӧۀ͕଍Λ࢖ͬͯσʔλΛूΊΔ 
ΫϥΠΞϯτͱపఈతʹ޲͖߹͍ʮউͪےʯΛݟ͚ͭΔ 
ͦ͏΍ͬͯϦΫϧʔτͷ 
͜Ε·Ͱͷ50೥Λ࡞͖ͬͯͨ 
ϝσΟΞΛ௨ͯ͠ੈքΛ࡞͖ͬͯͨ

View Slide

ΤϯδχΞ͕खΛ࢖ͬͯσʔλΛूΊΔ 
ΧελϚʔͱపఈతʹ޲͖߹͍ʮUVPʯΛݟ͚ͭΔ 
 
ͦ͏͢Δ͜ͱ͕ϦΫϧʔτͷ 
͜Ε͔Βͷ50೥Λ࡞Δ͸ͣ 
ϓϩμΫτΛ௨ͯ͠ੈքΛ࡞Δ͸ͣ

View Slide

ϦΫϧʔτʹݶͬͨ࿩Ͱ͸ͳ͍ 
 
͜Ε͔ΒͷੈքΛ࡞Δͷ͸ɺ 
୭ΑΓ΋ ੡඼ ͱ ސ٬ ʹਫ਼௨ͨ͠ 
ΤϯδχΞͷ໾ׂ Ͱ͢ 
͍·͜ͷ৔ʹ͍Δ ࢲͨͪࣗ਎ Ͱ͢
 
σʔλ׆༻͸ɺͦͷୈҰาͩͱࢥ͍ͬͯ·͢

View Slide

σʔλΛ׆༻͢ΔͨΊʹ৐Γӽ͑Δน
● େن໛૊৫ͰͲͷΑ͏ͳγεςϜɾϓϩηεΛ࠾༻͢Δ͔ʁ 
● Buildॏࢹݱ৔ʹͲ͏΍ͬͯMeasureɾLearnΛ૷ண͢Δ͔ʁ
1ͭͷϓϩμΫτʹ͓͚ΔࣄྫΛڞ༗͠·ͨ͠

ɹຊ೔ͷৼΓฦΓ

View Slide

σʔλ׆༻ͷਪਐʢ୅දͱͯ͠ൃද͠·͕࣮ͨ͠ࡍʹ͸൴Βͷޭ੷ʹΑΔͱ͜Ζ͕େ͖͍Ͱ͢ʣ
ɹNaoya OsugiɹLee HeesungɹTatsuya IshibeɹTakuya BeppuɹTakahiro Kato 
ɹKohei AotaɹRyo InoueɹHiroki SakamotoɹShinryo UchidaɹShinsaku Kouno 
ɹShin KanouchiɹGeeWook KimɹKoh Fujiwara 
ٕज़ΞυόΠεɾ૊৫ࢧԉ
ɹSho ItoɹShinya NishinakaɹNaoki Ainoya 
ɹKeisuke SoneɹSatoshi UejimaɹNorihisa Miyakawa
ൃදαϙʔτ
ɹShun OnoɹYusaku TokunagaɹKazutaka SakuraiɹSaki Kato
※ࢿྉɾ಺༰ʹޡΓ΍ෆద੾ͳදݱ͕͋Ε͹ൃදऀͷϛεͰ͢ɻ@yuzutas0 ʹ͝࿈བྷ͍͚ͨͩΔͱ޾͍Ͱ͢ɻ

ɹSpecial Thanks - ج൫ͷಓ͸1ਓʹͯ͠੒Βͣ

View Slide

গ͠Ͱ΋ଟ͘ͷ ։ൃݱ৔ ͕ 
ੈͷதʹ ྑ͍ϓϩμΫτ Λఏڙ͢Δ 
ώϯτ ʹͳΕͨΒͱࢥ͍ͬͯ·͢

ɹ఻͍͑ͨ͜ͱ

View Slide

͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
presented by @yuzutas0 
https://www.pexels.com/photo/close-up-of-computer-keyboard-257949/ɹhttps://www.pexels.com/photo/technology-computer-lines-board-50711/ɹhttps://www.pexels.com/photo/black-and-white-business-chart-computer-241544/

View Slide

Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyconjp / 20170909

Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyconjp / 20170909

yuzutas0
PRO

More Decks by yuzutas0

Other Decks in Technology

Featured

Transcript

Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyconjp / 20170909

Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyconjp / 20170909

yuzutas0 PRO

More Decks by yuzutas0

Other Decks in Technology

Featured

Transcript

yuzutas0
PRO