Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyconjp / 20170909

Jupyter + BigQuery ʹΑΔ  σʔλ෼ੳج൫ ͷ DevˍOps 2017-09-09 (Sat)  4:15
p.m. – 4:45 p.m. in Room 201  PyCon JP 2017  presented by @yuzutas0  https://www.pexels.com/photo/close-up-of-computer-keyboard-257949/ɹhttps://www.pexels.com/photo/technology-computer-lines-board-50711/ɹhttps://www.pexels.com/photo/black-and-white-business-chart-computer-241544/

ຊ೔ͷࢿྉ͸WEBʹެ։͠·͢  #pyconjp #pyconjp_201  ࡱӨɾه࿥ͷඞཁ͸͋Γ·ͤΜ  ͭ͘Ζ͍Ͱௌ͍͍͚ͯͨͩΕ͹ͱࢥ͍·͢

εϥΠυ 150+ / 30min  ʢ஌ݟΛ͕ͬͭΓڞ༗͠·͢ʣ   ͕Μ͕Μߦ͖·͢ʂ

ɹSho Yokoyamaɹ@yuzutas0 ɹɹ     • Recruit Technologies Co.,
Ltd.   ɹɹάϧʔϓͷITࢪࡦશൠΛ୲͏ byʰϦΫϧʔτɺਐԽΛࢭΊͳ͍ITݱ৔ྗʱ • Site Reliability Engineering  ɹɹج൫γεςϜͷӡ༻ɾվળ • Finance & Econometrics (Before)  ɹɹRʹΑΔ౷ܭॲཧ - ͋ͷͱ͖Jupyter͕͋Ε͹Α͔ͬͨͷʹʂ

https://github.com/yuzutas0 ෼ੳج൫ ɹޱ΋ख΋ಈ͔͢೿

ϓϩμΫτ։ൃ ʹ͓͚Δ  σʔλ׆༻ ͷ ࢀߟࣄྫ ͷڞ༗ ɹຊ೔ͷΰʔϧ

Python Λ׆༻ͯ͠  ։ൃݱ৔ ʹ σʔλจԽ Λਁಁ͍ͤͨ͞  ιϑτ΢ΣΞΤϯδχΞ ɹ૝ఆ͢Δର৅

ɹɹɹ ɹPythonͷ࿩ ɹɹɹ "ɹPythonΛۀ຿Ͱ࢖͏࿩ ɹPythonͷConferenceͰ͕͢ த൫·Ͱग़͖ͯ·ͤΜʢ׼ʣ

1. ϓϩμΫτ / νʔϜ   2. ෼ੳج൫ͷDev&Ops ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε   3.
σʔλจԽΛ૊৫ʹ૷ண͢Δ ɹΞδΣϯμ

ࠗ׆ ࿀׆

ֹ݄՝ۚϞσϧ  ※࿀݁ͼঁੑϢʔβʔ͸ຊਓ֬ೝྉͷΈ ɹ͝ར༻ͷྲྀΕ

※αϒγεςϜͷٕज़ཁૉ: Golang, Groovy, Python, Ruby, AWS, GCP ɹɹɹɹɹɹɹɹɹɹɹɹΦϯϓϨ ɹ֤छσόΠεରԠ
- γεςϜུ֓ (AP)

※ถࠃͰ͸݁ࠗͨ͠Χοϓϧͷ1/3͕ΦϯϥΠϯͰͷग़ձ͍ ɹσʔςΟϯάࢢ৔ͷ֦େ

ɹࣄۀ੒௕ → εςʔΫϗϧμʔͷଟ༷Խ σβΠϯ ηΩϡϦςΟ ๏຿ Πϯϑϥ SRE ΞϓϦج൫
ू٬ ΧελϚʔ  αϙʔτ σʔλ  αΠΤϯε ػցֶश  ΤϯδχΞ ։ൃӡ༻  νʔϜA ։ൃӡ༻  νʔϜB ։ൃӡ༻  νʔϜC PO PO PO σΟϨΫγϣϯ ޿ใ

ɹ֤෦ॺ΁ͷσʔλఏڙ σʔλج൫

ɹͲͷΑ͏ʹ׆༻͍ͯ͠Δ͔ σʔλج൫

ɹσʔλ׆༻ࣄྫ ɹ1. KPIϞχλϦϯάɹɹ  ɹ2. ϢʔβʔߦಈͷՄࢹԽ  ɹ3. ࢪࡦޮՌͷ༧ଌɾܭଌ Ϗδωε 4.
ػցֶशʹΑΔϨίϝϯυ  5. ۀऀɾεύϜͷࣗಈ൑ఆ ϓϩμΫτ ɹ6. ໰͍߹Θͤͷ܏޲෼ੳ  ɹ7. ޿ࠂ഑৴ͷࣗಈ࠷దԽ  ɹ8. ϓϨεϦϦʔε ΧελϚʔνϟωϧ ɹ9. γεςϜϞχλϦϯά  ɹ10. ো֐ൃੜ࣌ͷӨڹௐࠪ γεςϜ ɹ11. νʔϜͷΩϟύγςΟ νʔϜ

Ϗδωεࢦඪʢച্΍DAUʣͷਪҠ    ຖேͷSlack௨஌ɹɹɹɹSpreadsheetɹɹɹɹμογϡϘʔυ ɹɹɹKPIϞχλϦϯά 1

ओཁಋઢʢձһొ࿥ˠϝοηʔδʣͷར༻ɾ཭୤ ߦಈϑΝϯωϧɹɹɹɹɹɹδϟʔχʔϚοϓ ɹɹɹϢʔβʔߦಈͷՄࢹԽ 2

Split (AB) ςετ݁Ռɹɹ৽ػೳͷར༻౓߹͍ ɹɹɹࢪࡦͷޮՌ༧ଌˍܭଌ 3

ɹɹɹػցֶशʹΑΔϨίϝϯυ 4

ɹɹɹۀऀɾεύϜͷࣗಈ൑ఆ 5

ɹɹɹ໰͍߹Θͤͷ܏޲෼ੳ 6

ɹɹɹ޿ࠂ഑৴ͷࣗಈ࠷దԽ 7

σʔλ෼ੳ݁Ռͷެ։ → ࢢ৔ͷ׆ੑԽʹߩݙ ɹɹɹϓϨεϦϦʔε 8 IUUQXXXSFDSVJUNQDPKQOFXTSFMFBTF@IUNM

Ϋϥογϡ཰΍ϨεϙϯελΠϜ ɹɹɹγεςϜϞχλϦϯά 9 IUUQZV[VUBTIBUFOBCMPHDPNFOUSZ

ɹɹɹো֐ൃੜ࣌ͷӨڹௐࠪ 10

εΫϥϜνʔϜͷϕϩγςΟ ΍ νέοτফԽͷϝτϦΫε ɹɹɹνʔϜͷΩϟύγςΟ 11

σβΠϯ ηΩϡϦςΟ ๏຿ Πϯϑϥ SRE ΞϓϦج൫ ू٬ ΧελϚʔ  αϙʔτ σʔλ 
αΠΤϯε ػցֶश  ΤϯδχΞ ։ൃӡ༻  νʔϜA ։ൃӡ༻  νʔϜB ։ൃӡ༻  νʔϜC PO PO PO σΟϨΫγϣϯ ޿ใ ։ൃӡ༻  νʔϜB PO ɹ͜ΕΒͷࢪࡦ : ֤෦ॺ͕ϘτϜΞοϓͰਪਐ

σʔλ׆༻Ͱ࣮ݱͰ͖Δ͜ͱ ɹސ٬Ձ஋ͷ௥ٻ → σʔλ׆༻͸ඞવ 1. ࣗಈԽ 2. ՄࢹԽ ސ٬Ձ஋ͷఏڙ

“ΑΓૣ͘ɾ҆ఆͨ͠” αʔϏεͷఏڙ ɹ1. ࣗಈԽ ػցֶशͰީิऀϦετΛࣄલநग़ ໨ࢹ͚ͩͰۀऀɾεύϜΛ൑ఆ IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM

“ΑΓޮՌతͳ” ΤϯδχΞϦϯάͷ࣮ݱ ɹ2. ՄࢹԽ ͜ͷσβΠϯ͸Ϣʔβʔͷ  ຊԻʹ౴͍͑ͯΔͷ͔ʁ ͜ͷػೳ௥Ճ͸ޮՌ͕͋Δͷ͔ʁ  ࣮ࡍʹޮՌ͸͋ͬͨͷ͔ʁ IUUQHBUBHOFUGSFFJMMVTUSBJUJPO
IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM

σβΠϯ ηΩϡϦςΟ ๏຿ Πϯϑϥ SRE ΞϓϦج൫ ू٬ ΧελϚʔ  αϙʔτ σʔλ 
αΠΤϯε ػցֶश  ΤϯδχΞ ։ൃӡ༻  νʔϜA ։ൃӡ༻  νʔϜB ։ൃӡ༻  νʔϜC PO PO PO σΟϨΫγϣϯ ޿ใ ։ൃӡ༻  νʔϜB PO ɹ֤෦ॺʹΑΔੵۃతͳσʔλ׆༻

ɹͳͥج൫͕ඞཁʹͳ͔ͬͨ σʔλج൫

DBσʔλΛ࢖͍͍ͨͷͰ  ೔࣍ͰS3ʹCSVग़ྗ͍ͯͩ͘͠͞  ɹ࠷ॳ͸খ͞ͳσʔλૄ௨ґཔ ෦ॺA

ɹεέʔϧ → ֤෦ॺ͔Βґཔ ෦ॺB ෦ॺC ෦ॺD

ͬͪ͜Ͱαʔόͱૄ௨πʔϧΛ༻ҙ͠·͢  ϨϏϡʔˍຊ൪؀ڥʹஔ͍ͯ΋Β͑·͔͢ ɹͪΐͬͱֻ͕͔࣌ؒΓͦ͏ ੜσʔλΛ౉ͯ͠΋Β͑Ε͹  ͬͪ͜Ͱ࢖͑ΔΑ͏ʹՃ޻͢ΔͷͰ ෦ॺE

ɹ֤෦ॺ͝ͱʹσʔλૄ௨ ෦ॺA  πʔϧ ෦ॺB  πʔϧ ෦ॺA  αʔό ෦ॺB  αʔό

ɹ֤෦ॺ͝ͱʹσʔλՃ޻ ෦ॺA  நग़σʔλ ෦ॺB  நग़σʔλ ෦ॺA  Ճ޻ϩδοΫ ෦ॺB  Ճ޻ϩδοΫ

ɹମ੍ͷ··ʹγεςϜෳࡶԽ - ίϯ΢ΣΠͷ๏ଇ ݩσʔλ ֤෦ॺ޲͚ʹσʔλࢀরγεςϜΛܧ͗଍͠

  http://jp.techcrunch.com/2017/06/25/20170623ﬁve-building-blocks-of-a-data-driven-culture/ ελοϑ͕ͨͪҟͳΔγεςϜ͔ΒࣅͨΑ͏ͳϝτϦοΫΛ  Ҿ͖ग़͢͜ͱ͕Ͱ͖ͯ͠·͏ͳΒɺඞવతʹͦΕΒͷγεςϜ͔Β͸  ҟͳΔ਺ࣈ͕ੜ੒͞ΕΔɻ…ΑΓ·͠ͳ৘ใݯΛ࢖͑Δഺͩͬͨͷʹɺ  ݹͯ͘඼࣭ͷ௿͍ɺ͋Δ͍͸ؒҧͬͨσʔλ΍ࢦඪΛ஌Βͣʹ  ࢖ͬͯ͠·͍ɺѱ͍ܾఆΛԼ͢νʔϜ΋ग़ͯ͠·͏͔΋஌Εͳ͍ɻ ɹ෦ॺ͝ͱʹʮച্ʯ͕ζϨΔ໰୊

ɹΤϯδχΞʹௐࠪґཔᶃ - σʔλ࢓༷͸೔ʑมΘΔ ͜ͷϨίʔυ͕  ௥Ճ͞ΕΔλΠϛϯάɺ  ࠷ۙมߋ͞Ε·ͨ͠ʁ ͪΐͬͱมߋཤྺΛ  ௥ͬͯΈ·͢Ͷ ෦ॺF
ΤϯδχΞ

ɹΤϯδχΞʹௐࠪґཔᶄ - Өڹൣғ͸೔ʑ޿͕Δ Re:dashͷ਺ࣈ͕  ζϨΔΜͰ͚͢Ͳ ଞ෦ॺ͕࡞ͬͨγεςϜΛ  ௐ΂Δͱ͜Ζ͔Β…… ෦ॺG ΤϯδχΞ

ɹݸผରԠͷѱ॥؀ ಠࣗσʔλૄ௨ͷґཔ  ʢ͜ΕͳΒ͙͢Ͱ͖·͢ΑͶʣ ΤϯδχΞνʔϜͷෛՙˢ γεςϜෳࡶԽˢ ௐࠪґཔ

ɹ୭͔ͷ൑அ͕ѱ͔ͬͨΘ͚Ͱ͸ͳ͍ εέʔϧ΍ຊ֨Քಇ࣌ʹ  ໰୊͕ݦࡏԽ ɹɹ·ͣ͸খ͘͞ࢼ͢  ɹ • ࠷খݶͷσʔλૄ௨  ɹ •
ࣗલͰσʔλՃ޻ ෦ॺ͝ͱʹ  ৽πʔϧಋೖ΍ࢪࡦணख IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM

  http://jp.techcrunch.com/2017/06/25/20170623ﬁve-building-blocks-of-a-data-driven-culture/ ɹମ੍֦େ → σʔλ΢ΣΞϋ΢εͷඞཁੑ ࣄ࣮ͷ୯Ұ৘ใݯΛ͍࣋ͬͯΔ৔߹ʹ͸ɺΞφϦετ΍  ଞͷҙࢥܾఆऀͱ͍ͬͨΤϯυϢʔβʔͨͪʹɺ༏ΕͨՁ஋Λ  ఏڙ͢Δ͜ͱ͕Ͱ͖Δɻ൴Β͸૊৫಺ͰσʔλΛ୳͕࣌ؒ͢  গͳͯ͘ࡁΉΑ͏ʹͳΓɺσʔλͷར༻ʹΑΓଟ͘ͷ࣌ؒΛ 
ׂ͘͜ͱ͕Ͱ͖ΔΑ͏ʹͳΔ͔Βͩɻ

ɹͲͷΑ͏ͳγεςϜΛߏங͔ͨ͠ σʔλج൫

1. ModelͱViewΛ෼͚Δ΂͠  2. ͳΔ΂͘ϥΫͯ͠࡞Δ΂͠ ɹઃܭϙϦγʔ

ɹModelͱViewΛ෼͚Δ΂͠ ɹɹɹɹɹɹɹɹɹɹɹɹɹDWH ੹຿ʹԠͨ͡γεςϜͷ෼཭ɾ݁߹  Model (஝ੵ/Ճ޻) ͱView (ࢀর)

ɹ஫ҙɿ෦ॺ͝ͱʹ࠷దͳView͸ҟͳΔ Excel ؾܰʹ਺ࣈΛม͑ͯ  γϛϡϨʔγϣϯ Ϗδωε෦໳ Tableau ߴՁ֨ɾߴػೳ  ෼ੳཁٻʹରԠ Re:dash
SQL͕ॻ͚Δͻͱ  ͓खܰར༻ σΟϨΫλʔ Jupyter  GitͰίʔυ؅ཧ  ϓϩάϥϜͷԸܙ ΤϯδχΞ ෼ੳ෦໳

ɹจ໌ͷརث σʔλૢ࡞ͳΒPythonҰ୒ ΤίγεςϜʹ৐Δ σʔλอଘͳΒBigQueryҰ୒  GoogleʹཔΔ ΞϓϦ૚ Πϯϑϥ

ɹɹɹ• ʢRͱൺ΂ͯʣॻ͖΍͘͢ಡΈ΍࣮͘͢ߦ͠΍͍͢  ɹɹɹ• ൚༻తͳϓϩάϥϜॲཧΛॻ͚ΔεΫϦϓτݴޠ ɹPythonͷ࠾୒ཧ༝ - ൚༻LL

  ɹɹɹɹ• ʢଞͷ൚༻ݴޠʹൺ΂ͯʣσʔλͷՃ޻΍ඳը͕ಘҙ  ɹɹɹɹ• DB㱻Dataframeม׵ͳͲଞγεςϜ࿈ܞ΋༰қ ɹPythonͷ࠾୒ཧ༝ - σʔλૢ࡞޲͚ϥΠϒϥϦ

  ɹɹɹ• σόοά΍σʔλͷՄࢹԽ͕ίϚϯυ1ͭͰՄೳ  ɹɹɹ• ಺෦తʹ͸json → GitͰࠩ෼؅ཧ → ࠶ݱՄೳͳه࿥Λ࢒ͤΔ
ɹPythonͷ࠾୒ཧ༝ - ࠷ڧͷ࣮ߦ؀ڥ

ػցֶशࢪࡦεΫϦϓτ   Ҡ২Մೳ → γεςϜͷ෼཭ɾ݁߹͕༰қ → རศੑɾอकੑ ɹPythonͷ࠾୒ཧ༝ -
طଘεΫϦϓτͱͷ૬ੑ

ɹจ໌ͷརث σʔλૢ࡞ͳΒPythonҰ୒ ΤίγεςϜʹ৐Δ σʔλอଘͳΒBigQueryҰ୒  GoogleʹཔΔ Πϯϑϥ ΞϓϦ૚

ɹBQͷ࠾୒ཧ༝ɿ͍҆ IUUQZBQDBTJBPSHUBMLTIPXDFBGBDFDBCEDB • Googleͷσʔληϯλʔ  ɹεέʔϧϝϦοτʹΑΔ  ɹίετ༏Ґ • ϢʔβʔͷҙݟΛ΋ͱʹ  ɹྉۚϓϥϯΛݟ௚ͨ͠ܦҢ

ɹBQͷ࠾୒ཧ༝ɿૣ͍ IUUQZBQDBTJBPSHUBMLTIPXDFBGBDFDBCEDB • ΧϥϜϕʔεઃܭʹΑΔ  ɹΞΫηεߴ଎Խ • େن໛σʔλ׆༻࣌ʹ  ɹϘτϧωοΫͱͳΔ  ɹDisk
I/Oͷฒྻॲཧ

ɹBQͷ࠾୒ཧ༝ɿ҆৺ GCPɾGsuiteͰݖݶ؅ཧ - ΈΜͳ͕࢖͑Δ SQLΠϯλʔϑΣʔε - ಺෦ॲཧɾอकӡ༻Λؾʹͤͣʹ࢖͑Δ ඪ४SQL  ΫΤϦͷҠ২ੑ͕ߴ͍ 
ֶशίετ͕খ͍͞  ςετ͠΍͍͢ पลπʔϧ  ओཁBIπʔϧͳΒେ఍ܨ͕Δ  pandas.io.gbq → Jupyterσόοά͕༰қ

ɹจ໌ͷརث σʔλૢ࡞ͳΒPythonҰ୒ ΤίγεςϜʹ৐Δ σʔλอଘͳΒBigQueryҰ୒  GoogleʹཔΔ Πϯϑϥ ΞϓϦ૚

ɹσʔλϑϩʔͷߏங ऩू Ճ޻ ׆༻ ஝ੵ ύΠϓϥΠϯ؅ཧ Model View

ɹσʔλϑϩʔͷߏங ऩू Ճ޻ ׆༻ ஝ੵ Model View ऩू ύΠϓϥΠϯ؅ཧ

ɹσʔλऩू ݸਓ৘ใ  ϚεΫࡁΈView Recruit Ads Console Recruit Clash  Report
όον ετϦʔϛϯά WebAPI εΫϨΠϐϯά requests + beautiful soup

ओཁͳDBɾϩά͔Βͷసૹ → ࠷খઃఆͰ࣮ݱՄೳ ɹσʔλసૹΛࢧ͑Δٕज़

Python (requests + beautiful soup) ʹΑΔ  WebAPIίʔϧɾը໘εΫϨΠϐϯά ɹJupyterͰૉૣ͘ࢼ͢ ·ͣ͸ϩʔΧϧ؀ڥͰࡁ·ͤΔ
܁Γฦ͠࢖͏ͳΒεΫϦϓτԽ IUUQTXXXTIBSFJDPOOFUEPDVNFOUpMFQZ

ΤϯδχΞ͕෼ੳʹܞΘ͍ͬͯͳ͍ͱEventσʔλ͕ϩάཁ͔݅Βൈ͚࿙Ε͕ͪ  ෆཁσʔλͷա৒஝ੵʹΑΔετϨʔδѹഭΛݒ೦ʢͦ͜ͰBQͰ͢Αʣ ɹσʔλੑ࣭ʹԠͨ͡ऩूํ๏ 4UBUFঢ়ଶσʔλ &WFOUཤྺσʔλ ྫ Ϣʔβʔͷ࠷ऴϩάΠϯ೔࣌ʢ্ॻ͖͞ΕΔʣ աڈͷϩάΠϯཤྺʢ஝ੵ͞ΕΔʣ ओͳ༻్
ը໘දࣔ   ʮ͜ͷਓ͸೔લʹϩάΠϯ͠·ͨ͠Αʯ ෼ੳ   Ͳͷ͘Β͍ͷස౓ͰϩάΠϯ͢Δ͔ ڧΈɾಛ௃ ϓϩμΫτຊମͷύϑΥʔϚϯε޲্  ͍͍ͪͪཤྺςʔϒϧͱ+PJOͯ͠  ࠷৽ϨίʔυΛ୳͢ͷ͸%#ෛՙ σʔλ෼ੳͷ࠶ݱੑΛ୲อ    ຖճϩάΠϯ೔࣌Λ্ॻ͖͞ΕͨΒ  ෼ੳͰ͖ͳ͘ͳͬͯ͠·͏ ෼ੳج൫Ͱͷ  σʔλऩू ೔࣍ͰҰׅஔ͖׵͑   %#ϩάˠࠩ෼ه࿥ˠ׆༻͸ͻͱ޻෉ඞཁ ࠩ෼൓ө   લճҎ߱ͷϨίʔυ͚ͩΛநग़

ɹσʔλϑϩʔͷߏங ऩू Ճ޻ ׆༻ ஝ੵ Model View ஝ੵ ύΠϓϥΠϯ؅ཧ

3૚ߏ଄  ࢀߟʰ10೥ઓ͑Δσʔλ෼ੳೖ໳ - SQLΛ෢ثʹσʔλ׆༻࣌୅Λੜ͖ൈ͘ʱ ɹσʔλ஝ੵ ɹɹɹɹɹɹBigQuery - Google Cloud
Platform Source  ݩσʔλͷίϐʔ Warehouse  ओཁࢦඪɾසग़σʔλ  தؒςʔϒϧ App  ֤πʔϧ͔Β௚઀ࢀর  ඇٕज़ऀ޲͚IF ஝ੵ

ྫɿ enmusubi__source__db ΞϯμʔείΞ2ͭͰཁૉΛ۠੾Δ  BEMʢCSSͷઃܭख๏ʣϥΠΫͳنଇ ɹσʔληοτͷ໋໊نଇ ϓϩμΫτ໊ source warehouse app
ϝΠϯDB αʔόϩά ΞΫηεղੳπʔϧ

ɹσʔλࣙॻ IUUQKQUFDIDSVODIDPNpWFCVJMEJOHCMPDLTPGBEBUBESJWFODVMUVSF ར༻ऀ͸σʔλϑΟʔϧυͱͦͷ஋ʢϝτϦΫεʣͷҙຯΛ஌Δඞཁ͕͋Δɻ* ݩσʔλ৘ใ΁ͷϦϯΫूͷΈఏڙ  γεςϜʢʹσʔλ࢓༷ʣ͸೔ʑมԽ͢ΔͨΊ  refs. ϩθολετʔϯ ֤σʔλιʔεଆͰ࢓༷؅ཧ  e.g.
ຊମDB → MySQL Workbench  ɹERߏ੒/ίϝϯτ͸DDLͰৗʹ࠷৽Խ

ൃల్্ͷϓϩμΫτɹ → ɹػೳ௥ՃͰසൟͳERมߋɹ → ɹBQ΋௥ਵ͍ͨ͠ ɹ஝ੵσʔλʹϚΠάϨʔγϣϯ͕ඞཁ

ɹJupyter for Migration 1. ϩʔΧϧͷJupyterͰ  ɹϚΠάϨʔγϣϯεΫϦϓτΛॻ͘ 2. ͦͷ··JupyterͰ  ɹ1೔෼ͷσʔλΛର৅ʹಈ࡞֬ೝ
3. PythonϑΝΠϧΛग़ྗ  ɹ → αʔό্ͰશσʔλΛҠߦ IUUQTXXXTIBSFJDPOOFUEPDVNFOUpMFQZ

※ͨͩ͠Python2.7੍໿͕͋ΔͷͰҰ෦ॻ͖௚͕͠ඞཁ → ύλʔϯ͕ݟ͖͑ͯͨΒม׵εΫϦϓτ • Φʔτεέʔϧ → Dataﬂow  ɹGKE͸෼ࢄ޲͚ͷઃܭ͕ඞཁ  • σʔλྔ͕গͳ͚Ε͹Datalab 
ɹJupyterͷίʔυ͕··࢖͑Δ ɹGCPͷαʔόΛར༻ ɹ • Googleઐ༻ઢͰBQΞΫηε  ɹ • ಉ͡ถࠃϦʔδϣϯʹཱͯΔ ɹBQͰ100ԯϨίʔυΛҠߦ͢Δ ϘτϧωοΫᶃ N/W ϘτϧωοΫᶄ ϝϞϦ

ɹσʔλϑϩʔͷߏங ऩू Ճ޻ ׆༻ ஝ੵ Model View Ճ޻ ύΠϓϥΠϯ؅ཧ

3૚ߏ଄  ࢀߟʰ10೥ઓ͑Δσʔλ෼ੳೖ໳ - SQLΛ෢ثʹσʔλ׆༻࣌୅Λੜ͖ൈ͘ʱ ɹσʔλՃ޻ ɹɹɹɹɹɹBigQuery - Google Cloud
Platform Source  ݩσʔλͷίϐʔ Warehouse  ओཁࢦඪɾසग़σʔλ  தؒςʔϒϧ App  ֤πʔϧ͔Β௚઀ࢀর  ඇٕज़ऀ޲͚IF Ճ޻ Ճ޻

IUUQTHSPXUIIBDLKPVSOBMDPNLQJUSFFGPSBQQ ɹओཁࢦඪΛதؒςʔϒϧͰ؅ཧ

ɹPythonεΫϦϓτʹΑΔՃ޻

ɹsh → exec.py • γΣϧҾ਺Λ΋ͱʹॲཧΛಈతৼΓ෼͚  • ݸʑͷΤϯυϙΠϯτ͸ࡉ෼Խ  ɹˠ ΦʔτεέʔϧɺϦτϥΠɺಈ࡞֬ೝ͠΍͘͢

ɹController • ॲཧͷશମ૾Λهड़  • Facadeͱͯ͠ଞॲཧΛݺͼग़͢

ɹModel • Ϗδωεࢦඪͱ1ର1ͰରԠ͢ΔΫϥε  • Create/ReadॲཧͷΈ → ஋͸BQ؅ཧ  ɹ →
ϓϩάϥϜ্͸εςʔτϨεʹอͭ

ɹQuery • BigQueryͱͷR/WΛѻ͏SQL  • ςΩετॲཧͷࢧԉػೳΛ׆༻  ɹ → f-strings (3.6~)΍TypeHints
(3.5~)  • ϝιουͱͯ͠੾Γग़͠  ɹ → ஋ͷҾ͖౉͠ʹΑΔείʔϓ੍ޚ

ɹEnum • Enum(3.4~)Ͱσʔλݻ༗ͷίʔυ஋Λ؅ཧ  • ੡඼ຊମ (Java) ͷEnumΛҠ২ → Dictܕʹม׵ 
• ܧঝݩͷbaseEnumΛ༻ҙ → ڞ௨ͷσʔλऔಘॲཧΛ࡞੒

ɹResource • ֎෦γεςϜͱͷ࿈ܞॲཧ  • retryॲཧͷҰݩԽ → σʔλྔ͕૿͑ΔͱBQ͸Αࣦ͘ഊ͢ΔͷͰ  • dotenvͰ؀ڥม਺ΛಡΈࠐΉ
→ ຊ൪ɾ։ൃͰSlackνϟϯωϧΛ෼͚Δ

ɹσʔλϑϩʔͷߏங ऩू Ճ޻ ׆༻ ஝ੵ Model View ׆༻ ύΠϓϥΠϯ؅ཧ

ɹσʔλ׆༻ - Pushܕ IUUQGSFFJMMVTUSBUJPOTHBUBHOFUIUNM ػցֶशࢪࡦ

ΞυϗοΫ ࣗಈԽ ɹσʔλ׆༻ - Pullܕ

ɹ୭ʹεΫϦϓτΛ༻ҙ͢Δ͔ Yes No ͦͷ࡞ۀ͸  ܁Γฦ͢ʁ ϞχλϦϯά  ఆظతͳॲཧ  ʲPushܕʳʢҰ෦ྫ֎͋Γʣ εφοϓγϣοτ 
ΞυϗοΫͳ෼ੳɾௐࠪ  ʲPullܕʳ SQLΘ͔Δ  ͻͱɾ෦ॺʁ ELTɿૄ௨ˠՃ޻  ʢྫʣػցֶशΤϯδχΞ  ʲPullܕʳ ETLɿՃ޻ˠૄ௨  SpreadsheetͰI/F  ʲPushܕʳ ※Ճ޻ࡁΈσʔλʹରͯ͠γϯϓϧͳSELECTจΛൃߦ͢Δ͚ͩͰࡁΉέʔε͸ଟ͍

ɹDecorator • ֤View޲͚ͷσʔλՃ޻ʢग़ྗઌʹԠͨ͡ม׵ॲཧʹಛԽʣ  • ྫɿຖேSlackʹ౤͛ΔάϥϑΛ matplotlib Ͱඳը

ɹσʔλϑϩʔͷߏங ऩू Ճ޻ ׆༻ ஝ੵ Model View ύΠϓϥΠϯ؅ཧ

؅ཧπʔϧʹظ଴͢Δཁ݅ʢཧ૝ʣ     ɹσʔλύΠϓϥΠϯͷ؅ཧ 1. ίʔυͰόʔδϣϯ؅ཧ  2. εέδϡʔϧࣗಈىಈ  3.
ηοτΞοϓ΍ϝϯςφϯεͷָ͞(DB಺แͳͲ)  4. GUI/CUI྆ํͰϩάɾύϑΥʔϚϯεͷ؅ཧ  5. GUI/CUI྆ํͰม਺ࢦఆͷ࠶࣮ߦ 6. ࣗಈϦτϥΠॲཧɾεΩοϓػೳ  7. εΫϦϓτͷฒྻ࣮ߦ  8. αʔόͷΦʔτεέʔϧ  9. ༗޲ඇ८ճάϥϑͷࣗಈ࡞੒  10. Python3.6~ରԠʢςΩετॲཧ͕֨ஈʹָʣ

࿩୊ͷπʔϧΛܰ͘৮ͬͯࢼͯ͠ΈͨΓ  ɹɹɹɹɹɹ   ɹɹɹɹɹͲΕ΋ັྗత͚ͩͲҰேҰ୹ → ͔Ώ͍ͱ͜Ζʹख͕ಧ͔ͳ͍  ɹɹɹɹɹɹ • ίϯτϦϏϡʔτͷνϟϯε͔΋ʁ  ɹɹɹɹɹɹ
• ೔ਐ݄าͰັྗతͳͷͰ͍ͣΕҠ২͍ͨ͠ؾ࣋ͪ͸͋Δ ɹWorkﬂow Engine

ͳΜ͔ͩΜͩͰ࠷ॳʹ͓खܰߏஙͨ͠ jenkins Λ࢖͍ଓ͚͍ͯΔ    ɹɹɹɹɹɹ• 2ܥ + BlueOcean + Pipeline
of jenkinsﬁles  ɹɹɹɹɹɹ• ϦτϥΠػߏɾॲཧ࠷దԽ͸ଞʹྼΔ  ɹɹɹɹɹɹ• jenkinsδϣϒ͔ΒγΣϧܦ༝Ͱ֤छॲཧΛίʔϧ ɹ҆ఆͷJenkins

ɹσʔλϑϩʔͷߏஙʢ׬ʣ ऩू Ճ޻ ׆༻ ஝ੵ Model View ύΠϓϥΠϯ؅ཧ

ɹͲͷΑ͏ͳϓϩηεΛ࠾༻͔ͨ͠ σʔλج൫

ʮσʔλج൫Λ࡞Δͧʯܭը͸ ࣦഊ͔Β࢝·ͬͨ

μογϡϘʔυΛ  ෦԰ͷϞχλʔʹө͢ ɹ࠷ॳͷҰา

1. ຖ೔ͷKPIΛ͢Β͢Βݴ͑ΔΑ͏ʹͳΔ͔΋    ɹ2. ҟৗ஋ʹؾ͍ͮͯਝ଎ʹಈ͚ΔΑ͏ʹͳΔ͔΋ ɹߴ·Δظ଴

1िؒͰ୭΋ݟͳ͘ͳͬͨ ɹݱ࣮͸ඇ৘Ͱ͋Δ

ɹɹɹ׬ᘳͳاըॻʁ ɹɹɹѹ౗తͳ༧ࢉʁ ɹɹɹ1೥ؒͷ։ൃϓϩδΣΫτʁ ɹʮ࠷ڧͷσʔλج൫ʯΛࢧ͑Δϓϩηεʁ

ɹɹɹ׬ᘳͳاըॻʁ ɹɹɹѹ౗తͳ༧ࢉʁ ɹɹɹ1೥ؒͷ։ൃϓϩδΣΫτʁ ɹʮ࠷ڧͷσʔλج൫ʯΛࢧ͑Δϓϩηεʁ ❌

ɹɹ• ࢥ͍ͬͯͨͷͱҧͬͨ ɹɹ• ྑͦ͞͏͚ͩͲ݁ہ࢖Θͳ͔ͬͨ ɹ1೥ޙʹ଴ͭ΋ͷ

ɹʮԶͷߟ͑ͨ࠷ڧͷσʔλج൫ʯ͸࢖ΘΕͳ͍ IUUQXXXQSPKFDUDBSUPPODPN

→ ͜ΕͳΒ࣮֬ʹ࢖ͬͯ΋Β͑Δ ɹগͣͭ͠ஔ͖׵͑Α͏   ࣮͸͏ͪͷ෦ॺͰຖேSlackʹάϥϑΛ  खಈͰྲྀ͍ͯ͠ΔΜ͚ͩͲʂ   ͜ͷػʹࣗಈԽ͠·͔͢ʂ

ɹɹɹQ.ɹ࠷ॳʹཁ݅Λશ෦ચ͍ग़ͤ͹OKʁ    ɹɹɹA.ɹNGʂʮ΍Γ͍ͨ͜ͱʯ͸มΘΔʂ  ɹɹɹɹɹɹɹ • ଞࣾϒϩάΛݟͯʮ͏ͪ΋΍Γ͍ͨʂʯ  ɹɹɹɹɹɹɹ • ৽͍͠BIπʔϧͷఏҊʹʮ͜Ε͍͍ʂʯ  ɹɹɹɹɹɹɹ
• ࣮ࡍʹը໘Λ࡞ͬͨΒʮͳΜ͔ҧ͏ʂʯ   ɹɹ ɹɹ → มԽʹॊೈʹରԠ͢Δʂ ɹΠςϨʔγϣϯΛճ͢͜ͱ͕େࣄ

TodoΛνέοτ؅ཧ → ॊೈʹ༏ઌॱҐΛೖΕସ͑Δ ɹνέοτۦಈ։ൃ

ɹλεΫͷ༏ઌॱҐ 1 σʔλ͕·͕͍ͪͬͯΔ  ٙ࿭ͷௐࠪ ɹ1ͭؒҧ͍ͬͯΔͱશ෦৴༻Ͱ͖ͳ͘ͳΔͷͰ࠷༏ઌରԠɻ  ɹ͙͢ௐࠪ͢Ε͹ؔ܎ऀͷ৴༻࢒ߴΛ૿΍ͤΔɻ 2 Modelվम ɹσʔλ߲໨௥ՃͳͲɻ 
ɹଟগݟʹͯ͘͘΋࢖͑Δঢ়ଶʹ͢Δ͜ͱ͕༏ઌɻ  ɹ࠷ѱͷ৔߹Ͱ΋Ͳ͏ʹ͔࢖͏ଆͰ޻෉Ͱ͖ΔͷͰɻ 3 Viewվम ɹάϥϑͷݟӫ͑ͳͲɻ  ɹҰ൪ʮมԽ͍ͯ͠ΔʯΠϯύΫτ → ؔ܎ߏஙʹ͸د༩ɻ  ɹ࢖͍खʹͱͬͯ͸ॏཁͳͷͰՄೳͳൣғͰαϙʔτɻ 4 γεςϜͷอकੑͱ  ύϑΥʔϚϯε ɹϘτϧωοΫʹͳͬͨΒରॲ͢Δɻ  ɹ࠷ॳ͔ΒڽΔͱʮૣ͗͢Δ࠷దԽʯʹؕΓ͕ͪɻ  ɹ࡞ۀνέοτʹ͢Δ·Ͱ΋ͳ͍ͭ͘Ͱʹ௚͢श׳Λ͚ͭΔɻ

ɹTest • unittest ʹΑΔࣗಈςετΛ׆༻  • Excel಺༰Λظ଴஋ʹ൓ө → GreenʹͳΕ͹࠶ݱ׬ྃ -
·͞ʹ “TDD”  • SQLνϡʔχϯά΍ϦϑΝΫλ࣌ʹσάϨΛଈ࣌ݕ஌Ͱ͖ΔͷͰ૬ੑྑ͍

ɹॏް௕େͳExcelΛ࠶ݱ͢Δˠ·ͣղಡ ϚΠϯυϚοϓʹΑΔ੔ཧ  εϥΠυʹೖΔΑ͏ʹʢೖΓ͖ͬͯͳ͍͚ͲʣॖΊͨΒը૾͕௵Εͯ͠΋͏ͨ……

ɹɹ1. Jupyter Notebook Ͱௐࠪ಺༰ͱ  ɹɹɹσʔλΛ࠶ݱՄೳͳܗͰอଘ͢Δ     ɹɹ2. Github Preview
Λͦͷ··  ɹɹɹઆ໌ࢿྉͱͯ͠ར༻͢Δ ɹطଘͷܭࢉϛε΍σʔλෆ੔߹Λൃݟ

ɹχʔζ։୓ͷ޷॥؀ ֤෦ॺͷϢʔεέʔεΛ  ώΞϦϯά͠΍͘͢ͳΔ ʮσʔλͷ͜ͱ͸͍ͭ͜ʹฉ͚ʯ  ʹ ࣭໰΍૬ஊ͕དྷΔΑ͏ʹͳΔ طଘσʔλͷϛε΍໰୊ɺ  ͞ΒͳΔհࡏ༨஍ʹؾ෇͚Δ খ͞ͳ࣮੷ 
طଘσʔλͷܭࢉϛε΍ෆ੔߹Λൃݟ  → ใࠂͯ͠௚͢

ͱʹ͔͘਺ࣈ͕߹Θͳ͍ ɹ޷॥؀Λࢧ͑Δ೜଱ γεςϜ࿈ܞͷ్தͰ࢛ࣺޒೖ͕ੵΈॏͳΔ Excel͕طʹ 4 x 1990 = 7959
ͱͳ͍ͬͯΔ ࣮ߦλΠϛϯάͰूܭ݁Ռ͕มΘΔ  → طଘͷ਺஋͸୭΋࠶ݱͰ͖ͳ͍

࣮ࡍʹඅ΍ͨ࣌ؒ͠ͷ80% ʹ ͻͨ͢Β஍ಓͳ࡞ۀ   ·͊ɺେมͰͨ͠ɻ ɹ׼ͱྦͷτϥϒϧγϡʔςΟϯά

ɹ֤෦ॺ΁ͷσʔλఏڙΛ࣮ݱ σʔλج൫

ɹ͜Ε·ͰɿϏδωε෦໳ͷϝΠϯExcel • σʔλͷ஝ੵ → ॏ͍ϑΝΠϧ    • ؔ਺/ηϧͷଟॏࢀর →
ௐࠪ/ղऍ͕ࠔ೉    • macOSͰ։͘ͱͳΜ͔յΕΔ ։͘ͷʹ3෼ɻಈ͔ͦ͏ͱ͢Δͱ·ͨࢭ·Δɻ ΤϯδχΞ͕ؾܰʹݟΒΕͳ͍ঢ়گ

ɹɹɹɹɹ ʴ ɹ͜Ε͔ΒɿTech + Culture = Value ςΫϊϩδʔ  จԽɾϓϩηε
σʔλΛ׆༻ͨ͠  ੡඼։ൃ ੈͷதʹ ސ٬Ձ஋Λఏڙ Done

ɹɹɹɹɹ ʴ ɹ͜Ε͔ΒɿTech + Culture = Value ςΫϊϩδʔ  จԽɾϓϩηε
σʔλΛ׆༻ͨ͠  ੡඼։ൃ ੈͷதʹ ސ٬Ձ஋Λఏڙ Done Todo

ɹϓϩμΫτ։ൃͱԾઆݕূ Build Measure Learn

ɹϓϩμΫτ։ൃͱԾઆݕূ Build Measure Learn Ͱ͖͍ͯΔ ΍ͬͯΈΔ

ɹɹͲΜͳϢʔβʔ͕৽ػೳΛ࢖ͬͨͷ͔  ɹɹϓϩμΫτʹͲΜͳظ଴Λ΋͍ͬͯΔͷ͔ ɹɹϏδωεKPI͸Ͳͷఔ౓޲্ͨ͠ͷ͔  ɹɹ෭࡞༻͸͔͋ͬͨʢ͋ΔػೳΛԡ͠ग़͢ → ผͷػೳ͕࢖ΘΕͳ͘ͳΔՄೳੑʣ ɹ࣮ࢪͨ͠ࢪࡦͷ݁ՌΛ෼ੳ IUUQHBUBHOFUGSFFJMMVTUSBJUJPO

ɹ͜Μͳײ͡Ͱ΍Γ·ͨ͠ 1. ϗϫΠτϘʔυʹ෼ੳཁ݅  ɹɹΧελϚʔߦಈભҠΛϕʔεͱͨ͠ࢦඪπϦʔ  ɹɹϩάΠϯ཰͕޲্ → ΞΫγϣϯ͕… → ച্׵ࢉ
  ɹɹɹɹ2. JupyterͰ෼ੳ  ɹɹɹɹɹίʔυ͸શͯίϛοτཤྺͰ؅ཧ    ɹɹɹɹɹɹɹɹ3. ෼ੳ݁ՌΛؔ܎ऀʹڞ༗  ɹɹɹɹɹɹɹɹɹGithubͷϓϨϏϡʔը໘Λ׆༻

ɹKeepɹϏδωεࢹ఺Ͱ։ൃΛݟΔ ࣗ෼ͷ੒ՌΛࣗ෼Ͱݴ͑ΔΑ͏ʹͳͬͨ  ʢྫɿ1ϲֻ݄͚ͨҊ݅Ͱച্xxԁ/೥ʣ 

ɹProblemɹνʔϜͷվળ༨஍͕ݦࡏԽ ɹɹϝϯόʔ͕ࣗ෼ͷཧղෆ଍ʹڻ͘    ɹɹɹ • ੡඼࢓༷ - ͜ͷέʔεͩͱσʔλͷத਎͸Ͳ͏ͳΔʁ  ɹɹɹ •
ϑϩϯτدΓϝϯόʔ͕ෳࡶͳSQLʹखؒऔΔ  ɹɹɹ • ͦ΋ͦ΋ϏδωεKPIΛ೺Ѳ͍ͯ͠Δ͔  ɹɹɹ • ͖ͪΜͱ෼ੳ͢ΔͨΊʹຊ౰͸ඞཁͩͬͨϩάཁ݅ͷ࿙Ε 

ɹTryɹϓϩμΫτࢤ޲νʔϜ΁ͷୈҰา ݴΘΕͨ΋ͷΛ࡞Δ͚ͩʢBuildॏࢹʣࢥߟ͔Βͷ୤٫ ʮ͞Βʹ਺ࣈΛ৳͹͢ʹ͸Ͳ͏͢Δʁʯ ࢪࡦΛىҊ͢ΔΤϯδχΞ͕ग़࢝ΊΔ Ϗδωε෦໳ͱนଧͪ 㲗 ࢼߦࡨޡ   αϯϓϧ࣮૷ɹษڧձͰڝ߹ͷΤϯδχΞΛั·͑ͯ࿩Λฉ͘ 
ࣾ಺ؔ܎ऀʹ໰୊ҙࣝΛώΞϦϯάɹ਺ࣈΛ΋ͱʹޮՌݟཱͯ νʔϑϓϩσϡʔαʔ͔ΒGoαΠϯ

ɹϓϩμΫτ։ൃͱԾઆݕূ Build Measure Learn Ͱ͖͍ͯΔ ΍ͬͯΈΔ

ɹ͍͔ʹਁಁͤ͞Δ͔ Build Measure Learn Ͱ͖͍ͯΔ ΍ͬͯΈΔ ਁಁͤ͞Δ

͋ͷख͜ͷखͰʮσʔλ෼ੳ΍Ζ͏ʯͷۭؾΛৢ੒ ɹ • σʔλ෼ੳϋοΧιϯͷ։࠵ ɹ • ே׆ x ࣗ༝ݚڀʢϏοτίΠϯͷ஋্͕ΓཁҼΛ౷ܭղੳ͢Δۚͷ๢ऀ͕େྔൃੜʣ ɹ
• JupyterHubΛ֦ுͨ͠PythonֶशαΠτ΍  ɹɹ֎ʹஔ͚ͳ͍σʔλ޲͚ͷnbviewerΛࣾ಺ఏڙ ɹϝοηʔδϯά

1ͭͷϞχλʔΛғΜͰશһͰ࡞ۀ͢Δ ɹɹ • पғ͕ௐ΂ͨΓΞυόΠε͠ͳ͕ΒਐΊΔ  ɹɹɹ → ϋϚΒͳ͍ / ࠳ંΛ๷͙ɺTips΍ίπΛڞ༗͋͠͏  ɹɹ
• օ͕΍ΔͳΒࣗ෼΋΍Δ͔ʂͷޙԡ͠  ɹɹɹ → ʮ΍ͬͯΈͨΒࢥͬͨҎ্ʹྑ͔ͬͨʯͷମݧ ɹϞϒ ϓϩάϥϛϯά σʔλ෼ੳ

  σʔλ࢓༷ʹৄ͍͠ʢগͳ͘ͱ΋ௐࠪ͢ΔεΩϧ͸΋͍ͬͯΔʣΤϯδχΞ͕  ୲౰ྖҬΛ޿͛Δ͜ͱͰલޙ޻ఔͷϦʔυλΠϜΛ୹ॖ  ɹϓϩηε૷ண - ։ൃ޻ఔʹ૊ΈࠐΉ ੡଄ ઃܭ ࢼݧ
ϦϦʔε ޮՌ  ଌఆ ཁ݅  ఆٛ ཁٻ  ੔ཧ ੡଄ ઃܭ ࢼݧ ϦϦʔε ޮՌ  ଌఆ ཁ݅  ఆٛ ཁٻ  ੔ཧ طʹ΍͍ͬͯΔ ෼ੳཁٻ ϩάཁ݅ ܭଌ ΞΠσΞ

ΤϯδχΞ ʹ ͦͷ৔Ͱσʔλ࢓༷Λௐ΂Δ͜ͱ͕Ͱ͖Δ   σʔλαΠΤϯε෦໳͕࢓༷ώΞϦϯάΛॏͶͯ  1ि͔͚͍ؒͯͨ෼ੳ͕1࣌ؒͰ׬ྃʢϑϩʔޮ཰Խʣ ɹ࣮੷ɿΞδϦςΟͷ޲্ ϝΠϯϞχλʔ͸  Jupyter
Notebook αΠυϞχλʔ͸  ϓϩμΫτͷιʔείʔυ ʴ

ɹϓϩμΫτ։ൃͱԾઆݕূ Build Measure Learn Ͱ͖͍ͯΔ ΍ͬͯΈΔ Ͱ͖͍ͯΔ

ɹLearn ࠓޙͷ՝୊  ·ͩࢼߦࡨޡதͰ͢

ɹؾ෇͖ - “Ծઆ” Λ஌Βͳ͍ͱ “ݕূ” Ͱ͖ͳ͍ IUUQXXXOJLLFJCQDPKQBUDMQVCNLUCPPL1 Ϗδϣϯ ઓུ
ઓज़

“Ծઆ” Λνʔϑϓϩσϡʔαʔʹ࣭໰ ↓    ߟ͑Λฉ͍ͨΤϯδχΞʮ͍΍ɺԶ͸͜͏ࢥ͏ʂʯ ↓  ϒϥϯυ࠶ݕ౼ϫʔΫγϣοϓͷ։࠵  ͲΜͳސ٬ʹͲΜͳՁ஋Λఏڙ͢Δͷ͔  ෦ॺΛԣஅ͓ͯ͠ޓ͍ͷߟ͑ΛͿ͚ͭ߹͏
ɹର࿩ͷ׆ੑԽ

֤ࣗͷҙݟ͕ʮ͋ͳͨͷײ૝Ͱ͢ΑͶʯʮͳΜ͔σʔλͱ͔͋ΔΜͰ͔͢ʯঢ়ଶ ɹҙݟͷͿ͚ͭ߹͍ → ݟ͖͑ͯͨ՝୊ ࣗ෼ͨͪ͸ސ٬ͷ͜ͱΛ  ຊ౰ʹཧղ͍ͯ͠Δͷ͔ʁ উखͳࢥ͍ࠐΈͰ  ϓϩμΫτΛ  ։ൃ͍ͯ͠ͳ͍͔ʁ

ɹϦΫϧʔτͷྺ࢙ Ӧۀ͕௚઀ళฮʹग़޲͍ͯ  σʔλΛऩू͍ͯͨ͠ σʔλʢࣄ࣮ʣΛ࣋ͪدͬͯ  ෦ॺؒͰҙݟΛͿ͚ͭ߹͏ ࢢ৔ͷ࠶ఆٛ  ৽͍͠ϏδωεՁ஋ͷߏங  ྫɿঁੑ޲͚ब৬ϝσΟΞ →
ࣾձਐग़ͷࢧԉ  ྫɿΫʔϙϯࡶࢽ → ༧໿αΠτ ΁ͷస׵ IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM IUUQXXXJSBTVUPZBDPNOFXQPQIUNM

4&$*Ϟσϧ ɹσʔλ͕ࢧ͑Δ஌ࣝ૑଄ ސ٬Ձ஋ উͪےɾUVP ΤϯδχΞ ू٬ σβΠϯ ηΩϡϦςΟ ๏຿
ΧελϚʔαϙʔτ ޿ใ ࢧԉऀɾ։୓ऀͱͯ͠ͷ  σʔλαΠΤϯςΟετ σʔλ

શͯ͸ ސ٬ཧղ ͔Β࢝·Δ σʔλ͸ͦͷͨΊͷ൑அࡐྉ

Ӧۀ͕଍Λ࢖ͬͯσʔλΛूΊΔ  ΫϥΠΞϯτͱపఈతʹ޲͖߹͍ʮউͪےʯΛݟ͚ͭΔ  ͦ͏΍ͬͯϦΫϧʔτͷ  ͜Ε·Ͱͷ50೥Λ࡞͖ͬͯͨ  ϝσΟΞΛ௨ͯ͠ੈքΛ࡞͖ͬͯͨ

ΤϯδχΞ͕खΛ࢖ͬͯσʔλΛूΊΔ  ΧελϚʔͱపఈతʹ޲͖߹͍ʮUVPʯΛݟ͚ͭΔ    ͦ͏͢Δ͜ͱ͕ϦΫϧʔτͷ  ͜Ε͔Βͷ50೥Λ࡞Δ͸ͣ  ϓϩμΫτΛ௨ͯ͠ੈքΛ࡞Δ͸ͣ

ϦΫϧʔτʹݶͬͨ࿩Ͱ͸ͳ͍    ͜Ε͔ΒͷੈքΛ࡞Δͷ͸ɺ  ୭ΑΓ΋ ੡඼ ͱ ސ٬ ʹਫ਼௨ͨ͠  ΤϯδχΞͷ໾ׂ
Ͱ͢  ͍·͜ͷ৔ʹ͍Δ ࢲͨͪࣗ਎ Ͱ͢   σʔλ׆༻͸ɺͦͷୈҰาͩͱࢥ͍ͬͯ·͢

σʔλΛ׆༻͢ΔͨΊʹ৐Γӽ͑Δน • େن໛૊৫ͰͲͷΑ͏ͳγεςϜɾϓϩηεΛ࠾༻͢Δ͔ʁ  • Buildॏࢹݱ৔ʹͲ͏΍ͬͯMeasureɾLearnΛ૷ண͢Δ͔ʁ 1ͭͷϓϩμΫτʹ͓͚ΔࣄྫΛڞ༗͠·ͨ͠ ɹຊ೔ͷৼΓฦΓ

σʔλ׆༻ͷਪਐʢ୅දͱͯ͠ൃද͠·͕࣮ͨ͠ࡍʹ͸൴Βͷޭ੷ʹΑΔͱ͜Ζ͕େ͖͍Ͱ͢ʣ ɹNaoya OsugiɹLee HeesungɹTatsuya IshibeɹTakuya BeppuɹTakahiro Kato  ɹKohei AotaɹRyo InoueɹHiroki
SakamotoɹShinryo UchidaɹShinsaku Kouno  ɹShin KanouchiɹGeeWook KimɹKoh Fujiwara  ٕज़ΞυόΠεɾ૊৫ࢧԉ ɹSho ItoɹShinya NishinakaɹNaoki Ainoya  ɹKeisuke SoneɹSatoshi UejimaɹNorihisa Miyakawa ൃදαϙʔτ ɹShun OnoɹYusaku TokunagaɹKazutaka SakuraiɹSaki Kato ※ࢿྉɾ಺༰ʹޡΓ΍ෆద੾ͳදݱ͕͋Ε͹ൃදऀͷϛεͰ͢ɻ@yuzutas0 ʹ͝࿈བྷ͍͚ͨͩΔͱ޾͍Ͱ͢ɻ ɹSpecial Thanks - ج൫ͷಓ͸1ਓʹͯ͠੒Βͣ

গ͠Ͱ΋ଟ͘ͷ ։ൃݱ৔ ͕  ੈͷதʹ ྑ͍ϓϩμΫτ Λఏڙ͢Δ  ώϯτ ʹͳΕͨΒͱࢥ͍ͬͯ·͢ ɹ఻͍͑ͨ͜ͱ

͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ presented by @yuzutas0  https://www.pexels.com/photo/close-up-of-computer-keyboard-257949/ɹhttps://www.pexels.com/photo/technology-computer-lines-board-50711/ɹhttps://www.pexels.com/photo/black-and-white-business-chart-computer-241544/

Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyco...

Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyconjp / 20170909

More Decks by yuzutas0

Other Decks in Technology

Featured

Transcript