Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyconjp / 20170909

yuzutas0
September 09, 2017

Jupyter(Python)とBigQueryによるデータ分析基盤のDevOps #pyconjp / 20170909

PyCon JP 2017 の発表資料です。ジャンルは「業務利用事例」となります。

追記1. PyCon JP 2017 ベストトークアワード優秀賞を受賞しました!
追記2. http://yuzutas0.hatenablog.com/entry/2017/09/12/203000 に補足を掲載しています!

yuzutas0

September 09, 2017
Tweet

More Decks by yuzutas0

Other Decks in Technology

Transcript

  1. Jupyter + BigQuery ʹΑΔ

    σʔλ෼ੳج൫ ͷ DevˍOps
    2017-09-09 (Sat)

    4:15 p.m. – 4:45 p.m. in Room 201

    PyCon JP 2017

    presented by @yuzutas0

    https://www.pexels.com/photo/close-up-of-computer-keyboard-257949/ɹhttps://www.pexels.com/photo/technology-computer-lines-board-50711/ɹhttps://www.pexels.com/photo/black-and-white-business-chart-computer-241544/

    View full-size slide

  2. ຊ೔ͷࢿྉ͸WEBʹެ։͠·͢

    #pyconjp #pyconjp_201


    ࡱӨɾه࿥ͷඞཁ͸͋Γ·ͤΜ

    ͭ͘Ζ͍Ͱௌ͍͍͚ͯͨͩΕ͹ͱࢥ͍·͢

    View full-size slide

  3. εϥΠυ 150+ / 30min

    ʢ஌ݟΛ͕ͬͭΓڞ༗͠·͢ʣ

    ͕Μ͕Μߦ͖·͢ʂ

    View full-size slide

  4. ɹSho Yokoyamaɹ@yuzutas0
    ɹɹ



    ● Recruit Technologies Co., Ltd. 

    ɹɹάϧʔϓͷITࢪࡦશൠΛ୲͏ byʰϦΫϧʔτɺਐԽΛࢭΊͳ͍ITݱ৔ྗʱ
    ● Site Reliability Engineering

    ɹɹج൫γεςϜͷӡ༻ɾվળ
    ● Finance & Econometrics (Before)

    ɹɹRʹΑΔ౷ܭॲཧ - ͋ͷͱ͖Jupyter͕͋Ε͹Α͔ͬͨͷʹʂ

    View full-size slide

  5. https://github.com/yuzutas0

    ෼ੳج൫
    ɹޱ΋ख΋ಈ͔͢೿

    View full-size slide

  6. ϓϩμΫτ։ൃ ʹ͓͚Δ

    σʔλ׆༻ ͷ ࢀߟࣄྫ ͷڞ༗

    ɹຊ೔ͷΰʔϧ

    View full-size slide

  7. Python Λ׆༻ͯ͠

    ։ൃݱ৔ ʹ σʔλจԽ Λਁಁ͍ͤͨ͞

    ιϑτ΢ΣΞΤϯδχΞ

    ɹ૝ఆ͢Δର৅

    View full-size slide

  8. ɹɹɹ
    ɹPythonͷ࿩
    ɹɹɹ
    "ɹPythonΛۀ຿Ͱ࢖͏࿩

    ɹPythonͷConferenceͰ͕͢
    த൫·Ͱग़͖ͯ·ͤΜʢ׼ʣ

    View full-size slide

  9. 1. ϓϩμΫτ / νʔϜ

    2. ෼ੳج൫ͷDev&Ops
    ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε

    3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

    ɹΞδΣϯμ

    View full-size slide

  10. 1. ϓϩμΫτ / νʔϜ

    2. ෼ੳج൫ͷDev&Ops
    ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε

    3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

    ɹΞδΣϯμ

    View full-size slide

  11. ֹ݄՝ۚϞσϧ

    ※࿀݁ͼঁੑϢʔβʔ͸ຊਓ֬ೝྉͷΈ

    ɹ͝ར༻ͷྲྀΕ

    View full-size slide

  12. ※αϒγεςϜͷٕज़ཁૉ: Golang, Groovy, Python, Ruby, AWS, GCP
    ɹɹɹɹɹɹɹɹɹɹɹɹΦϯϓϨ

    ɹ֤छσόΠεରԠ - γεςϜུ֓ (AP)

    View full-size slide

  13. ※ถࠃͰ͸݁ࠗͨ͠Χοϓϧͷ1/3͕ΦϯϥΠϯͰͷग़ձ͍

    ɹσʔςΟϯάࢢ৔ͷ֦େ

    View full-size slide


  14. ɹࣄۀ੒௕ → εςʔΫϗϧμʔͷଟ༷Խ
    σβΠϯ
    ηΩϡϦςΟ
    ๏຿
    Πϯϑϥ
    SRE
    ΞϓϦج൫
    ू٬
    ΧελϚʔ

    αϙʔτ
    σʔλ

    αΠΤϯε
    ػցֶश

    ΤϯδχΞ
    ։ൃӡ༻

    νʔϜA
    ։ൃӡ༻

    νʔϜB
    ։ൃӡ༻

    νʔϜC
    PO PO PO
    σΟϨΫγϣϯ
    ޿ใ

    View full-size slide

  15. 1. ϓϩμΫτ / νʔϜ

    2. ෼ੳج൫ͷDev&Ops
    ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε

    3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

    ɹΞδΣϯμ

    View full-size slide


  16. ɹ֤෦ॺ΁ͷσʔλఏڙ
    σʔλج൫

    View full-size slide

  17. 1. ϓϩμΫτ / νʔϜ

    2. ෼ੳج൫ͷDev&Ops
    ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε

    3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

    ɹΞδΣϯμ

    View full-size slide


  18. ɹͲͷΑ͏ʹ׆༻͍ͯ͠Δ͔
    σʔλج൫

    View full-size slide


  19. ɹσʔλ׆༻ࣄྫ
    ɹ1. KPIϞχλϦϯάɹɹ

    ɹ2. ϢʔβʔߦಈͷՄࢹԽ

    ɹ3. ࢪࡦޮՌͷ༧ଌɾܭଌ
    Ϗδωε
    4. ػցֶशʹΑΔϨίϝϯυ

    5. ۀऀɾεύϜͷࣗಈ൑ఆ
    ϓϩμΫτ
    ɹ6. ໰͍߹Θͤͷ܏޲෼ੳ

    ɹ7. ޿ࠂ഑৴ͷࣗಈ࠷దԽ

    ɹ8. ϓϨεϦϦʔε
    ΧελϚʔνϟωϧ
    ɹ9. γεςϜϞχλϦϯά

    ɹ10. ো֐ൃੜ࣌ͷӨڹௐࠪ
    γεςϜ
    ɹ11. νʔϜͷΩϟύγςΟ
    νʔϜ

    View full-size slide

  20. Ϗδωεࢦඪʢച্΍DAUʣͷਪҠ


    ຖேͷSlack௨஌ɹɹɹɹSpreadsheetɹɹɹɹμογϡϘʔυ

    ɹɹɹKPIϞχλϦϯά
    1

    View full-size slide

  21. ओཁಋઢʢձһొ࿥ˠϝοηʔδʣͷར༻ɾ཭୤
    ߦಈϑΝϯωϧɹɹɹɹɹɹδϟʔχʔϚοϓ

    ɹɹɹϢʔβʔߦಈͷՄࢹԽ
    2

    View full-size slide

  22. Split (AB) ςετ݁Ռɹɹ৽ػೳͷར༻౓߹͍

    ɹɹɹࢪࡦͷޮՌ༧ଌˍܭଌ
    3

    View full-size slide


  23. ɹɹɹػցֶशʹΑΔϨίϝϯυ
    4

    View full-size slide


  24. ɹɹɹۀऀɾεύϜͷࣗಈ൑ఆ
    5

    View full-size slide


  25. ɹɹɹ໰͍߹Θͤͷ܏޲෼ੳ
    6

    View full-size slide


  26. ɹɹɹ޿ࠂ഑৴ͷࣗಈ࠷దԽ
    7

    View full-size slide

  27. σʔλ෼ੳ݁Ռͷެ։ → ࢢ৔ͷ׆ੑԽʹߩݙ

    ɹɹɹϓϨεϦϦʔε
    8
    IUUQXXXSFDSVJUNQDPKQOFXTSFMFBTF@IUNM

    View full-size slide

  28. Ϋϥογϡ཰΍ϨεϙϯελΠϜ

    ɹɹɹγεςϜϞχλϦϯά
    9
    IUUQZV[VUBTIBUFOBCMPHDPNFOUSZ

    View full-size slide


  29. ɹɹɹো֐ൃੜ࣌ͷӨڹௐࠪ
    10

    View full-size slide

  30. εΫϥϜνʔϜͷϕϩγςΟ ΍ νέοτফԽͷϝτϦΫε

    ɹɹɹνʔϜͷΩϟύγςΟ
    11

    View full-size slide

  31. σβΠϯ
    ηΩϡϦςΟ
    ๏຿
    Πϯϑϥ
    SRE
    ΞϓϦج൫
    ू٬
    ΧελϚʔ

    αϙʔτ
    σʔλ

    αΠΤϯε
    ػցֶश

    ΤϯδχΞ
    ։ൃӡ༻

    νʔϜA
    ։ൃӡ༻

    νʔϜB
    ։ൃӡ༻

    νʔϜC
    PO PO PO
    σΟϨΫγϣϯ
    ޿ใ
    ։ൃӡ༻

    νʔϜB
    PO

    ɹ͜ΕΒͷࢪࡦ : ֤෦ॺ͕ϘτϜΞοϓͰਪਐ

    View full-size slide

  32. σʔλ׆༻Ͱ࣮ݱͰ͖Δ͜ͱ

    ɹސ٬Ձ஋ͷ௥ٻ → σʔλ׆༻͸ඞવ
    1. ࣗಈԽ 2. ՄࢹԽ
    ސ٬Ձ஋ͷఏڙ

    View full-size slide

  33. “ΑΓૣ͘ɾ҆ఆͨ͠” αʔϏεͷఏڙ

    ɹ1. ࣗಈԽ
    ػցֶशͰީิऀϦετΛࣄલநग़
    ໨ࢹ͚ͩͰۀऀɾεύϜΛ൑ఆ
    IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM

    View full-size slide

  34. “ΑΓޮՌతͳ” ΤϯδχΞϦϯάͷ࣮ݱ

    ɹ2. ՄࢹԽ
    ͜ͷσβΠϯ͸Ϣʔβʔͷ

    ຊԻʹ౴͍͑ͯΔͷ͔ʁ
    ͜ͷػೳ௥Ճ͸ޮՌ͕͋Δͷ͔ʁ

    ࣮ࡍʹޮՌ͸͋ͬͨͷ͔ʁ
    IUUQHBUBHOFUGSFFJMMVTUSBJUJPO IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM

    View full-size slide

  35. σβΠϯ
    ηΩϡϦςΟ
    ๏຿
    Πϯϑϥ
    SRE
    ΞϓϦج൫
    ू٬
    ΧελϚʔ

    αϙʔτ
    σʔλ

    αΠΤϯε
    ػցֶश

    ΤϯδχΞ
    ։ൃӡ༻

    νʔϜA
    ։ൃӡ༻

    νʔϜB
    ։ൃӡ༻

    νʔϜC
    PO PO PO
    σΟϨΫγϣϯ
    ޿ใ
    ։ൃӡ༻

    νʔϜB
    PO

    ɹ֤෦ॺʹΑΔੵۃతͳσʔλ׆༻

    View full-size slide

  36. 1. ϓϩμΫτ / νʔϜ

    2. ෼ੳج൫ͷDev&Ops
    ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε

    3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

    ɹΞδΣϯμ

    View full-size slide


  37. ɹͳͥج൫͕ඞཁʹͳ͔ͬͨ
    σʔλج൫

    View full-size slide

  38. DBσʔλΛ࢖͍͍ͨͷͰ

    ೔࣍ͰS3ʹCSVग़ྗ͍ͯͩ͘͠͞


    ɹ࠷ॳ͸খ͞ͳσʔλૄ௨ґཔ
    ෦ॺA

    View full-size slide


  39. ɹεέʔϧ → ֤෦ॺ͔Βґཔ
    ෦ॺB
    ෦ॺC
    ෦ॺD

    View full-size slide

  40. ͬͪ͜Ͱαʔόͱૄ௨πʔϧΛ༻ҙ͠·͢

    ϨϏϡʔˍຊ൪؀ڥʹஔ͍ͯ΋Β͑·͔͢

    ɹͪΐͬͱֻ͕͔࣌ؒΓͦ͏
    ੜσʔλΛ౉ͯ͠΋Β͑Ε͹

    ͬͪ͜Ͱ࢖͑ΔΑ͏ʹՃ޻͢ΔͷͰ
    ෦ॺE

    View full-size slide


  41. ɹ֤෦ॺ͝ͱʹσʔλૄ௨
    ෦ॺA

    πʔϧ
    ෦ॺB

    πʔϧ
    ෦ॺA

    αʔό
    ෦ॺB

    αʔό

    View full-size slide


  42. ɹ֤෦ॺ͝ͱʹσʔλՃ޻
    ෦ॺA

    நग़σʔλ
    ෦ॺB

    நग़σʔλ
    ෦ॺA

    Ճ޻ϩδοΫ
    ෦ॺB

    Ճ޻ϩδοΫ

    View full-size slide


  43. ɹମ੍ͷ··ʹγεςϜෳࡶԽ - ίϯ΢ΣΠͷ๏ଇ
    ݩσʔλ
    ֤෦ॺ޲͚ʹσʔλࢀরγεςϜΛܧ͗଍͠

    View full-size slide


  44. http://jp.techcrunch.com/2017/06/25/20170623five-building-blocks-of-a-data-driven-culture/
    ελοϑ͕ͨͪҟͳΔγεςϜ͔ΒࣅͨΑ͏ͳϝτϦοΫΛ

    Ҿ͖ग़͢͜ͱ͕Ͱ͖ͯ͠·͏ͳΒɺඞવతʹͦΕΒͷγεςϜ͔Β͸

    ҟͳΔ਺ࣈ͕ੜ੒͞ΕΔɻ…ΑΓ·͠ͳ৘ใݯΛ࢖͑Δഺͩͬͨͷʹɺ

    ݹͯ͘඼࣭ͷ௿͍ɺ͋Δ͍͸ؒҧͬͨσʔλ΍ࢦඪΛ஌Βͣʹ

    ࢖ͬͯ͠·͍ɺѱ͍ܾఆΛԼ͢νʔϜ΋ग़ͯ͠·͏͔΋஌Εͳ͍ɻ

    ɹ෦ॺ͝ͱʹʮച্ʯ͕ζϨΔ໰୊

    View full-size slide


  45. ɹΤϯδχΞʹௐࠪґཔᶃ - σʔλ࢓༷͸೔ʑมΘΔ
    ͜ͷϨίʔυ͕

    ௥Ճ͞ΕΔλΠϛϯάɺ

    ࠷ۙมߋ͞Ε·ͨ͠ʁ
    ͪΐͬͱมߋཤྺΛ

    ௥ͬͯΈ·͢Ͷ
    ෦ॺF ΤϯδχΞ

    View full-size slide


  46. ɹΤϯδχΞʹௐࠪґཔᶄ - Өڹൣғ͸೔ʑ޿͕Δ
    Re:dashͷ਺ࣈ͕

    ζϨΔΜͰ͚͢Ͳ
    ଞ෦ॺ͕࡞ͬͨγεςϜΛ

    ௐ΂Δͱ͜Ζ͔Β……
    ෦ॺG ΤϯδχΞ

    View full-size slide


  47. ɹݸผରԠͷѱ॥؀
    ಠࣗσʔλૄ௨ͷґཔ

    ʢ͜ΕͳΒ͙͢Ͱ͖·͢ΑͶʣ
    ΤϯδχΞνʔϜͷෛՙˢ
    γεςϜෳࡶԽˢ
    ௐࠪґཔ

    View full-size slide


  48. ɹ୭͔ͷ൑அ͕ѱ͔ͬͨΘ͚Ͱ͸ͳ͍
    εέʔϧ΍ຊ֨Քಇ࣌ʹ

    ໰୊͕ݦࡏԽ
    ɹɹ·ͣ͸খ͘͞ࢼ͢

    ɹ ● ࠷খݶͷσʔλૄ௨

    ɹ ● ࣗલͰσʔλՃ޻
    ෦ॺ͝ͱʹ

    ৽πʔϧಋೖ΍ࢪࡦணख
    IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM
    IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM

    View full-size slide


  49. http://jp.techcrunch.com/2017/06/25/20170623five-building-blocks-of-a-data-driven-culture/

    ɹମ੍֦େ → σʔλ΢ΣΞϋ΢εͷඞཁੑ
    ࣄ࣮ͷ୯Ұ৘ใݯΛ͍࣋ͬͯΔ৔߹ʹ͸ɺΞφϦετ΍

    ଞͷҙࢥܾఆऀͱ͍ͬͨΤϯυϢʔβʔͨͪʹɺ༏ΕͨՁ஋Λ

    ఏڙ͢Δ͜ͱ͕Ͱ͖Δɻ൴Β͸૊৫಺ͰσʔλΛ୳͕࣌ؒ͢

    গͳͯ͘ࡁΉΑ͏ʹͳΓɺσʔλͷར༻ʹΑΓଟ͘ͷ࣌ؒΛ

    ׂ͘͜ͱ͕Ͱ͖ΔΑ͏ʹͳΔ͔Βͩɻ

    View full-size slide

  50. 1. ϓϩμΫτ / νʔϜ

    2. ෼ੳج൫ͷDev&Ops
    ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε

    3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

    ɹΞδΣϯμ

    View full-size slide


  51. ɹͲͷΑ͏ͳγεςϜΛߏங͔ͨ͠
    σʔλج൫

    View full-size slide

  52. 1. ModelͱViewΛ෼͚Δ΂͠

    2. ͳΔ΂͘ϥΫͯ͠࡞Δ΂͠

    ɹઃܭϙϦγʔ

    View full-size slide

  53. 1. ModelͱViewΛ෼͚Δ΂͠

    2. ͳΔ΂͘ϥΫͯ͠࡞Δ΂͠

    ɹઃܭϙϦγʔ

    View full-size slide


  54. ɹModelͱViewΛ෼͚Δ΂͠
    ɹɹɹɹɹɹɹɹɹɹɹɹɹDWH
    ੹຿ʹԠͨ͡γεςϜͷ෼཭ɾ݁߹

    Model (஝ੵ/Ճ޻) ͱView (ࢀর)

    View full-size slide


  55. ɹ஫ҙɿ෦ॺ͝ͱʹ࠷దͳView͸ҟͳΔ
    Excel
    ؾܰʹ਺ࣈΛม͑ͯ

    γϛϡϨʔγϣϯ
    Ϗδωε෦໳
    Tableau
    ߴՁ֨ɾߴػೳ

    ෼ੳཁٻʹରԠ
    Re:dash
    SQL͕ॻ͚Δͻͱ

    ͓खܰར༻
    σΟϨΫλʔ
    Jupyter

    GitͰίʔυ؅ཧ

    ϓϩάϥϜͷԸܙ
    ΤϯδχΞ
    ෼ੳ෦໳

    View full-size slide

  56. 1. ModelͱViewΛ෼͚Δ΂͠

    2. ͳΔ΂͘ϥΫͯ͠࡞Δ΂͠

    ɹઃܭϙϦγʔ

    View full-size slide


  57. ɹจ໌ͷརث
    σʔλૢ࡞ͳΒPythonҰ୒
    ΤίγεςϜʹ৐Δ
    σʔλอଘͳΒBigQueryҰ୒

    GoogleʹཔΔ
    ΞϓϦ૚ Πϯϑϥ

    View full-size slide


  58. ɹจ໌ͷརث
    σʔλૢ࡞ͳΒPythonҰ୒
    ΤίγεςϜʹ৐Δ
    σʔλอଘͳΒBigQueryҰ୒

    GoogleʹཔΔ
    ΞϓϦ૚ Πϯϑϥ

    View full-size slide

  59. ɹɹɹ● ʢRͱൺ΂ͯʣॻ͖΍͘͢ಡΈ΍࣮͘͢ߦ͠΍͍͢

    ɹɹɹ● ൚༻తͳϓϩάϥϜॲཧΛॻ͚ΔεΫϦϓτݴޠ

    ɹPythonͷ࠾୒ཧ༝ - ൚༻LL

    View full-size slide


  60. ɹɹɹɹ● ʢଞͷ൚༻ݴޠʹൺ΂ͯʣσʔλͷՃ޻΍ඳը͕ಘҙ

    ɹɹɹɹ● DB㱻Dataframeม׵ͳͲଞγεςϜ࿈ܞ΋༰қ

    ɹPythonͷ࠾୒ཧ༝ - σʔλૢ࡞޲͚ϥΠϒϥϦ

    View full-size slide


  61. ɹɹɹ● σόοά΍σʔλͷՄࢹԽ͕ίϚϯυ1ͭͰՄೳ

    ɹɹɹ● ಺෦తʹ͸json → GitͰࠩ෼؅ཧ → ࠶ݱՄೳͳه࿥Λ࢒ͤΔ

    ɹPythonͷ࠾୒ཧ༝ - ࠷ڧͷ࣮ߦ؀ڥ

    View full-size slide

  62. ػցֶशࢪࡦεΫϦϓτ

    Ҡ২Մೳ → γεςϜͷ෼཭ɾ݁߹͕༰қ → རศੑɾอकੑ

    ɹPythonͷ࠾୒ཧ༝ - طଘεΫϦϓτͱͷ૬ੑ

    View full-size slide


  63. ɹจ໌ͷརث
    σʔλૢ࡞ͳΒPythonҰ୒
    ΤίγεςϜʹ৐Δ
    σʔλอଘͳΒBigQueryҰ୒

    GoogleʹཔΔ
    Πϯϑϥ
    ΞϓϦ૚

    View full-size slide


  64. ɹBQͷ࠾୒ཧ༝ɿ͍҆
    IUUQZBQDBTJBPSHUBMLTIPXDFBGBDFDBCEDB
    ● Googleͷσʔληϯλʔ

    ɹεέʔϧϝϦοτʹΑΔ

    ɹίετ༏Ґ
    ● ϢʔβʔͷҙݟΛ΋ͱʹ

    ɹྉۚϓϥϯΛݟ௚ͨ͠ܦҢ

    View full-size slide


  65. ɹBQͷ࠾୒ཧ༝ɿૣ͍
    IUUQZBQDBTJBPSHUBMLTIPXDFBGBDFDBCEDB
    ● ΧϥϜϕʔεઃܭʹΑΔ

    ɹΞΫηεߴ଎Խ
    ● େن໛σʔλ׆༻࣌ʹ

    ɹϘτϧωοΫͱͳΔ

    ɹDisk I/Oͷฒྻॲཧ

    View full-size slide


  66. ɹBQͷ࠾୒ཧ༝ɿ҆৺
    GCPɾGsuiteͰݖݶ؅ཧ - ΈΜͳ͕࢖͑Δ
    SQLΠϯλʔϑΣʔε - ಺෦ॲཧɾอकӡ༻Λؾʹͤͣʹ࢖͑Δ
    ඪ४SQL

    ΫΤϦͷҠ২ੑ͕ߴ͍

    ֶशίετ͕খ͍͞

    ςετ͠΍͍͢
    पลπʔϧ

    ओཁBIπʔϧͳΒେ఍ܨ͕Δ

    pandas.io.gbq → Jupyterσόοά͕༰қ

    View full-size slide


  67. ɹจ໌ͷརث
    σʔλૢ࡞ͳΒPythonҰ୒
    ΤίγεςϜʹ৐Δ
    σʔλอଘͳΒBigQueryҰ୒

    GoogleʹཔΔ
    Πϯϑϥ
    ΞϓϦ૚

    View full-size slide

  68. 1. ModelͱViewΛ෼͚Δ΂͠

    2. ͳΔ΂͘ϥΫͯ͠࡞Δ΂͠

    ɹઃܭϙϦγʔ

    View full-size slide

  69. 1. ϓϩμΫτ / νʔϜ

    2. ෼ੳج൫ͷDev&Ops
    ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε

    3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

    ɹΞδΣϯμ

    View full-size slide


  70. ɹσʔλϑϩʔͷߏங
    ऩू Ճ޻ ׆༻
    ஝ੵ
    ύΠϓϥΠϯ؅ཧ
    Model View

    View full-size slide


  71. ɹσʔλϑϩʔͷߏங
    ऩू Ճ޻ ׆༻
    ஝ੵ
    Model View
    ऩू
    ύΠϓϥΠϯ؅ཧ

    View full-size slide


  72. ɹσʔλऩू
    ݸਓ৘ใ

    ϚεΫࡁΈView
    Recruit Ads
    Console
    Recruit Clash

    Report
    όον
    ετϦʔϛϯά
    WebAPI
    εΫϨΠϐϯά
    requests +
    beautiful soup

    View full-size slide

  73. ओཁͳDBɾϩά͔Βͷసૹ → ࠷খઃఆͰ࣮ݱՄೳ

    ɹσʔλసૹΛࢧ͑Δٕज़

    View full-size slide

  74. Python (requests + beautiful soup) ʹΑΔ

    WebAPIίʔϧɾը໘εΫϨΠϐϯά

    ɹJupyterͰૉૣ͘ࢼ͢
    ·ͣ͸ϩʔΧϧ؀ڥͰࡁ·ͤΔ ܁Γฦ͠࢖͏ͳΒεΫϦϓτԽ
    IUUQTXXXTIBSFJDPOOFUEPDVNFOUpMFQZ

    View full-size slide

  75. ΤϯδχΞ͕෼ੳʹܞΘ͍ͬͯͳ͍ͱEventσʔλ͕ϩάཁ͔݅Βൈ͚࿙Ε͕ͪ

    ෆཁσʔλͷա৒஝ੵʹΑΔετϨʔδѹഭΛݒ೦ʢͦ͜ͰBQͰ͢Αʣ

    ɹσʔλੑ࣭ʹԠͨ͡ऩूํ๏
    4UBUFঢ়ଶσʔλ &WFOUཤྺσʔλ
    ྫ Ϣʔβʔͷ࠷ऴϩάΠϯ೔࣌ʢ্ॻ͖͞ΕΔʣ աڈͷϩάΠϯཤྺʢ஝ੵ͞ΕΔʣ
    ओͳ༻్
    ը໘දࣔ

    ʮ͜ͷਓ͸೔લʹϩάΠϯ͠·ͨ͠Αʯ
    ෼ੳ

    Ͳͷ͘Β͍ͷස౓ͰϩάΠϯ͢Δ͔
    ڧΈɾಛ௃
    ϓϩμΫτຊମͷύϑΥʔϚϯε޲্

    ͍͍ͪͪཤྺςʔϒϧͱ+PJOͯ͠

    ࠷৽ϨίʔυΛ୳͢ͷ͸%#ෛՙ
    σʔλ෼ੳͷ࠶ݱੑΛ୲อ


    ຖճϩάΠϯ೔࣌Λ্ॻ͖͞ΕͨΒ

    ෼ੳͰ͖ͳ͘ͳͬͯ͠·͏
    ෼ੳج൫Ͱͷ

    σʔλऩू
    ೔࣍ͰҰׅஔ͖׵͑

    %#ϩάˠࠩ෼ه࿥ˠ׆༻͸ͻͱ޻෉ඞཁ
    ࠩ෼൓ө

    લճҎ߱ͷϨίʔυ͚ͩΛநग़

    View full-size slide


  76. ɹσʔλϑϩʔͷߏங
    ऩू Ճ޻ ׆༻
    ஝ੵ
    Model View
    ஝ੵ
    ύΠϓϥΠϯ؅ཧ

    View full-size slide

  77. 3૚ߏ଄

    ࢀߟʰ10೥ઓ͑Δσʔλ෼ੳೖ໳ - SQLΛ෢ثʹσʔλ׆༻࣌୅Λੜ͖ൈ͘ʱ

    ɹσʔλ஝ੵ
    ɹɹɹɹɹɹBigQuery - Google Cloud Platform
    Source

    ݩσʔλͷίϐʔ
    Warehouse

    ओཁࢦඪɾසग़σʔλ

    தؒςʔϒϧ
    App

    ֤πʔϧ͔Β௚઀ࢀর

    ඇٕज़ऀ޲͚IF
    ஝ੵ

    View full-size slide

  78. ྫɿ enmusubi__source__db
    ΞϯμʔείΞ2ͭͰཁૉΛ۠੾Δ

    BEMʢCSSͷઃܭख๏ʣϥΠΫͳنଇ

    ɹσʔληοτͷ໋໊نଇ
    ϓϩμΫτ໊ source
    warehouse
    app
    ϝΠϯDB
    αʔόϩά
    ΞΫηεղੳπʔϧ

    View full-size slide


  79. ɹσʔλࣙॻ
    IUUQKQUFDIDSVODIDPNpWFCVJMEJOHCMPDLTPGBEBUBESJWFODVMUVSF
    ར༻ऀ͸σʔλϑΟʔϧυͱͦͷ஋ʢϝτϦΫεʣͷҙຯΛ஌Δඞཁ͕͋Δɻ*
    ݩσʔλ৘ใ΁ͷϦϯΫूͷΈఏڙ

    γεςϜʢʹσʔλ࢓༷ʣ͸೔ʑมԽ͢ΔͨΊ

    refs. ϩθολετʔϯ
    ֤σʔλιʔεଆͰ࢓༷؅ཧ

    e.g. ຊମDB → MySQL Workbench

    ɹERߏ੒/ίϝϯτ͸DDLͰৗʹ࠷৽Խ

    View full-size slide

  80. ൃల్্ͷϓϩμΫτɹ → ɹػೳ௥ՃͰසൟͳERมߋɹ → ɹBQ΋௥ਵ͍ͨ͠

    ɹ஝ੵσʔλʹϚΠάϨʔγϣϯ͕ඞཁ

    View full-size slide


  81. ɹJupyter for Migration
    1. ϩʔΧϧͷJupyterͰ

    ɹϚΠάϨʔγϣϯεΫϦϓτΛॻ͘
    2. ͦͷ··JupyterͰ

    ɹ1೔෼ͷσʔλΛର৅ʹಈ࡞֬ೝ
    3. PythonϑΝΠϧΛग़ྗ

    ɹ → αʔό্ͰશσʔλΛҠߦ
    IUUQTXXXTIBSFJDPOOFUEPDVNFOUpMFQZ

    View full-size slide

  82. ※ͨͩ͠Python2.7੍໿͕͋ΔͷͰҰ෦ॻ͖௚͕͠ඞཁ → ύλʔϯ͕ݟ͖͑ͯͨΒม׵εΫϦϓτ
    ● Φʔτεέʔϧ → Dataflow

    ɹGKE͸෼ࢄ޲͚ͷઃܭ͕ඞཁ

    ● σʔλྔ͕গͳ͚Ε͹Datalab

    ɹJupyterͷίʔυ͕··࢖͑Δ
    ɹGCPͷαʔόΛར༻
    ɹ ● Googleઐ༻ઢͰBQΞΫηε

    ɹ ● ಉ͡ถࠃϦʔδϣϯʹཱͯΔ

    ɹBQͰ100ԯϨίʔυΛҠߦ͢Δ
    ϘτϧωοΫᶃ N/W ϘτϧωοΫᶄ ϝϞϦ

    View full-size slide


  83. ɹσʔλϑϩʔͷߏங
    ऩू Ճ޻ ׆༻
    ஝ੵ
    Model View
    Ճ޻
    ύΠϓϥΠϯ؅ཧ

    View full-size slide

  84. 3૚ߏ଄

    ࢀߟʰ10೥ઓ͑Δσʔλ෼ੳೖ໳ - SQLΛ෢ثʹσʔλ׆༻࣌୅Λੜ͖ൈ͘ʱ

    ɹσʔλՃ޻
    ɹɹɹɹɹɹBigQuery - Google Cloud Platform
    Source

    ݩσʔλͷίϐʔ
    Warehouse

    ओཁࢦඪɾසग़σʔλ

    தؒςʔϒϧ
    App

    ֤πʔϧ͔Β௚઀ࢀর

    ඇٕज़ऀ޲͚IF
    Ճ޻ Ճ޻

    View full-size slide


  85. IUUQTHSPXUIIBDLKPVSOBMDPNLQJUSFFGPSBQQ
    ɹओཁࢦඪΛதؒςʔϒϧͰ؅ཧ

    View full-size slide


  86. ɹPythonεΫϦϓτʹΑΔՃ޻

    View full-size slide


  87. ɹsh → exec.py
    ● γΣϧҾ਺Λ΋ͱʹॲཧΛಈతৼΓ෼͚

    ● ݸʑͷΤϯυϙΠϯτ͸ࡉ෼Խ

    ɹˠ ΦʔτεέʔϧɺϦτϥΠɺಈ࡞֬ೝ͠΍͘͢

    View full-size slide


  88. ɹController
    ● ॲཧͷશମ૾Λهड़

    ● Facadeͱͯ͠ଞॲཧΛݺͼग़͢

    View full-size slide


  89. ɹModel
    ● Ϗδωεࢦඪͱ1ର1ͰରԠ͢ΔΫϥε

    ● Create/ReadॲཧͷΈ → ஋͸BQ؅ཧ

    ɹ → ϓϩάϥϜ্͸εςʔτϨεʹอͭ

    View full-size slide


  90. ɹQuery
    ● BigQueryͱͷR/WΛѻ͏SQL

    ● ςΩετॲཧͷࢧԉػೳΛ׆༻

    ɹ → f-strings (3.6~)΍TypeHints (3.5~)

    ● ϝιουͱͯ͠੾Γग़͠

    ɹ → ஋ͷҾ͖౉͠ʹΑΔείʔϓ੍ޚ

    View full-size slide


  91. ɹEnum
    ● Enum(3.4~)Ͱσʔλݻ༗ͷίʔυ஋Λ؅ཧ

    ● ੡඼ຊମ (Java) ͷEnumΛҠ২ → Dictܕʹม׵

    ● ܧঝݩͷbaseEnumΛ༻ҙ → ڞ௨ͷσʔλऔಘॲཧΛ࡞੒

    View full-size slide


  92. ɹResource
    ● ֎෦γεςϜͱͷ࿈ܞॲཧ

    ● retryॲཧͷҰݩԽ → σʔλྔ͕૿͑ΔͱBQ͸Αࣦ͘ഊ͢ΔͷͰ

    ● dotenvͰ؀ڥม਺ΛಡΈࠐΉ → ຊ൪ɾ։ൃͰSlackνϟϯωϧΛ෼͚Δ

    View full-size slide


  93. ɹσʔλϑϩʔͷߏங
    ऩू Ճ޻ ׆༻
    ஝ੵ
    Model View
    ׆༻
    ύΠϓϥΠϯ؅ཧ

    View full-size slide


  94. ɹσʔλ׆༻ - Pushܕ
    IUUQGSFFJMMVTUSBUJPOTHBUBHOFUIUNM
    ػցֶशࢪࡦ

    View full-size slide

  95. ΞυϗοΫ
    ࣗಈԽ

    ɹσʔλ׆༻ - Pullܕ

    View full-size slide


  96. ɹ୭ʹεΫϦϓτΛ༻ҙ͢Δ͔
    Yes No
    ͦͷ࡞ۀ͸

    ܁Γฦ͢ʁ
    ϞχλϦϯά

    ఆظతͳॲཧ

    ʲPushܕʳʢҰ෦ྫ֎͋Γʣ
    εφοϓγϣοτ

    ΞυϗοΫͳ෼ੳɾௐࠪ

    ʲPullܕʳ
    SQLΘ͔Δ

    ͻͱɾ෦ॺʁ
    ELTɿૄ௨ˠՃ޻

    ʢྫʣػցֶशΤϯδχΞ

    ʲPullܕʳ
    ETLɿՃ޻ˠૄ௨

    SpreadsheetͰI/F

    ʲPushܕʳ
    ※Ճ޻ࡁΈσʔλʹରͯ͠γϯϓϧͳSELECTจΛൃߦ͢Δ͚ͩͰࡁΉέʔε͸ଟ͍

    View full-size slide


  97. ɹDecorator
    ● ֤View޲͚ͷσʔλՃ޻ʢग़ྗઌʹԠͨ͡ม׵ॲཧʹಛԽʣ

    ● ྫɿຖேSlackʹ౤͛ΔάϥϑΛ matplotlib Ͱඳը

    View full-size slide


  98. ɹσʔλϑϩʔͷߏங
    ऩू Ճ޻ ׆༻
    ஝ੵ
    Model View
    ύΠϓϥΠϯ؅ཧ

    View full-size slide

  99. ؅ཧπʔϧʹظ଴͢Δཁ݅ʢཧ૝ʣ



    ɹσʔλύΠϓϥΠϯͷ؅ཧ
    1. ίʔυͰόʔδϣϯ؅ཧ

    2. εέδϡʔϧࣗಈىಈ

    3. ηοτΞοϓ΍ϝϯςφϯεͷָ͞(DB಺แͳͲ)

    4. GUI/CUI྆ํͰϩάɾύϑΥʔϚϯεͷ؅ཧ

    5. GUI/CUI྆ํͰม਺ࢦఆͷ࠶࣮ߦ
    6. ࣗಈϦτϥΠॲཧɾεΩοϓػೳ

    7. εΫϦϓτͷฒྻ࣮ߦ

    8. αʔόͷΦʔτεέʔϧ

    9. ༗޲ඇ८ճάϥϑͷࣗಈ࡞੒

    10. Python3.6~ରԠʢςΩετॲཧ͕֨ஈʹָʣ

    View full-size slide

  100. ࿩୊ͷπʔϧΛܰ͘৮ͬͯࢼͯ͠ΈͨΓ

    ɹɹɹɹɹɹ

    ɹɹɹɹɹͲΕ΋ັྗత͚ͩͲҰேҰ୹ → ͔Ώ͍ͱ͜Ζʹख͕ಧ͔ͳ͍

    ɹɹɹɹɹɹ ● ίϯτϦϏϡʔτͷνϟϯε͔΋ʁ

    ɹɹɹɹɹɹ ● ೔ਐ݄าͰັྗతͳͷͰ͍ͣΕҠ২͍ͨ͠ؾ࣋ͪ͸͋Δ

    ɹWorkflow Engine

    View full-size slide

  101. ͳΜ͔ͩΜͩͰ࠷ॳʹ͓खܰߏஙͨ͠ jenkins Λ࢖͍ଓ͚͍ͯΔ


    ɹɹɹɹɹɹ● 2ܥ + BlueOcean + Pipeline of jenkinsfiles

    ɹɹɹɹɹɹ● ϦτϥΠػߏɾॲཧ࠷దԽ͸ଞʹྼΔ

    ɹɹɹɹɹɹ● jenkinsδϣϒ͔ΒγΣϧܦ༝Ͱ֤छॲཧΛίʔϧ

    ɹ҆ఆͷJenkins

    View full-size slide


  102. ɹσʔλϑϩʔͷߏஙʢ׬ʣ
    ऩू Ճ޻ ׆༻
    ஝ੵ
    Model View
    ύΠϓϥΠϯ؅ཧ

    View full-size slide

  103. 1. ϓϩμΫτ / νʔϜ

    2. ෼ੳج൫ͷDev&Ops
    ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε

    3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

    ɹΞδΣϯμ

    View full-size slide


  104. ɹͲͷΑ͏ͳϓϩηεΛ࠾༻͔ͨ͠
    σʔλج൫

    View full-size slide

  105. ʮσʔλج൫Λ࡞Δͧʯܭը͸
    ࣦഊ͔Β࢝·ͬͨ

    View full-size slide


  106. μογϡϘʔυΛ

    ෦԰ͷϞχλʔʹө͢
    ɹ࠷ॳͷҰา

    View full-size slide


  107. 1. ຖ೔ͷKPIΛ͢Β͢Βݴ͑ΔΑ͏ʹͳΔ͔΋


    ɹ2. ҟৗ஋ʹؾ͍ͮͯਝ଎ʹಈ͚ΔΑ͏ʹͳΔ͔΋
    ɹߴ·Δظ଴

    View full-size slide


  108. 1िؒͰ୭΋ݟͳ͘ͳͬͨ
    ɹݱ࣮͸ඇ৘Ͱ͋Δ

    View full-size slide

  109. ɹɹɹ׬ᘳͳاըॻʁ
    ɹɹɹѹ౗తͳ༧ࢉʁ
    ɹɹɹ1೥ؒͷ։ൃϓϩδΣΫτʁ

    ɹʮ࠷ڧͷσʔλج൫ʯΛࢧ͑Δϓϩηεʁ

    View full-size slide

  110. ɹɹɹ׬ᘳͳاըॻʁ
    ɹɹɹѹ౗తͳ༧ࢉʁ
    ɹɹɹ1೥ؒͷ։ൃϓϩδΣΫτʁ

    ɹʮ࠷ڧͷσʔλج൫ʯΛࢧ͑Δϓϩηεʁ

    View full-size slide

  111. ɹɹ● ࢥ͍ͬͯͨͷͱҧͬͨ
    ɹɹ● ྑͦ͞͏͚ͩͲ݁ہ࢖Θͳ͔ͬͨ

    ɹ1೥ޙʹ଴ͭ΋ͷ

    View full-size slide


  112. ɹʮԶͷߟ͑ͨ࠷ڧͷσʔλج൫ʯ͸࢖ΘΕͳ͍
    IUUQXXXQSPKFDUDBSUPPODPN

    View full-size slide

  113. → ͜ΕͳΒ࣮֬ʹ࢖ͬͯ΋Β͑Δ

    ɹগͣͭ͠ஔ͖׵͑Α͏

    ࣮͸͏ͪͷ෦ॺͰຖேSlackʹάϥϑΛ

    खಈͰྲྀ͍ͯ͠ΔΜ͚ͩͲʂ

    ͜ͷػʹࣗಈԽ͠·͔͢ʂ

    View full-size slide

  114. ɹɹɹQ.ɹ࠷ॳʹཁ݅Λશ෦ચ͍ग़ͤ͹OKʁ


    ɹɹɹA.ɹNGʂʮ΍Γ͍ͨ͜ͱʯ͸มΘΔʂ

    ɹɹɹɹɹɹɹ ● ଞࣾϒϩάΛݟͯʮ͏ͪ΋΍Γ͍ͨʂʯ

    ɹɹɹɹɹɹɹ ● ৽͍͠BIπʔϧͷఏҊʹʮ͜Ε͍͍ʂʯ

    ɹɹɹɹɹɹɹ ● ࣮ࡍʹը໘Λ࡞ͬͨΒʮͳΜ͔ҧ͏ʂʯ

    ɹɹ ɹɹ → มԽʹॊೈʹରԠ͢Δʂ

    ɹΠςϨʔγϣϯΛճ͢͜ͱ͕େࣄ

    View full-size slide

  115. TodoΛνέοτ؅ཧ → ॊೈʹ༏ઌॱҐΛೖΕସ͑Δ

    ɹνέοτۦಈ։ൃ

    View full-size slide


  116. ɹλεΫͷ༏ઌॱҐ
    1 σʔλ͕·͕͍ͪͬͯΔ

    ٙ࿭ͷௐࠪ
    ɹ1ͭؒҧ͍ͬͯΔͱશ෦৴༻Ͱ͖ͳ͘ͳΔͷͰ࠷༏ઌରԠɻ

    ɹ͙͢ௐࠪ͢Ε͹ؔ܎ऀͷ৴༻࢒ߴΛ૿΍ͤΔɻ
    2 Modelվम
    ɹσʔλ߲໨௥ՃͳͲɻ

    ɹଟগݟʹͯ͘͘΋࢖͑Δঢ়ଶʹ͢Δ͜ͱ͕༏ઌɻ

    ɹ࠷ѱͷ৔߹Ͱ΋Ͳ͏ʹ͔࢖͏ଆͰ޻෉Ͱ͖ΔͷͰɻ
    3 Viewվम
    ɹάϥϑͷݟӫ͑ͳͲɻ

    ɹҰ൪ʮมԽ͍ͯ͠ΔʯΠϯύΫτ → ؔ܎ߏஙʹ͸د༩ɻ

    ɹ࢖͍खʹͱͬͯ͸ॏཁͳͷͰՄೳͳൣғͰαϙʔτɻ
    4 γεςϜͷอकੑͱ

    ύϑΥʔϚϯε
    ɹϘτϧωοΫʹͳͬͨΒରॲ͢Δɻ

    ɹ࠷ॳ͔ΒڽΔͱʮૣ͗͢Δ࠷దԽʯʹؕΓ͕ͪɻ

    ɹ࡞ۀνέοτʹ͢Δ·Ͱ΋ͳ͍ͭ͘Ͱʹ௚͢श׳Λ͚ͭΔɻ

    View full-size slide


  117. ɹTest
    ● unittest ʹΑΔࣗಈςετΛ׆༻

    ● Excel಺༰Λظ଴஋ʹ൓ө → GreenʹͳΕ͹࠶ݱ׬ྃ - ·͞ʹ “TDD”

    ● SQLνϡʔχϯά΍ϦϑΝΫλ࣌ʹσάϨΛଈ࣌ݕ஌Ͱ͖ΔͷͰ૬ੑྑ͍

    View full-size slide

  118. ɹॏް௕େͳExcelΛ࠶ݱ͢Δˠ·ͣղಡ
    ϚΠϯυϚοϓʹΑΔ੔ཧ

    εϥΠυʹೖΔΑ͏ʹʢೖΓ͖ͬͯͳ͍͚ͲʣॖΊͨΒը૾͕௵Εͯ͠΋͏ͨ……

    View full-size slide

  119. ɹɹ1. Jupyter Notebook Ͱௐࠪ಺༰ͱ

    ɹɹɹσʔλΛ࠶ݱՄೳͳܗͰอଘ͢Δ


    ɹɹ2. Github Preview Λͦͷ··

    ɹɹɹઆ໌ࢿྉͱͯ͠ར༻͢Δ

    ɹطଘͷܭࢉϛε΍σʔλෆ੔߹Λൃݟ

    View full-size slide


  120. ɹχʔζ։୓ͷ޷॥؀
    ֤෦ॺͷϢʔεέʔεΛ

    ώΞϦϯά͠΍͘͢ͳΔ
    ʮσʔλͷ͜ͱ͸͍ͭ͜ʹฉ͚ʯ

    ʹ ࣭໰΍૬ஊ͕དྷΔΑ͏ʹͳΔ
    طଘσʔλͷϛε΍໰୊ɺ

    ͞ΒͳΔհࡏ༨஍ʹؾ෇͚Δ
    খ͞ͳ࣮੷

    طଘσʔλͷܭࢉϛε΍ෆ੔߹Λൃݟ

    → ใࠂͯ͠௚͢

    View full-size slide

  121. ͱʹ͔͘਺ࣈ͕߹Θͳ͍

    ɹ޷॥؀Λࢧ͑Δ೜଱
    γεςϜ࿈ܞͷ్தͰ࢛ࣺޒೖ͕ੵΈॏͳΔ
    Excel͕طʹ 4 x 1990 = 7959 ͱͳ͍ͬͯΔ
    ࣮ߦλΠϛϯάͰूܭ݁Ռ͕มΘΔ

    → طଘͷ਺஋͸୭΋࠶ݱͰ͖ͳ͍

    View full-size slide

  122. ࣮ࡍʹඅ΍ͨ࣌ؒ͠ͷ80% ʹ ͻͨ͢Β஍ಓͳ࡞ۀ

    ·͊ɺେมͰͨ͠ɻ

    ɹ׼ͱྦͷτϥϒϧγϡʔςΟϯά

    View full-size slide


  123. ɹ֤෦ॺ΁ͷσʔλఏڙΛ࣮ݱ
    σʔλج൫

    View full-size slide

  124. 1. ϓϩμΫτ / νʔϜ

    2. ෼ੳج൫ͷDev&Ops
    ɹɹϢʔεέʔεɹߏஙͷഎܠɹઃܭϙϦγʔɹσʔλϑϩʔɹ։ൃϓϩηε

    3. σʔλจԽΛ૊৫ʹ૷ண͢Δ

    ɹΞδΣϯμ

    View full-size slide


  125. ɹ͜Ε·ͰɿϏδωε෦໳ͷϝΠϯExcel
    ● σʔλͷ஝ੵ → ॏ͍ϑΝΠϧ


    ● ؔ਺/ηϧͷଟॏࢀর → ௐࠪ/ղऍ͕ࠔ೉


    ● macOSͰ։͘ͱͳΜ͔յΕΔ
    ։͘ͷʹ3෼ɻಈ͔ͦ͏ͱ͢Δͱ·ͨࢭ·Δɻ
    ΤϯδχΞ͕ؾܰʹݟΒΕͳ͍ঢ়گ

    View full-size slide

  126. ɹɹɹɹɹ ʴ
    ɹ͜Ε͔ΒɿTech + Culture = Value

    ςΫϊϩδʔ

    จԽɾϓϩηε
    σʔλΛ׆༻ͨ͠

    ੡඼։ൃ
    ੈͷதʹ
    ސ٬Ձ஋Λఏڙ
    Done

    View full-size slide

  127. ɹɹɹɹɹ ʴ
    ɹ͜Ε͔ΒɿTech + Culture = Value

    ςΫϊϩδʔ

    จԽɾϓϩηε
    σʔλΛ׆༻ͨ͠

    ੡඼։ൃ
    ੈͷதʹ
    ސ٬Ձ஋Λఏڙ
    Done Todo

    View full-size slide


  128. ɹϓϩμΫτ։ൃͱԾઆݕূ
    Build
    Measure
    Learn

    View full-size slide


  129. ɹϓϩμΫτ։ൃͱԾઆݕূ
    Build
    Measure
    Learn
    Ͱ͖͍ͯΔ
    ΍ͬͯΈΔ

    View full-size slide

  130. ɹɹͲΜͳϢʔβʔ͕৽ػೳΛ࢖ͬͨͷ͔

    ɹɹϓϩμΫτʹͲΜͳظ଴Λ΋͍ͬͯΔͷ͔
    ɹɹϏδωεKPI͸Ͳͷఔ౓޲্ͨ͠ͷ͔

    ɹɹ෭࡞༻͸͔͋ͬͨʢ͋ΔػೳΛԡ͠ग़͢ → ผͷػೳ͕࢖ΘΕͳ͘ͳΔՄೳੑʣ

    ɹ࣮ࢪͨ͠ࢪࡦͷ݁ՌΛ෼ੳ
    IUUQHBUBHOFUGSFFJMMVTUSBJUJPO

    View full-size slide


  131. ɹ͜Μͳײ͡Ͱ΍Γ·ͨ͠
    1. ϗϫΠτϘʔυʹ෼ੳཁ݅

    ɹɹΧελϚʔߦಈભҠΛϕʔεͱͨ͠ࢦඪπϦʔ

    ɹɹϩάΠϯ཰͕޲্ → ΞΫγϣϯ͕… → ച্׵ࢉ

    ɹɹɹɹ2. JupyterͰ෼ੳ

    ɹɹɹɹɹίʔυ͸શͯίϛοτཤྺͰ؅ཧ


    ɹɹɹɹɹɹɹɹ3. ෼ੳ݁ՌΛؔ܎ऀʹڞ༗

    ɹɹɹɹɹɹɹɹɹGithubͷϓϨϏϡʔը໘Λ׆༻

    View full-size slide

  132. ɹKeepɹϏδωεࢹ఺Ͱ։ൃΛݟΔ
    ࣗ෼ͷ੒ՌΛࣗ෼Ͱݴ͑ΔΑ͏ʹͳͬͨ

    ʢྫɿ1ϲֻ݄͚ͨҊ݅Ͱച্xxԁ/೥ʣ


    View full-size slide

  133. ɹProblemɹνʔϜͷվળ༨஍͕ݦࡏԽ
    ɹɹϝϯόʔ͕ࣗ෼ͷཧղෆ଍ʹڻ͘


    ɹɹɹ ● ੡඼࢓༷ - ͜ͷέʔεͩͱσʔλͷத਎͸Ͳ͏ͳΔʁ

    ɹɹɹ ● ϑϩϯτدΓϝϯόʔ͕ෳࡶͳSQLʹखؒऔΔ

    ɹɹɹ ● ͦ΋ͦ΋ϏδωεKPIΛ೺Ѳ͍ͯ͠Δ͔

    ɹɹɹ ● ͖ͪΜͱ෼ੳ͢ΔͨΊʹຊ౰͸ඞཁͩͬͨϩάཁ݅ͷ࿙Ε


    View full-size slide

  134. ɹTryɹϓϩμΫτࢤ޲νʔϜ΁ͷୈҰา
    ݴΘΕͨ΋ͷΛ࡞Δ͚ͩʢBuildॏࢹʣࢥߟ͔Βͷ୤٫

    ʮ͞Βʹ਺ࣈΛ৳͹͢ʹ͸Ͳ͏͢Δʁʯ
    ࢪࡦΛىҊ͢ΔΤϯδχΞ͕ग़࢝ΊΔ
    Ϗδωε෦໳ͱนଧͪ 㲗 ࢼߦࡨޡ

    αϯϓϧ࣮૷ɹษڧձͰڝ߹ͷΤϯδχΞΛั·͑ͯ࿩Λฉ͘

    ࣾ಺ؔ܎ऀʹ໰୊ҙࣝΛώΞϦϯάɹ਺ࣈΛ΋ͱʹޮՌݟཱͯ
    νʔϑϓϩσϡʔαʔ͔ΒGoαΠϯ

    View full-size slide


  135. ɹϓϩμΫτ։ൃͱԾઆݕূ
    Build
    Measure
    Learn
    Ͱ͖͍ͯΔ
    ΍ͬͯΈΔ

    View full-size slide


  136. ɹ͍͔ʹਁಁͤ͞Δ͔
    Build
    Measure
    Learn
    Ͱ͖͍ͯΔ
    ΍ͬͯΈΔ ਁಁͤ͞Δ

    View full-size slide

  137. ͋ͷख͜ͷखͰʮσʔλ෼ੳ΍Ζ͏ʯͷۭؾΛৢ੒
    ɹ ● σʔλ෼ੳϋοΧιϯͷ։࠵
    ɹ ● ே׆ x ࣗ༝ݚڀʢϏοτίΠϯͷ஋্͕ΓཁҼΛ౷ܭղੳ͢Δۚͷ๢ऀ͕େྔൃੜʣ
    ɹ ● JupyterHubΛ֦ுͨ͠PythonֶशαΠτ΍

    ɹɹ֎ʹஔ͚ͳ͍σʔλ޲͚ͷnbviewerΛࣾ಺ఏڙ

    ɹϝοηʔδϯά

    View full-size slide

  138. 1ͭͷϞχλʔΛғΜͰશһͰ࡞ۀ͢Δ
    ɹɹ ● पғ͕ௐ΂ͨΓΞυόΠε͠ͳ͕ΒਐΊΔ

    ɹɹɹ → ϋϚΒͳ͍ / ࠳ંΛ๷͙ɺTips΍ίπΛڞ༗͋͠͏

    ɹɹ ● օ͕΍ΔͳΒࣗ෼΋΍Δ͔ʂͷޙԡ͠

    ɹɹɹ → ʮ΍ͬͯΈͨΒࢥͬͨҎ্ʹྑ͔ͬͨʯͷମݧ

    ɹϞϒ ϓϩάϥϛϯά σʔλ෼ੳ

    View full-size slide


  139. σʔλ࢓༷ʹৄ͍͠ʢগͳ͘ͱ΋ௐࠪ͢ΔεΩϧ͸΋͍ͬͯΔʣΤϯδχΞ͕

    ୲౰ྖҬΛ޿͛Δ͜ͱͰલޙ޻ఔͷϦʔυλΠϜΛ୹ॖ


    ɹϓϩηε૷ண - ։ൃ޻ఔʹ૊ΈࠐΉ
    ੡଄
    ઃܭ ࢼݧ ϦϦʔε
    ޮՌ

    ଌఆ
    ཁ݅

    ఆٛ
    ཁٻ

    ੔ཧ
    ੡଄
    ઃܭ ࢼݧ ϦϦʔε
    ޮՌ

    ଌఆ
    ཁ݅

    ఆٛ
    ཁٻ

    ੔ཧ
    طʹ΍͍ͬͯΔ
    ෼ੳཁٻ ϩάཁ݅ ܭଌ
    ΞΠσΞ

    View full-size slide

  140. ΤϯδχΞ ʹ ͦͷ৔Ͱσʔλ࢓༷Λௐ΂Δ͜ͱ͕Ͱ͖Δ

    σʔλαΠΤϯε෦໳͕࢓༷ώΞϦϯάΛॏͶͯ

    1ि͔͚͍ؒͯͨ෼ੳ͕1࣌ؒͰ׬ྃʢϑϩʔޮ཰Խʣ

    ɹ࣮੷ɿΞδϦςΟͷ޲্
    ϝΠϯϞχλʔ͸

    Jupyter Notebook
    αΠυϞχλʔ͸

    ϓϩμΫτͷιʔείʔυ
    ʴ

    View full-size slide


  141. ɹϓϩμΫτ։ൃͱԾઆݕূ
    Build
    Measure
    Learn
    Ͱ͖͍ͯΔ
    ΍ͬͯΈΔ Ͱ͖͍ͯΔ

    View full-size slide

  142. ɹLearn
    ࠓޙͷ՝୊

    ·ͩࢼߦࡨޡதͰ͢

    View full-size slide


  143. ɹؾ෇͖ - “Ծઆ” Λ஌Βͳ͍ͱ “ݕূ” Ͱ͖ͳ͍
    IUUQXXXOJLLFJCQDPKQBUDMQVCNLUCPPL1
    Ϗδϣϯ
    ઓུ
    ઓज़

    View full-size slide

  144. “Ծઆ” Λνʔϑϓϩσϡʔαʔʹ࣭໰
    ↓


    ߟ͑Λฉ͍ͨΤϯδχΞʮ͍΍ɺԶ͸͜͏ࢥ͏ʂʯ
    ↓

    ϒϥϯυ࠶ݕ౼ϫʔΫγϣοϓͷ։࠵

    ͲΜͳސ٬ʹͲΜͳՁ஋Λఏڙ͢Δͷ͔

    ෦ॺΛԣஅ͓ͯ͠ޓ͍ͷߟ͑ΛͿ͚ͭ߹͏

    ɹର࿩ͷ׆ੑԽ

    View full-size slide

  145. ֤ࣗͷҙݟ͕ʮ͋ͳͨͷײ૝Ͱ͢ΑͶʯʮͳΜ͔σʔλͱ͔͋ΔΜͰ͔͢ʯঢ়ଶ

    ɹҙݟͷͿ͚ͭ߹͍ → ݟ͖͑ͯͨ՝୊
    ࣗ෼ͨͪ͸ސ٬ͷ͜ͱΛ

    ຊ౰ʹཧղ͍ͯ͠Δͷ͔ʁ
    উखͳࢥ͍ࠐΈͰ

    ϓϩμΫτΛ

    ։ൃ͍ͯ͠ͳ͍͔ʁ

    View full-size slide


  146. ɹϦΫϧʔτͷྺ࢙
    Ӧۀ͕௚઀ళฮʹग़޲͍ͯ

    σʔλΛऩू͍ͯͨ͠
    σʔλʢࣄ࣮ʣΛ࣋ͪدͬͯ

    ෦ॺؒͰҙݟΛͿ͚ͭ߹͏
    ࢢ৔ͷ࠶ఆٛ

    ৽͍͠ϏδωεՁ஋ͷߏங

    ྫɿঁੑ޲͚ब৬ϝσΟΞ → ࣾձਐग़ͷࢧԉ

    ྫɿΫʔϙϯࡶࢽ → ༧໿αΠτ ΁ͷస׵
    IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM IUUQXXXJSBTVUPZBDPNCMPHQPTU@IUNM IUUQXXXJSBTVUPZBDPNOFXQPQIUNM

    View full-size slide

  147. 4&$*Ϟσϧ

    ɹσʔλ͕ࢧ͑Δ஌ࣝ૑଄
    ސ٬Ձ஋ উͪےɾUVP
    ΤϯδχΞ ू٬
    σβΠϯ ηΩϡϦςΟ
    ๏຿
    ΧελϚʔαϙʔτ ޿ใ
    ࢧԉऀɾ։୓ऀͱͯ͠ͷ

    σʔλαΠΤϯςΟετ
    σʔλ

    View full-size slide

  148. શͯ͸ ސ٬ཧղ ͔Β࢝·Δ
    σʔλ͸ͦͷͨΊͷ൑அࡐྉ

    View full-size slide

  149. Ӧۀ͕଍Λ࢖ͬͯσʔλΛूΊΔ

    ΫϥΠΞϯτͱపఈతʹ޲͖߹͍ʮউͪےʯΛݟ͚ͭΔ

    ͦ͏΍ͬͯϦΫϧʔτͷ

    ͜Ε·Ͱͷ50೥Λ࡞͖ͬͯͨ

    ϝσΟΞΛ௨ͯ͠ੈքΛ࡞͖ͬͯͨ

    View full-size slide

  150. ΤϯδχΞ͕खΛ࢖ͬͯσʔλΛूΊΔ

    ΧελϚʔͱపఈతʹ޲͖߹͍ʮUVPʯΛݟ͚ͭΔ


    ͦ͏͢Δ͜ͱ͕ϦΫϧʔτͷ

    ͜Ε͔Βͷ50೥Λ࡞Δ͸ͣ

    ϓϩμΫτΛ௨ͯ͠ੈքΛ࡞Δ͸ͣ

    View full-size slide


  151. ϦΫϧʔτʹݶͬͨ࿩Ͱ͸ͳ͍


    ͜Ε͔ΒͷੈքΛ࡞Δͷ͸ɺ

    ୭ΑΓ΋ ੡඼ ͱ ސ٬ ʹਫ਼௨ͨ͠

    ΤϯδχΞͷ໾ׂ Ͱ͢

    ͍·͜ͷ৔ʹ͍Δ ࢲͨͪࣗ਎ Ͱ͢

    σʔλ׆༻͸ɺͦͷୈҰาͩͱࢥ͍ͬͯ·͢

    View full-size slide

  152. σʔλΛ׆༻͢ΔͨΊʹ৐Γӽ͑Δน
    ● େن໛૊৫ͰͲͷΑ͏ͳγεςϜɾϓϩηεΛ࠾༻͢Δ͔ʁ

    ● Buildॏࢹݱ৔ʹͲ͏΍ͬͯMeasureɾLearnΛ૷ண͢Δ͔ʁ
    1ͭͷϓϩμΫτʹ͓͚ΔࣄྫΛڞ༗͠·ͨ͠

    ɹຊ೔ͷৼΓฦΓ

    View full-size slide

  153. σʔλ׆༻ͷਪਐʢ୅දͱͯ͠ൃද͠·͕࣮ͨ͠ࡍʹ͸൴Βͷޭ੷ʹΑΔͱ͜Ζ͕େ͖͍Ͱ͢ʣ
    ɹNaoya OsugiɹLee HeesungɹTatsuya IshibeɹTakuya BeppuɹTakahiro Kato

    ɹKohei AotaɹRyo InoueɹHiroki SakamotoɹShinryo UchidaɹShinsaku Kouno

    ɹShin KanouchiɹGeeWook KimɹKoh Fujiwara

    ٕज़ΞυόΠεɾ૊৫ࢧԉ
    ɹSho ItoɹShinya NishinakaɹNaoki Ainoya

    ɹKeisuke SoneɹSatoshi UejimaɹNorihisa Miyakawa
    ൃදαϙʔτ
    ɹShun OnoɹYusaku TokunagaɹKazutaka SakuraiɹSaki Kato
    ※ࢿྉɾ಺༰ʹޡΓ΍ෆద੾ͳදݱ͕͋Ε͹ൃදऀͷϛεͰ͢ɻ@yuzutas0 ʹ͝࿈བྷ͍͚ͨͩΔͱ޾͍Ͱ͢ɻ

    ɹSpecial Thanks - ج൫ͷಓ͸1ਓʹͯ͠੒Βͣ

    View full-size slide

  154. গ͠Ͱ΋ଟ͘ͷ ։ൃݱ৔ ͕

    ੈͷதʹ ྑ͍ϓϩμΫτ Λఏڙ͢Δ

    ώϯτ ʹͳΕͨΒͱࢥ͍ͬͯ·͢

    ɹ఻͍͑ͨ͜ͱ

    View full-size slide

  155. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
    presented by @yuzutas0

    https://www.pexels.com/photo/close-up-of-computer-keyboard-257949/ɹhttps://www.pexels.com/photo/technology-computer-lines-board-50711/ɹhttps://www.pexels.com/photo/black-and-white-business-chart-computer-241544/

    View full-size slide