$30 off During Our Annual Pro Sale. View Details »

月額コーヒー3.34杯分のコストでオオタニサンの活躍を見守るデータ基盤のはなし / Pyhack Con

月額コーヒー3.34杯分のコストでオオタニサンの活躍を見守るデータ基盤のはなし / Pyhack Con

#pyhack conference 2023/06/17

Shinichi Nakagawa

June 17, 2023
Tweet

More Decks by Shinichi Nakagawa

Other Decks in Technology

Transcript

  1. ίʔώʔ໿3.34ഋͷֹ݄ྉۚͰ


    σʔλج൫Λ࡞ͬͯ࢖͏࿩
    Google Cloudͷ͋Ε͜ΕͱࣆJAPANͷͦͷޙ.
    @shinyorke 2023/06/17 Python mini Hack-a-thon Conference

    View Slide

  2. Who am I?(ࣗݾ঺հ)
    • Shinichi Nakagawa@shinyorke


    • ๭େख֎ࢿITίϯαϧاۀϚωʔδϟʔʢݱ৬ʣ


    • #Pyhack ॳࢀՃ: 2013/02/16ʢ#28ʣ, ଟ෼ৗ࿈ʢͱ͍͏ೝࣝʣ


    • ໺ٿ͕Ұ൪ͷझຯͱݟ͔͚ͤͯͦ͏Ͱ΋ͳ͍ਓ.


    • ωλ࿮ͱͯ͠ͷʮ໺ٿͷਓʯ͸طʹଔۀ͍ͯ͠·͢.

    ͨ·ͨ·ݸਓ։ൃωλ͕໺ٿʹدͬͯΔ͚ͩͰ͢.

    View Slide

  3. ࠓ೔ͷ͓࿩
    • #PyHack #148ʢ2023/02/04ʣͰൃද࿅शͨ͠σϒαϛ2023ొஃωλ

    …ͷμΠδΣετ൛ʢ10෼Ver.ʣΛ͓൸࿐໨͠·͢.


    • ొஃͨ͠ͷ͕2݄Ͱࠓ͸4݄ͳͷͰΞοϓσʔτ΋গ͋͠Γ·͢.


    • σϒαϛຊฤͷಈըʢ໿40෼ʣ͸ެ։͞Ε͍ͯ·͢.

    https://codezine.jp/devonline/archive/session/131

    View Slide

  4. σϒαϛͰ࿩ͨ͜͠ͱʢཁ໿ʣ
    • Ϋϥ΢υͱσʔλΤϯδχΞϦϯάͰ࣮ݱ͢ΔԶͷDX


    • αʔόϨεͳΞʔΩςΫνϟͰ࣮ݱ͢Δʮ͍͍ײ͡ͳσʔλج൫ʯ


    • ʮਪ͠ͷ໺ٿબखʯͷʮਪ࣌͠ʯ͸Ͳ͜ͳͷ͔ΛݟۃΊΔ⽁

    View Slide

  5. ͜͏͍͏ϞϊΛ࢖ͬͯ…
    ※σϒαϛࢿྉΑΓൈਮ

    View Slide

  6. ϥʔζɾψʔτόʔ


    ʢࣆJAPANʣ
    • ࣆJAPANॳͷΞϝϦΧग़਎બख


    • ଧ཰͸௿͍΋ͷͷग़ྥ཰ͱ௕ଧ཰

    ͦͯ͠ଧٿ଎౓͕଎͘, 3ϙδγϣϯ

    ͦͭͳ͘कΕΔ֎໺कඋ΋˕


    • Ωϟϥཱ͍͍ͪͯͯ͠Ϡπͱ͍͏ᷚ,

    ࣆδϟύϯͷηϯλʔͱͯ͠ظ଴ʂ
    ※σϒαϛࢿྉΑΓൈਮ

    View Slide

  7. ϥʔζɾψʔτόʔ


    ʢࣆJAPANʣ
    • ࣆJAPANॳͷΞϝϦΧग़਎બख


    • ଧ཰͸௿͍΋ͷͷग़ྥ཰ͱ௕ଧ཰

    ͦͯ͠ଧٿ଎౓͕଎͘, 3ϙδγϣϯ

    ͦͭͳ͘कΕΔ֎໺कඋ΋˕


    • Ωϟϥཱ͍͍ͪͯͯ͠Ϡπͱ͍͏ᷚ,

    ࣆδϟύϯͷηϯλʔͱͯ͠ظ଴ʂ
    8#$͕։ນ͢Δલʢʣ
    ͜ͷ࣌఺Ͱψʔτόʔͷ׆༂Λ༧ݴ͢Δ͜ͱʹ੒ޭʂ
    ˞*5ۀքΤϯδχΞք۾࠷଎ͷൃදʢଟ෼ʣ
    Ұ൪όολʔͱͯ͠ͷ׆༂͸༧૝֎͕ͩͬͨͦΕ͸͓͖ͯ͞স

    View Slide

  8. ࠓ೔ͷ10෼LTͰશ෦৮ΕΔͷ͸ແཧ.


    Google Cloudͱ໺ٿͷ࿩Λগʑ.

    View Slide

  9. ͓͠ͳ͕͖
    • DashͱCloud RunͰσʔλɾμογϡϘʔυΛ࡞ͬͯಈ͔͢


    • WBCઓ࢜ͨͪͷͦͷޙ⽁


    • ͦͯ͠ؾʹͳΔ͓஋ஈ͸?

    View Slide

  10. ࡞ͬͨσʔλɾμογϡϘʔυʢͬ͘͟Γʣ
    •DashΛ࢖ͬͯʮPythonॻ͚͹μογϡϘʔυ࡞ΕΔʯΑ͏ʹ͠·ͨ͠.
    •σʔλຊମ͸BigQuery, σʔλ΁ͷΞΫηε͸RESTful APIʢGoʣܦ༝.
    •ೝূ໨తͰAPI Gateway, Cacheͱͯ͠StorageΛར༻ʢ͜Ε͸Ϛετ͡Όͳ͍ʣ.

    View Slide

  11. Dash is Կऀ🤔
    • PlotlyΛLow-codeπʔϧͱͯ͠

    ࢖͑ΔΑ͏ʹ͍͍ͨ͠ײ͡ͷౕ.


    • ϓϩτλΠϓͰ͸෼ੳɾՄࢹԽʹ

    Jupyter LabͱPlotlyΛ࢖͓ͬͯΓ,

    ͜ΕΛͦͷ··Ҡ২Ͱ͖ΔखஈͰ

    ࣮૷͔ͨͬͨ͠.


    • https://dash.plotly.com/

    View Slide

  12. ࠓճ࡞ͬͨίʔυ

    ʢҰ෦ൈਮʣ
    • Python͚ͩͰॻ͚Δͷ͸ΊͬͪΌศརͩͬͨ.


    • HTMLʹ໛ͨ͠ίϯϙʔωϯτΛPythonͰ

    ΰϦͬͱॻ͍ͯ࡞੒͢Δͱ͍͍ײ͡ʹಈ͘.


    • ΠϕϯτۦಈͰͷॻ͖׵͑͸Callbackͳ

    σίϨʔλʔͰ࣮૷.


    • ReactͬΆ͋ͬͯ͞ݸਓతʹ͸େ޷͖͕ͩ,

    ςετ͕௒ॻ͖ʹ͍͘ҹ৅…

    ※গͳ͘ͱ΋UI ίϯϙʔωϯτʹؔͯ͠

    View Slide

  13. Cloud RunΛબ୒ͨ͠ʢPythonicͳʣཧ༝
    • μογϡϘʔυΞϓϦͷґଘϥΠϒϥϦతʹApp EngineͰಈ͔͘ո͔ͬͨͨ͠Ί


    • PandasʹDashʢPlotlyʣͱσʔλܥͷϥΠϒϥϦ͕੝Γͩ͘͞ΜͷWebΞϓϦ


    • ͜ͷ࣌఺ͰͪΌΜͱͨ͠ContainerܥͰಈ͔͞ͳ͍ͱବ໨ͩͳͬͯ൑அˠͦͷ௨Γͩͬͨ


    • બ୒͢Δ·Ͱͷࢥߟͱͯ͠͸…


    1.ґଘϥΠϒϥϦͳͲͷ౎߹Ͱ΄΅PythonʢҰ෦GoʣͰͷ։ൃ -> Firebase͕ީิ͔Βফ͑Δ


    2.Cloud FunctionsͰಈ͔͢ʹ͸ඍົͳ࢓༷ͱͳͬͨͨΊબ୒ࢶͱͯ͠ফڈ


    3.App EngineʢStandardʣ͸Container͡Όͳ͍ -> Cloud Runʹܾఆ

    ※ContainerͰಈ͔ͤΔApp EngineʢFlexibleʣΛ࢖Θͳ͔ͬͨཧ༝͸…ͦΜͳબ୒ࢶΛೖΕͨ͘ͳ͔͔ͬͨΒͰ͢ʢ௚ٿʣ

    View Slide

  14. WBC2023ࣆδϟύϯઓ࢜ͷͦͷޙ
    • ೋ౛ྲྀͷ͋ͷํ


    • ϕετφΠϯʹબ͹ΕͨMLB৽ਓԦʢީิʣ

    View Slide

  15. େ୩ᠳฏʢϩαϯθϧεɾΤϯδΣϧε, TWPʢTwo Way Playerʣʣ


    ϗʔϜϥϯྔ࢈ͷൿີ͕໌Β͔ʹ

    View Slide

  16. େ୩ᠳฏʢϩαϯθϧεɾΤϯδΣϧε, TWPʣ


    ӈʹࠨʹଧͪ෼͚͍ͯΔͷ΋͍͍ײ͡

    View Slide

  17. େ୩ᠳฏʢϩαϯθϧεɾΤϯδΣϧε, TWPʣ


    ౤खͱͯ͠͸͜Μͳײ͡

    View Slide

  18. ٢ాਖ਼ঘʢϘετϯɾϨουιοΫε, LFʣ


    ώοτྔ࢈ͷൿີ͕໌Β͔ʹ

    View Slide

  19. ʁʁʁʮσʔλج൫ࣗ෼Ͱ࣋ͭͱ͓ߴ͍ͷͰ͸ʁʯ

    View Slide

  20. ʲ౴ʳ࢖͏αʔϏεΛ޻෉ͯ͠ίετ཈੍
    • ʮ࢖͏ͱ্ཱ͖͚͕ͩͪΕ͹͍͍ʯલఏͰશαʔϏεΛίʔϧυελϯόΠʹ.


    • ͢΂ͯͷΞϓϦɾػೳΛαʔόϨεͳΫϥ΢υʢCloud RunͳͲʣΛར༻.


    • ʮಈ͍͍ͯͳ͍࣌΋͓͕͔͔ۚΔʯͷ͸BigQueryͱετϨʔδͷΈʢ͘͝গ
    ֹʣ


    • σʔλอ؅͸࢖͍ํΛ޻෉͠ίετΛ཈͑Δ


    • ແྉ࿮ͷϧʔϧΛཧղ͠, ͦͷൣᙝͰۃྗ࢖͏ʢ㲈ඞཁͳ΋ͷʹ͸͓ۚΛ෷͏ʣ


    • BigQuery΋Cloud Storage΋ແྉ࿮ΛΊͬͪΌ׆༻

    View Slide

  21. Ұϲ݄Ͱ$3ະຬ, ίʔώʔ3.34ഋ෼ఔ౓Ͱӡ༻͍ͯ͠·͢🐯

    ※σʔλج൫ϓϩδΣΫτͷ࣮ίετΑΓࢉग़ʢίʔώʔ୅͸ίϯϏχίʔώʔج४ʣ, υϝΠϯऔಘྉ౳֎෦ίετΛআ͘

    View Slide

  22. ͓͠ͳ͕͖
    • DashΛGoogle CloudͰಈ͔͢ͳΒCloud Run


    • ΦΦλχαϯ΋Ϛονϣ͞Μ΋৭ʑͱΤά͍


    • ࣗ෼Ͱσʔλج൫Λ࣋ͬͯ΋ରͯۚ͠͸͔͔Βͳ͍


    ※AWS౳, ଞͷΫϥ΢υαʔϏεͰ΋ಉ͜͡ͱ͕Ͱ͖·͢Αʂ

    ʢGoogle Cloudʹ͍ͯ͠Δͷ͸ࢲͷ޷ΈͰ͢ʣ

    View Slide

  23. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠⽁


    @shinyorke

    View Slide