Upgrade to Pro — share decks privately, control downloads, hide ads and more …

実践Streamlit & Flask - AIプロジェクトをいい感じにする技術 / Serv...

実践Streamlit & Flask - AIプロジェクトをいい感じにする技術 / Service development with Streamlit and Flask

PyCon JP 2021「実践Streamlit & Flask - AIプロジェクトのプロトタイピングから本番運用までをいい感じにするPythonicなやりかた」登壇資料

# 「AIオオタニサン本塁打予測」のサンプルコード

https://github.com/Shinichi-Nakagawa/streamlit-sample-ohtani-san

# 参考文献

## JX通信社エンジニアブログ

- https://tech.jxpress.net/entry/data-app-for-streamlit
- https://tech.jxpress.net/entry/vaccine-gae
- https://tech.jxpress.net/entry/ogp-generator

## Lean Baseball

- https://shinyorke.hatenablog.com/entry/flask-and-gae-and-github-actions
- https://shinyorke.hatenablog.com/entry/engineer-and-datascientist
- https://shinyorke.hatenablog.com/entry/jupyter-to-streamlit

## Agile Manifest

https://agilemanifesto.org/iso/ja/manifesto.html

## 仕事ではじめる機械学習 第2版

https://www.oreilly.co.jp/books/9784873119472/

Shinichi Nakagawa

October 16, 2021
Tweet

More Decks by Shinichi Nakagawa

Other Decks in Technology

Transcript

  1. ͜ͷ࿩ͷείʔϓʢ࿩͢͜ͱ&ର৅ऀʣ • JX௨৴ࣾʮNewsDigestʯͷظؒݶఆίϯςϯπͱͯ͠ग़͍ͯͨ͠ 
 ʮAIϫΫνϯ઀छ༧ଌʯͷ։ൃ͔Βӡ༻ͷ࣮࿩Λݩʹ • Streamlit΍FlaskͳͲΛ࢖ͬͯ೗ԿʹPython࢖͍Β͘͠΍͔ͬͨ?Λ࿩͠·͢ • ͳ͓, طʹAIϫΫνϯ઀छ༧ଌͷαʔϏε͸ऴ͍ྃͯ͠ΔͨΊ,

    
 αϯϓϧͱͳΔΑ͏ͳΞϓϦΛ࡞͖ͬͯ·ͨ͠&ͦΕΛݩʹղઆ • WebΞϓϦ։ൃऀ͔ΒσʔλαΠΤϯςΟετ·Ͱָ͓͠Έ͍͚ͨͩ·͢ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ ㅟ
  2. Who am I ?ʢ͓લ୭Αʣ • Shinichi Nakagawaʢத઒ ৳Ұʣ • େ఍ͷSNSͰʮshinyorkeʢ͠ΜΑʔ͘ʣʯͱ໊৐͍ͬͯ·͢

    • Software Engineer & Data ScientistʢؚΉ⽁ʣ • ݩɾJX௨৴ࣾγχΞΤϯδχΞ 
 ※9݄·Ͱ, 10݄͔Βస৬͠·ͨ͠ • #Python #DataScience #Baseball⚾ #SABRmetrics
  3. ࠓ೔ͷ͓౔࢈ • AIϓϩδΣΫτΛ੒ޭͤ͞ΔͨΊʹඞཁͳߟ͑ํͱελϯε • PoCʢ֓೦࣮ূʣͷ͸ͳ͠ - StreamlitͰ͍͍ײ͡ʹ΍Δ • ϓϩμΫτ։ൃ͔Βϩʔϯν, ӡ༻·ͰΛPythonicʹ΍Δ

    • ͪΐͬͱͨ͠খωλ - ΞΠΩϟονը૾ੜ੒ͱCIपΓ ͜ΕΒΛJX௨৴ࣾ࣌୅ͷ࢓ࣄʢͷެ։৘ใʣ͓Αͼݸਓ։ൃͨ͠಺༰͔Β঺հ
  4. ຊ೔࿩͞ͳ͍͜ͱʢ౴͑ͳ͍͜ͱʣ • ݱ৬ͷ࿩ʢͲ͜Ͱಇ͍͍ͯΔɾ࢓ࣄԿͯ͠Δͱ͔ʣ 
 ask the speakerؚΊͯҰ੾࿩͠·ͤΜ🙇ʢḨࡧ΋φγͰʣ • લ৬ʢJX௨৴ࣾʣͷࠓͷ࿩ɾ࠾༻ͷ࿩ɾެ։͞Εͯͳ͍৘ใ 


    JX௨৴ࣾͷࣄ͸εϙϯαʔϒʔεͰதͷਓʹฉ͍ͯʂ 
 ͍͍࿩Λͯ͘͠ΕΔͱࢥ͏ͷͰ👍 • ͦͷଞ, ࢲʢshinyorkeʣ͕ෆద੾ͱ൑அͨ͠಺༰
  5. ෆ࣮֬ੑͱ޲͖߹͏ͨΊͷAgile • ෆ࣮֬ੑ͕ߴ͍, AIɾػցֶशͳϓϩδΣΫτͦ͜ 
 ʮಈ͘ιϑτ΢ΣΞʯΛݩʹ։ൃ͢ΔΞϓϩʔν͕େࣄ • PoCͱ͸͍͑, ͳΔ΂͘ຊ൪͕ΠϝʔδͰ͖Δಈ͘ 


    ΞϓϦέʔγϣϯͰձ࿩͢Δͷ͕͍͍ͱࢥ͏ʢೝࣝҧ͍ͷ๷ࢭʣ • Ұํ, खΛൈ͚Δॴ͸खΛൈ͖·͘Δ΂͖ʢ㲈࡞Βͳ͍ɾ΍Βͳ͍ʣ 
 υΩϡϝϯτ΍ςετΑΓٞ࿦Ͱ͖ΔΞϓϦ͕ඞཁͳ࣌΋͋Δ
  6. ਏ͘ͳ͍Streamlitɹ - ͻͱ·ͣೖ໳͠Α͏ ㅟ ㅟ ㅟ ㅟ • ͻͱ·ͣಈ͔͢ •

    ͻͱ·ͣݟͤΔʢProxy͢ΔɾΫϥ΢υʹUp͢Δʣ • WebΞϓϦέʔγϣϯͬΆ͘࢓্͛Δ
  7. ͻͱ·ͣಈ͔͢ $ pip install streamli t $ streamlit hello •

    ಋೖ͸ී௨ʹpip installͰ͍͚·͢ • ʮstreamlit helloʯͱ͔ଧͭͱ, αϯϓϧ্ཱ͕͕ͪΔ • ↑ͷαϯϓϧͷίʔυ΋͙͢ಡΊ·͢, ͜ΕͰงғؾ͸௫ΊΔ https://streamlit.io/
  8. ͻͱ·ͣݟͤΔ - ϩʔΧϧ؀ڥ͔Β $ ngrok http 8501 • streamlitͷਖ਼ମ͸WebΞϓϦέʔγϣϯͳͷͰ, Կ͔͠ΒͷखஈͰProxyͯ͋͛͠Ε͹ྑ͍

    • ࣗ෼ͷϚγϯʢϩʔΧϧʣͰಈ͔͢ͳΒngrok౳ͰProxyͰߦ͚Δ 
 ʢࣾ಺ͷηΩϡϦςΟϧʔϧͱ͔Ͱ໰୊ͳ͚Ε͹ʣ • ײ֮తʹ͸Slack Botͷ։ൃʹ͍ۙʢngrokͰProxyͬͯΑ͘΍ΔͰ͠ΐʣ
  9. ϩʔΧϧ͡Όͪΐͬͱ 🤔ͬͯ࣌͸ • ࣗલͷ؀ڥͰ΍Δ • ContainerʢDockerʣ΍Virtual MaschineతͳIaaSʢGCE΍AWS EC2ͳͲʣΛ४උ • ४උͨ͠؀ڥʹσϓϩΠ

    • StreamlitͷΫϥ΢υαʔϏεΛ࢖͏ • https://streamlit.io/cloud • શੈքʹࡽͯ͠OKͳΒ͜ΕͰ΋ྑ͍͔΋ʢͨͩ͠βతͳαʔϏεͬΆ͍ʣ
  10. ʲྫʳαΠυόʔʹFormΛදࣔ͢Δ import streamlit as s t # PITCH_TYPES, THROWS͸DictionaryͰ͢ from

    interfaces.view.label import PITCH_TYPES, THROW S st.sidebar.markdown ( "" " # ϘʔϧΛܾΊΔ "" " ) # valueΛselectboxͷϥϕϧʹ͍ͨ࣌͠͸lambdaؔ਺ΛڬΉ p_throw = st.sidebar.selectbox ( "ར͖࿹" , THROWS.keys() , format_func=lambda x: THROWS[x ] ) pitch_speed = st.sidebar.slider ( 'ٿ଎ʢkm/hʣ', 70, 170, 150, 5 ) pitch_type = st.sidebar.selectbox ( "ٿछ" , PITCH_TYPES.keys() , format_func=lambda x: PITCH_TYPES[x ] )
  11. ͜͜·ͰͰ͖Δͱ • νʔϜϝΠτ, Ғ͍ਓͱ͍ͬͨεςʔΫϗϧμʔʹ੒ՌΛ 
 ݟͤΔ͜ͱ͕͍͍ײ͡ʹͰ͖Δ • Jupyter notebook΍ColabͰݟͤΔҎ্ʹΞϓϦͬΆ͍ 


    UIͱಈ͖ͰϓϩτλΠϓΛ։ൃͰ͖Δ • ͜͜·Ͱ͍͚͹, ٞ࿦͠ͳ͕ΒϑΟʔυόοΫ΋Βͬͯ 
 ख௚͠Ͱ͖ΔͰ͠ΐ͏👍
  12. ίʔυΛ෼ׂ͢Δ • Jupyter notebook΍StreamlitͰॻ͍ͨίʔυΛҙຯ͋Δ୯Ґʹ෼͚Δ • σʔλͷॲཧʢೖग़ྗ, લॲཧʣ • Ϟσϧͷॲཧʢֶशɾ༧ଌʣ •

    ༧ଌ݁Ռͷ֬ೝɾςετ • ී௨ͷpythonεΫϦϓτͰߏΘͳ͍ʢΦϒδΣΫτࢦ޲͡Όͳ͍͍ͯ͘ʣ 
 ଞͷਓʢίʔυΛ΋Βͬͨਓʣ͕ࣗྗͰಈ͔ͤΔ͜ͱ͕ॏཁ
  13. ΞʔΩςΫνϟબͼͷࡶͳجຊ • εέʔϦϯάͷཁ݅ • ϦΫΤετ਺ɾτϥϑΟοΫʹ೾͕͋Δ࣌͸App EngineΈ͍ͨͳϑϧϚωʔδυͳαʔϏε͕͍͍ײ͡ • ϦΫΤετ਺͕ൺֱత҆ఆʢ㲈αʔόʔͷ୆਺͕ܾ·ͬͯΔʣͦ͠͏ͳ࣌͸ଞͷཁ݅࣍ୈͰࣗલͰαʔόʔ؅ཧ͍͍͔ͯ͠΋? • ࢖͍͍ͨϥΠϒϥϦɾFramework͕࢖͑Δ؀ڥ͔Ͳ͏͔

    • ඪ४తͳϥΠϒϥϦɾFrameworkͰࡁΉ৔߹͸ϑϧϚωʔδυͳΫϥ΢υαʔϏε͕ಘҙͳͷͰͦͬͪʹدͤͨ΄͏͕޾ͤ. • ͨͩ͠, ϑϧϚωʔδυͳΫϥ΢υαʔϏε͸ϥΠϒϥϦɾFrameworkʹ͋Δఔ౓੍໿͕͋ΔͷͰ஫ҙ͕ඞཁ, ಛघͳཁ͕݅͋Δ࣌͸ಛʹ. • ྉۚ💰 • ϑϧϚωʔδυͳαʔϏεͱͦ͏͡Όͳ͍ͷ, Ͳ͕ͬͪඒຯ͍͔͠ݟۃΊΔʢଛӹ෼ذ఺΋ؚΊͯʣ • Ծ૝ϚγϯܥͷαʔϏεͳΒ֬໿ར༻ׂҾʢϦβʔϒυΠϯελϯεͳͲʣΛ࢖͑ΔαʔϏεʹ͢Δ͜ͱ΋ݕ౼
  14. ͻͱ·ͣಈ͔͢ - ελϯμʔυ؀ڥฤ • ͸͡ΊͯGAEΛ৮Δʢorٱ͠ͿΓʹ৮Δʣํ͸Google CloudͷΫΠοΫελʔτΛࢀߟʹ΍Δͱ ྑ͍Ͱ͠ΐ͏, ࣸܦͯ͠ਅࣅΔͷ͕ͪΐ͏ͲΑ͍͔΋. • https://cloud.google.com/appengine/docs/standard/python3/quickstart?hl=ja

    • ্هΛ௨͠Ͱ΍ΔͱFlask੡ͷhello world͕ര஀, ࢓૊Έ΋ཧղͰ͖Δ͸ͣ. • ͋ͱ͸࡞Γ͍ͨϞϊͷ࢓༷ʹ߹ΘͤͯFlaskͷίʔυΛ͍͡Ε͹OK • ྫ͑͹RESTfulʹ͢ΔͳΒResponseΛJSONʹม͑Δͱ͔ • FastAPIͳͲ, ଞͷFWΛ࢖͏࣌͸ͦ͜ʹ߹Θͤͯ࡞Γ௚ͤ͹
  15. ϥΠϒϥϦ΁ͷґଘΛݮΒ͢ # AIΦΦλχαϯຊྥଧ༧ଌͷύοέʔδߏ੒ʢΠϝʔδʣ app / !"" entities / !"" interfaces/

    # FlaskҎ֎ͷαʔυύʔςΟʔɾϥΠϒϥϦΛimportͯ͠OKͳॴ !"" tests / # !"" dataset / # !"" interfaces / # $"" usecase / !"" usecase / main.py # Flaskʹґଘͯ͠OKͳͱ͜Ζ • αʔυύʔςΟʔͷϥΠϒϥϦɾFramework΁ͷґଘ͸ہॴతʹ͢Δͱྑ͍ • Կ͔ͷ౎߹Ͱ࡞Γม͑Δʢྫ͑͹GAEҎ֎Ͱಈ͔͢ʣΈ͍ͨͳ࣌ʹมߋՕॴ͕গͳͯ͘ࡁΉ • ΫϦʔϯɾΞʔΩςΫνϟͳͲ͕ͦͷΑ͏ͳߟ͑ํͩͬͨΓ͢ΔͷͰਅࣅ͢Δͱ͍͍͔΋͠Εͳ͍
  16. ߴෛՙରࡦʢapp.yamlهड़ྫʣ runtime: python3 9 automatic_scaling : max_instances: 33 # ࠷େͷΠϯελϯε਺ʢ͜ΕҎ্૿͑ͳ͍ʣ

    min_instances: 4 # ࠷খͷΠϯελϯε਺ʢ͜ΕҎ্ݮΒͳ͍ʣ • app.yamlͷΦϓγϣϯΛ͍ͬͯ͡deployͨ͠Β͍͍ײ͡ʹ͍͚·͢ʢ্هͷྫ͸ελϯμʔυ؀ڥʣ • TVʹग़Δ, SNSͰόζΔ౳͕༧૝͞ΕΔ࣌͸ࣄલʹύϥϝʔλΛ͍ͬͯ͡σϓϩΠͯ͠ղܾ 
 ͪͳΈʹ૿͑ͨΠϯελϯε͸app.yamlΛ௚ͯ͠࠶σϓϩΠ or GCPͷίϯιʔϧͰखಈͰফ͢౳ͰରԠͰ͖·͢ • min_instancesΛ0ʹͨ͠ΒʮϦΫΤετ͕͖ͨΒ্ཱͪ͛ΔʯΈ͍ͨͳײ͡ʹ 
 ʢϨεϙϯε͕དྷΔ·Ͱ࣌ؒ͸͔͔Δ͕ྉۚ͸અ໿Ͱ͖Δʣ • max_instancesΛ0ʹͨ͠Βύϥϝʔλ͕ແޮʹͳΓ·͢…͕͓͢͢Ί͸͠·ͤΜ 
 ࣄ্࣮࠷େ஋Λఆٛ͢Δ͜ͱʹͳΔͨΊʢ㲈ͳʹ͔ͷΞΫγσϯτͰΠϯελϯε͕૿͑Δʣ
  17. αʔόʔαΠυͰαΫοͱը૾Λͭ͘Δ # ࣮ࡍͷίʔυͱ͸ҟͳΓ·͢ʢ͋͘·ͰαϯϓϧͰ͢ʣ import cairosv g def convert_ogp(area: str, age:

    int, period: str) : # Կ͔͠Βͷํ๏ͰSVGϑΥʔϚοτͳςΩετΛೖख ogp_context = 'TODO: ͜͜ʹSVGͷத਎͕ೖΔ ' # TemporaryFileͱͯ͠ॻ͖ग़ͯ͠ޙbyteͰ΋Β͏ with NamedTemporaryFile('w') as f : f.write(ogp_context ) f.flush( ) image_bytes = svg2png(url=f.name, write_to=None ) return image_byte s image = convert_ogp('౦ژ౎', 40, '10݄Լ०ʙ2্݄०') ↑͜͏͍͏ը૾Λͭ͘Δํ๏ʢαϯϓϧίʔυ→ʣ • σβΠϯΛSVGͰىͯ͜͠cairosvgͰpngʹม׵ 
 SVGΛJinja2ͱ͔ͰςϯϓϨʹͨ͠Βಈతʹߦ͚Δ • ಈ͔͢؀ڥ͸஫ҙ͕ඞཁ. GAEͰϑΥϯτͷΠϯετʔϧ͸Ͱ͖ͳ͍. 
 ͦ͜ͰϋϚͬͨͷͰDockerԽͯ͠Cloud RunͰӡ༻͠·ͨ͠. 
 ৄ͘͠͸, https://tech.jxpress.net/entry/ogp-generator
  18. ͜Μͳײ͡ʢงғؾʣ name: C I on: pus h jobs : test

    : name: tes t runs-on: ubuntu-lates t steps : - uses: actions/checkout@maste r - name: Instal l uses: abatilo/[email protected]. 0 with : python_version: 3.9. 7 poetry_version: 1.1. 7 args: instal l - name: Run pytes t uses: abatilo/[email protected]. 0 with : python_version: 3.9. 7 poetry_version: 1.1. 7 args: run python -m pytest . deploy : name: Deplo y needs: tes t if: github.ref == 'refs/heads/main ' runs-on: ubuntu-lates t steps : - uses: actions/checkout@maste r - name: Deploy a Cloud Run servic e id: deploy-servic e uses: google-github-actions/deploy-cloudrun@mai n with : image: asia.gcr.io/${{ secrets.GCP_PROJECT_ID }}/ohtani-hr-predic t service: ohtani-hr-predic t region: asia-northeast 1 project_id: ${{ secrets.GCP_PROJECT_ID } } credentials: ${{ secrets.GCP_SA_KEY }}
  19. खஈͱϓϩηε, ྆ํͰAgilityΛߴΊΔ • ࠷ॳ͸ࡶͰ΋͍͍͔Βͬ͞͞ͱಈ͘΋ͷΛ࡞ͬͯݟͤΔ • MVP͔ΒຊϓϩμΫτ΁ͷҠߦ…ͷલʹςετͱϦϑΝΫλϦϯά 
 ͜͜Ͱ͍͍ײ͡ʹͰ͖Δ͔Ͳ͏͔͕͜ͷઌͷAgilityʹӨڹ͢Δ • ϦϦʔε΍ߴෛՙରࡦͳͲ,

    ܁Γฦ͠΍Δ΋ͷΛCIʹͨ͠Γ࢓૊Έʹ৐͔ͬͬͨΓ 
 ͜Ε͸ΤϯδχΞͳࢹ఺Ͱͷ࿹ͷݟͤͲ͜Ζ💪 • ͜ΕΒΛ͢΂ͯͷϑΣʔζΛҰ؏ͯ͠ಉ͡ݴޠͰ΍ΕΔPython͸, 
 ࠷ॳʹϓϩδΣΫτΛ࢝ΊΔͱ͖ͷखஈͱͯ͠ྑ͍ؾ͕͢Δ. 
 ʢͨͩ͠, ͦͷޙผݴޠͰ࡞Γ௚͢ͳͲͳͲ͸શવ͋Γʣ
  20. ΤϯδχΞͱσʔλαΠΤϯςΟετͷҧ͍ • ϓϩμΫτΛ։ൃʢӡ༻ʣ͢ΔΤϯδχΞͱ, AIͳ࢓ࣄΛ͢ΔσʔλαΠΤϯςΟετ͸ಘҙෆಘҙ͕ҧ͏ 
 ಉ͡Pythonͱ͍͏ݴޠΛૢ͍ͬͯͯ΋ҧ͏ੜଶܥͰੜ͖͍ͯΔੜ͖෺Ͱ͋Δʢͱࢥ͓ͬͯ͘΂͖ʣ • ྫ͑͹ΤϯδχΞ͸όʔδϣϯ؅ཧɾςετɾCIΛίπίπͱ΍Δʢ΍Δ΂͖ͩʣ͕ 
 σʔλαΠΤϯςΟετ͸Ͱ͖Δਓ΋͍Ε͹Ͱ͖ͳ͍ਓ΋͍Δ

    
 ʢͳ͓, ྆ํͰ͖ΔԽ͚෺͸গ਺Ͱ͕͢ଘࡏ͢Δʣ • ྫ͑͹σʔλαΠΤϯςΟετ͸਺ֶɾ਺ࣜʹڧ͍ʢڧ͘ͳ͍ͱ͍͚ͳ͍ʣ 
 ͕, ΤϯδχΞ͸ผʹͦ͜·ͰٻΊΒΕͳ͍ʢͳ͓྆ํͰ͖ΔԽ͚෺͸গ਺͕ͩҎԼུʣ • ͭ·Δͱ͜Ζɺ྆ऀʹٻΊΒΕΔεΩϧηοτɾελϯε͸ڞ௨఺͸͋ΕͲ༏ઌ౓͕શ͘ҟͳΔ 
 ಉ͡PythonΛಡΈॻ͖͢Δʹͯ͠΋ࢲʢ͋ͳͨʣ͕େ੾ʹ͢Δॱং͸͋ͳͨʢࢲʣͱશવҧ͏ͱ৺ಘΔ΂͖
  21. ँࣙ • JX௨৴ࣾͰ৭ʑ΍Βͤͯ΋Βͬͨܦݧ͕ࠓճͷൃදʹ׆͖ͨ • ௅ઓͱֶͼͷػձΛ͘ΕͨJX௨৴ࣾʹ͸ϗϯτʹຊ౰ʹ͓ੈ࿩ʹͳΓ·ͨ͠🙇 • ςοΫϒϩά https://tech.jxpress.net/ Έͯ΍͍ͬͯͩ͘͞📝 


    ͋ͱ, εϙϯαʔϒʔε΋ͥͻ༡ͼʹߦͬͯ΋Β͑Δͱ👍 • ͳ͓ຊ೔16:20͔ΒαʔόʔαΠυΤϯδχΞ @TatchNicolas ͕ 
 ʮLocust࣮ફฤʯͱ͍͏τʔΫ͢ΔͷͰؾʹͳΔํ͸νΣοΫΛʂ
  22. ʲAppendixʳࢀߟจݙɾϒϩά • JX௨৴ࣾΤϯδχΞϒϩά • https://tech.jxpress.net/entry/data-app-for-streamlit • https://tech.jxpress.net/entry/vaccine-gae • https://tech.jxpress.net/entry/ogp-generator •

    Lean Baseball • https://shinyorke.hatenablog.com/entry/ fl ask-and-gae-and-github-actions • https://shinyorke.hatenablog.com/entry/engineer-and-datascientist • https://shinyorke.hatenablog.com/entry/jupyter-to-streamlit • Agile Manifest • https://agilemanifesto.org/iso/ja/manifesto.html • ࢓ࣄͰ͸͡ΊΔػցֶश ୈ2൛ • https://www.oreilly.co.jp/books/9784873119472/