Upgrade to Pro — share decks privately, control downloads, hide ads and more …

事業横断組織でのMLシステム開発・運用と基盤設計

 事業横断組織でのMLシステム開発・運用と基盤設計

MACHINE LEARNING Meetup KANSAI #5
2019/6/17

Livesense Inc.

June 17, 2019
Tweet

More Decks by Livesense Inc.

Other Decks in Technology

Transcript

  1. ࣄۀԣஅ૊৫Ͱͷ
    MLγεςϜ։ൃɾӡ༻ͱج൫ઃܭ
    Shotaro Tanaka / @yubessy / Ϧϒηϯε (ژ౎ΦϑΟε)
    MACHINE LEARNING Meetup KANSAI #5

    View full-size slide

  2. ࿩͢͜ͱ
    • ෳ਺ͷWebαʔϏεΛӡӦ͍ͯ͠Δاۀ
    • MLγεςϜͷ։ൃɾӡ༻νʔϜ͸ԣஅ૊৫ͱͯ͠αʔϏε͔Βಠཱ
    • ֤αʔϏεʹϨίϝϯυ΍ޮՌ༧ଌϞσϧͳͲෳ਺ͷMLγεςϜΛఏڙ
    ͱ͍͏؀ڥͰɺMLγεςϜΛ։ൃɾӡ༻͢ΔͨΊʹͲΜͳ޻෉Λ͍ͯ͠Δ͔
    ͋Δ͍͸ɺͦͷͨΊͷMLج൫ΛͲ͏ઃܭ͍ͯ͠Δ͔

    View full-size slide

  3. αʔϏεɾ૊৫ɾνʔϜ

    View full-size slide

  4. ૊৫ߏ੒
    • ࣄۀ෦
    • ΞϧόΠτࣄۀ෦ (ϚοϋόΠτ)
    • ΩϟϦΞࣄۀ෦ (స৬ձٞ, స৬υϥϑτ, ...)
    • ෆಈ࢈Ϣχοτ (Door ௞ି, IESHIL)
    • ...
    • ԣஅ૊৫
    • ςΫϊϩδΧϧϚʔέςΟϯά෦
    • σʔλϓϥοτϑΥʔϜάϧʔϓ ← MLؔ࿈ͷνʔϜ͸͜͜ʹॴଐ
    • σʔλϚʔέςΟϯάάϧʔϓ
    • ΠϯϑϥετϥΫνϟάϧʔϓ

    View full-size slide

  5. MLνʔϜɾMLج൫νʔϜ
    ࣾһ
    • MLΤϯδχΞ: 2໊
    • MLج൫ΤϯδχΞ: 2໊
    • ׬શʹ෼ۀ͍ͯ͠ΔΘ͚Ͱ͸ͳ͘ɺॏͳΔ෦෼΋͋Δ
    ֶੜΞϧόΠτ
    • ژ౎ΦϑΟε: 4໊
    • ౦ژΦϑΟε: 1໊
    • ML/Web/ΠϯϑϥͳͲͷεΩϧΛ׆͔ͯ͠։ൃΛิॿ

    View full-size slide

  6. ओͳMLγεςϜ

    View full-size slide

  7. ٻਓϨίϝϯυΤϯδϯ
    • ϚοϋόΠτɾస৬φϏͳͲͷϢʔβʹٻਓΛਪન
    • ϝʔϧɾWebαΠτɾωΠςΟϒΞϓϦͳͲ༷ʑͳ৔ॴͰಋೖ

    View full-size slide

  8. ٻਓϨίϝϯυΤϯδϯͷ಺෦
    • ධՁ஋ɾίϯςϯπΛ΋ͱʹ MF, FM ͳͲͷΞϧΰϦζϜΛద༻͠είΞΛࢉग़
    • user-item ΍ item-item ͷϦετΛόονॲཧͰੜ੒֤͠αʔϏεʹఏڙ

    View full-size slide

  9. Ԡืɾ࠾༻ͷޮՌਪఆɾ༧ଌϞσϧ
    • ϚοϋόΠτɾస৬φϏͳͲͷٻਓͷCVR΍Ԡื୯ՁΛࢉग़
    • ݕࡧ݁ՌͷॱҐ੍ޚ΍޿ࠂग़ߘͷ࠷దԽʹ׆༻

    View full-size slide

  10. Ԡืɾ࠾༻ͷޮՌਪఆɾ༧ଌϞσϧͷ಺෦
    • ϩάΛ༻͍ͯϕΠζਪఆɾϩδεςΟοΫճؼ౳Ͱ༧ଌɾਪఆ
    • σΟϨΫλʔ޲͚ʹ؆қతͳϏϡʔϫΛWebΞϓϦͱͯ͠։ൃ

    View full-size slide

  11. A/BςετɾόϯσΟοτπʔϧ
    • A/Bςετͷύλʔϯ഑৴ൺ཰ΛόϯσΟοτΞϧΰϦζϜͰࣗಈௐ੔
    • WebαΠτɾωΠςΟϒΞϓϦͷ࠷దԽΛޮ཰Խ

    View full-size slide

  12. A/BςετɾόϯσΟοτπʔϧͷ಺෦
    • ؅ཧը໘͔ΒύλʔϯΛొ࿥͠ɺWeb APIͰϥϯμϜʹ഑৴
    • CVϩάΛ෼ੳج൫Ͱूܭ͠ɺύλʔϯ഑৴ൺ཰Λࣗಈߋ৽

    View full-size slide

  13. MLج൫͕ͳ͔ͬͨ࣌୅

    View full-size slide

  14. ࠷ॳظͷߏ੒
    • 2014ࠒʹ࠷ॳͷγεςϜ (ϨίϝϯυΤϯδϯ) Λ։ൃ
    • σʔλऔಘ͔Βσʔλอଘ·ͰΛҰؾ௨؏Ͱߦ͏δϣϒ

    View full-size slide

  15. ෳࡶԽ͢Δ࣮૷
    • ධՁσʔλͷΈΛ࢖͏ΞϧΰϦζϜΛ௥Ճ
    • ϑΟϧλϦϯά΍ϦετͷϚʔδΛߦͬͯϨίϝϯυϦετΛੜ੒

    View full-size slide

  16. ࣅͨΑ͏ͳγεςϜͷ૿Ճ
    • ಉαʔϏεͰA/BςετͷͨΊϨϙδτϦΛෳ੡ͯ͠ΞϧΰϦζϜ͚ͩมߋ
    • ผαʔϏε΁ͷԣల։ͷࡍʹΞϧΰϦζϜΛίϐʔͯ͠ར༻

    View full-size slide

  17. ଟछଟ༷ͳ࣮ߦ؀ڥ
    • ֤γεςϜ͕αʔϏεͷ؀ڥʹ૬৐Γ͠ɺσϓϩΠϑϩʔ΋ҧ͏
    • ΦϯϓϨͷϦιʔεʹݶΓ͕͋ΓɺVM͝ͱʹεϖοΫ͕ඍົʹҧ͏

    View full-size slide

  18. खʹෛ͑ͳ͘ͳ͖ͬͯͨ

    View full-size slide

  19. ͜ͷ··Ͱ͸͍ͣΕ։ൃ΋ӡ༻΋ഁ୼͢Δ
    → ͦ͏ͳΔલʹMLج൫Λߏங͢Δ͜ͱʹ
    MLج൫ Livesense Brain
    ※ͱΓ͋͑ͣ࠷ॳʹ໊લ͚ܾͩΊͨ

    View full-size slide

  20. MLج൫ԽͷͶΒ͍
    ։ൃ໘ͷݟ௚͠
    • γεςϜ಺ͷϞδϡʔϧؒͷີ݁߹Λͳ͘͢
    • γεςϜؒͰڞ௨ԽͰ͖Δ෦෼͸ڞ௨Խ
    ӡ༻໘ͷݟ௚͠
    • νʔϜ͕ࣗલӡ༻Ͱ͖ΔΠϯϑϥ΁ू໿
    • δϣϒ؅ཧ΍σϓϩΠϑϩʔΛڞ௨Խ

    View full-size slide

  21. ։ൃ໘ͷݟ௚͠

    View full-size slide

  22. ։ൃ্ͷ໰୊఺
    MLͷίΞ෦෼ͱͦΕҎ֎͕ີ݁߹
    • MLΤϯδχΞɾMLج൫ΤϯδχΞͷ୲౰Λ෼཭͠ʹ͍͘
    • ༻్ʹԠͯ͡ݴޠ΍ϥΠϒϥϦΛ࢖͍෼͚ΒΕͳ͍
    ֤γεςϜͷڞ௨ॲཧΛίϐϖར༻
    • ڠௐϑΟϧλϦϯάͳͲͷΞϧΰϦζϜ࣮૷
    • લॲཧ΍DBೖग़ྗ༻ͷΫϥε
    → γεςϜΛػೳ͝ͱʹ෼ׂ͠࠶ߏ੒Ͱ͖ͳ͍͔ʁ

    View full-size slide

  23. ίϯϙʔωϯτͷ෼ׂ
    ֤εςοϓΛ୯ػೳίϯϙʔωϯτʹ੾Γग़͠
    • ֤ίϯϙʔωϯτ͸ CLI Ͱ୯ಠ࣮ߦͰ͖Δ
    • ίϯϙʔωϯτؒͷೖग़ྗ͸͢΂ͯϑΝΠϧΛհ͢Δ
    ྫ:
    • I/Oίϯϙʔωϯτ: SQLϑΝΠϧΛೖྗ → σʔλΛCSVͱͯ͠ग़ྗ
    • લॲཧίϯϙʔωϯτ: ςΩετσʔλΛೖྗ → BoWϕΫτϧΛग़ྗ
    • Ϩίϝϯυίϯϙʔωϯτ: ධՁσʔλΛೖྗ → ਪનείΞΛग़ྗ

    View full-size slide

  24. ίϯϙʔωϯτͷίϯςφԽ
    • ֤ίϯϙʔωϯτΛ୯Ұͷ Docker Πϝʔδʹ͢Δ
    • γεςϜ͝ͱͷࠩ෼͸΄΅ઃఆϑΝΠϧɾSQL͚ͩͰදݱ
    • ୯७ͳόονॲཧͳΒ docker run ͷஞ࣮࣍ߦ͚ͩͰॻ͚Δ
    • ࣮ࡍʹ͜ΕͰຊ൪Քಇ͍ͯ͠ΔγεςϜ΋͋Δ
    # load dataset
    docker run -v $(pwd):/workdir sqlkit select ratings.sql /workdir/ratings.csv
    docker run -v $(pwd):/workdir sqlkit select content.sql /workdir/content.csv
    # preprocess
    docker run -v $(pwd):/workdir nlpkit vectorize /workdir/content.csv /workdir/features.csv
    # run recommender
    docker run -v $(pwd):/workdir recommender predict config.yaml /workdir

    View full-size slide

  25. Pros
    • ΞϧΰϦζϜ࣮૷ͳͲΛෳ਺γεςϜؒͰ࠶ར༻Ͱ͖Δ
    • ίϯϙʔωϯτ͝ͱʹݸผʹ։ൃɾվળ͕Ͱ͖Δ
    • ผͷݴޠ΍ϥΠϒϥϦΛ࢖͏͜ͱ΋༰қ
    • ৽نϝϯόʔ΍ΞϧόΠτͷࢀೖোน͕௿͘ͳͬͨ
    • ͋Δίϯϙʔωϯτʹ͍ͭͯΩϟονΞοϓ͢Ε͹࢓ࣄ͕Ͱ͖Δ
    Cons
    • MLΤϯδχΞ΋ؚΊશһʹίϯςφͷ஌͕ࣝ͋Δఔ౓ඞཁ

    View full-size slide

  26. ӡ༻໘ͷݟ௚͠

    View full-size slide

  27. ӡ༻্ͷ໰୊఺
    Πϯϑϥͷมߋͷࣗ༝౓͕௿͍
    • MLγεςϜ͕֤αʔϏεͷΠϯϑϥʹ૬৐Γ
    • धཁʹԠͨ͡Ϧιʔεมߋ͕ॊೈʹͰ͖ͳ͍
    ͦΕͧΕͷγεςϜͷ؀ڥ͕શ͘ผ
    • 10Ҏ্ͷδϣϒ͕֤γεςϜઐ༻ͷVMͰ cron ࣮ߦ
    • δϣϒ؅ཧπʔϧ΍σϓϩΠϑϩʔ΋όϥόϥ
    → νʔϜࣗલͷΠϯϑϥͰγεςϜΛҰݩ؅ཧͰ͖ͳ͍͔ʁ

    View full-size slide

  28. Google Kubernetes Engine (GKE) ͷ࠾༻
    νʔϜࣗલͷίϯϐϡʔςΟϯάج൫ΛखʹೖΕΔ
    • ߴෛՙͳMLॲཧͷ࣮ߦத͸ΦʔτεέʔϧͰϦιʔεΛ૿ڧ
    • γεςϜͷߏ੒Λ༰қʹ௥ՃɾมߋͰ͖Δ
    • ྫ: ༧ଌϞσϧͷ؆қϏϡʔϫΛ Deployment ͱཱͯͯ͠Δ
    ෳ਺γεςϜΛ౷Ұ͞Εͨ؀ڥͰҰݩ؅ཧͰ͖Δ
    • WebΞϓϦ΋όονॲཧ΋୯ҰΫϥελʹಉډ
    • δϣϒ؅ཧ΍σϓϩΠ΋ k8s ͷΤίγεςϜ্Ͱ࣮ݱ

    View full-size slide

  29. ิ଍: গਓ਺νʔϜͰ Kubernetes ?
    γεςϜͷܗଶ͕Ұ༷ɾ୯७ͳΒଞʹྑ͍બ୒ࢶ͕͋Δ
    • όονॲཧ͚ͩ → GCE, Fargate, ...
    • WebΞϓϦ͚ͩ → GAE, Beanstalk, ...
    ͔͠͠ɺࠓճ͸༷ʑͳγεςϜΛѻ͏ඞཁ͕͋ͬͨ
    • Ҡߦର৅ͷطଘγεςϜʹ͸όονॲཧ΋WebΞϓϦ΋ؚ·ΕΔ
    • ίϯςφԽ͞Εͨෳ਺ͷίϯϙʔωϯτΛ࿈ܞ͍ͤͨ͞
    → ͜ΕΒΛ౷Ұతʹ؅ཧ͢Δͷʹ Kubernetes ͕ద͍ͯͨ͠

    View full-size slide

  30. ϫʔΫϑϩʔΤϯδϯͷಋೖ
    • δϣϒ؅ཧͦͷ΋ͷ͸ k8s ͰҰݩԽͰ͖ͨ
    • ίϯϙʔωϯτΛ࿈ܞͤͯ͞ϫʔΫϑϩʔԽ͍ͨ͠
    → Argo Workflow Λಋೖ
    • k8s ͰίϯςφΛ௚ྻɾฒྻɾDAG࣮ߦ͢ΔϫʔΫϑϩʔΤϯδϯ
    • ϦτϥΠɾΤϥʔॲཧͳͲ΋؆୯ʹ࣮ݱ
    • ϫʔΫϑϩʔ࣮ߦΛ Web UI ͰϞχλϦϯάͰ͖Δ
    લճLTͰ঺հ: Argo Workflow ʹΑΔMLδϣϒ؅ཧ - Speaker Deck

    View full-size slide

  31. σϓϩΠϑϩʔͷࣗಈԽ
    • σϓϩΠͦͷ΋ͷ͸ k8s manifest Λ apply ͢Δ͚ͩ
    • ߋʹखؒΛলͨ͘ΊσϓϩΠΛࣗಈԽ͍ͨ͠
    → Cloud Build Ͱ GitOps Λ࣮ݱ
    • Manifest ϨϙδτϦͷϓϧϦΫϚʔδΛτϦΨʔͱ͢Δ
    • Cloud Build Ͱ kustomize build ~ kubectl apply
    ※ Argo CD ΋ݕ౼͕ͨ͠ɺࠓͷن໛ͳΒ Cloud Build Ͱे෼ͱ൑அ

    View full-size slide

  32. Pros
    • νʔϜ಺Ͱͷӡ༻ͷॊೈੑɾࣗ༝౓͕޲্
    • ඞཁʹԠͯ͡ϦιʔεΛ૿ڧ
    • PoC ࣮૷ΛαʔϏεӨڹΛݶఆͯ͠ࢼݧӡ༻
    • γεςϜ͕૿͑ͯ΋؅ཧίετΛ཈͑΍͍͢
    Cons
    • ಉ͡Πϯϑϥʹෳ਺γεςϜ͕ಉډ → Χεέʔυো֐ͷϦεΫ
    • δϣϒͷႈ౳ੑͳͲ k8s ಛ༗ͷੑ࣭ʹཹҙ͢Δඞཁ

    View full-size slide

  33. MLج൫લޙͷશମൺֱ

    View full-size slide

  34. Before
    • ਨ௚ߏ଄ = ݽཱͨ͠γεςϜΛݸผʹ։ൃɾӡ༻
    • γεςϜͷ਺ʹൺྫͯ͠޻਺ɾਓ਺͕ඞཁʹͳΓ͕ͪ
    After
    • ਫฏߏ଄ = ػೳίϯϙʔωϯτΛ࠶ར༻͠ɺΠϯϑϥϨΠϠΛڞ௨Խ
    • γεςϜͷ਺͕૿͑ͯ΋؅ཧ΍໾ׂ෼୲͕͠΍͍͢

    View full-size slide

  35. ·ͱΊ
    • গਓ਺ͷԣஅνʔϜͰෳ਺ͷMLγεςϜΛ։ൃɾӡ༻
    • MLج൫Ҏલ͸֤γεςϜ͕ݽཱͨ͠ঢ়ଶͰଟ͘ͷ໰୊Λ๊͍͑ͯͨ
    • MLج൫ߏஙΛ௨ͯ͡։ൃɾӡ༻Λશମతʹݟ௚͢͜ͱʹ
    • ίϯϙʔωϯτ෼ׂͱࣗલΠϯϑϥ΁ͷҰݩԽΛߦͬͨ
    • ਨ௚ߏ଄ → ਫฏߏ଄ ΁ͷస׵Ͱ؅ཧ΍໾ׂ෼୲͕͠΍͘͢ͳͬͨ

    View full-size slide