Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction of Machine Learning1 Team

Introduction of Machine Learning1 Team

菊地 悠/Machine Learning1チーム/マネージャー

2020.06.24
LINE Data Labs 採用説明会 〜機械学習・データ分析〜
https://line.connpass.com/event/174842/

LINE Developers

June 24, 2020
Tweet

More Decks by LINE Developers

Other Decks in Technology

Transcript

  1. MACHINE LEARNING1 TEAM
    Haruka Kikuchi

    View full-size slide

  2. Agenda • Mission
    • Project, System, Tool
    • Position
    • Team Members' Voices
    • Reference

    View full-size slide

  3. MISSION STATEMENT
    ● ػցֶशΛ௨ͯ͡ɺLINEͷ༷ʑͳαʔϏεͷڝ૪ྗɾརӹʹߩݙ͢Δɹɹɹ
    Provide service competencies and profits through ML solutions to various LINE services
    ● ػցֶशʹؔ͢ΔLINEࣾ಺Ͱͷඪ४ԽɾຽओԽͷਪਐʢάϧʔϓձࣾΛؚΉʣ
    Lead standardization/democratization process of ML use at scale across LINE (inc. group companies)
    together w/ other data-related teams/depts.

    View full-size slide

  4. PROJECT, SYSTEM, TOOL

    View full-size slide

  5. Item2Item
    User2Item
    LINEελϯϓͷϨίϝϯυ

    View full-size slide

  6. ଟ༷ͳαʔϏε΁ͷΤϯδϯఏڙ
    Sticker, etc. Manga
    NEWS Live Parttime Fortune-telling
    Music
    Store
    ࠶ར༻ՄೳͳMLΤϯδϯ ʴ ಋೖ࣌ͷݸผνϡʔχϯά
    ※աڈʹ։ൃͨ͠αʔϏεʢൈਮʣ

    View full-size slide

  7. ελϯϓը૾΁ͷλάͷࣗಈ෇༩
    ਂ૚ֶशϕʔεͷ෼ྨ໰୊

    View full-size slide

  8. ը૾ͷྨࣅ౓Λར༻ͨ͠Ϩίϝϯυ

    View full-size slide

  9. εϚʔτνϟϯωϧ
    ● αʔϏεԣஅͷίϯςϯπΛڞ௨ͷදࣔྖҬʹ഑৴
    ● ݸผαʔϏεͷ௨஌΍ϨίϝϯυΤϯδϯ + ίϯςϯπԣஅͰͷ഑৴࠷దԽ
    ● ݸผαʔϏεͷΤϯδϯ։ൃ͸ࣾ಺ΦʔϓϯԽͷ࿮૊ΈΛ੔උ
    ● ഑৴࠷దԽ͸ɺෳ਺ͷContextual banditsΞϧΰϦζϜΛఏڙ

    View full-size slide

  10. FEATURE AS A SERVICE
    ● ML༻్ͰͷαʔϏεԣஅσʔλͷඪ४తͳ؅ཧͷ࢓૊Έ

    View full-size slide

  11. 2018೥10݄࣌఺
    SYSTEM OVERVIEW
    ࣄۀผͷγεςϜ ML1 TEAMͷ؅ཧ͢ΔγεςϜ
    ࣾ಺σʔλج൫γεςϜ

    View full-size slide

  12. 2020೥6݄࣌఺
    SYSTEM OVERVIEW
    Hadoop 3.x
    k8s
    port to external system
    ࣄۀผͷγεςϜ ML1 TEAMͷ؅ཧ͢ΔγεςϜ
    ࣾ಺σʔλج൫γεςϜ

    View full-size slide

  13. AB TEST TOOLSET
    ● ML LogicվળͷͨΊɺνʔϜ಺ͷ಺੡։ൃ͔Βελʔτʢޙʹࣾ಺ඪ४΁ʣ
    ● ౰ॳ͸A/BςετͷઃఆͷΈ؅ཧɾ഑෍͢ΔCMSͩͬͨ
    ● ࠷ۙɺϦΞϧλΠϜͰͷμογϡϘʔυػೳ΋αϙʔτ

    View full-size slide

  14. ϓϩδΣΫτମ੍ʢؔ܎ऀҰཡʣ
    ࣄۀαΠυʹ
    اըɾ։ൃɾ෼ੳ౳ͷؔ
    ܎ऀ͕ଘࡏ
    Hadoop,
    Kafka, etc.ͷࣾ಺σ
    ʔλΠϯϑϥɺϛυϧ
    ΢ΣΞ։ൃ
    ߴ౓ͳσʔλ
    ෼ੳʹΑΔࣄۀ෦ଆͷҙࢥ
    ܾఆࢧԉ΍ɺϞχλϦϯά΍
    ϨϙʔςΟϯά
    Data
    ͷऔΓճ͠ʹؔΘ
    Δ֤छͷ੔ཧ΍ௐ੔
    ͳͲ
    1

    View full-size slide

  15. ༷ʑͳ໾ׂͷϝϯό͕࿈ܞ͠ɺϓϩμΫτͷ։ൃɾอकӡ༻Λ࣮ࢪ
    ֤ϙδγϣϯͷ໾ׂ
    ML Engineer ML Server-side Engineer ML Product Manager ML Project Manager
    ػցֶशͷΞϧΰϦζϜͷ
    ݕ౼΍બఆͱ࣮૷
    MLΤϯδϯΛ૊ΈࠐΜͩγ
    εςϜ΍ϓϥοτϑΥʔϜɾ
    APIͳͲͷ։ൃɾอक
    MLΤϯδϯΛಈ͔ͨ͢Ίͷ
    ܭࢉػ؀ڥͷߏஙʙӡ༻
    MLΛ༻͍ͨγεςϜ΍ϓϥ
    οτϑΥʔϜͷاը
    ։ൃ෺ʹؔ͢Δࣄۀ෦౳΁
    ͷఏҊ΍ಋೖͷਪਐ
    MLΛ༻͍ͨγεςϜ΍ϓϥ
    οτϑΥʔϜͷ։ൃ؅ཧ
    ϦϦʔεɾಋೖޙͷMLαʔ
    Ϗεͷܧଓతվળͷਪਐ
    ※࣮຿Ͱ͸Ұ໊͕ෳ਺ϩʔϧͰͷۀ຿਱ߦΛߦ͏৔߹΋͋Γ·͢ʢຊਓͷదੑ΍ܦݧʹΑΔʣ

    View full-size slide

  16. ʢJob DescriptionͷهࡌΑΓൈਮʣ
    ඞਢͷܦݧɾεΩϧ
    ML Engineer ML Server-side Engineer ML Product Manager ML Project Manager
    ػցֶशɺίϯϐϡʔλʔαΠ
    Τϯεɺ਺ֶͷઐ໳తͳ஌ࣝ

    ࣄۀ/ϏδωεΛཧղ্ͨ͠
    Ͱɺ෼ੳɾఏҊ͕Ͱ͖Δ͜ͱ
    σʔλͷ୳ࡧɺಛ௃ྔͷม׵ɺ
    Ϟσϧͷಋग़ɺγεςϜͷ࣮
    ૷ɺύϑΥʔϚϯεධՁͷҰ௨
    ΓͷߦఔΛ TerabyteʙPetabyte
    ن໛ͷେن໛σʔλͰ࣮ࢪͰ͖
    ΔεΩϧ

    ෼ࢄॲཧγεςϜʢHadoop,
    Spark, MPI, etc.ʣͷ஌ࣝɾܦݧ
    ίϯϐϡʔλʔαΠΤϯεʹؔ
    ͢Δશൠతͳ஌ࣝ

    ԿΒ͔ͷαʔϏεɾϓϩδΣΫ
    τʹ͓͚ΔAPI΍γεςϜͷ։
    ൃɺ͓Αͼӡ༻ܦݧ
    ෼ࢄॲཧγεςϜʢHadoop,
    Spark, etc.ʣ্ͰͷσʔλՃ
    ޻ɺγεςϜͷ࣮૷ɺύϑΥʔ
    ϚϯεධՁͳͲͷߦఔΛ
    TerabyteʙPetabyteن໛ͷେن
    ໛σʔλͰ࣮ࢪͰ͖ΔεΩϧ

    ػցֶशΤϯδχΞͱڠಇͯ͠
    ۀ຿Λ਱ߦ͢ΔͨΊͷɺػցֶ
    शʹؔ͢Δجຊత஌ࣝ

    ػցֶशٕज़Λ༻͍ͨαʔϏε
    ͷ։ൃɺ·ͨ͸։ൃ؅ཧͷܦݧ

    ࣄۀ/Ϗδωεɾػցֶशٕज़
    Λཧղ্ͨ͠Ͱɺجૅతͳ෼ੳ
    ʙاըɾఏҊ͕Ͱ͖Δೳྗ

    αʔϏεͷ։ൃऀ΍Ϗδωε෦
    ໳౳ͷؔ܎ऀͱ࿈ܞ͠ɺεϜʔ
    ζʹۀ຿ΛਐΊΔͨΊͷίϛϡ
    χέʔγϣϯೳྗ
    ίϯϐϡʔλαΠΤϯεઐ߈
    and/or ιϑτ΢ΣΞ։ൃͷܦݧ
    ϓϩδΣΫτϚωʔδϟʔ΍ͦ
    Εʹ४ͣΔ৬຿ܦݧ

    αʔϏεͷ։ൃऀ΍Ϗδωε෦
    ໳౳ͷؔ܎ऀͱ࿈ܞ͠ɺεϜʔ
    ζʹۀ຿ΛਐΊΔͨΊͷίϛϡ
    χέʔγϣϯೳྗ

    View full-size slide

  17. TEAM MEMBERS' VOCICES

    View full-size slide

  18. ΍Γ͕͍ɾ޲͍͍ͯΔਓ
    ML ENGINEERING AT LINE
    σʔλαΠζ͕େ͖͍ͷͰɺܭࢉ؀ڥߏஙʹ͓͍ͯάάͬͨ಺༰͕໾
    ʹཱͨͣɺࣗ෼Ͱߟ͑ͯ࡞Βͳ͚Ε͹ͳΒͳ͍ɻͦͷ෼ɺࡋྔ͕େ͖
    ͍ͷͰָ͍͠
    ࣗ෼Ͱߟ͑ͨ΋ͷΛߏஙɺҡ࣋Ͱ͖Δɻࣾ಺ͷHadoop؀ڥͳͲ΋
    ࢼߦࡨޡͰߏங͞Ε͍ͯΔɻҠߦఫୀͳͲ΋ଟ͍ͨΊɺؒҧͬͨ
    ΋ͷ΍΋͏࢖Θͳ͘ͳͬͨ΋ͷΛ࡞ͬͯ΋Ԝ·ͣʹ࡞Γ௚ͤΔ
    ๲େʹσʔλ͕͋ΔʢʮϞσϧߏஙΛؤுͬͯ΋ͦ΋ͦ΋σʔλ͕
    গͳͯ͘μϝʯͱ͍͏͜ͱ͕গͳ͍ʣ
    ߴ͍ٕज़΍ࢹ࠲Λ࣋ͬͨϝϯόʔ͕ଟ͍தͰɺ͔ͳΓࣗ༝ʹ։
    ൃ͕ߦ͑Δ
    ཧ૝ͱݱ࣮ͷόϥϯεΛऔΓͳ͕Βɺࣗ෼ͳΓͷࠜڌΛ࣋ͬͯ
    ։ൃ͕ߦ͑Δ
    ੹೚ײ͕͋Δਓʢಉ྅΍ࣾ಺ؔ܎ऀͱͷ৴པΛੵΈ্͍͛ͯ͘
    ͜ͱͰɺେ͖ͳ࢓ࣄ͕Ͱ͖ΔΑ͏ʹͳ͍ͬͯ͘ʣ
    ઌਐతͳख๏Λҙཉతʹࢼͭͭ͠΋ɺैདྷͷख๏ͷྑ͞΋େࣄʹͰ
    ͖Δਓ ʢσʔλ͕๲େͩͬͨΓɺܭࢉ࣌ؒʹ੍໿͕͋ͬͨΓ͢Δ
    ͷͰɺ ࠷৽ͷख๏ʹͩ͜ΘΓ͗͢Δͱ͏·͍͔͘ͳ͍͜ͱ΋ʣ
    ෼ࢄॲཧج൫Λ༻͍ͨϨίϝϯυΤϯδϯͷઃܭɺ࣮૷
    ԣஅతʹσʔλ׆༻͠ɺ਎ۙͳαʔϏεʹߩݙ͢Δ
    ࣗൃతʢSelf-motivatedʣͳਓʢৗʹ࢓ࣄ͕༩͑ΒΕΔΘ͚Ͱ͸ͳ
    ͍ɻࣗݾम࿅ΛߦͬͨΓɺࣗΒఏҊ͢Δ͜ͱ΋ඞཁʣ
    ίϛϡχέʔγϣϯʹରͯ͠લ޲͖ͳਓʢձࣾશମͱͯ͠ݟͯ΋ɺ
    ݞॻͰ࢓ࣄ͢Δਓ͸গͳ͍ɻ๬Ί͹ɺ୭ͱͰ΋࢓ࣄ͕Ͱ͖Δʣ
    ఏҊ͔Βɺ։ൃɺ෼ੳɺӡ༻ʢSREʣ·Ͱ͢΂ͯؔΘΔ

    View full-size slide

  19. ● Building a smart recommender system across LINE services (LINE DEVELOPER DAY 2019)
    ● https://logmi.jp/tech/articles/322247
    ● LINEॳͷΤϯδχΞϑΣϩʔฒ઒३͞Μ͸ɺͲΜͳ࢓ࣄΛ͍ͯ͠Δͷ͔
    ● https://engineering.linecorp.com/ja/blog/line-engineer-insights-vol-10-line-fellow/
    ● Machine Learning at LINE (LINE DEVELOPER DAY 2018ʣ
    ● https://logmi.jp/tech/articles/320435
    ● LINE's AB Test Standardization Process with Our Own Toolset (LINE DEVELOPER DAY 2018)
    ● https://www.slideshare.net/linecorp/lines-ab-test-standardization-with-our-own-
    toolset-124023353
    ࢀߟࢿྉ

    View full-size slide