Recommendation systems on microservices - productivity & reproducibility

Data Driven Developer Meetup #1 https://d3m.connpass.com/event/98934/ で発表した資料です。Wantedly People では、推薦システムを microservices として実装することで productivity を向上しつつ、再現性を担保するために project template を導入しています。#d3m

Transcript

1. ©2018 Wantedly, Inc. ਪનͱ microservices Productivity & Reproducibility 11.Sep.2018 - Naomichi Agata
2. ©2018 Wantedly, Inc. Naomichi Agata Software engineer at Wantedly, inc. Server side + Machine learning @ Wantedly People Github Twitter @agatan @agatan_
3. ©2018 Wantedly, Inc. Recommendations in Wantedly People ⾣
   ͭͳ͕Γ
   ⾣
   ʰ஌Γ߹͍Ͱ͔͢ʁʱ
   ⾣
   χϡʔε
   ⾣
   ਓؾͷχϡʔε
   ⾣
   ޿ࠂͷදࣔ
4. ©2018 Wantedly, Inc. ͲΜͳ࢓૊ΈͰਪન͢Δ͔ ⾣
   ਓؾ౓
   ⾣
   ڠௐϑΟϧλϦϯά
   ⾣
   Like ਺ʹԠͨ͡ਪન
   ⾣
   ίϯςϯπϕʔεͷػցֶशʹΑΔਪન
   ⾣
   ಉ࣌͡ظʹಉ͡ձࣾʹॴଐ͍ͯͨ͠
   ⾣
   ڞ௨ͷͭͳ͕Γ਺΍ Adamic/Adar
   ⾣
   ͭͳ͕Γάϥϑʹ஫໨ͨ͠
   Network Embedding ϕʔεͷਪન
5. ©2018 Wantedly, Inc. ࢼ͍ͨ͠΋ͷ͸ແݶʹ͋Δ ⾣
   ͢͹΍͘ޮ཰తʹࢼ͍ͨ͠
   ⾣
   ڞ௨͢Δॲཧ͕ͳ͍෦෼͸ಠཱʹ։ൃ͍ͨ͠
   ⾣
   ࣮ݧతͳ feature/algorithm Λؾܰʹ࣮૷͍ͨ͠
   ⾣ A/B ςετΛ؆୯ʹߦ͍͍ͨ
6. ©2018 Wantedly, Inc. Strategies as microservices
7. ©2018 Wantedly, Inc. aggregator Strategy Strategy Strategy Strategy ༑ୡͷ
   -JLF
   ͨ͠ΞΠςϜΛఏࣔ ڠௐϑΟϧλϦϯά ਓؾ౓ ͭͳ͕Γ৘ใ͔Βͷਪન Recommendation Overview ݁ՌΛू໿
   re-ordering logging ฒߦॲཧ
   A/B
8. ©2018 Wantedly, Inc. Pros. ⾣
   ։ൃͷฒྻ౓͕ߴ͍
   ⾣
   A/B ςετ͕͠΍͍͢
   ⾣
   લஈαʔόͰৼΓ෼͚Δ͚ͩɻධՁͷͨΊͷ
   Logging
   ΋લஈαʔόʹू໿Ͱ͖Δɻ
   ⾣
   ݸʑͷਪનϩδοΫͰ͸ࣗ෼ͷਪન಺༰ʹ͍͚ͭͯͩߟ͑Ε͹ྑ͍
   ⾣
   ͩΊͩͬͨͱ͖ʹࣺͯ΍͍͢
   ⾣
   lೖΓ૊Μͩ࢓૊ΈΛ࡞ͬͯ
   merge → ධՁͷ݁ՌɺࣺͯΔ͜ͱʹͳͬͨz
   ⾣
   ։ൃ଎౓͕͸΍͍ͱ
   revert ΋Ұۤ࿑
   lݴޠ͕͕ͪ͏͔Β
   microservicez
   Ͱ͸ͳ͍
9. ©2018 Wantedly, Inc. Cons. ⾣
   ߴ଎
   
   ฒྻʹࢼߦࡨޡ͕૸Δ
   ⾣
   ࢼߦࡨޡ͕؅ཧෆೳʹͳΔ
   ⾣
   Ұਓͷ։ൃऀʹ͔͍͠͡Εͳ͍ϞσϧʹͳΔ
   ⾣
   ஌ݟͷڞ༗͕೉͍͠
   ⾣
   αʔόͱͯ͠ػೳͤ͞Δͷ͕໘౗
10. ©2018 Wantedly, Inc. Project Template for ML ⾣ cookiecutter-data-science
    ϕʔεͷ
    Project Template
    Λ࡞ͬͨ
    ⾣
    IUUQTHJUIVCDPNESJWFOEBUBDPPLJFDVUUFSEBUBTDJFODF
    ⾣
    ෼ੳج൫τʔΫͰฉ͍ͨࣄྫΛࢀߟʹࢼݧతʹಋೖத
    ⾣
    IUUQTEBGUDPOOQBTTDPNFWFOU
    ⾣
    IUUQTTQFBLFSEFDLDPNBZFNPTIPNFNBEFNBDIJOFMFBSOJOHUPPMLJUT
11. ©2018 Wantedly, Inc. Project Template for ML ⾣ σʔλͷऔಘɼલॲཧɼֶशɼධՁͷ entry point
    Λنఆ͢Δ
    ⾣
    σʔλ͸
    S3 ʹஔ͘લఏ
    ⾣
    "1*
    αʔόͱͯ͠ಈ͔ͨ͢Ίͷ
    template
    Λఏڙ
    ⾣
    ࣾ಺
    microservices
    ༻
    utility library
    ͷ༻ҙ
    ⾣
    యܕతͳ
    Dockerfile
    Λ༻ҙ
    ⾣
    *.py
    ͷத਎ʹ͍ͭͯ͸΄΅੍ݶΛ՝͞ͳ͍
12. ©2018 Wantedly, Inc. Makefile for ML Project
13. ©2018 Wantedly, Inc. Conclusion ⾣
    ਪન͸ධՁͱվળ͕ॏཁ
    ⾣
    ͨ͘͞Μࢼͯ͠ධՁ͍ͨ͠
    ⾣
    Microservices ͱ࣮ͯ͠૷
    ⾣
    ࡞Γ΍͍͢
    
    ࣺͯ΍͍͢
    ⾣
    lݴޠ͕͕ͪ͏͔Βz
    Ҏ্ͷϝϦοτ͕͋Δ
    ⾣
    ΋ͪΖΜͭΒ͍ͱ͜Ζɾ೰·͍͠ͱ͜Ζ΋ͨ͘͞Μ͋Δ
    ⾣
    Project Template
    Λಋೖ
    
    ▶ Productivity++ ▶ Reproducibility++