Recommendation systems on microservices - productivity & reproducibility

Data Driven Developer Meetup #1 https://d3m.connpass.com/event/98934/ で発表した資料です。Wantedly People では、推薦システムを microservices として実装することで productivity を向上しつつ、再現性を担保するために project template を導入しています。#d3m

Transcript

1. ©2018 Wantedly, Inc.
   ਪનͱ microservices
   Productivity & Reproducibility
   11.Sep.2018 - Naomichi Agata
   

   View Slide

2. ©2018 Wantedly, Inc.
   Naomichi Agata
   Software engineer at Wantedly, inc.
   Server side + Machine learning @ Wantedly People
   Github
   Twitter
   @agatan
   @agatan_
   

   View Slide

3. ©2018 Wantedly, Inc.
   Recommendations in Wantedly People
   ⾣ͭͳ͕Γ
   ⾣
   ʰ஌Γ߹͍Ͱ͔͢ʁʱ
   ⾣χϡʔε
   ⾣
   ਓؾͷχϡʔε
   ⾣
   ޿ࠂͷදࣔ
   

   View Slide

4. ©2018 Wantedly, Inc.
   ͲΜͳ࢓૊ΈͰਪન͢Δ͔
   ⾣ਓؾ౓
   ⾣ڠௐϑΟϧλϦϯά
   ⾣Like ਺ʹԠͨ͡ਪન
   ⾣ίϯςϯπϕʔεͷػցֶशʹΑΔਪન
   ⾣ಉ࣌͡ظʹಉ͡ձࣾʹॴଐ͍ͯͨ͠
   ⾣ڞ௨ͷͭͳ͕Γ਺΍ Adamic/Adar
   ⾣ͭͳ͕Γάϥϑʹ஫໨ͨ͠Network Embedding ϕʔεͷਪન
   

   View Slide

5. ©2018 Wantedly, Inc.
   ࢼ͍ͨ͠΋ͷ͸ແݶʹ͋Δ
   ⾣͢͹΍͘ޮ཰తʹࢼ͍ͨ͠
   ⾣ڞ௨͢Δॲཧ͕ͳ͍෦෼͸ಠཱʹ։ൃ͍ͨ͠
   ⾣࣮ݧతͳ feature/algorithm Λؾܰʹ࣮૷͍ͨ͠
   ⾣ A/B ςετΛ؆୯ʹߦ͍͍ͨ
   

   View Slide

6. ©2018 Wantedly, Inc.
   Strategies as microservices
   

   View Slide

7. ©2018 Wantedly, Inc.
   aggregator
   Strategy
   Strategy
   Strategy
   Strategy
   ༑ୡͷ-JLFͨ͠ΞΠςϜΛఏࣔ
   ڠௐϑΟϧλϦϯά
   ਓؾ౓
   ͭͳ͕Γ৘ใ͔Βͷਪન
   Recommendation Overview
   ݁ՌΛू໿
   re-ordering
   logging
   ฒߦॲཧ
   A/B
   

   View Slide

8. ©2018 Wantedly, Inc.
   Pros.
   ⾣։ൃͷฒྻ౓͕ߴ͍
   ⾣A/B ςετ͕͠΍͍͢
   ⾣
   લஈαʔόͰৼΓ෼͚Δ͚ͩɻධՁͷͨΊͷLogging΋લஈαʔόʹू໿Ͱ͖Δɻ
   ⾣
   ݸʑͷਪનϩδοΫͰ͸ࣗ෼ͷਪન಺༰ʹ͍͚ͭͯͩߟ͑Ε͹ྑ͍
   ⾣ͩΊͩͬͨͱ͖ʹࣺͯ΍͍͢
   ⾣
   lೖΓ૊Μͩ࢓૊ΈΛ࡞ͬͯmerge → ධՁͷ݁ՌɺࣺͯΔ͜ͱʹͳͬͨz
   ⾣
   ։ൃ଎౓͕͸΍͍ͱrevert ΋Ұۤ࿑
   lݴޠ͕͕ͪ͏͔ΒmicroservicezͰ͸ͳ͍
   

   View Slide

9. ©2018 Wantedly, Inc.
   Cons.
   ⾣ߴ଎ฒྻʹࢼߦࡨޡ͕૸Δ
   ⾣
   ࢼߦࡨޡ͕؅ཧෆೳʹͳΔ
   ⾣
   Ұਓͷ։ൃऀʹ͔͍͠͡Εͳ͍ϞσϧʹͳΔ
   ⾣
   ஌ݟͷڞ༗͕೉͍͠
   ⾣αʔόͱͯ͠ػೳͤ͞Δͷ͕໘౗
   

   View Slide

10. ©2018 Wantedly, Inc.
    Project Template for ML
    ⾣ cookiecutter-data-scienceϕʔεͷProject TemplateΛ࡞ͬͨ
    ⾣
    IUUQTHJUIVCDPNESJWFOEBUBDPPLJFDVUUFSEBUBTDJFODF
    ⾣෼ੳج൫τʔΫͰฉ͍ͨࣄྫΛࢀߟʹࢼݧతʹಋೖத
    ⾣
    IUUQTEBGUDPOOQBTTDPNFWFOU
    ⾣
    IUUQTTQFBLFSEFDLDPNBZFNPTIPNFNBEFNBDIJOFMFBSOJOHUPPMLJUT
    

    View Slide

11. ©2018 Wantedly, Inc.
    Project Template for ML
    ⾣ σʔλͷऔಘɼલॲཧɼֶशɼධՁͷ entry pointΛنఆ͢Δ
    ⾣σʔλ͸S3 ʹஔ͘લఏ
    ⾣"1*αʔόͱͯ͠ಈ͔ͨ͢ΊͷtemplateΛఏڙ
    ⾣ࣾ಺microservices༻utility libraryͷ༻ҙ
    ⾣యܕతͳDockerfileΛ༻ҙ
    ⾣*.pyͷத਎ʹ͍ͭͯ͸΄΅੍ݶΛ՝͞ͳ͍
    

    View Slide

12. ©2018 Wantedly, Inc.
    Makefile for ML Project
    

    View Slide

13. ©2018 Wantedly, Inc.
    Conclusion
    ⾣ਪન͸ධՁͱվળ͕ॏཁ
    ⾣
    ͨ͘͞Μࢼͯ͠ධՁ͍ͨ͠
    ⾣Microservices ͱ࣮ͯ͠૷
    ⾣
    ࡞Γ΍͍ࣺͯ͢΍͍͢
    ⾣
    lݴޠ͕͕ͪ͏͔ΒzҎ্ͷϝϦοτ͕͋Δ
    ⾣
    ΋ͪΖΜͭΒ͍ͱ͜Ζɾ೰·͍͠ͱ͜Ζ΋ͨ͘͞Μ͋Δ
    ⾣Project TemplateΛಋೖ
    ▶
    Productivity++
    ▶
    Reproducibility++
    

    View Slide