The Web Conference2020 参加報告会

https://connpass.com/event/174856/?utm_campaign=event_publish_to_follower&utm_source=notifications&utm_medium=twitter

Transcript

1. ©2020 Wantedly, Inc.
   ࿦จɾηογϣϯ঺հ
   The Web Conference2020 ࢀՃใࠂձ
   Apr 30, 2020 - Shuhei Goda - @jy_msc
   

   View Slide

2. ©2020 Wantedly, Inc.
   ࣗݾ঺հ
   Shuhei Goda
   Twitter: hakubishin3 (@jy_msc)
   Visit Engineering Team
   Kaggle Master
   

   View Slide

3. ©2020 Wantedly, Inc.
   1. Deep Transfer Learning for Search and Recommendation
   ɾݕࡧɾਪનγεςϜʹ͓͚ΔసҠֶशͷԠ༻ͱLinkedinͰͷࣄྫΛ঺հ͍ͯ͠Δ (Tutorial)
   2. Clustering and Constructing User Coresets to Accelerate Large-
   scale Top-K Recommender Systems
   ɾਪનର৅ͷϢʔβΛᅂ޷ͷ͍ۙ΋ͷಉ࢜ͰΫϥελϦϯά͠, ֤Ϋϥελͷ୅දϕΫτϧΛ

   ɹ ࢖ͬͯۙࣅ࠷ۙ๣୳ࡧΛ͢Δ͜ͱͰਪનγεςϜͷεϐʔυΞοϓΛ࣮ݱ
   3. Adaptive Hierarchical Translation-based Sequential Recommendation
   ɾϢʔβߦಈͷ࣌ܥྻΛߟྀͨ͠Translation-based Reccommendationʹ, ΞΠςϜؒͷ

   ɹ ؔ܎ੑͱ࣌ؒґଘͷuser translation vectorΛՃ͑ͨख๏ΛఏҊ
   ຊൃදͰ঺հ͢Δ࿦จɾηογϣϯ
   

   View Slide

4. INTERNAL ONLY
   ©2020 Wantedly, Inc.
   %FFQ5SBOTGFS-FBSOJOHGPS4FBSDIBOE3FDPNNFOEBUJPO
   IUUQTTJUFTHPPHMFDPNWJFXXXXEUMUVUPSJBMIPNF
   

   View Slide

5. ©2020 Wantedly, Inc.
   Motivation
   ݕࡧɾਪનγεςϜͰసҠֶशΛ࢖͏ཧ༝͸ʁ
   ɾαʔϏε಺Ͱ܇࿅σʔλΛ͋·ΓऔಘͰ͖ͳ͍ঢ়گԼͰ΋ੑೳΛߴΊ͍ͨ
   ɹɾྫ͑͹, αʔϏεΛ্ཱͪ͛ͨ௚ޙ΍ར༻Ϣʔβ͕গͳ͍Α͏ͳ৔߹
   

   View Slide

6. ©2020 Wantedly, Inc.
   Deep Transfer Learning
   1. Model Transfer
   ɹɾsource domainsͰͷֶशࡁΈϞσϧΛtarget domainͰ࢖͏
   ɹɾfine-tuning (e.g. BERT) ΍ ֤domainͰͷtaskͷmulti-task learning
   2. Feature Representation Transfer
   ɹɾҟͳΔdomainؒͰڞ༗Մೳͳಛ௃Λֶश
   ɹɾsource domainͰͷֶशࡁΈϞσϧ͔Βͷಛ௃நग़ ΍ domain adaptation
   3. Instance Transfer
   ɹɾsource domain͔Β܇࿅σʔλΛ࣋ͬͯ͘Δ
   

   View Slide

7. ©2020 Wantedly, Inc.
   Deep Transfer Learning for Search and Recommendation
   ݕࡧɾਪનγεςϜͷͲͷ෦෼ͰసҠֶशΛ࢖͏͔
   Raw Query
   Query
   Understanding
   User Profile,
   Activities
   Raw Document
   User
   Understanding
   Document
   Understanding
   Document
   Retrieval
   Ranking
   ←ΑΓྑ͍ཧղͷͨΊʹ, pretrained model

   ͔ΒຒΊࠐΈදݱΛྲྀ༻ (Feature Transfer)
   ←ຒΊࠐΈۭؒ಺ͰͷީิΞΠςϜͷ֦ு
   (Feature Transfer)
   ←ؔ࿈͢ΔαʔϏεͷσʔλ΍ϞσϧΛ

   ྲྀ༻͢Δ(Model Transfer & Instance Transfer)
   

   View Slide

8. ©2020 Wantedly, Inc.
   Case study in Linkedin (1)
   LinkedinͰ͸ෳ਺ͷϓϩμΫτʹ͓͚ΔϢʔβͷߦಈཤྺ͕औಘͰ͖Δ
   Job Search People Search
   ଞϢʔβ΁ͷ

   ܨ͕ΓϦΫΤετ
   ืू΁ͷԠื
   https://www.linkedin.com/
   

   View Slide

9. ©2020 Wantedly, Inc.
   Case study in Linkedin (2)
   People Search + Careers Job Recommendation
   ɾ֤υϝΠϯͷਪનλεΫΛղ͍ͯuser, itemͷදݱΛֶश͠, ಛ௃ྔͱͯ͠ར༻
   Add embeddings trained with...
   AUC Lift
   NDCG@25
   Lift
   Tree Model Baseline

   (no embedding added)
   0.00% 0.00%
   + People Search Embedding
   (Single-task) 1.25% 0.85%
   + Multi-task Embedding 1.82% 1.50%
   

   View Slide

10. INTERNAL ONLY
    ©2020 Wantedly, Inc.
    $MVTUFSJOHBOE$POTUSVDUJOH6TFS$PSFTFUTUP"DDFMFSBUF

    -BSHFTDBMF5PQ,3FDPNNFOEFS4ZTUFNT
    +ZVO:V+JBOH 1BUSJDL)$IFO $IP+VJ)TJFI 8FJ8BOH
    IUUQTEMBDNPSHEPJ
    

    View Slide

11. ©2020 Wantedly, Inc.
    Motivation
    ྨࣅΞΠςϜͷ୳ࡧͷߴ଎Խ
    ɾϢʔβʹਪન͢ΔΞΠςϜΛ୳͢ࡍ, PredictionʹؔΘΔ࣮ߦ࣌ؒΛશମతʹ୹͍ͨ͘͠
    ɹɾPrediction time = Preparation time + Inference time
    ɹɾطଘͷ୳ࡧΞϧΰϦζϜͰ͸Inferece timeͷ୹ॖʹয఺Λ౰͍ͯͯΔ

    ɾطଘͷख๏Ͱ͸Ϣʔβؒͷؔ܎ੑΛߟྀ͍ͯ͠ͳ͍ͷͰ, ߴ଎Խʹ׆༻͍ͨ͠
    ɹɾ࠷ۙ๣୳ࡧ͸೚ҙͷΫΤϦϕΫτϧ͕༩͑ΒΕͨ࣌ʹ͍ۙϕΫτϧΛ୳͢໰୊͕ͩ, 

    ɹɹਪનγεςϜͰ͸ΫΤϦϕΫτϧ͸ϢʔβϕΫτϧͰ΋͋Γ, ϢʔβϕΫτϧ͸ڧ͍

    ɹɹΫϥελϦϯάߏ଄Λ͍࣋ͬͯΔ
    ɹɾ͍ۙϢʔβ͸ࣅͨؔ৺Λ͍࣋ͬͯΔͨΊ, ୳ࡧ࣌ʹ·ͱΊͯ͠·͑͹ޮ཰తͰ͸?
    

    View Slide

12. ©2020 Wantedly, Inc.
    Framework Overview
    Preparation Stage:
    1. αϒαϯϓϧͨ͠ϢʔβΛΫϥελϦϯά
    2. άϧʔϓ಺ͷϢʔβΛΧόʔͰ͖ΔΑ͏ͳ

    ෳ਺ͷ୅දϕΫτϧΛϢʔβάϧʔϓຖʹٻΊΔ
    3. ΞΠςϜͷ֊૚ܕάϥϑߏ଄Λߏங͠,
    ɹ֤Ϣʔβάϧʔϓͷ୅දϕΫτϧΛ༻͍ͨ

    ɹgreedyͳάϥϑ୳ࡧʹΑΓ, άϧʔϓຖʹ

    ɹީิΞΠςϜू߹Λऔಘ
    Prediction Stage:
    4. ༧ଌର৅Ϣʔβͷॴଐ͢ΔάϧʔϓΛ୳͠,
    ɹͦͷάϧʔϓʹରԠ͢ΔީิΞΠςϜू߹ͱ
    ɹϢʔβϕΫτϧͷ಺ੵΛͱͬͯϥϯΫ෇͚
    

    View Slide

13. ©2020 Wantedly, Inc.
    User Group Clutering
    ࣅͨΑ͏ͳڵຯΛ࣋ͭϢʔβͷू߹Λ࡞Δ
    ɾϢʔβΛࢦఆͷΫϥελ਺ͰΫϥελϦϯά
    ɾ൓෮ܭࢉΛߦ͏ͷͰ, ࣄલʹϢʔβΛαϒαϯϓϦϯά
    z͸, Ϣʔβͷॴଐ͢ΔάϧʔϓΛࣔ͢ࢦࣔؔ਺
    p_i͕ϢʔβiͷϕΫτϧͰ, v_r ͸άϧʔϓrͷॏ৺ϕΫτϧ
    

    View Slide

14. ©2020 Wantedly, Inc.
    Coreset Vectors
    ϢʔβάϧʔϓͷڵຯΛΧόʔͰ͖ΔΑ͏ͳ୅දϕΫτϧΛऔಘ
    ɾॏ৺ϕΫτϧͩͱ໢ཏͰ͖ͳ͍ͷͰ, ΫϥελͷൣғʹԠͨ͡਺ͷ୅දϕΫτϧ͕ඞཁ
    ɾఆٛ4Λຬͨ͢Α͏ʹ୅දϕΫτϧΛऔಘ
    

    View Slide

15. ©2020 Wantedly, Inc.
    Experiments (1)
    Ϟσϧ͸NMFΛ࢖༻, Ϋϥελ਺͸8
    SU͸O(nm)ͷφΠʔϒͳΞϓϩʔν(SVDS)͔Βͷશମ଎౓ͷվળ཰ (n͸userͷ਺, m͸itemͷ਺)
    PT͸Preparation time, IT͸Inference timeͷ͜ͱ
    ਫ਼౓ྼԽΛ཈͑ͭͭPrediction Time͕୹ॖ͞Ε͍ͯΔ
    

    View Slide

16. ©2020 Wantedly, Inc.
    Experiments (2)
    ਫ਼౓ͱ଎౓ͷཁٻ࢓༷ʹԠͯ͡, Ϋϥελ਺Λௐ੔͢Δ
    

    View Slide

17. INTERNAL ONLY
    ©2020 Wantedly, Inc.
    "EBQUJWF)JFSBSDIJDBM5SBOTMBUJPOCBTFE4FRVFOUJBM
    3FDPNNFOEBUJPO
    :JO;IBOH :VO)F +JBOMJOH8BOH +BNFT$BWFSMFF
    IUUQTEMBDNPSHEPJBCT
    

    View Slide

18. ©2020 Wantedly, Inc.
    Motivation
    ΞΠςϜؒͷؔ࿈ੑͱϢʔβͷߦಈܥྻΛߟྀͨ͠ਪનΛ͍ͨ͠
    ɾϢʔβͷߦಈܥྻʹج͍ͮͨਪન (Sequential recommendation)
    ɹɾe.g. RNN, Ϛϧίϑ࿈࠯, CNN, Translated-based Recommendation
    ɾϢʔβͷߪങܥྻͷཧղʹ͸ΞΠςϜؒͷ૬ิؔ܎ͱ୅ସؔ܎͕ॏཁ
    ɹɾPCΛߪೖͨ͠௚ޙʹผͷPCΛങ͏͜ͱ͸ͳ͍͕, ͕࣌ؒܦͬͨΒങ͍׵͑ͨ͘ͳΔ
    

    View Slide

19. ©2020 Wantedly, Inc.
    Model Overview
    Inputs:
    User Sequences Item Relations
    ิ׬ؔ܎
    ୅ସؔ܎
    Ouputs:
    Ϣʔβ͕࣍ʹڵຯΛ࣋ͪͦ͏ͳΞΠςϜ
    

    View Slide

20. ©2020 Wantedly, Inc.
    Item Relations
    ΞΠςϜ͸ೖΕସΘΓͷස౓͕ߴ͍, ·ͨϢʔβ͔Βߪങ͞ΕΔΞΠςϜ͸εύʔε.
    ͦ͜ͰΧςΰϦϨϕϧͷؔ܎ΛݟΔ͜ͱʹ͢Δ.
    ←ҟͳΔΞΠςϜΛ

    ɹߪೖ͍ͯ͠Δ
    ←ΧςΰϦΛݟΔͱ
    ɹߪങύλʔϯ͕ಉ͡
    ΞΠςϜؒͷؔ࿈ੑ → ΧςΰϦϨϕϧͰݟΔ
    Ϣʔβͷߦಈܥྻ → ΞΠςϜϨϕϧͰݟΔ
    

    View Slide

21. ©2020 Wantedly, Inc.
    Proposed Method: HierTrans
    ֊૚ܕͷ࣌ؒάϥϑΛߏங
    ɾิ׬ؔ܎, ΋͘͠͸୅ସؔ܎Ͱ઀ଓ͞ΕΔΧςΰϦϨϕϧͷάϥϑ
    ɾϢʔβͷߦಈܥྻͷάϥϑ
    

    View Slide

22. ©2020 Wantedly, Inc.
    Recommendation with HierTrans
    ←Ϣʔβͷߦಈܥྻ
    T࣌఺·ͰͷܥྻΛ΋ͱʹ
    ֤αϒάϥϑ͔Β৘ใΛऔಘ
    (Head͸GIͱGCΛ

    ݁߹ͤ͞ΔͨΊʹؔ਺)
    T࣌఺·Ͱͷܥྻͱ

    ΞΠςϜͷؔ܎ੑΛ࢖ͬͯ,
    ϢʔβͷTranslation vectorΛ
    ٻΊΔ
    

    View Slide

23. ©2020 Wantedly, Inc.
    Experiments
    BPR: Bayesian personalized ranking
    TransE, TransFM, TransRec: Translation-based Methods
    ….
    

    View Slide

24. INTERNAL ONLY
    ©2020 Wantedly, Inc.
    'JO
    

    View Slide