The Web Conference2020 参加報告会

Transcript

1. ©2020 Wantedly, Inc. ࿦จɾηογϣϯ঺հ The Web Conference2020 ࢀՃใࠂձ Apr 30,

   2020 - Shuhei Goda - @jy_msc

2. ©2020 Wantedly, Inc. ࣗݾ঺հ Shuhei Goda Twitter: hakubishin3 (@jy_msc) Visit

   Engineering Team Kaggle Master

3. ©2020 Wantedly, Inc. 1. Deep Transfer Learning for Search and

   Recommendation ɾݕࡧɾਪનγεςϜʹ͓͚ΔసҠֶशͷԠ༻ͱLinkedinͰͷࣄྫΛ঺հ͍ͯ͠Δ (Tutorial) 2. Clustering and Constructing User Coresets to Accelerate Large- scale Top-K Recommender Systems ɾਪનର৅ͷϢʔβΛᅂ޷ͷ͍ۙ΋ͷಉ࢜ͰΫϥελϦϯά͠, ֤Ϋϥελͷ୅දϕΫτϧΛ
 ɹ ࢖ͬͯۙࣅ࠷ۙ๣୳ࡧΛ͢Δ͜ͱͰਪનγεςϜͷεϐʔυΞοϓΛ࣮ݱ 3. Adaptive Hierarchical Translation-based Sequential Recommendation ɾϢʔβߦಈͷ࣌ܥྻΛߟྀͨ͠Translation-based Reccommendationʹ, ΞΠςϜؒͷ
 ɹ ؔ܎ੑͱ࣌ؒґଘͷuser translation vectorΛՃ͑ͨख๏ΛఏҊ ຊൃදͰ঺հ͢Δ࿦จɾηογϣϯ

4. INTERNAL ONLY ©2020 Wantedly, Inc. %FFQ
   5SBOTGFS
   -FBSOJOH
   GPS
   4FBSDI
   BOE
   3FDPNNFOEBUJPO IUUQTTJUFTHPPHMFDPNWJFXXXXEUMUVUPSJBMIPNF

5. ©2020 Wantedly, Inc. Motivation ݕࡧɾਪનγεςϜͰసҠֶशΛ࢖͏ཧ༝͸ʁ ɾαʔϏε಺Ͱ܇࿅σʔλΛ͋·ΓऔಘͰ͖ͳ͍ঢ়گԼͰ΋ੑೳΛߴΊ͍ͨ ɹɾྫ͑͹, αʔϏεΛ্ཱͪ͛ͨ௚ޙ΍ར༻Ϣʔβ͕গͳ͍Α͏ͳ৔߹

6. ©2020 Wantedly, Inc. Deep Transfer Learning 1. Model Transfer ɹɾsource

   domainsͰͷֶशࡁΈϞσϧΛtarget domainͰ࢖͏ ɹɾfine-tuning (e.g. BERT) ΍ ֤domainͰͷtaskͷmulti-task learning 2. Feature Representation Transfer ɹɾҟͳΔdomainؒͰڞ༗Մೳͳಛ௃Λֶश ɹɾsource domainͰͷֶशࡁΈϞσϧ͔Βͷಛ௃நग़ ΍ domain adaptation 3. Instance Transfer ɹɾsource domain͔Β܇࿅σʔλΛ࣋ͬͯ͘Δ

7. ©2020 Wantedly, Inc. Deep Transfer Learning for Search and Recommendation

   ݕࡧɾਪનγεςϜͷͲͷ෦෼ͰసҠֶशΛ࢖͏͔ Raw Query Query Understanding User Profile, Activities Raw Document User Understanding Document Understanding Document Retrieval Ranking ←ΑΓྑ͍ཧղͷͨΊʹ, pretrained model
 ͔ΒຒΊࠐΈදݱΛྲྀ༻ (Feature Transfer) ←ຒΊࠐΈۭؒ಺ͰͷީิΞΠςϜͷ֦ு (Feature Transfer) ←ؔ࿈͢ΔαʔϏεͷσʔλ΍ϞσϧΛ
 ྲྀ༻͢Δ(Model Transfer & Instance Transfer)

8. ©2020 Wantedly, Inc. Case study in Linkedin (1) LinkedinͰ͸ෳ਺ͷϓϩμΫτʹ͓͚ΔϢʔβͷߦಈཤྺ͕औಘͰ͖Δ Job

   Search People Search ଞϢʔβ΁ͷ
 ܨ͕ΓϦΫΤετ ืू΁ͷԠื https://www.linkedin.com/

9. ©2020 Wantedly, Inc. Case study in Linkedin (2) People Search

   + Careers Job Recommendation ɾ֤υϝΠϯͷਪનλεΫΛղ͍ͯuser, itemͷදݱΛֶश͠, ಛ௃ྔͱͯ͠ར༻ Add embeddings trained with... AUC Lift NDCG@25 Lift Tree Model Baseline
 (no embedding added) 0.00% 0.00% + People Search Embedding (Single-task) 1.25% 0.85% + Multi-task Embedding 1.82% 1.50%

10. INTERNAL ONLY ©2020 Wantedly, Inc. $MVTUFSJOH
    BOE
    $POTUSVDUJOH
    6TFS
    $PSFTFUT
    UP
    "DDFMFSBUF
    
 -BSHFTDBMF
    5PQ,
    3FDPNNFOEFS
    4ZTUFNT +ZVO:V
    +JBOH
    1BUSJDL
    )
    $IFO
    $IP+VJ
    )TJFI

    
    8FJ
    8BOH
    
    IUUQTEMBDNPSHEPJ

11. ©2020 Wantedly, Inc. Motivation ྨࣅΞΠςϜͷ୳ࡧͷߴ଎Խ ɾϢʔβʹਪન͢ΔΞΠςϜΛ୳͢ࡍ, PredictionʹؔΘΔ࣮ߦ࣌ؒΛશମతʹ୹͍ͨ͘͠ ɹɾPrediction time =

    Preparation time + Inference time ɹɾطଘͷ୳ࡧΞϧΰϦζϜͰ͸Inferece timeͷ୹ॖʹয఺Λ౰͍ͯͯΔ
 ɾطଘͷख๏Ͱ͸Ϣʔβؒͷؔ܎ੑΛߟྀ͍ͯ͠ͳ͍ͷͰ, ߴ଎Խʹ׆༻͍ͨ͠ ɹɾ࠷ۙ๣୳ࡧ͸೚ҙͷΫΤϦϕΫτϧ͕༩͑ΒΕͨ࣌ʹ͍ۙϕΫτϧΛ୳͢໰୊͕ͩ, 
 ɹɹਪનγεςϜͰ͸ΫΤϦϕΫτϧ͸ϢʔβϕΫτϧͰ΋͋Γ, ϢʔβϕΫτϧ͸ڧ͍
 ɹɹΫϥελϦϯάߏ଄Λ͍࣋ͬͯΔ ɹɾ͍ۙϢʔβ͸ࣅͨؔ৺Λ͍࣋ͬͯΔͨΊ, ୳ࡧ࣌ʹ·ͱΊͯ͠·͑͹ޮ཰తͰ͸?

12. ©2020 Wantedly, Inc. Framework Overview Preparation Stage: 1. αϒαϯϓϧͨ͠ϢʔβΛΫϥελϦϯά 2.

    άϧʔϓ಺ͷϢʔβΛΧόʔͰ͖ΔΑ͏ͳ
 ෳ਺ͷ୅දϕΫτϧΛϢʔβάϧʔϓຖʹٻΊΔ 3. ΞΠςϜͷ֊૚ܕάϥϑߏ଄Λߏங͠, ɹ֤Ϣʔβάϧʔϓͷ୅දϕΫτϧΛ༻͍ͨ
 ɹgreedyͳάϥϑ୳ࡧʹΑΓ, άϧʔϓຖʹ
 ɹީิΞΠςϜू߹Λऔಘ Prediction Stage: 4. ༧ଌର৅Ϣʔβͷॴଐ͢ΔάϧʔϓΛ୳͠, ɹͦͷάϧʔϓʹରԠ͢ΔީิΞΠςϜू߹ͱ ɹϢʔβϕΫτϧͷ಺ੵΛͱͬͯϥϯΫ෇͚

13. ©2020 Wantedly, Inc. User Group Clutering ࣅͨΑ͏ͳڵຯΛ࣋ͭϢʔβͷू߹Λ࡞Δ ɾϢʔβΛࢦఆͷΫϥελ਺ͰΫϥελϦϯά ɾ൓෮ܭࢉΛߦ͏ͷͰ, ࣄલʹϢʔβΛαϒαϯϓϦϯά

    z͸, Ϣʔβͷॴଐ͢ΔάϧʔϓΛࣔ͢ࢦࣔؔ਺ p_i͕ϢʔβiͷϕΫτϧͰ, v_r ͸άϧʔϓrͷॏ৺ϕΫτϧ

14. ©2020 Wantedly, Inc. Coreset Vectors ϢʔβάϧʔϓͷڵຯΛΧόʔͰ͖ΔΑ͏ͳ୅දϕΫτϧΛऔಘ ɾॏ৺ϕΫτϧͩͱ໢ཏͰ͖ͳ͍ͷͰ, ΫϥελͷൣғʹԠͨ͡਺ͷ୅දϕΫτϧ͕ඞཁ ɾఆٛ4Λຬͨ͢Α͏ʹ୅දϕΫτϧΛऔಘ

15. ©2020 Wantedly, Inc. Experiments (1) Ϟσϧ͸NMFΛ࢖༻, Ϋϥελ਺͸8 SU͸O(nm)ͷφΠʔϒͳΞϓϩʔν(SVDS)͔Βͷશମ଎౓ͷվળ཰ (n͸userͷ਺, m͸itemͷ਺)

    PT͸Preparation time, IT͸Inference timeͷ͜ͱ ਫ਼౓ྼԽΛ཈͑ͭͭPrediction Time͕୹ॖ͞Ε͍ͯΔ

16. ©2020 Wantedly, Inc. Experiments (2) ਫ਼౓ͱ଎౓ͷཁٻ࢓༷ʹԠͯ͡, Ϋϥελ਺Λௐ੔͢Δ

17. INTERNAL ONLY ©2020 Wantedly, Inc. "EBQUJWF
    )JFSBSDIJDBM
    5SBOTMBUJPOCBTFE
    4FRVFOUJBM
    3FDPNNFOEBUJPO :JO
    ;IBOH
    :VO
    )F
    +JBOMJOH
    8BOH

    
    +BNFT
    $BWFSMFF
    
    IUUQTEMBDNPSHEPJBCT

18. ©2020 Wantedly, Inc. Motivation ΞΠςϜؒͷؔ࿈ੑͱϢʔβͷߦಈܥྻΛߟྀͨ͠ਪનΛ͍ͨ͠ ɾϢʔβͷߦಈܥྻʹج͍ͮͨਪન (Sequential recommendation) ɹɾe.g. RNN,

    Ϛϧίϑ࿈࠯, CNN, Translated-based Recommendation ɾϢʔβͷߪങܥྻͷཧղʹ͸ΞΠςϜؒͷ૬ิؔ܎ͱ୅ସؔ܎͕ॏཁ ɹɾPCΛߪೖͨ͠௚ޙʹผͷPCΛങ͏͜ͱ͸ͳ͍͕, ͕࣌ؒܦͬͨΒങ͍׵͑ͨ͘ͳΔ

19. ©2020 Wantedly, Inc. Model Overview Inputs: User Sequences Item Relations

    ิ׬ؔ܎ ୅ସؔ܎ Ouputs: Ϣʔβ͕࣍ʹڵຯΛ࣋ͪͦ͏ͳΞΠςϜ

20. ©2020 Wantedly, Inc. Item Relations ΞΠςϜ͸ೖΕସΘΓͷස౓͕ߴ͍, ·ͨϢʔβ͔Βߪങ͞ΕΔΞΠςϜ͸εύʔε. ͦ͜ͰΧςΰϦϨϕϧͷؔ܎ΛݟΔ͜ͱʹ͢Δ. ←ҟͳΔΞΠςϜΛ
 ɹߪೖ͍ͯ͠Δ

    ←ΧςΰϦΛݟΔͱ ɹߪങύλʔϯ͕ಉ͡ ΞΠςϜؒͷؔ࿈ੑ → ΧςΰϦϨϕϧͰݟΔ Ϣʔβͷߦಈܥྻ → ΞΠςϜϨϕϧͰݟΔ

21. ©2020 Wantedly, Inc. Proposed Method: HierTrans ֊૚ܕͷ࣌ؒάϥϑΛߏங ɾิ׬ؔ܎, ΋͘͠͸୅ସؔ܎Ͱ઀ଓ͞ΕΔΧςΰϦϨϕϧͷάϥϑ ɾϢʔβͷߦಈܥྻͷάϥϑ

22. ©2020 Wantedly, Inc. Recommendation with HierTrans ←Ϣʔβͷߦಈܥྻ T࣌఺·ͰͷܥྻΛ΋ͱʹ ֤αϒάϥϑ͔Β৘ใΛऔಘ (Head͸GIͱGCΛ


    ݁߹ͤ͞ΔͨΊʹؔ਺) T࣌఺·Ͱͷܥྻͱ
 ΞΠςϜͷؔ܎ੑΛ࢖ͬͯ, ϢʔβͷTranslation vectorΛ ٻΊΔ

23. ©2020 Wantedly, Inc. Experiments BPR: Bayesian personalized ranking TransE, TransFM,

    TransRec: Translation-based Methods ….

24. INTERNAL ONLY ©2020 Wantedly, Inc. 'JO