The Web Conference2020 参加報告会

The Web Conference2020 参加報告会

99e9e6d2de62c373990ac1bd7c4defc5?s=128

Shuhei Goda

April 30, 2020
Tweet

Transcript

  1. ©2020 Wantedly, Inc. ࿦จɾηογϣϯ঺հ The Web Conference2020 ࢀՃใࠂձ Apr 30,

    2020 - Shuhei Goda - @jy_msc
  2. ©2020 Wantedly, Inc. ࣗݾ঺հ Shuhei Goda Twitter: hakubishin3 (@jy_msc) Visit

    Engineering Team Kaggle Master
  3. ©2020 Wantedly, Inc. 1. Deep Transfer Learning for Search and

    Recommendation ɾݕࡧɾਪનγεςϜʹ͓͚ΔసҠֶशͷԠ༻ͱLinkedinͰͷࣄྫΛ঺հ͍ͯ͠Δ (Tutorial) 2. Clustering and Constructing User Coresets to Accelerate Large- scale Top-K Recommender Systems ɾਪનର৅ͷϢʔβΛᅂ޷ͷ͍ۙ΋ͷಉ࢜ͰΫϥελϦϯά͠, ֤Ϋϥελͷ୅දϕΫτϧΛ
 ɹ ࢖ͬͯۙࣅ࠷ۙ๣୳ࡧΛ͢Δ͜ͱͰਪનγεςϜͷεϐʔυΞοϓΛ࣮ݱ 3. Adaptive Hierarchical Translation-based Sequential Recommendation ɾϢʔβߦಈͷ࣌ܥྻΛߟྀͨ͠Translation-based Reccommendationʹ, ΞΠςϜؒͷ
 ɹ ؔ܎ੑͱ࣌ؒґଘͷuser translation vectorΛՃ͑ͨख๏ΛఏҊ ຊൃදͰ঺հ͢Δ࿦จɾηογϣϯ
  4. INTERNAL ONLY ©2020 Wantedly, Inc. %FFQ5SBOTGFS-FBSOJOHGPS4FBSDIBOE3FDPNNFOEBUJPO IUUQTTJUFTHPPHMFDPNWJFXXXXEUMUVUPSJBMIPNF

  5. ©2020 Wantedly, Inc. Motivation ݕࡧɾਪનγεςϜͰసҠֶशΛ࢖͏ཧ༝͸ʁ ɾαʔϏε಺Ͱ܇࿅σʔλΛ͋·ΓऔಘͰ͖ͳ͍ঢ়گԼͰ΋ੑೳΛߴΊ͍ͨ ɹɾྫ͑͹, αʔϏεΛ্ཱͪ͛ͨ௚ޙ΍ར༻Ϣʔβ͕গͳ͍Α͏ͳ৔߹

  6. ©2020 Wantedly, Inc. Deep Transfer Learning 1. Model Transfer ɹɾsource

    domainsͰͷֶशࡁΈϞσϧΛtarget domainͰ࢖͏ ɹɾfine-tuning (e.g. BERT) ΍ ֤domainͰͷtaskͷmulti-task learning 2. Feature Representation Transfer ɹɾҟͳΔdomainؒͰڞ༗Մೳͳಛ௃Λֶश ɹɾsource domainͰͷֶशࡁΈϞσϧ͔Βͷಛ௃நग़ ΍ domain adaptation 3. Instance Transfer ɹɾsource domain͔Β܇࿅σʔλΛ࣋ͬͯ͘Δ
  7. ©2020 Wantedly, Inc. Deep Transfer Learning for Search and Recommendation

    ݕࡧɾਪનγεςϜͷͲͷ෦෼ͰసҠֶशΛ࢖͏͔ Raw Query Query Understanding User Profile, Activities Raw Document User Understanding Document Understanding Document Retrieval Ranking ←ΑΓྑ͍ཧղͷͨΊʹ, pretrained model
 ͔ΒຒΊࠐΈදݱΛྲྀ༻ (Feature Transfer) ←ຒΊࠐΈۭؒ಺ͰͷީิΞΠςϜͷ֦ு (Feature Transfer) ←ؔ࿈͢ΔαʔϏεͷσʔλ΍ϞσϧΛ
 ྲྀ༻͢Δ(Model Transfer & Instance Transfer)
  8. ©2020 Wantedly, Inc. Case study in Linkedin (1) LinkedinͰ͸ෳ਺ͷϓϩμΫτʹ͓͚ΔϢʔβͷߦಈཤྺ͕औಘͰ͖Δ Job

    Search People Search ଞϢʔβ΁ͷ
 ܨ͕ΓϦΫΤετ ืू΁ͷԠื https://www.linkedin.com/
  9. ©2020 Wantedly, Inc. Case study in Linkedin (2) People Search

    + Careers Job Recommendation ɾ֤υϝΠϯͷਪનλεΫΛղ͍ͯuser, itemͷදݱΛֶश͠, ಛ௃ྔͱͯ͠ར༻ Add embeddings trained with... AUC Lift NDCG@25 Lift Tree Model Baseline
 (no embedding added) 0.00% 0.00% + People Search Embedding (Single-task) 1.25% 0.85% + Multi-task Embedding 1.82% 1.50%
  10. INTERNAL ONLY ©2020 Wantedly, Inc. $MVTUFSJOHBOE$POTUSVDUJOH6TFS$PSFTFUTUP"DDFMFSBUF
 -BSHFTDBMF5PQ,3FDPNNFOEFS4ZTUFNT +ZVO:V+JBOH 1BUSJDL)$IFO $IP+VJ)TJFI

    8FJ8BOH  IUUQTEMBDNPSHEPJ
  11. ©2020 Wantedly, Inc. Motivation ྨࣅΞΠςϜͷ୳ࡧͷߴ଎Խ ɾϢʔβʹਪન͢ΔΞΠςϜΛ୳͢ࡍ, PredictionʹؔΘΔ࣮ߦ࣌ؒΛશମతʹ୹͍ͨ͘͠ ɹɾPrediction time =

    Preparation time + Inference time ɹɾطଘͷ୳ࡧΞϧΰϦζϜͰ͸Inferece timeͷ୹ॖʹয఺Λ౰͍ͯͯΔ
 ɾطଘͷख๏Ͱ͸Ϣʔβؒͷؔ܎ੑΛߟྀ͍ͯ͠ͳ͍ͷͰ, ߴ଎Խʹ׆༻͍ͨ͠ ɹɾ࠷ۙ๣୳ࡧ͸೚ҙͷΫΤϦϕΫτϧ͕༩͑ΒΕͨ࣌ʹ͍ۙϕΫτϧΛ୳͢໰୊͕ͩ, 
 ɹɹਪનγεςϜͰ͸ΫΤϦϕΫτϧ͸ϢʔβϕΫτϧͰ΋͋Γ, ϢʔβϕΫτϧ͸ڧ͍
 ɹɹΫϥελϦϯάߏ଄Λ͍࣋ͬͯΔ ɹɾ͍ۙϢʔβ͸ࣅͨؔ৺Λ͍࣋ͬͯΔͨΊ, ୳ࡧ࣌ʹ·ͱΊͯ͠·͑͹ޮ཰తͰ͸?
  12. ©2020 Wantedly, Inc. Framework Overview Preparation Stage: 1. αϒαϯϓϧͨ͠ϢʔβΛΫϥελϦϯά 2.

    άϧʔϓ಺ͷϢʔβΛΧόʔͰ͖ΔΑ͏ͳ
 ෳ਺ͷ୅දϕΫτϧΛϢʔβάϧʔϓຖʹٻΊΔ 3. ΞΠςϜͷ֊૚ܕάϥϑߏ଄Λߏங͠, ɹ֤Ϣʔβάϧʔϓͷ୅දϕΫτϧΛ༻͍ͨ
 ɹgreedyͳάϥϑ୳ࡧʹΑΓ, άϧʔϓຖʹ
 ɹީิΞΠςϜू߹Λऔಘ Prediction Stage: 4. ༧ଌର৅Ϣʔβͷॴଐ͢ΔάϧʔϓΛ୳͠, ɹͦͷάϧʔϓʹରԠ͢ΔީิΞΠςϜू߹ͱ ɹϢʔβϕΫτϧͷ಺ੵΛͱͬͯϥϯΫ෇͚
  13. ©2020 Wantedly, Inc. User Group Clutering ࣅͨΑ͏ͳڵຯΛ࣋ͭϢʔβͷू߹Λ࡞Δ ɾϢʔβΛࢦఆͷΫϥελ਺ͰΫϥελϦϯά ɾ൓෮ܭࢉΛߦ͏ͷͰ, ࣄલʹϢʔβΛαϒαϯϓϦϯά

    z͸, Ϣʔβͷॴଐ͢ΔάϧʔϓΛࣔ͢ࢦࣔؔ਺ p_i͕ϢʔβiͷϕΫτϧͰ, v_r ͸άϧʔϓrͷॏ৺ϕΫτϧ
  14. ©2020 Wantedly, Inc. Coreset Vectors ϢʔβάϧʔϓͷڵຯΛΧόʔͰ͖ΔΑ͏ͳ୅දϕΫτϧΛऔಘ ɾॏ৺ϕΫτϧͩͱ໢ཏͰ͖ͳ͍ͷͰ, ΫϥελͷൣғʹԠͨ͡਺ͷ୅දϕΫτϧ͕ඞཁ ɾఆٛ4Λຬͨ͢Α͏ʹ୅දϕΫτϧΛऔಘ

  15. ©2020 Wantedly, Inc. Experiments (1) Ϟσϧ͸NMFΛ࢖༻, Ϋϥελ਺͸8 SU͸O(nm)ͷφΠʔϒͳΞϓϩʔν(SVDS)͔Βͷશମ଎౓ͷվળ཰ (n͸userͷ਺, m͸itemͷ਺)

    PT͸Preparation time, IT͸Inference timeͷ͜ͱ ਫ਼౓ྼԽΛ཈͑ͭͭPrediction Time͕୹ॖ͞Ε͍ͯΔ
  16. ©2020 Wantedly, Inc. Experiments (2) ਫ਼౓ͱ଎౓ͷཁٻ࢓༷ʹԠͯ͡, Ϋϥελ਺Λௐ੔͢Δ

  17. INTERNAL ONLY ©2020 Wantedly, Inc. "EBQUJWF)JFSBSDIJDBM5SBOTMBUJPOCBTFE4FRVFOUJBM 3FDPNNFOEBUJPO :JO;IBOH :VO)F +JBOMJOH8BOH

    +BNFT$BWFSMFF  IUUQTEMBDNPSHEPJBCT
  18. ©2020 Wantedly, Inc. Motivation ΞΠςϜؒͷؔ࿈ੑͱϢʔβͷߦಈܥྻΛߟྀͨ͠ਪનΛ͍ͨ͠ ɾϢʔβͷߦಈܥྻʹج͍ͮͨਪન (Sequential recommendation) ɹɾe.g. RNN,

    Ϛϧίϑ࿈࠯, CNN, Translated-based Recommendation ɾϢʔβͷߪങܥྻͷཧղʹ͸ΞΠςϜؒͷ૬ิؔ܎ͱ୅ସؔ܎͕ॏཁ ɹɾPCΛߪೖͨ͠௚ޙʹผͷPCΛങ͏͜ͱ͸ͳ͍͕, ͕࣌ؒܦͬͨΒങ͍׵͑ͨ͘ͳΔ
  19. ©2020 Wantedly, Inc. Model Overview Inputs: User Sequences Item Relations

    ิ׬ؔ܎ ୅ସؔ܎ Ouputs: Ϣʔβ͕࣍ʹڵຯΛ࣋ͪͦ͏ͳΞΠςϜ
  20. ©2020 Wantedly, Inc. Item Relations ΞΠςϜ͸ೖΕସΘΓͷස౓͕ߴ͍, ·ͨϢʔβ͔Βߪങ͞ΕΔΞΠςϜ͸εύʔε. ͦ͜ͰΧςΰϦϨϕϧͷؔ܎ΛݟΔ͜ͱʹ͢Δ. ←ҟͳΔΞΠςϜΛ
 ɹߪೖ͍ͯ͠Δ

    ←ΧςΰϦΛݟΔͱ ɹߪങύλʔϯ͕ಉ͡ ΞΠςϜؒͷؔ࿈ੑ → ΧςΰϦϨϕϧͰݟΔ Ϣʔβͷߦಈܥྻ → ΞΠςϜϨϕϧͰݟΔ
  21. ©2020 Wantedly, Inc. Proposed Method: HierTrans ֊૚ܕͷ࣌ؒάϥϑΛߏங ɾิ׬ؔ܎, ΋͘͠͸୅ସؔ܎Ͱ઀ଓ͞ΕΔΧςΰϦϨϕϧͷάϥϑ ɾϢʔβͷߦಈܥྻͷάϥϑ

  22. ©2020 Wantedly, Inc. Recommendation with HierTrans ←Ϣʔβͷߦಈܥྻ T࣌఺·ͰͷܥྻΛ΋ͱʹ ֤αϒάϥϑ͔Β৘ใΛऔಘ (Head͸GIͱGCΛ


    ݁߹ͤ͞ΔͨΊʹؔ਺) T࣌఺·Ͱͷܥྻͱ
 ΞΠςϜͷؔ܎ੑΛ࢖ͬͯ, ϢʔβͷTranslation vectorΛ ٻΊΔ
  23. ©2020 Wantedly, Inc. Experiments BPR: Bayesian personalized ranking TransE, TransFM,

    TransRec: Translation-based Methods ….
  24. INTERNAL ONLY ©2020 Wantedly, Inc. 'JO