Slide 1

Slide 1 text

©2020 Wantedly, Inc. ࿦จɾηογϣϯ঺հ The Web Conference2020 ࢀՃใࠂձ Apr 30, 2020 - Shuhei Goda - @jy_msc

Slide 2

Slide 2 text

©2020 Wantedly, Inc. ࣗݾ঺հ Shuhei Goda Twitter: hakubishin3 (@jy_msc) Visit Engineering Team Kaggle Master

Slide 3

Slide 3 text

©2020 Wantedly, Inc. 1. Deep Transfer Learning for Search and Recommendation ɾݕࡧɾਪનγεςϜʹ͓͚ΔసҠֶशͷԠ༻ͱLinkedinͰͷࣄྫΛ঺հ͍ͯ͠Δ (Tutorial) 2. Clustering and Constructing User Coresets to Accelerate Large- scale Top-K Recommender Systems ɾਪનର৅ͷϢʔβΛᅂ޷ͷ͍ۙ΋ͷಉ࢜ͰΫϥελϦϯά͠, ֤Ϋϥελͷ୅දϕΫτϧΛ
 ɹ ࢖ͬͯۙࣅ࠷ۙ๣୳ࡧΛ͢Δ͜ͱͰਪનγεςϜͷεϐʔυΞοϓΛ࣮ݱ 3. Adaptive Hierarchical Translation-based Sequential Recommendation ɾϢʔβߦಈͷ࣌ܥྻΛߟྀͨ͠Translation-based Reccommendationʹ, ΞΠςϜؒͷ
 ɹ ؔ܎ੑͱ࣌ؒґଘͷuser translation vectorΛՃ͑ͨख๏ΛఏҊ ຊൃදͰ঺հ͢Δ࿦จɾηογϣϯ

Slide 4

Slide 4 text

INTERNAL ONLY ©2020 Wantedly, Inc. %FFQ5SBOTGFS-FBSOJOHGPS4FBSDIBOE3FDPNNFOEBUJPO IUUQTTJUFTHPPHMFDPNWJFXXXXEUMUVUPSJBMIPNF

Slide 5

Slide 5 text

©2020 Wantedly, Inc. Motivation ݕࡧɾਪનγεςϜͰసҠֶशΛ࢖͏ཧ༝͸ʁ ɾαʔϏε಺Ͱ܇࿅σʔλΛ͋·ΓऔಘͰ͖ͳ͍ঢ়گԼͰ΋ੑೳΛߴΊ͍ͨ ɹɾྫ͑͹, αʔϏεΛ্ཱͪ͛ͨ௚ޙ΍ར༻Ϣʔβ͕গͳ͍Α͏ͳ৔߹

Slide 6

Slide 6 text

©2020 Wantedly, Inc. Deep Transfer Learning 1. Model Transfer ɹɾsource domainsͰͷֶशࡁΈϞσϧΛtarget domainͰ࢖͏ ɹɾfine-tuning (e.g. BERT) ΍ ֤domainͰͷtaskͷmulti-task learning 2. Feature Representation Transfer ɹɾҟͳΔdomainؒͰڞ༗Մೳͳಛ௃Λֶश ɹɾsource domainͰͷֶशࡁΈϞσϧ͔Βͷಛ௃நग़ ΍ domain adaptation 3. Instance Transfer ɹɾsource domain͔Β܇࿅σʔλΛ࣋ͬͯ͘Δ

Slide 7

Slide 7 text

©2020 Wantedly, Inc. Deep Transfer Learning for Search and Recommendation ݕࡧɾਪનγεςϜͷͲͷ෦෼ͰసҠֶशΛ࢖͏͔ Raw Query Query Understanding User Profile, Activities Raw Document User Understanding Document Understanding Document Retrieval Ranking ←ΑΓྑ͍ཧղͷͨΊʹ, pretrained model
 ͔ΒຒΊࠐΈදݱΛྲྀ༻ (Feature Transfer) ←ຒΊࠐΈۭؒ಺ͰͷީิΞΠςϜͷ֦ு (Feature Transfer) ←ؔ࿈͢ΔαʔϏεͷσʔλ΍ϞσϧΛ
 ྲྀ༻͢Δ(Model Transfer & Instance Transfer)

Slide 8

Slide 8 text

©2020 Wantedly, Inc. Case study in Linkedin (1) LinkedinͰ͸ෳ਺ͷϓϩμΫτʹ͓͚ΔϢʔβͷߦಈཤྺ͕औಘͰ͖Δ Job Search People Search ଞϢʔβ΁ͷ
 ܨ͕ΓϦΫΤετ ืू΁ͷԠื https://www.linkedin.com/

Slide 9

Slide 9 text

©2020 Wantedly, Inc. Case study in Linkedin (2) People Search + Careers Job Recommendation ɾ֤υϝΠϯͷਪનλεΫΛղ͍ͯuser, itemͷදݱΛֶश͠, ಛ௃ྔͱͯ͠ར༻ Add embeddings trained with... AUC Lift NDCG@25 Lift Tree Model Baseline
 (no embedding added) 0.00% 0.00% + People Search Embedding (Single-task) 1.25% 0.85% + Multi-task Embedding 1.82% 1.50%

Slide 10

Slide 10 text

INTERNAL ONLY ©2020 Wantedly, Inc. $MVTUFSJOHBOE$POTUSVDUJOH6TFS$PSFTFUTUP"DDFMFSBUF
 -BSHFTDBMF5PQ,3FDPNNFOEFS4ZTUFNT +ZVO:V+JBOH 1BUSJDL)$IFO $IP+VJ)TJFI 8FJ8BOH IUUQTEMBDNPSHEPJ

Slide 11

Slide 11 text

©2020 Wantedly, Inc. Motivation ྨࣅΞΠςϜͷ୳ࡧͷߴ଎Խ ɾϢʔβʹਪન͢ΔΞΠςϜΛ୳͢ࡍ, PredictionʹؔΘΔ࣮ߦ࣌ؒΛશମతʹ୹͍ͨ͘͠ ɹɾPrediction time = Preparation time + Inference time ɹɾطଘͷ୳ࡧΞϧΰϦζϜͰ͸Inferece timeͷ୹ॖʹয఺Λ౰͍ͯͯΔ
 ɾطଘͷख๏Ͱ͸Ϣʔβؒͷؔ܎ੑΛߟྀ͍ͯ͠ͳ͍ͷͰ, ߴ଎Խʹ׆༻͍ͨ͠ ɹɾ࠷ۙ๣୳ࡧ͸೚ҙͷΫΤϦϕΫτϧ͕༩͑ΒΕͨ࣌ʹ͍ۙϕΫτϧΛ୳͢໰୊͕ͩ, 
 ɹɹਪનγεςϜͰ͸ΫΤϦϕΫτϧ͸ϢʔβϕΫτϧͰ΋͋Γ, ϢʔβϕΫτϧ͸ڧ͍
 ɹɹΫϥελϦϯάߏ଄Λ͍࣋ͬͯΔ ɹɾ͍ۙϢʔβ͸ࣅͨؔ৺Λ͍࣋ͬͯΔͨΊ, ୳ࡧ࣌ʹ·ͱΊͯ͠·͑͹ޮ཰తͰ͸?

Slide 12

Slide 12 text

©2020 Wantedly, Inc. Framework Overview Preparation Stage: 1. αϒαϯϓϧͨ͠ϢʔβΛΫϥελϦϯά 2. άϧʔϓ಺ͷϢʔβΛΧόʔͰ͖ΔΑ͏ͳ
 ෳ਺ͷ୅දϕΫτϧΛϢʔβάϧʔϓຖʹٻΊΔ 3. ΞΠςϜͷ֊૚ܕάϥϑߏ଄Λߏங͠, ɹ֤Ϣʔβάϧʔϓͷ୅දϕΫτϧΛ༻͍ͨ
 ɹgreedyͳάϥϑ୳ࡧʹΑΓ, άϧʔϓຖʹ
 ɹީิΞΠςϜू߹Λऔಘ Prediction Stage: 4. ༧ଌର৅Ϣʔβͷॴଐ͢ΔάϧʔϓΛ୳͠, ɹͦͷάϧʔϓʹରԠ͢ΔީิΞΠςϜू߹ͱ ɹϢʔβϕΫτϧͷ಺ੵΛͱͬͯϥϯΫ෇͚

Slide 13

Slide 13 text

©2020 Wantedly, Inc. User Group Clutering ࣅͨΑ͏ͳڵຯΛ࣋ͭϢʔβͷू߹Λ࡞Δ ɾϢʔβΛࢦఆͷΫϥελ਺ͰΫϥελϦϯά ɾ൓෮ܭࢉΛߦ͏ͷͰ, ࣄલʹϢʔβΛαϒαϯϓϦϯά z͸, Ϣʔβͷॴଐ͢ΔάϧʔϓΛࣔ͢ࢦࣔؔ਺ p_i͕ϢʔβiͷϕΫτϧͰ, v_r ͸άϧʔϓrͷॏ৺ϕΫτϧ

Slide 14

Slide 14 text

©2020 Wantedly, Inc. Coreset Vectors ϢʔβάϧʔϓͷڵຯΛΧόʔͰ͖ΔΑ͏ͳ୅දϕΫτϧΛऔಘ ɾॏ৺ϕΫτϧͩͱ໢ཏͰ͖ͳ͍ͷͰ, ΫϥελͷൣғʹԠͨ͡਺ͷ୅දϕΫτϧ͕ඞཁ ɾఆٛ4Λຬͨ͢Α͏ʹ୅දϕΫτϧΛऔಘ

Slide 15

Slide 15 text

©2020 Wantedly, Inc. Experiments (1) Ϟσϧ͸NMFΛ࢖༻, Ϋϥελ਺͸8 SU͸O(nm)ͷφΠʔϒͳΞϓϩʔν(SVDS)͔Βͷશମ଎౓ͷվળ཰ (n͸userͷ਺, m͸itemͷ਺) PT͸Preparation time, IT͸Inference timeͷ͜ͱ ਫ਼౓ྼԽΛ཈͑ͭͭPrediction Time͕୹ॖ͞Ε͍ͯΔ

Slide 16

Slide 16 text

©2020 Wantedly, Inc. Experiments (2) ਫ਼౓ͱ଎౓ͷཁٻ࢓༷ʹԠͯ͡, Ϋϥελ਺Λௐ੔͢Δ

Slide 17

Slide 17 text

INTERNAL ONLY ©2020 Wantedly, Inc. "EBQUJWF)JFSBSDIJDBM5SBOTMBUJPOCBTFE4FRVFOUJBM 3FDPNNFOEBUJPO :JO;IBOH :VO)F +JBOMJOH8BOH +BNFT$BWFSMFF IUUQTEMBDNPSHEPJBCT

Slide 18

Slide 18 text

©2020 Wantedly, Inc. Motivation ΞΠςϜؒͷؔ࿈ੑͱϢʔβͷߦಈܥྻΛߟྀͨ͠ਪનΛ͍ͨ͠ ɾϢʔβͷߦಈܥྻʹج͍ͮͨਪન (Sequential recommendation) ɹɾe.g. RNN, Ϛϧίϑ࿈࠯, CNN, Translated-based Recommendation ɾϢʔβͷߪങܥྻͷཧղʹ͸ΞΠςϜؒͷ૬ิؔ܎ͱ୅ସؔ܎͕ॏཁ ɹɾPCΛߪೖͨ͠௚ޙʹผͷPCΛങ͏͜ͱ͸ͳ͍͕, ͕࣌ؒܦͬͨΒങ͍׵͑ͨ͘ͳΔ

Slide 19

Slide 19 text

©2020 Wantedly, Inc. Model Overview Inputs: User Sequences Item Relations ิ׬ؔ܎ ୅ସؔ܎ Ouputs: Ϣʔβ͕࣍ʹڵຯΛ࣋ͪͦ͏ͳΞΠςϜ

Slide 20

Slide 20 text

©2020 Wantedly, Inc. Item Relations ΞΠςϜ͸ೖΕସΘΓͷස౓͕ߴ͍, ·ͨϢʔβ͔Βߪങ͞ΕΔΞΠςϜ͸εύʔε. ͦ͜ͰΧςΰϦϨϕϧͷؔ܎ΛݟΔ͜ͱʹ͢Δ. ←ҟͳΔΞΠςϜΛ
 ɹߪೖ͍ͯ͠Δ ←ΧςΰϦΛݟΔͱ ɹߪങύλʔϯ͕ಉ͡ ΞΠςϜؒͷؔ࿈ੑ → ΧςΰϦϨϕϧͰݟΔ Ϣʔβͷߦಈܥྻ → ΞΠςϜϨϕϧͰݟΔ

Slide 21

Slide 21 text

©2020 Wantedly, Inc. Proposed Method: HierTrans ֊૚ܕͷ࣌ؒάϥϑΛߏங ɾิ׬ؔ܎, ΋͘͠͸୅ସؔ܎Ͱ઀ଓ͞ΕΔΧςΰϦϨϕϧͷάϥϑ ɾϢʔβͷߦಈܥྻͷάϥϑ

Slide 22

Slide 22 text

©2020 Wantedly, Inc. Recommendation with HierTrans ←Ϣʔβͷߦಈܥྻ T࣌఺·ͰͷܥྻΛ΋ͱʹ ֤αϒάϥϑ͔Β৘ใΛऔಘ (Head͸GIͱGCΛ
 ݁߹ͤ͞ΔͨΊʹؔ਺) T࣌఺·Ͱͷܥྻͱ
 ΞΠςϜͷؔ܎ੑΛ࢖ͬͯ, ϢʔβͷTranslation vectorΛ ٻΊΔ

Slide 23

Slide 23 text

©2020 Wantedly, Inc. Experiments BPR: Bayesian personalized ranking TransE, TransFM, TransRec: Translation-based Methods ….

Slide 24

Slide 24 text

INTERNAL ONLY ©2020 Wantedly, Inc. 'JO