Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Web Conference2020 参加報告会

The Web Conference2020 参加報告会

Shuhei Goda

April 30, 2020
Tweet

More Decks by Shuhei Goda

Other Decks in Technology

Transcript

  1. ©2020 Wantedly, Inc.
    ࿦จɾηογϣϯ঺հ
    The Web Conference2020 ࢀՃใࠂձ
    Apr 30, 2020 - Shuhei Goda - @jy_msc

    View Slide

  2. ©2020 Wantedly, Inc.
    ࣗݾ঺հ
    Shuhei Goda
    Twitter: hakubishin3 (@jy_msc)
    Visit Engineering Team
    Kaggle Master

    View Slide

  3. ©2020 Wantedly, Inc.
    1. Deep Transfer Learning for Search and Recommendation
    ɾݕࡧɾਪનγεςϜʹ͓͚ΔసҠֶशͷԠ༻ͱLinkedinͰͷࣄྫΛ঺հ͍ͯ͠Δ (Tutorial)
    2. Clustering and Constructing User Coresets to Accelerate Large-
    scale Top-K Recommender Systems
    ɾਪનର৅ͷϢʔβΛᅂ޷ͷ͍ۙ΋ͷಉ࢜ͰΫϥελϦϯά͠, ֤Ϋϥελͷ୅දϕΫτϧΛ

    ɹ ࢖ͬͯۙࣅ࠷ۙ๣୳ࡧΛ͢Δ͜ͱͰਪનγεςϜͷεϐʔυΞοϓΛ࣮ݱ
    3. Adaptive Hierarchical Translation-based Sequential Recommendation
    ɾϢʔβߦಈͷ࣌ܥྻΛߟྀͨ͠Translation-based Reccommendationʹ, ΞΠςϜؒͷ

    ɹ ؔ܎ੑͱ࣌ؒґଘͷuser translation vectorΛՃ͑ͨख๏ΛఏҊ
    ຊൃදͰ঺հ͢Δ࿦จɾηογϣϯ

    View Slide

  4. INTERNAL ONLY
    ©2020 Wantedly, Inc.
    %FFQ5SBOTGFS-FBSOJOHGPS4FBSDIBOE3FDPNNFOEBUJPO
    IUUQTTJUFTHPPHMFDPNWJFXXXXEUMUVUPSJBMIPNF

    View Slide

  5. ©2020 Wantedly, Inc.
    Motivation
    ݕࡧɾਪનγεςϜͰసҠֶशΛ࢖͏ཧ༝͸ʁ
    ɾαʔϏε಺Ͱ܇࿅σʔλΛ͋·ΓऔಘͰ͖ͳ͍ঢ়گԼͰ΋ੑೳΛߴΊ͍ͨ
    ɹɾྫ͑͹, αʔϏεΛ্ཱͪ͛ͨ௚ޙ΍ར༻Ϣʔβ͕গͳ͍Α͏ͳ৔߹

    View Slide

  6. ©2020 Wantedly, Inc.
    Deep Transfer Learning
    1. Model Transfer
    ɹɾsource domainsͰͷֶशࡁΈϞσϧΛtarget domainͰ࢖͏
    ɹɾfine-tuning (e.g. BERT) ΍ ֤domainͰͷtaskͷmulti-task learning
    2. Feature Representation Transfer
    ɹɾҟͳΔdomainؒͰڞ༗Մೳͳಛ௃Λֶश
    ɹɾsource domainͰͷֶशࡁΈϞσϧ͔Βͷಛ௃நग़ ΍ domain adaptation
    3. Instance Transfer
    ɹɾsource domain͔Β܇࿅σʔλΛ࣋ͬͯ͘Δ

    View Slide

  7. ©2020 Wantedly, Inc.
    Deep Transfer Learning for Search and Recommendation
    ݕࡧɾਪનγεςϜͷͲͷ෦෼ͰసҠֶशΛ࢖͏͔
    Raw Query
    Query
    Understanding
    User Profile,
    Activities
    Raw Document
    User
    Understanding
    Document
    Understanding
    Document
    Retrieval
    Ranking
    ←ΑΓྑ͍ཧղͷͨΊʹ, pretrained model

    ͔ΒຒΊࠐΈදݱΛྲྀ༻ (Feature Transfer)
    ←ຒΊࠐΈۭؒ಺ͰͷީิΞΠςϜͷ֦ு
    (Feature Transfer)
    ←ؔ࿈͢ΔαʔϏεͷσʔλ΍ϞσϧΛ

    ྲྀ༻͢Δ(Model Transfer & Instance Transfer)

    View Slide

  8. ©2020 Wantedly, Inc.
    Case study in Linkedin (1)
    LinkedinͰ͸ෳ਺ͷϓϩμΫτʹ͓͚ΔϢʔβͷߦಈཤྺ͕औಘͰ͖Δ
    Job Search People Search
    ଞϢʔβ΁ͷ

    ܨ͕ΓϦΫΤετ
    ืू΁ͷԠื
    https://www.linkedin.com/

    View Slide

  9. ©2020 Wantedly, Inc.
    Case study in Linkedin (2)
    People Search + Careers Job Recommendation
    ɾ֤υϝΠϯͷਪનλεΫΛղ͍ͯuser, itemͷදݱΛֶश͠, ಛ௃ྔͱͯ͠ར༻
    Add embeddings trained with...
    AUC Lift
    NDCG@25
    Lift
    Tree Model Baseline

    (no embedding added)
    0.00% 0.00%
    + People Search Embedding
    (Single-task) 1.25% 0.85%
    + Multi-task Embedding 1.82% 1.50%

    View Slide

  10. INTERNAL ONLY
    ©2020 Wantedly, Inc.
    $MVTUFSJOHBOE$POTUSVDUJOH6TFS$PSFTFUTUP"DDFMFSBUF

    -BSHFTDBMF5PQ,3FDPNNFOEFS4ZTUFNT
    +ZVO:V+JBOH 1BUSJDL)$IFO $IP+VJ)TJFI 8FJ8BOH
    IUUQTEMBDNPSHEPJ

    View Slide

  11. ©2020 Wantedly, Inc.
    Motivation
    ྨࣅΞΠςϜͷ୳ࡧͷߴ଎Խ
    ɾϢʔβʹਪન͢ΔΞΠςϜΛ୳͢ࡍ, PredictionʹؔΘΔ࣮ߦ࣌ؒΛશମతʹ୹͍ͨ͘͠
    ɹɾPrediction time = Preparation time + Inference time
    ɹɾطଘͷ୳ࡧΞϧΰϦζϜͰ͸Inferece timeͷ୹ॖʹয఺Λ౰͍ͯͯΔ

    ɾطଘͷख๏Ͱ͸Ϣʔβؒͷؔ܎ੑΛߟྀ͍ͯ͠ͳ͍ͷͰ, ߴ଎Խʹ׆༻͍ͨ͠
    ɹɾ࠷ۙ๣୳ࡧ͸೚ҙͷΫΤϦϕΫτϧ͕༩͑ΒΕͨ࣌ʹ͍ۙϕΫτϧΛ୳͢໰୊͕ͩ, 

    ɹɹਪનγεςϜͰ͸ΫΤϦϕΫτϧ͸ϢʔβϕΫτϧͰ΋͋Γ, ϢʔβϕΫτϧ͸ڧ͍

    ɹɹΫϥελϦϯάߏ଄Λ͍࣋ͬͯΔ
    ɹɾ͍ۙϢʔβ͸ࣅͨؔ৺Λ͍࣋ͬͯΔͨΊ, ୳ࡧ࣌ʹ·ͱΊͯ͠·͑͹ޮ཰తͰ͸?

    View Slide

  12. ©2020 Wantedly, Inc.
    Framework Overview
    Preparation Stage:
    1. αϒαϯϓϧͨ͠ϢʔβΛΫϥελϦϯά
    2. άϧʔϓ಺ͷϢʔβΛΧόʔͰ͖ΔΑ͏ͳ

    ෳ਺ͷ୅දϕΫτϧΛϢʔβάϧʔϓຖʹٻΊΔ
    3. ΞΠςϜͷ֊૚ܕάϥϑߏ଄Λߏங͠,
    ɹ֤Ϣʔβάϧʔϓͷ୅දϕΫτϧΛ༻͍ͨ

    ɹgreedyͳάϥϑ୳ࡧʹΑΓ, άϧʔϓຖʹ

    ɹީิΞΠςϜू߹Λऔಘ
    Prediction Stage:
    4. ༧ଌର৅Ϣʔβͷॴଐ͢ΔάϧʔϓΛ୳͠,
    ɹͦͷάϧʔϓʹରԠ͢ΔީิΞΠςϜू߹ͱ
    ɹϢʔβϕΫτϧͷ಺ੵΛͱͬͯϥϯΫ෇͚

    View Slide

  13. ©2020 Wantedly, Inc.
    User Group Clutering
    ࣅͨΑ͏ͳڵຯΛ࣋ͭϢʔβͷू߹Λ࡞Δ
    ɾϢʔβΛࢦఆͷΫϥελ਺ͰΫϥελϦϯά
    ɾ൓෮ܭࢉΛߦ͏ͷͰ, ࣄલʹϢʔβΛαϒαϯϓϦϯά
    z͸, Ϣʔβͷॴଐ͢ΔάϧʔϓΛࣔ͢ࢦࣔؔ਺
    p_i͕ϢʔβiͷϕΫτϧͰ, v_r ͸άϧʔϓrͷॏ৺ϕΫτϧ

    View Slide

  14. ©2020 Wantedly, Inc.
    Coreset Vectors
    ϢʔβάϧʔϓͷڵຯΛΧόʔͰ͖ΔΑ͏ͳ୅දϕΫτϧΛऔಘ
    ɾॏ৺ϕΫτϧͩͱ໢ཏͰ͖ͳ͍ͷͰ, ΫϥελͷൣғʹԠͨ͡਺ͷ୅දϕΫτϧ͕ඞཁ
    ɾఆٛ4Λຬͨ͢Α͏ʹ୅දϕΫτϧΛऔಘ

    View Slide

  15. ©2020 Wantedly, Inc.
    Experiments (1)
    Ϟσϧ͸NMFΛ࢖༻, Ϋϥελ਺͸8
    SU͸O(nm)ͷφΠʔϒͳΞϓϩʔν(SVDS)͔Βͷશମ଎౓ͷվળ཰ (n͸userͷ਺, m͸itemͷ਺)
    PT͸Preparation time, IT͸Inference timeͷ͜ͱ
    ਫ਼౓ྼԽΛ཈͑ͭͭPrediction Time͕୹ॖ͞Ε͍ͯΔ

    View Slide

  16. ©2020 Wantedly, Inc.
    Experiments (2)
    ਫ਼౓ͱ଎౓ͷཁٻ࢓༷ʹԠͯ͡, Ϋϥελ਺Λௐ੔͢Δ

    View Slide

  17. INTERNAL ONLY
    ©2020 Wantedly, Inc.
    "EBQUJWF)JFSBSDIJDBM5SBOTMBUJPOCBTFE4FRVFOUJBM
    3FDPNNFOEBUJPO
    :JO;IBOH :VO)F +JBOMJOH8BOH +BNFT$BWFSMFF
    IUUQTEMBDNPSHEPJBCT

    View Slide

  18. ©2020 Wantedly, Inc.
    Motivation
    ΞΠςϜؒͷؔ࿈ੑͱϢʔβͷߦಈܥྻΛߟྀͨ͠ਪનΛ͍ͨ͠
    ɾϢʔβͷߦಈܥྻʹج͍ͮͨਪન (Sequential recommendation)
    ɹɾe.g. RNN, Ϛϧίϑ࿈࠯, CNN, Translated-based Recommendation
    ɾϢʔβͷߪങܥྻͷཧղʹ͸ΞΠςϜؒͷ૬ิؔ܎ͱ୅ସؔ܎͕ॏཁ
    ɹɾPCΛߪೖͨ͠௚ޙʹผͷPCΛങ͏͜ͱ͸ͳ͍͕, ͕࣌ؒܦͬͨΒങ͍׵͑ͨ͘ͳΔ

    View Slide

  19. ©2020 Wantedly, Inc.
    Model Overview
    Inputs:
    User Sequences Item Relations
    ิ׬ؔ܎
    ୅ସؔ܎
    Ouputs:
    Ϣʔβ͕࣍ʹڵຯΛ࣋ͪͦ͏ͳΞΠςϜ

    View Slide

  20. ©2020 Wantedly, Inc.
    Item Relations
    ΞΠςϜ͸ೖΕସΘΓͷස౓͕ߴ͍, ·ͨϢʔβ͔Βߪങ͞ΕΔΞΠςϜ͸εύʔε.
    ͦ͜ͰΧςΰϦϨϕϧͷؔ܎ΛݟΔ͜ͱʹ͢Δ.
    ←ҟͳΔΞΠςϜΛ

    ɹߪೖ͍ͯ͠Δ
    ←ΧςΰϦΛݟΔͱ
    ɹߪങύλʔϯ͕ಉ͡
    ΞΠςϜؒͷؔ࿈ੑ → ΧςΰϦϨϕϧͰݟΔ
    Ϣʔβͷߦಈܥྻ → ΞΠςϜϨϕϧͰݟΔ

    View Slide

  21. ©2020 Wantedly, Inc.
    Proposed Method: HierTrans
    ֊૚ܕͷ࣌ؒάϥϑΛߏங
    ɾิ׬ؔ܎, ΋͘͠͸୅ସؔ܎Ͱ઀ଓ͞ΕΔΧςΰϦϨϕϧͷάϥϑ
    ɾϢʔβͷߦಈܥྻͷάϥϑ

    View Slide

  22. ©2020 Wantedly, Inc.
    Recommendation with HierTrans
    ←Ϣʔβͷߦಈܥྻ
    T࣌఺·ͰͷܥྻΛ΋ͱʹ
    ֤αϒάϥϑ͔Β৘ใΛऔಘ
    (Head͸GIͱGCΛ

    ݁߹ͤ͞ΔͨΊʹؔ਺)
    T࣌఺·Ͱͷܥྻͱ

    ΞΠςϜͷؔ܎ੑΛ࢖ͬͯ,
    ϢʔβͷTranslation vectorΛ
    ٻΊΔ

    View Slide

  23. ©2020 Wantedly, Inc.
    Experiments
    BPR: Bayesian personalized ranking
    TransE, TransFM, TransRec: Translation-based Methods
    ….

    View Slide

  24. INTERNAL ONLY
    ©2020 Wantedly, Inc.
    'JO

    View Slide