Upgrade to Pro — share decks privately, control downloads, hide ads and more …

user-behaviour-vol1

KARAKURI Inc.
November 16, 2021

 user-behaviour-vol1

ユーザー行動予測に関する研究のサーベイ

KARAKURI Inc.

November 16, 2021
Tweet

More Decks by KARAKURI Inc.

Other Decks in Research

Transcript

  1. Ϣʔβʔߦಈ༧ଌʹؔ͢ΔݚڀͷαʔϕΠ
    ߴ໦ࢤ࿠
    1

    View Slide

  2. ͜ͷαʔϕΠͰ΍Δ͜ͱ
    2
    w Ϣʔβʔߦಈܥྻ͔ΒϢʔβʔߦಈ༧ଌʹؔ͢Δݚڀͷ঺հ
    w Ϣʔβʔߦಈ༧ଌͱͦͷࢦඪͷ঺հ
    w Ϣʔβʔߦಈͱͯ͠ͲͷΑ͏ͳ΋ͷ͕࢖ΘΕ͍ͯΔ͔঺հ
    w ϢʔβʔΤϯήʔδϝϯτʹؔ͢Δݚڀͷྫͷ঺հ঺հ
    w Ϣʔβʔ͕͍ͭԿΛ͢Δ͔Λ༧ଌ͢Δݚڀͷྫͷ঺հ
    *Ҿ༻͕ͳ͍΋ͷ͸֘౰࿦จ͔ΒҾ༻

    View Slide

  3. ࠓ೔ͷྲྀΕ
    3
    ̍ɽϢʔβʔߦಈ༧ଌͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ

    View Slide

  4. ࠓ೔ͷྲྀΕ
    4
    ̍ɽϢʔβʔߦಈ༧ଌͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ

    View Slide

  5. ໨త͋Γ͖ͷϢʔβʔߦಈ༧ଌ
    5
    ໨త
    Ϣʔβʔߦಈ

    View Slide

  6. Ͳ͏΍ͬͯଌΔͷ͔ʁ
    6
    ܭଌ
    ʢྫɿΫϦοΫ਺ʣ
    ج४
    ʢྫɿ$53ʣ
    ,1*
    ʢྫɿ$73ʣ
    [Tutorial on Online User Engagement KDD 2020]

    View Slide

  7. ྑ͍ධՁࢦඪͱ͸ʁ
    7
    • SensitivityɿվળΛͪΌΜͱݕ஌Ͱ͖Δ͔ʁ
    • Trustworthinessɿ݁Ռ͸৴པͰ͖Δ͔ʁ
    • Efficiencyɿܭࢉɾܭଌͷίετ͕௿͍͔ʁ
    • Debuggability and ActionabilityɿࢦඪͷมԽͷཧ༝ͷಛఆ͕Մೳ͔ʁ
    • Interpretability and DirectionalityɿࢦඪͷมԽ͸໨తΛୡ੒͢Δ͔ʁ
    [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]

    View Slide

  8. Overall Evaluation Criteria (OEC)
    8
    • ࠷ऴతͳ໨తͱͳΔࢦඪ

    • DirectivityͱSensitivity͕ॏཁ

    • ͜ΕΒ2ͭΛຬͨ͢ͷ͸ࠔ೉

    • ྫɿ1Ϣʔβ͋ͨΓͷฏۉऩӹ
    [Dmitriev+ KDD 2017]
    [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]

    View Slide

  9. ͦͷଞͷࢦඪ
    9
    • Guardrail Metricsɿ੍໿৚݅

    • Operational Metricsɿσόοά༻ͷࢦඪ

    • Data Quality Metricsɿσʔλ͕৴པͰ͖Δ͜ͱΛอূ͢Δࢦඪ
    [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]

    View Slide

  10. ࠓ೔ͷྲྀΕ
    10
    ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ

    View Slide

  11. ΫϦοΫ
    11
    • ඇৗʹΑ͘࢖ΘΕΔϢʔβʔߦಈ
    • ޡΫϦοΫͷՄೳੑ͕͋ΔʢಛʹεϚϗʣ
    • $MJDL%FQUI
    • ϢʔβʔΤϯήʔδϝϯτɹPSɹ໎͍ͬͯΔ
    • $MJDL5ISPVHI3BUF
    • ޿ࠂͷධՁࢦඪͱͯ͠࢖ΘΕΔ
    [Tolomei + 2018]

    View Slide

  12. Ϛ΢ευϥοάɾϖʔδεΫϩʔϧ
    12
    ɾυϥοάͱεΫϩʔϧʹҙਤ͕ग़Δ
    ɾεϚϗͰ͸ಘΒΕͳ͍ ୅ΘΓʹεϫΠϓ

    ॎυϥοά ԣυϥοά υϥοά͕஗͍
    ؔ࿈ੑ × ○ ○
    εΫϩʔϧྔ εΫϩʔϧස౓ εΫϩʔϧ଎౓
    ؔ࿈ੑ × ○ ○
    [Guo & Agichtein WWW 2012, Arapakis & Leiva SIGIR 2020]
    [Guo+ SIGIR 2013, Lagun+ SIGIR 2014]
    ݕࡧ݁Ռͷจॻ͕ཉ͍͠৘ใͱؔ܎ͯ͠
    ͍Δ͔͕Χʔιϧͷಈ͖͔ΒΘ͔Δ

    View Slide

  13. ଺ࡏ࣌ؒʢDwell Timeʣ
    13
    [Yi+ RecSys 2014], [Lu+ WWW 2019]
    • ଺ࡏ࣌ؒ͸ϢʔβʔҙਤΛΑ͘൓ө͍ͯ͠Δ
    • UIʹґଘ͢Δ
    • ΦϑϥΠϯͷ܇࿅࣌ʹ͔͠࢖͑ͳ͍
    [Lalmas + KDD 2015]
    [Ouyang+ KDD 2019]
    %XFMM5JNF
    8FCQBHF
    [Yom-Tovi+ BigData 2013], [Yi+ RecSys 2014]
    • ϖʔδ΍σόΠε͝ͱʹ͹Βͭ͘

    View Slide

  14. ஫ҙ఺·ͱΊ
    14
    • Ϣʔβʔߦಈ͸ϢʔβʔɾσόΠεɾϖʔδຖʹ͹Βͭ͘
    ˠ֤ཁૉຖʹਖ਼نԽͨ͠ྔΛ࢖༻͢Δͷ͕ॏཁ
    • ΦϑϥΠϯͱΦϯϥΠϯͰ؀ڥ͕ҟͳΔ
    ˠ؀ڥมಈΛߟྀͨ͠ΦϑϥΠϯͰͷֶश͕ॏཁ

    View Slide

  15. Ϣʔβʔߦಈ͔Βͷࢦඪͷ࡞੒
    15
    ̍ɽૉ๿ͳ؍࡯͔ΒԾઆΛཱͯΔ
    ྫɿ଺ࡏ͕࣌ؒ௕͍͜ͱ͸Ϣʔβʔ͕ຬ଍ͯ͠Δ͜ͱΛද͢

    ̎ɽ࣮ݧΛܭը͠ɼԾઆΛཱͯɼԾઆΛݕূ͢Δ
    ྫɿຬ଍౓ͷαʔϕΠͱ଺ࡏ࣌ؒͷܭଌΛߦ͏

    ̏ɽΦϯϥΠϯͷࢦඪΛઃܭ͠ɼͦͷଥ౰ੑΛݕূ͢Δ
    ྫɿ଺ࡏ͕࣌ؒ30ඵҎ্ΛϢʔβʔ͕ຬ଍͍ͯ͠Δͱ൑அ͢Δ
    [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]

    View Slide

  16. ͍͔ͭ͘ͷࢦඪΛ૊Έ߹ΘͤΔ
    16
    ࠶ݕࡧ͞Ε͔ͨʁ
    ଺ࡏ͕࣌ؒ௕͍͔ʁ
    :&4
    [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments KDD 2019]

    View Slide

  17. ࠓ೔ͷྲྀΕ
    17
    ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ

    View Slide

  18. ཭୤ͦ͠͏ͳϢʔβʔͷಛ௃
    18
    ཭୤ͦ͠͏ͳϢʔβʔ
    • αʔϏεΛ࢖͏ͭ΋Γ͕͕͋ͬͨఘΊͨʢ୹ظʣ
    • αʔϏεΛ࢖͍͕ͬͯͨ࢖Θͳ͘ͳͬͨʢ௕ظʣ
    ˠɹԿ͔͠ΒͷཁҼ͕αʔϏεͷັྗΛଛͳΘͤͨ
    ˠɹϢʔβʔΤϯήʔδϝϯτͷ໰୊

    View Slide

  19. ϢʔβʔΤϯήʔδϝϯτͷࢦඪ
    19
    • Click Depth IndexɿϖʔδϏϡʔ
    • Duration IndexɿαΠτ଺ࡏ࣌ؒ
    • Interaction IndexɿߪೖɾΞοϓϩʔυ
    • Recency Indexɿස౓
    • Loyalty Indexɿ௕ظؒ
    [Tutorial on Online User Engagement KDD 2020]

    View Slide

  20. ηογϣϯຖͷࢦඪ
    20
    intra-session metrics
    7JTJU 7JTJU 7JTJU
    inter-session metrics
    long-term value metrics
    [Tutorial on Online User Engagement KDD 2020]

    View Slide

  21. Ϣʔβʔຬ଍౓༧ଌ
    21
    • Jointly Leveraging Intent and Interaction Signals to Predict User
    Satisfaction with Slate Recommendations [Mehrotra+ WWW 2019]

    • 16ͷϢʔβʔߦಈ͔ΒϢʔβʔຬ଍౓Λ༧ଌʢSpotifyʣ

    • ΠϯλϏϡʔɼαʔϕΠɼϊϯύϥϕΠζͰϢʔβʔͷҙਤΛಉఆ

    • ϢʔβʔҙਤΛߟྀͨ͠΄͏͕ΑΓΑ͘Ϣʔβʔຬ଍౓Λ༧ଌͰ͖ͨ

    • Ϣʔβʔҙਤ͝ͱʹॏཁͱͳΔϢʔβʔߦಈ͕͹Βͭ͘

    View Slide

  22. 22
    ࢀߟ

    View Slide

  23. Ϣʔβʔͷߪങ཭୤༧ଌ
    23
    • Predicting Shopping Behavior with Mixture of RNNs [Toh+ SIGIR 2017]

    • ߪങ͢Δ͔൱͔ɼ·ͨ͸Ӿཡ͍ͯ͠Δ͚͔ͩΛ༧ଌʢָఱʣ

    • ΫϦοΫετϦʔϜσʔλɹ→ɹ଺ࡏ࣌ؒͱϖʔδλΠϓͷϖΞͷྻ

    • RNNͰߴ͍ਫ਼౓Ͱ༧ଌͰ͖Δ͜ͱΛ֬ೝ

    • ߪങ͢Δ͔൱͔ͷ൑அ͸Ӿཡ͍ͯ͠Δ͔൱͔ͷ൑அΑΓ೉͍͠

    View Slide

  24. ྫɿCart Abandonment Rate Statistics
    24
    [44 Cart Abandonment Rate Statistics by Baymard Institute]

    View Slide

  25. 25
    ࢀߟ

    View Slide

  26. 26
    ࢀߟ

    View Slide

  27. ໨తࢥߟͳϢʔβʔͷ෼ੳɾ༧ଌ
    27
    • Predicting Intent Using Activity Logs: How Goal Specificity and
    Temporal Range Affect User Behavior [Cheng+ WWW 2017]

    • PinterestϢʔβʔ͕໨తࢥߟ͔ɼͦΕ͕ߦಈ͔Β༧ଌͰ͖Δ͔
    • ໨తࢤ޲ͷ৔߹ɼݕࡧ͕ଟ͘ෳࡶͰɼݕࡧʹ͔͔Δ·Ͱ͕ૣ͘ɼݟΔ
    ίϯςϯπ͸গͳ͍͕ΑΓ۩ମతͰɼը૾ͷιʔεʹඈͿ܏޲͕͋Δ
    • Ӿཡɼ֦େɼΫϦοΫɼݕࡧͳͲͷߦಈ͔ΒϥϯμϜϑΥϨετͰ
    Ϣʔβʔ͕໨తࢥߟ͔Λ༧ଌ
    • ݕࡧ͕༧ଌʹॏཁͰɼ࠷ॳͷ෼ͷߦಈͰ໨తࢥߟ͔͕༧ଌͰ͖ͨ

    View Slide

  28. 28
    ࢀߟ

    View Slide

  29. νϟʔϯϨʔτ༧ଌ
    29
    • I Know You’ll Be Back: Interpretable New User Clustering and Churn
    Prediction on a Mobile Social Application [Yang+ KDD 2018]

    • SnapϢʔβʔͷνϟʔϯϨʔτͷ༧ଌ
    • ϢʔβʔΛΫϥελϦϯάͰηάϝϯτΘ͚ͯ͠཭୤཰͕ߴ͍Ϣʔ
    βʔΛಛఆ͠ɼAttention͖ͭLSTMͰνϟʔϯϨʔτΛ༧ଌ

    View Slide

  30. 30
    ࢀߟ

    View Slide

  31. 31
    ࢀߟ

    View Slide

  32. ࠓ೔ͷྲྀΕ
    32
    ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ

    View Slide

  33. จ຺͖ͭਪન
    33
    ௨ৗͷਪન จ຺͖ͭਪન
    User
    Item
    User
    Item
    Context

    View Slide

  34. CTR༧ଌ
    34
    ɾ$53ΫϦοΫ਺ɹɹ޿ࠂදࣔճ਺
    ɾ6TFSº*UFNº5JNFΛݩʹΫϦοΫ͢Δ͔Λ൑அ͢Δೋ஋෼ྨ
    [࣮຿ʢ$53༧ଌʣͱػցֶशίϯϖͷൺֱ :BIPP
    ]

    View Slide

  35. Ϣʔβʔͷ࣭໰ͷ༧ଌ
    35
    • Reinforcement Learning for User Intent Prediction in Customer
    Service Bots [Chen+ SIGIR 2019]

    • νϟοτϘοτͷͨΊͷਪનΞϧΰϦζϜʢAnt financialʣ
    • ࣭໰͞ΕΔલʹߦಈཤྺ͔Β࣭໰Λ༧ଌ͠ɼ࣭໰ީิΛఏҊ
    ʢUser Intent Predictionʣ

    View Slide

  36. ઃఆ
    36
    • 6TFSJOUFOUQSFEJDUJPOΛϚϧίϑܾఆաఔͱͯ͠ఆࣜԽ͠3-Ͱֶश
    • ࣭໰จͷఏҊΛUPQ/ਪન໰୊ͱͯ͠ఆࣜԽ

    View Slide

  37. ख๏
    37
    • CTR༧ଌ
    • ϢʔβʔͷΫϦοΫܥྻͱϢʔβʔ৘ใΛೖྗͱͯ͠࢖༻
    • લऀ͸CNNɼޙऀ͸FCNNͰಛ௃ྔΛ࡞੒͠ɼͦΕΒΛFCNN΁ೖྗ
    • Question popularity

    • աڈҰఆظؒʹ࣭໰͕ΫϦοΫ͞Εͨઈର਺ͱදࣔ͞Εͨճ਺͔Βɼ
    ࠓΑ͘ฉ͔Ε͍ͯΔ࣭໰Λࢉग़
    • Question Diversity

    • ݱࡏͷ࣭໰ͱੲͷ࣭໰ͷྨࣅ౓͔Β࣭໰ͷଟ༷ੑΛܭࢉ
    • ͜ΕΒͷείΞΛ࢖ͬͯtop-Nͷ࣭໰ީิΛϥϯΫ͚ͮ

    View Slide

  38. ݁Ռ
    38
    • ΦϑϥΠϯͷධՁͰϕʔεϥΠϯΛ૬ରతʹվળ
    • AntͷαʔϏεϘοτͷ೔ͷABςετͰCTRΛ૬ରతʹվળ

    View Slide

  39. ࣮ࡍʹγεςϜͱͯ͠ӡ༻͞Ε͍ͯΔ
    39
    • AntProphet: an Intention Mining
    System behind Alipay’s Intelligent
    Customer Service Bot [Chen+ IJCAI
    2020 (demo)]

    • Ϣʔβʔͷ࣭໰ͷʹରԠ
    • ଟ͘ͷ࣭໰͕ϢʔβʔߦಈͷΈ͔Β
    ਪଌͰ͖Δ͜ͱΛ֬ೝ

    View Slide

  40. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ
    40
    • Deep Interest Network for Click-Through Rate Prediction [Zhou +
    KDD 2018]

    • AttentionΛ༻͍Δ͜ͱͰϢʔβʔͷΞΠςϜ΁ͷؔ৺Λදݱͨ͠
    CTR༧ଌʢAlibabaʣ

    View Slide

  41. ಛ௃ྔͷ࡞੒
    41
    • ಛ௃ྔͷཁૉຖͷຒΊࠐΈͷ݁߹ΛϓʔϦϯάͯ͠ߦಈಛ௃ྔΛ࡞੒
    • ͦͷଞͷಛ௃ྔͱ݁߹ͯ͠ฏ׈Խͨ͠΋ͷΛ//ʹೖྗ

    View Slide

  42. Deep Interest Network
    42
    • ީิͷ޿ࠂͱϢʔβʔߦಈʹ͍ͭͯͷattentionΛಋೖ

    View Slide

  43. ࣮ݧͱ݁Ռ
    43
    • N-1εςοϓ·Ͱͷߦಈσʔλ͔Β/εςοϓͷߦಈΛ༧ଌ
    • ABςετͷ݁ՌɼCTR͕ɼΠϯϓϨογϣϯऩӹ͕վળ

    View Slide

  44. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ
    44
    • User Behavior Retrieval for Click-Through Rate Prediction [Qin +
    SIGIR 2020]

    • ௕͍ϢʔβʔߦಈܥྻΛ׆༻͢ΔCTR༧ଌʢAlibabaʣ
    • AttentionϕʔεͷωοτϫʔΫͰੲͷߦಈཤྺ͔Βݕࡧ
    • ͋Δจ຺Ͱ࠷΋͋Γ͏ΔUser-ItemͷϖΞΛ༧ଌ

    View Slide

  45. ख๏
    45
    • ߦಈܥྻʹΫΤϦΛ౤͛ͯ࠷΋ؔ࿈͢ΔϢʔβʔߦಈΛऔಘ
    • ϢʔβʔߦಈΛdocumentɼ֤ಛ௃Λtermͱͯ͠දݱ͔ͯͦ͜͠Β#.
    ͰϢʔβʔߦಈΛݕࡧɼ͜ΕΛREINFORCEͰֶश
    • attentionϕʔεͷNNΛ༻͍ͨର਺໬౓࠷େԽͰ༧ଌ

    View Slide

  46. ࣮ݧͱ݁Ռ
    46
    • TmallɼTaobaoɼAlipayͷϢʔβߦಈΛ༧ଌ
    • શମͷͷߦಈΛ࢖͏ͱAlipayͰAUC͕ɼlog loss͕վળ
    • શߦಈΛ࢖͏ͱɼఏҊ๏͸ͦͷ͏͔ͪ͠࢖ΘͣɼAUC͸େࠩͳ͍
    ͕ɼlog loss͸େ͖͘ݮͬͨ

    View Slide

  47. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ
    47
    • Practice on Long Sequential User Behavior Modeling for Click-
    Through Rate Prediction [Pi+ KDD 2019]

    • ௕͍ϢʔβʔߦಈܥྻΛར༻ͨ͠ΦϯϥΠϯ޿ࠂCTR༧ଌͷվળ
    ʢAlibabaʣ

    • Neural Turing MachineΛར༻͠storageͷ੍໿ͱlatencyͷ੍໿ʹରԠ

    View Slide

  48. ϞσϧͷΠϝʔδਤ
    48
    w ੜͷߦಈͱ/5.ʹهԱ͞Εͨ৘ใΛ྆ํอ࣋ͯ͠ؼೲతਪ࿦Λߦ͏

    View Slide

  49. ݁Ռ
    49
    • AlibabaͷΦϯϥΠϯσΟεϓϨΠ޿ࠂͷσʔλΛ࢖༻ʢΠϯϓϨογ
    ϣϯͷϩάͱɼΫϦοΫ͔ͨ͠൱͔ͷσʔλʣ

    • 49೔෼ͷσʔλ͔Β࣍ͷ೔ͷ΋ͷΛ༧ଌ

    • 2019-03-30͔Β2019-05-10·ͰͷظؒͰABςετΛ࣮ࢪ

    • warm upͷͨΊʹɼ120೔ؒΦϑϥΠϯͰֶश

    • ABςετͰCTR͕7.5%૿Ճ͠ΠϯϓϨογϣϯऩӹ͕6%૿Ճ

    View Slide

  50. ద੾ͳλΠϛϯάͰద੾ͳ΋ͷΛਪન
    50
    • Temporal-Contextual Recommendation in Real-Time [Ma+ KDD 2020]

    • ֊૚తͳRNNΛ࢖͏͜ͱͰҙਤͷมԽΛଊ͑ͯɼద੾ͳλΠϛϯάͰద
    ੾ͳ΋ͷΛਪન͢ΔϞσϧʢAmazonʣ

    View Slide

  51. શମͷγεςϜ
    51
    • ηογϣϯ಺ͷRNNͱηογϣϯؒͷRNNΛཅʹ۠ผ͠ɼηογϣϯ
    ಺ͷRNN͸౎౓Ϧηοτ͢Δ

    View Slide

  52. ࣮ݧ݁Ռ
    52
    • ϢʔβʔߦಈΛ࢖ΘͣҰͭલͷΞΠςϜ͚ͩͰ΋͋Δఔ౓͏·͍͘͘

    • Ϟσϧ͕Ͱ͔͍ͱੑೳࣗମʹվળ͸ͳ͍͕ૣ͞Ͱେ͖͘উΔ

    • ίʔϧυελʔτʹͦͦ͜͜ରԠͰ͖Δ

    • ΞΠςϜ਺͕ଟ͍࣌ʹ͸ॏ఺αϯϓϦϯά͸༗ޮ

    View Slide

  53. ϢʔβʔͷߪങߦಈΛཅʹϞσϦϯά
    53
    • Opportunity Models for E-commerce Recommendation: Right
    Product, Right Time [Wang & Zhang SIGIR 2013]

    • ੜଘ࣌ؒ෼ੳʹΑΓ͋Δ࣌ؒͰ͋Δ঎඼ΛϢʔβʔ͕ങ͏֬཰Λදݱ

    • shop.comͰͷߪങߦಈͷ༧ଌʹ੒ޭ͠CVRͱϢʔβʔຬ଍౓΋޲্

    View Slide

  54. ύʔνΣεϑΝωϧΛҙࣝͨ͠Ϟσϧ
    54
    • Understanding Consumer Journey using Attention based Recurrent
    Neural Networks [Zhou + KDD 2019]

    • ύʔνΣεϑΝωϧͷͲ͜ʹ͍Δ͔Λֶश͢Δattention͖ͭRNNʢYahooʣ

    View Slide

  55. ϩδεςΟοΫճؼʁ
    55
    ɾSimple and Scalable Response Prediction for Display Advertising
    [Chapelle+ 2014] (Criteo)
    ɾAd Click Prediction: a View from the Trenches [McMahan + KDD
    2013] (Google)
    ɾϚΠΫϩΞυʹ͓͚ΔCTR༧ଌ΁ͷऔΓ૊Έ

    View Slide

  56. ࣮຿ʹ͓͚ΔCTR༧ଌͱKaggle
    56
    • KaggleͰΑ͘༻͍ΒΕΔ౷ܭྔ΍௚લͷϩάΛ࢖༻ͨ͠ಛ௃ྔ͸ɼਫ਼
    ౓վળʹͭͳ͕Δ͕ΦϯϥΠϯਪ࿦؀ڥ੔උ͕େม

    • Target EncodingͰ࣮੷CTRΛ༻͍Δͷ͕ɼਫ਼౓ʹͱͯͭ΋ͳ͘ޮ͘
    [࣮຿ʢ$53༧ଌʣͱػցֶशίϯϖͷൺֱ :BIPP
    ]

    View Slide

  57. CTRͷಛ௃ྔΤϯδχΞϦϯά
    57
    • ϋογϡؔ਺Λ༻͍ͯΧςΰϦม਺Λ௿࣍ݩʹຒΊࠐΉ

    • ࣍ݩ͕ΧςΰϦ਺ΑΓ௿͍ͱিಥ͕ൃੜ
    [ϚΠΫϩΞυʹ͓͚Δ$53༧ଌ΁ͷऔΓ૊Έ MicroAd)]
    Feature Hashing

    View Slide

  58. CTRʹ͓͚ΔෆۉߧσʔλͷऔΓѻ͍
    58
    • ΠϯϓϨογϣϯʹର͢ΔΫϦοΫ਺ͷׂ߹͸গͳ͍

    • ෛྫͷΞϯμʔαϯϓϦϯά͕ඞਢ

    • ҎԼͷࣜͰิਖ਼
    [ϚΠΫϩΞυʹ͓͚Δ$53༧ଌ΁ͷऔΓ૊Έ MicroAd) Pozzolo+ 2015]

    View Slide

  59. ࠓ೔ͷྲྀΕ
    59
    ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ

    View Slide

  60. ײ૝
    60
    • Ϣʔβʔߦಈ͸ϢʔβʔҙਤΛ൓ө͢Δ͕͹Β͖͕ͭେ͖ͦ͏
    ˠɹػցֶश͸Մೳ͕ͩސ٬ຖͷࡉ͔͍ௐ੔͕ॏཁ
    • ػցֶशʹΑΔվળͷ༨஍͸͋Δͱࢥ͏

    View Slide

  61. ͦͷଞ
    61
    • Time-Aware Prospective Modeling of Users for Online Display
    Advertising [Gligorijevic + AdKDD 2019] (Yahoo)

    • Latent Cross: Making Use of Context in Recurrent Recommender
    Systems [Beutel + WSDM 2019] (Google)

    • Contextual Sequence Modeling for Recommendation with
    Recurrent Neural Networks [Smirnova + RecSys 2017] (Criteo)

    • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona?
    Recommendation in a Two-Sided Travel Marketplace [Wu + SIGIR
    2017] (Airbnb)

    View Slide

  62. اۀͷڭ܇
    62
    • Ad Click Prediction: a View from the Trenches [McMahan + KDD 2013]
    (Google)

    • Practical Lessons from Predicting Clicks on Ads at Facebookɹ[He +
    AdKDD 2014] (Facebook)

    • 150 Successful Machine Learning Models: 6 Lessons Learned at
    Booking.comɹ[Bernandi + KDD 2019] (Booking.com)

    • Applying Deep Learning for Airbnb Search [Halder+ KDD 2019]
    (Airbnb)

    View Slide

  63. ࢿྉͷ঺հʢਪનʣ
    63
    • Recommender Systems Handbook

    • RecSys 2020 Tutorial: Feature Engineering for Recommender Systems

    • ʮΦϯϥΠϯ޿ࠂؔ࿈ͷ࿦จΛຊ͘Β͍ࡶʹ঺հ͢ΔAdKDDฤʯ
    • ʮΦϯϥΠϯ޿ࠂʹ͓͚Δ$53$73ਪఆؔ܎ͷ࿦จΛຊ͘Β͍ࡶʹ঺
    հ͢Δʯ
    • CyberAgent Developers Blog

    • AWS Recommendation Engine Seminar ࢀՃϨϙʔτʢલ൒ʣ
    • DeepCTR-Torch

    View Slide

  64. ࢿྉͷ঺հʢςετʣ
    64
    • Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing

    • ʮւ֎ͷ༗໊*5اۀͷ"#ςετϒϩά·ͱΊʯ
    • Innovating Faster on Personalization Algorithms at Netflix Using Interleaving

    • A Dirty Dozen: Twelve Common Metric Interpretation Pitfalls in Online
    Controlled Experiments

    • Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained

    • Challenges, Best Practices and Pitfalls in Evaluating Results of Online
    Controlled Experiments

    View Slide

  65. ࢀߟจݙɾࢀߟࢿྉ
    65
    • Maximizing the Engagement: Exploring New Signals of Implicit Feedback in Music Recommendations

    • Intention Nets: Psychology-Inspired User Choice Behavior Modeling for Next-Basket Prediction

    • Shopper intent prediction from clickstream e‑commerce data with minimal browsing information

    • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona? Recommendation in a Two-Sided Travel Marketplace

    • A better clickthrough rate: How Pinterest upgraded everyone’s favorite engagement metric

    • Real-time User Signal Serving for Feature Engineering

    • Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction

    • Deep Interest Network for Click-Through Rate Prediction

    • Opportunity model for e-commerce recommendation: right product; right time

    • Temporal-Contextual Recommendation in Real-Time

    • Learning Efficient Representations of Mouse Movements to Predict User Attention

    • Time-Aware Prospective Modeling of Users for Online Display Advertising

    • Understanding Consumer Journey using Attention based Recurrent Neural Networks

    • User Response Prediction in Online Advertising

    • Predicting Shopping Behavior with Mixture of RNNs

    • Beyond Dwell Time: Estimating Document Relevance from Cursor Movements and other Post-click Searcher Behavior

    • AliMe Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience

    • Reinforcement Learning for User Intent Prediction in Customer Service Bots

    • AntProphet: an Intention Mining System behind Alipay’s Intelligent Customer Service Bot

    • Predicting Intent Using Activity Logs: How Goal Specificity and Temporal Range Affect User Behavior

    View Slide