Save 37% off PRO during our Black Friday Sale! »

user-behaviour-vol1

D8b77b4b3b0373eaee6c6077c4d7330a?s=47 KARAKURI Inc.
November 16, 2021

 user-behaviour-vol1

ユーザー行動予測に関する研究のサーベイ

D8b77b4b3b0373eaee6c6077c4d7330a?s=128

KARAKURI Inc.

November 16, 2021
Tweet

Transcript

  1. Ϣʔβʔߦಈ༧ଌʹؔ͢ΔݚڀͷαʔϕΠ ߴ໦ࢤ࿠ 1

  2. ͜ͷαʔϕΠͰ΍Δ͜ͱ 2 w Ϣʔβʔߦಈܥྻ͔ΒϢʔβʔߦಈ༧ଌʹؔ͢Δݚڀͷ঺հ w Ϣʔβʔߦಈ༧ଌͱͦͷࢦඪͷ঺հ w Ϣʔβʔߦಈͱͯ͠ͲͷΑ͏ͳ΋ͷ͕࢖ΘΕ͍ͯΔ͔঺հ w ϢʔβʔΤϯήʔδϝϯτʹؔ͢Δݚڀͷྫͷ঺հ঺հ

    w Ϣʔβʔ͕͍ͭԿΛ͢Δ͔Λ༧ଌ͢Δݚڀͷྫͷ঺հ *Ҿ༻͕ͳ͍΋ͷ͸֘౰࿦จ͔ΒҾ༻
  3. ࠓ೔ͷྲྀΕ 3 ̍ɽϢʔβʔߦಈ༧ଌͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

  4. ࠓ೔ͷྲྀΕ 4 ̍ɽϢʔβʔߦಈ༧ଌͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

  5. ໨త͋Γ͖ͷϢʔβʔߦಈ༧ଌ 5 ໨త Ϣʔβʔߦಈ

  6. Ͳ͏΍ͬͯଌΔͷ͔ʁ 6 ܭଌ ʢྫɿΫϦοΫ਺ʣ ج४ ʢྫɿ$53ʣ ,1* ʢྫɿ$73ʣ [Tutorial on

    Online User Engagement KDD 2020]
  7. ྑ͍ධՁࢦඪͱ͸ʁ 7 • SensitivityɿվળΛͪΌΜͱݕ஌Ͱ͖Δ͔ʁ • Trustworthinessɿ݁Ռ͸৴པͰ͖Δ͔ʁ • Efficiencyɿܭࢉɾܭଌͷίετ͕௿͍͔ʁ • Debuggability

    and ActionabilityɿࢦඪͷมԽͷཧ༝ͷಛఆ͕Մೳ͔ʁ • Interpretability and DirectionalityɿࢦඪͷมԽ͸໨తΛୡ੒͢Δ͔ʁ [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
  8. Overall Evaluation Criteria (OEC) 8 • ࠷ऴతͳ໨తͱͳΔࢦඪ • DirectivityͱSensitivity͕ॏཁ •

    ͜ΕΒ2ͭΛຬͨ͢ͷ͸ࠔ೉ • ྫɿ1Ϣʔβ͋ͨΓͷฏۉऩӹ [Dmitriev+ KDD 2017] [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
  9. ͦͷଞͷࢦඪ 9 • Guardrail Metricsɿ੍໿৚݅ • Operational Metricsɿσόοά༻ͷࢦඪ • Data

    Quality Metricsɿσʔλ͕৴པͰ͖Δ͜ͱΛอূ͢Δࢦඪ [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
  10. ࠓ೔ͷྲྀΕ 10 ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

  11. ΫϦοΫ 11 • ඇৗʹΑ͘࢖ΘΕΔϢʔβʔߦಈ • ޡΫϦοΫͷՄೳੑ͕͋ΔʢಛʹεϚϗʣ • $MJDL%FQUI • ϢʔβʔΤϯήʔδϝϯτɹPSɹ໎͍ͬͯΔ

    • $MJDL5ISPVHI3BUF • ޿ࠂͷධՁࢦඪͱͯ͠࢖ΘΕΔ [Tolomei + 2018]
  12. Ϛ΢ευϥοάɾϖʔδεΫϩʔϧ 12 ɾυϥοάͱεΫϩʔϧʹҙਤ͕ग़Δ ɾεϚϗͰ͸ಘΒΕͳ͍ ୅ΘΓʹεϫΠϓ ॎυϥοά ԣυϥοά υϥοά͕஗͍ ؔ࿈ੑ ×

    ◦ ◦ εΫϩʔϧྔ εΫϩʔϧස౓ εΫϩʔϧ଎౓ ؔ࿈ੑ × ◦ ◦ [Guo & Agichtein WWW 2012, Arapakis & Leiva SIGIR 2020] [Guo+ SIGIR 2013, Lagun+ SIGIR 2014] ݕࡧ݁Ռͷจॻ͕ཉ͍͠৘ใͱؔ܎ͯ͠ ͍Δ͔͕Χʔιϧͷಈ͖͔ΒΘ͔Δ
  13. ଺ࡏ࣌ؒʢDwell Timeʣ 13 [Yi+ RecSys 2014], [Lu+ WWW 2019] •

    ଺ࡏ࣌ؒ͸ϢʔβʔҙਤΛΑ͘൓ө͍ͯ͠Δ • UIʹґଘ͢Δ • ΦϑϥΠϯͷ܇࿅࣌ʹ͔͠࢖͑ͳ͍ [Lalmas + KDD 2015] [Ouyang+ KDD 2019] %XFMM5JNF 8FCQBHF [Yom-Tovi+ BigData 2013], [Yi+ RecSys 2014] • ϖʔδ΍σόΠε͝ͱʹ͹Βͭ͘
  14. ஫ҙ఺·ͱΊ 14 • Ϣʔβʔߦಈ͸ϢʔβʔɾσόΠεɾϖʔδຖʹ͹Βͭ͘ ˠ֤ཁૉຖʹਖ਼نԽͨ͠ྔΛ࢖༻͢Δͷ͕ॏཁ • ΦϑϥΠϯͱΦϯϥΠϯͰ؀ڥ͕ҟͳΔ ˠ؀ڥมಈΛߟྀͨ͠ΦϑϥΠϯͰͷֶश͕ॏཁ

  15. Ϣʔβʔߦಈ͔Βͷࢦඪͷ࡞੒ 15 ̍ɽૉ๿ͳ؍࡯͔ΒԾઆΛཱͯΔ ྫɿ଺ࡏ͕࣌ؒ௕͍͜ͱ͸Ϣʔβʔ͕ຬ଍ͯ͠Δ͜ͱΛද͢ ̎ɽ࣮ݧΛܭը͠ɼԾઆΛཱͯɼԾઆΛݕূ͢Δ ྫɿຬ଍౓ͷαʔϕΠͱ଺ࡏ࣌ؒͷܭଌΛߦ͏ ̏ɽΦϯϥΠϯͷࢦඪΛઃܭ͠ɼͦͷଥ౰ੑΛݕূ͢Δ ྫɿ଺ࡏ͕࣌ؒ30ඵҎ্ΛϢʔβʔ͕ຬ଍͍ͯ͠Δͱ൑அ͢Δ [Challenges, Best

    Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
  16. ͍͔ͭ͘ͷࢦඪΛ૊Έ߹ΘͤΔ 16 ࠶ݕࡧ͞Ε͔ͨʁ ଺ࡏ͕࣌ؒ௕͍͔ʁ :&4 [Challenges, Best Practices and Pitfalls

    in Evaluating Results of Online Controlled Experiments KDD 2019]
  17. ࠓ೔ͷྲྀΕ 17 ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

  18. ཭୤ͦ͠͏ͳϢʔβʔͷಛ௃ 18 ཭୤ͦ͠͏ͳϢʔβʔ • αʔϏεΛ࢖͏ͭ΋Γ͕͕͋ͬͨఘΊͨʢ୹ظʣ • αʔϏεΛ࢖͍͕ͬͯͨ࢖Θͳ͘ͳͬͨʢ௕ظʣ ˠɹԿ͔͠ΒͷཁҼ͕αʔϏεͷັྗΛଛͳΘͤͨ ˠɹϢʔβʔΤϯήʔδϝϯτͷ໰୊

  19. ϢʔβʔΤϯήʔδϝϯτͷࢦඪ 19 • Click Depth IndexɿϖʔδϏϡʔ • Duration IndexɿαΠτ଺ࡏ࣌ؒ •

    Interaction IndexɿߪೖɾΞοϓϩʔυ • Recency Indexɿස౓ • Loyalty Indexɿ௕ظؒ [Tutorial on Online User Engagement KDD 2020]
  20. ηογϣϯຖͷࢦඪ 20 intra-session metrics 7JTJU 7JTJU 7JTJU inter-session metrics long-term

    value metrics [Tutorial on Online User Engagement KDD 2020]
  21. Ϣʔβʔຬ଍౓༧ଌ 21 • Jointly Leveraging Intent and Interaction Signals to

    Predict User Satisfaction with Slate Recommendations [Mehrotra+ WWW 2019] • 16ͷϢʔβʔߦಈ͔ΒϢʔβʔຬ଍౓Λ༧ଌʢSpotifyʣ • ΠϯλϏϡʔɼαʔϕΠɼϊϯύϥϕΠζͰϢʔβʔͷҙਤΛಉఆ • ϢʔβʔҙਤΛߟྀͨ͠΄͏͕ΑΓΑ͘Ϣʔβʔຬ଍౓Λ༧ଌͰ͖ͨ • Ϣʔβʔҙਤ͝ͱʹॏཁͱͳΔϢʔβʔߦಈ͕͹Βͭ͘
  22. 22 ࢀߟ

  23. Ϣʔβʔͷߪങ཭୤༧ଌ 23 • Predicting Shopping Behavior with Mixture of RNNs

    [Toh+ SIGIR 2017] • ߪങ͢Δ͔൱͔ɼ·ͨ͸Ӿཡ͍ͯ͠Δ͚͔ͩΛ༧ଌʢָఱʣ • ΫϦοΫετϦʔϜσʔλɹ→ɹ଺ࡏ࣌ؒͱϖʔδλΠϓͷϖΞͷྻ • RNNͰߴ͍ਫ਼౓Ͱ༧ଌͰ͖Δ͜ͱΛ֬ೝ • ߪങ͢Δ͔൱͔ͷ൑அ͸Ӿཡ͍ͯ͠Δ͔൱͔ͷ൑அΑΓ೉͍͠
  24. ྫɿCart Abandonment Rate Statistics 24 [44 Cart Abandonment Rate Statistics

    by Baymard Institute]
  25. 25 ࢀߟ

  26. 26 ࢀߟ

  27. ໨తࢥߟͳϢʔβʔͷ෼ੳɾ༧ଌ 27 • Predicting Intent Using Activity Logs: How Goal

    Specificity and Temporal Range Affect User Behavior [Cheng+ WWW 2017] • PinterestϢʔβʔ͕໨తࢥߟ͔ɼͦΕ͕ߦಈ͔Β༧ଌͰ͖Δ͔ • ໨తࢤ޲ͷ৔߹ɼݕࡧ͕ଟ͘ෳࡶͰɼݕࡧʹ͔͔Δ·Ͱ͕ૣ͘ɼݟΔ ίϯςϯπ͸গͳ͍͕ΑΓ۩ମతͰɼը૾ͷιʔεʹඈͿ܏޲͕͋Δ • Ӿཡɼ֦େɼΫϦοΫɼݕࡧͳͲͷߦಈ͔ΒϥϯμϜϑΥϨετͰ Ϣʔβʔ͕໨తࢥߟ͔Λ༧ଌ • ݕࡧ͕༧ଌʹॏཁͰɼ࠷ॳͷ෼ͷߦಈͰ໨తࢥߟ͔͕༧ଌͰ͖ͨ
  28. 28 ࢀߟ

  29. νϟʔϯϨʔτ༧ଌ 29 • I Know You’ll Be Back: Interpretable New

    User Clustering and Churn Prediction on a Mobile Social Application [Yang+ KDD 2018] • SnapϢʔβʔͷνϟʔϯϨʔτͷ༧ଌ • ϢʔβʔΛΫϥελϦϯάͰηάϝϯτΘ͚ͯ͠཭୤཰͕ߴ͍Ϣʔ βʔΛಛఆ͠ɼAttention͖ͭLSTMͰνϟʔϯϨʔτΛ༧ଌ
  30. 30 ࢀߟ

  31. 31 ࢀߟ

  32. ࠓ೔ͷྲྀΕ 32 ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

  33. จ຺͖ͭਪન 33 ௨ৗͷਪન จ຺͖ͭਪન User Item User Item Context

  34. CTR༧ଌ 34 ɾ$53ΫϦοΫ਺ɹɹ޿ࠂදࣔճ਺ ɾ6TFSº*UFNº5JNFΛݩʹΫϦοΫ͢Δ͔Λ൑அ͢Δೋ஋෼ྨ [࣮຿ʢ$53༧ଌʣͱػցֶशίϯϖͷൺֱ :BIPP ]

  35. Ϣʔβʔͷ࣭໰ͷ༧ଌ 35 • Reinforcement Learning for User Intent Prediction in

    Customer Service Bots [Chen+ SIGIR 2019] • νϟοτϘοτͷͨΊͷਪનΞϧΰϦζϜʢAnt financialʣ • ࣭໰͞ΕΔલʹߦಈཤྺ͔Β࣭໰Λ༧ଌ͠ɼ࣭໰ީิΛఏҊ ʢUser Intent Predictionʣ
  36. ઃఆ 36 • 6TFSJOUFOUQSFEJDUJPOΛϚϧίϑܾఆաఔͱͯ͠ఆࣜԽ͠3-Ͱֶश • ࣭໰จͷఏҊΛUPQ/ਪન໰୊ͱͯ͠ఆࣜԽ

  37. ख๏ 37 • CTR༧ଌ • ϢʔβʔͷΫϦοΫܥྻͱϢʔβʔ৘ใΛೖྗͱͯ͠࢖༻ • લऀ͸CNNɼޙऀ͸FCNNͰಛ௃ྔΛ࡞੒͠ɼͦΕΒΛFCNN΁ೖྗ • Question

    popularity • աڈҰఆظؒʹ࣭໰͕ΫϦοΫ͞Εͨઈର਺ͱදࣔ͞Εͨճ਺͔Βɼ ࠓΑ͘ฉ͔Ε͍ͯΔ࣭໰Λࢉग़ • Question Diversity • ݱࡏͷ࣭໰ͱੲͷ࣭໰ͷྨࣅ౓͔Β࣭໰ͷଟ༷ੑΛܭࢉ • ͜ΕΒͷείΞΛ࢖ͬͯtop-Nͷ࣭໰ީิΛϥϯΫ͚ͮ
  38. ݁Ռ 38 • ΦϑϥΠϯͷධՁͰϕʔεϥΠϯΛ૬ରతʹվળ • AntͷαʔϏεϘοτͷ೔ͷABςετͰCTRΛ૬ରతʹվળ

  39. ࣮ࡍʹγεςϜͱͯ͠ӡ༻͞Ε͍ͯΔ 39 • AntProphet: an Intention Mining System behind Alipay’s

    Intelligent Customer Service Bot [Chen+ IJCAI 2020 (demo)] • Ϣʔβʔͷ࣭໰ͷʹରԠ • ଟ͘ͷ࣭໰͕ϢʔβʔߦಈͷΈ͔Β ਪଌͰ͖Δ͜ͱΛ֬ೝ
  40. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ 40 • Deep Interest Network for Click-Through Rate Prediction

    [Zhou + KDD 2018] • AttentionΛ༻͍Δ͜ͱͰϢʔβʔͷΞΠςϜ΁ͷؔ৺Λදݱͨ͠ CTR༧ଌʢAlibabaʣ
  41. ಛ௃ྔͷ࡞੒ 41 • ಛ௃ྔͷཁૉຖͷຒΊࠐΈͷ݁߹ΛϓʔϦϯάͯ͠ߦಈಛ௃ྔΛ࡞੒ • ͦͷଞͷಛ௃ྔͱ݁߹ͯ͠ฏ׈Խͨ͠΋ͷΛ//ʹೖྗ

  42. Deep Interest Network 42 • ީิͷ޿ࠂͱϢʔβʔߦಈʹ͍ͭͯͷattentionΛಋೖ

  43. ࣮ݧͱ݁Ռ 43 • N-1εςοϓ·Ͱͷߦಈσʔλ͔Β/εςοϓͷߦಈΛ༧ଌ • ABςετͷ݁ՌɼCTR͕ɼΠϯϓϨογϣϯऩӹ͕վળ

  44. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ 44 • User Behavior Retrieval for Click-Through Rate Prediction

    [Qin + SIGIR 2020] • ௕͍ϢʔβʔߦಈܥྻΛ׆༻͢ΔCTR༧ଌʢAlibabaʣ • AttentionϕʔεͷωοτϫʔΫͰੲͷߦಈཤྺ͔Βݕࡧ • ͋Δจ຺Ͱ࠷΋͋Γ͏ΔUser-ItemͷϖΞΛ༧ଌ
  45. ख๏ 45 • ߦಈܥྻʹΫΤϦΛ౤͛ͯ࠷΋ؔ࿈͢ΔϢʔβʔߦಈΛऔಘ • ϢʔβʔߦಈΛdocumentɼ֤ಛ௃Λtermͱͯ͠දݱ͔ͯͦ͜͠Β#. ͰϢʔβʔߦಈΛݕࡧɼ͜ΕΛREINFORCEͰֶश • attentionϕʔεͷNNΛ༻͍ͨର਺໬౓࠷େԽͰ༧ଌ

  46. ࣮ݧͱ݁Ռ 46 • TmallɼTaobaoɼAlipayͷϢʔβߦಈΛ༧ଌ • શମͷͷߦಈΛ࢖͏ͱAlipayͰAUC͕ɼlog loss͕վળ • શߦಈΛ࢖͏ͱɼఏҊ๏͸ͦͷ͏͔ͪ͠࢖ΘͣɼAUC͸େࠩͳ͍ ͕ɼlog

    loss͸େ͖͘ݮͬͨ
  47. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ 47 • Practice on Long Sequential User Behavior Modeling

    for Click- Through Rate Prediction [Pi+ KDD 2019] • ௕͍ϢʔβʔߦಈܥྻΛར༻ͨ͠ΦϯϥΠϯ޿ࠂCTR༧ଌͷվળ ʢAlibabaʣ • Neural Turing MachineΛར༻͠storageͷ੍໿ͱlatencyͷ੍໿ʹରԠ
  48. ϞσϧͷΠϝʔδਤ 48 w ੜͷߦಈͱ/5.ʹهԱ͞Εͨ৘ใΛ྆ํอ࣋ͯ͠ؼೲతਪ࿦Λߦ͏

  49. ݁Ռ 49 • AlibabaͷΦϯϥΠϯσΟεϓϨΠ޿ࠂͷσʔλΛ࢖༻ʢΠϯϓϨογ ϣϯͷϩάͱɼΫϦοΫ͔ͨ͠൱͔ͷσʔλʣ • 49೔෼ͷσʔλ͔Β࣍ͷ೔ͷ΋ͷΛ༧ଌ • 2019-03-30͔Β2019-05-10·ͰͷظؒͰABςετΛ࣮ࢪ •

    warm upͷͨΊʹɼ120೔ؒΦϑϥΠϯͰֶश • ABςετͰCTR͕7.5%૿Ճ͠ΠϯϓϨογϣϯऩӹ͕6%૿Ճ
  50. ద੾ͳλΠϛϯάͰద੾ͳ΋ͷΛਪન 50 • Temporal-Contextual Recommendation in Real-Time [Ma+ KDD 2020]

    • ֊૚తͳRNNΛ࢖͏͜ͱͰҙਤͷมԽΛଊ͑ͯɼద੾ͳλΠϛϯάͰద ੾ͳ΋ͷΛਪન͢ΔϞσϧʢAmazonʣ
  51. શମͷγεςϜ 51 • ηογϣϯ಺ͷRNNͱηογϣϯؒͷRNNΛཅʹ۠ผ͠ɼηογϣϯ ಺ͷRNN͸౎౓Ϧηοτ͢Δ

  52. ࣮ݧ݁Ռ 52 • ϢʔβʔߦಈΛ࢖ΘͣҰͭલͷΞΠςϜ͚ͩͰ΋͋Δఔ౓͏·͍͘͘ • Ϟσϧ͕Ͱ͔͍ͱੑೳࣗମʹվળ͸ͳ͍͕ૣ͞Ͱେ͖͘উΔ • ίʔϧυελʔτʹͦͦ͜͜ରԠͰ͖Δ • ΞΠςϜ਺͕ଟ͍࣌ʹ͸ॏ఺αϯϓϦϯά͸༗ޮ

  53. ϢʔβʔͷߪങߦಈΛཅʹϞσϦϯά 53 • Opportunity Models for E-commerce Recommendation: Right Product,

    Right Time [Wang & Zhang SIGIR 2013] • ੜଘ࣌ؒ෼ੳʹΑΓ͋Δ࣌ؒͰ͋Δ঎඼ΛϢʔβʔ͕ങ͏֬཰Λදݱ • shop.comͰͷߪങߦಈͷ༧ଌʹ੒ޭ͠CVRͱϢʔβʔຬ଍౓΋޲্
  54. ύʔνΣεϑΝωϧΛҙࣝͨ͠Ϟσϧ 54 • Understanding Consumer Journey using Attention based Recurrent

    Neural Networks [Zhou + KDD 2019] • ύʔνΣεϑΝωϧͷͲ͜ʹ͍Δ͔Λֶश͢Δattention͖ͭRNNʢYahooʣ
  55. ϩδεςΟοΫճؼʁ 55 ɾSimple and Scalable Response Prediction for Display Advertising

    [Chapelle+ 2014] (Criteo) ɾAd Click Prediction: a View from the Trenches [McMahan + KDD 2013] (Google) ɾϚΠΫϩΞυʹ͓͚ΔCTR༧ଌ΁ͷऔΓ૊Έ<MicroAd>
  56. ࣮຿ʹ͓͚ΔCTR༧ଌͱKaggle 56 • KaggleͰΑ͘༻͍ΒΕΔ౷ܭྔ΍௚લͷϩάΛ࢖༻ͨ͠ಛ௃ྔ͸ɼਫ਼ ౓վળʹͭͳ͕Δ͕ΦϯϥΠϯਪ࿦؀ڥ੔උ͕େม • Target EncodingͰ࣮੷CTRΛ༻͍Δͷ͕ɼਫ਼౓ʹͱͯͭ΋ͳ͘ޮ͘ [࣮຿ʢ$53༧ଌʣͱػցֶशίϯϖͷൺֱ :BIPP

    ]
  57. CTRͷಛ௃ྔΤϯδχΞϦϯά 57 • ϋογϡؔ਺Λ༻͍ͯΧςΰϦม਺Λ௿࣍ݩʹຒΊࠐΉ • ࣍ݩ͕ΧςΰϦ਺ΑΓ௿͍ͱিಥ͕ൃੜ [ϚΠΫϩΞυʹ͓͚Δ$53༧ଌ΁ͷऔΓ૊Έ MicroAd)] Feature Hashing

  58. CTRʹ͓͚ΔෆۉߧσʔλͷऔΓѻ͍ 58 • ΠϯϓϨογϣϯʹର͢ΔΫϦοΫ਺ͷׂ߹͸গͳ͍ • ෛྫͷΞϯμʔαϯϓϦϯά͕ඞਢ • ҎԼͷࣜͰิਖ਼ [ϚΠΫϩΞυʹ͓͚Δ$53༧ଌ΁ͷऔΓ૊Έ MicroAd)

    Pozzolo+ 2015]
  59. ࠓ೔ͷྲྀΕ 59 ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

  60. ײ૝ 60 • Ϣʔβʔߦಈ͸ϢʔβʔҙਤΛ൓ө͢Δ͕͹Β͖͕ͭେ͖ͦ͏ ˠɹػցֶश͸Մೳ͕ͩސ٬ຖͷࡉ͔͍ௐ੔͕ॏཁ • ػցֶशʹΑΔվળͷ༨஍͸͋Δͱࢥ͏

  61. ͦͷଞ 61 • Time-Aware Prospective Modeling of Users for Online

    Display Advertising [Gligorijevic + AdKDD 2019] (Yahoo) • Latent Cross: Making Use of Context in Recurrent Recommender Systems [Beutel + WSDM 2019] (Google) • Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks [Smirnova + RecSys 2017] (Criteo) • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona? Recommendation in a Two-Sided Travel Marketplace [Wu + SIGIR 2017] (Airbnb)
  62. اۀͷڭ܇ 62 • Ad Click Prediction: a View from the

    Trenches [McMahan + KDD 2013] (Google) • Practical Lessons from Predicting Clicks on Ads at Facebookɹ[He + AdKDD 2014] (Facebook) • 150 Successful Machine Learning Models: 6 Lessons Learned at Booking.comɹ[Bernandi + KDD 2019] (Booking.com) • Applying Deep Learning for Airbnb Search [Halder+ KDD 2019] (Airbnb)
  63. ࢿྉͷ঺հʢਪનʣ 63 • Recommender Systems Handbook • RecSys 2020 Tutorial:

    Feature Engineering for Recommender Systems • ʮΦϯϥΠϯ޿ࠂؔ࿈ͷ࿦จΛຊ͘Β͍ࡶʹ঺հ͢ΔAdKDDฤʯ • ʮΦϯϥΠϯ޿ࠂʹ͓͚Δ$53$73ਪఆؔ܎ͷ࿦จΛຊ͘Β͍ࡶʹ঺ հ͢Δʯ • CyberAgent Developers Blog • AWS Recommendation Engine Seminar ࢀՃϨϙʔτʢલ൒ʣ • DeepCTR-Torch
  64. ࢿྉͷ঺հʢςετʣ 64 • Trustworthy Online Controlled Experiments: A Practical Guide

    to A/B Testing • ʮւ֎ͷ༗໊*5اۀͷ"#ςετϒϩά·ͱΊʯ • Innovating Faster on Personalization Algorithms at Netflix Using Interleaving • A Dirty Dozen: Twelve Common Metric Interpretation Pitfalls in Online Controlled Experiments • Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained • Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments
  65. ࢀߟจݙɾࢀߟࢿྉ 65 • Maximizing the Engagement: Exploring New Signals of

    Implicit Feedback in Music Recommendations • Intention Nets: Psychology-Inspired User Choice Behavior Modeling for Next-Basket Prediction • Shopper intent prediction from clickstream e‑commerce data with minimal browsing information • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona? Recommendation in a Two-Sided Travel Marketplace • A better clickthrough rate: How Pinterest upgraded everyone’s favorite engagement metric • Real-time User Signal Serving for Feature Engineering • Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction • Deep Interest Network for Click-Through Rate Prediction • Opportunity model for e-commerce recommendation: right product; right time • Temporal-Contextual Recommendation in Real-Time • Learning Efficient Representations of Mouse Movements to Predict User Attention • Time-Aware Prospective Modeling of Users for Online Display Advertising • Understanding Consumer Journey using Attention based Recurrent Neural Networks • User Response Prediction in Online Advertising • Predicting Shopping Behavior with Mixture of RNNs • Beyond Dwell Time: Estimating Document Relevance from Cursor Movements and other Post-click Searcher Behavior • AliMe Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience • Reinforcement Learning for User Intent Prediction in Customer Service Bots • AntProphet: an Intention Mining System behind Alipay’s Intelligent Customer Service Bot • Predicting Intent Using Activity Logs: How Goal Specificity and Temporal Range Affect User Behavior