user-behaviour-vol1

ユーザー行動予測に関する研究のサーベイ

Transcript

1. Ϣʔβʔߦಈ༧ଌʹؔ͢ΔݚڀͷαʔϕΠ
   ߴ໦ࢤ࿠
   1
   

   View Slide

2. ͜ͷαʔϕΠͰ΍Δ͜ͱ
   2
   w Ϣʔβʔߦಈܥྻ͔ΒϢʔβʔߦಈ༧ଌʹؔ͢Δݚڀͷ঺հ
   w Ϣʔβʔߦಈ༧ଌͱͦͷࢦඪͷ঺հ
   w Ϣʔβʔߦಈͱͯ͠ͲͷΑ͏ͳ΋ͷ͕࢖ΘΕ͍ͯΔ͔঺հ
   w ϢʔβʔΤϯήʔδϝϯτʹؔ͢Δݚڀͷྫͷ঺հ঺հ
   w Ϣʔβʔ͕͍ͭԿΛ͢Δ͔Λ༧ଌ͢Δݚڀͷྫͷ঺հ
   *Ҿ༻͕ͳ͍΋ͷ͸֘౰࿦จ͔ΒҾ༻
   

   View Slide

3. ࠓ೔ͷྲྀΕ
   3
   ̍ɽϢʔβʔߦಈ༧ଌͱͦͷࢦඪ
   ̎ɽϢʔβʔߦಈ
   ̏ɽϢʔβʔΤϯήʔδϝϯτ
   ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
   ̑ɽ·ͱΊɾͦͷଞ
   

   View Slide

4. ࠓ೔ͷྲྀΕ
   4
   ̍ɽϢʔβʔߦಈ༧ଌͱͦͷࢦඪ
   ̎ɽϢʔβʔߦಈ
   ̏ɽϢʔβʔΤϯήʔδϝϯτ
   ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
   ̑ɽ·ͱΊɾͦͷଞ
   

   View Slide

5. ໨త͋Γ͖ͷϢʔβʔߦಈ༧ଌ
   5
   ໨త
   Ϣʔβʔߦಈ
   

   View Slide

6. Ͳ͏΍ͬͯଌΔͷ͔ʁ
   6
   ܭଌ
   ʢྫɿΫϦοΫ਺ʣ
   ج४
   ʢྫɿ$53ʣ
   ,1*
   ʢྫɿ$73ʣ
   [Tutorial on Online User Engagement KDD 2020]
   

   View Slide

7. ྑ͍ධՁࢦඪͱ͸ʁ
   7
   • SensitivityɿվળΛͪΌΜͱݕ஌Ͱ͖Δ͔ʁ
   • Trustworthinessɿ݁Ռ͸৴པͰ͖Δ͔ʁ
   • Efficiencyɿܭࢉɾܭଌͷίετ͕௿͍͔ʁ
   • Debuggability and ActionabilityɿࢦඪͷมԽͷཧ༝ͷಛఆ͕Մೳ͔ʁ
   • Interpretability and DirectionalityɿࢦඪͷมԽ͸໨తΛୡ੒͢Δ͔ʁ
   [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
   

   View Slide

8. Overall Evaluation Criteria (OEC)
   8
   • ࠷ऴతͳ໨తͱͳΔࢦඪ
   
   • DirectivityͱSensitivity͕ॏཁ
   
   • ͜ΕΒ2ͭΛຬͨ͢ͷ͸ࠔ೉
   
   • ྫɿ1Ϣʔβ͋ͨΓͷฏۉऩӹ
   [Dmitriev+ KDD 2017]
   [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
   

   View Slide

9. ͦͷଞͷࢦඪ
   9
   • Guardrail Metricsɿ੍໿৚݅
   
   • Operational Metricsɿσόοά༻ͷࢦඪ
   
   • Data Quality Metricsɿσʔλ͕৴པͰ͖Δ͜ͱΛอূ͢Δࢦඪ
   [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
   

   View Slide

10. ࠓ೔ͷྲྀΕ
    10
    ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ
    

    View Slide

11. ΫϦοΫ
    11
    • ඇৗʹΑ͘࢖ΘΕΔϢʔβʔߦಈ
    • ޡΫϦοΫͷՄೳੑ͕͋ΔʢಛʹεϚϗʣ
    • $MJDL%FQUI
    • ϢʔβʔΤϯήʔδϝϯτɹPSɹ໎͍ͬͯΔ
    • $MJDL5ISPVHI3BUF
    • ޿ࠂͷධՁࢦඪͱͯ͠࢖ΘΕΔ
    [Tolomei + 2018]
    

    View Slide

12. Ϛ΢ευϥοάɾϖʔδεΫϩʔϧ
    12
    ɾυϥοάͱεΫϩʔϧʹҙਤ͕ग़Δ
    ɾεϚϗͰ͸ಘΒΕͳ͍ ୅ΘΓʹεϫΠϓ
    
    ॎυϥοά ԣυϥοά υϥοά͕஗͍
    ؔ࿈ੑ × ○ ○
    εΫϩʔϧྔ εΫϩʔϧස౓ εΫϩʔϧ଎౓
    ؔ࿈ੑ × ○ ○
    [Guo & Agichtein WWW 2012, Arapakis & Leiva SIGIR 2020]
    [Guo+ SIGIR 2013, Lagun+ SIGIR 2014]
    ݕࡧ݁Ռͷจॻ͕ཉ͍͠৘ใͱؔ܎ͯ͠
    ͍Δ͔͕Χʔιϧͷಈ͖͔ΒΘ͔Δ
    

    View Slide

13. ଺ࡏ࣌ؒʢDwell Timeʣ
    13
    [Yi+ RecSys 2014], [Lu+ WWW 2019]
    • ଺ࡏ࣌ؒ͸ϢʔβʔҙਤΛΑ͘൓ө͍ͯ͠Δ
    • UIʹґଘ͢Δ
    • ΦϑϥΠϯͷ܇࿅࣌ʹ͔͠࢖͑ͳ͍
    [Lalmas + KDD 2015]
    [Ouyang+ KDD 2019]
    %XFMM5JNF
    8FCQBHF
    [Yom-Tovi+ BigData 2013], [Yi+ RecSys 2014]
    • ϖʔδ΍σόΠε͝ͱʹ͹Βͭ͘
    

    View Slide

14. ஫ҙ఺·ͱΊ
    14
    • Ϣʔβʔߦಈ͸ϢʔβʔɾσόΠεɾϖʔδຖʹ͹Βͭ͘
    ˠ֤ཁૉຖʹਖ਼نԽͨ͠ྔΛ࢖༻͢Δͷ͕ॏཁ
    • ΦϑϥΠϯͱΦϯϥΠϯͰ؀ڥ͕ҟͳΔ
    ˠ؀ڥมಈΛߟྀͨ͠ΦϑϥΠϯͰͷֶश͕ॏཁ
    

    View Slide

15. Ϣʔβʔߦಈ͔Βͷࢦඪͷ࡞੒
    15
    ̍ɽૉ๿ͳ؍࡯͔ΒԾઆΛཱͯΔ
    ྫɿ଺ࡏ͕࣌ؒ௕͍͜ͱ͸Ϣʔβʔ͕ຬ଍ͯ͠Δ͜ͱΛද͢
    
    ̎ɽ࣮ݧΛܭը͠ɼԾઆΛཱͯɼԾઆΛݕূ͢Δ
    ྫɿຬ଍౓ͷαʔϕΠͱ଺ࡏ࣌ؒͷܭଌΛߦ͏
    
    ̏ɽΦϯϥΠϯͷࢦඪΛઃܭ͠ɼͦͷଥ౰ੑΛݕূ͢Δ
    ྫɿ଺ࡏ͕࣌ؒ30ඵҎ্ΛϢʔβʔ͕ຬ଍͍ͯ͠Δͱ൑அ͢Δ
    [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]
    

    View Slide

16. ͍͔ͭ͘ͷࢦඪΛ૊Έ߹ΘͤΔ
    16
    ࠶ݕࡧ͞Ε͔ͨʁ
    ଺ࡏ͕࣌ؒ௕͍͔ʁ
    :&4
    [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments KDD 2019]
    

    View Slide

17. ࠓ೔ͷྲྀΕ
    17
    ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ
    

    View Slide

18. ཭୤ͦ͠͏ͳϢʔβʔͷಛ௃
    18
    ཭୤ͦ͠͏ͳϢʔβʔ
    • αʔϏεΛ࢖͏ͭ΋Γ͕͕͋ͬͨఘΊͨʢ୹ظʣ
    • αʔϏεΛ࢖͍͕ͬͯͨ࢖Θͳ͘ͳͬͨʢ௕ظʣ
    ˠɹԿ͔͠ΒͷཁҼ͕αʔϏεͷັྗΛଛͳΘͤͨ
    ˠɹϢʔβʔΤϯήʔδϝϯτͷ໰୊
    

    View Slide

19. ϢʔβʔΤϯήʔδϝϯτͷࢦඪ
    19
    • Click Depth IndexɿϖʔδϏϡʔ
    • Duration IndexɿαΠτ଺ࡏ࣌ؒ
    • Interaction IndexɿߪೖɾΞοϓϩʔυ
    • Recency Indexɿස౓
    • Loyalty Indexɿ௕ظؒ
    [Tutorial on Online User Engagement KDD 2020]
    

    View Slide

20. ηογϣϯຖͷࢦඪ
    20
    intra-session metrics
    7JTJU 7JTJU 7JTJU
    inter-session metrics
    long-term value metrics
    [Tutorial on Online User Engagement KDD 2020]
    

    View Slide

21. Ϣʔβʔຬ଍౓༧ଌ
    21
    • Jointly Leveraging Intent and Interaction Signals to Predict User
    Satisfaction with Slate Recommendations [Mehrotra+ WWW 2019]
    
    • 16ͷϢʔβʔߦಈ͔ΒϢʔβʔຬ଍౓Λ༧ଌʢSpotifyʣ
    
    • ΠϯλϏϡʔɼαʔϕΠɼϊϯύϥϕΠζͰϢʔβʔͷҙਤΛಉఆ
    
    • ϢʔβʔҙਤΛߟྀͨ͠΄͏͕ΑΓΑ͘Ϣʔβʔຬ଍౓Λ༧ଌͰ͖ͨ
    
    • Ϣʔβʔҙਤ͝ͱʹॏཁͱͳΔϢʔβʔߦಈ͕͹Βͭ͘
    

    View Slide

22. 22
    ࢀߟ
    

    View Slide

23. Ϣʔβʔͷߪങ཭୤༧ଌ
    23
    • Predicting Shopping Behavior with Mixture of RNNs [Toh+ SIGIR 2017]
    
    • ߪങ͢Δ͔൱͔ɼ·ͨ͸Ӿཡ͍ͯ͠Δ͚͔ͩΛ༧ଌʢָఱʣ
    
    • ΫϦοΫετϦʔϜσʔλɹ→ɹ଺ࡏ࣌ؒͱϖʔδλΠϓͷϖΞͷྻ
    
    • RNNͰߴ͍ਫ਼౓Ͱ༧ଌͰ͖Δ͜ͱΛ֬ೝ
    
    • ߪങ͢Δ͔൱͔ͷ൑அ͸Ӿཡ͍ͯ͠Δ͔൱͔ͷ൑அΑΓ೉͍͠
    

    View Slide

24. ྫɿCart Abandonment Rate Statistics
    24
    [44 Cart Abandonment Rate Statistics by Baymard Institute]
    

    View Slide

25. 25
    ࢀߟ
    

    View Slide

26. 26
    ࢀߟ
    

    View Slide

27. ໨తࢥߟͳϢʔβʔͷ෼ੳɾ༧ଌ
    27
    • Predicting Intent Using Activity Logs: How Goal Specificity and
    Temporal Range Affect User Behavior [Cheng+ WWW 2017]
    
    • PinterestϢʔβʔ͕໨తࢥߟ͔ɼͦΕ͕ߦಈ͔Β༧ଌͰ͖Δ͔
    • ໨తࢤ޲ͷ৔߹ɼݕࡧ͕ଟ͘ෳࡶͰɼݕࡧʹ͔͔Δ·Ͱ͕ૣ͘ɼݟΔ
    ίϯςϯπ͸গͳ͍͕ΑΓ۩ମతͰɼը૾ͷιʔεʹඈͿ܏޲͕͋Δ
    • Ӿཡɼ֦େɼΫϦοΫɼݕࡧͳͲͷߦಈ͔ΒϥϯμϜϑΥϨετͰ
    Ϣʔβʔ͕໨తࢥߟ͔Λ༧ଌ
    • ݕࡧ͕༧ଌʹॏཁͰɼ࠷ॳͷ෼ͷߦಈͰ໨తࢥߟ͔͕༧ଌͰ͖ͨ
    

    View Slide

28. 28
    ࢀߟ
    

    View Slide

29. νϟʔϯϨʔτ༧ଌ
    29
    • I Know You’ll Be Back: Interpretable New User Clustering and Churn
    Prediction on a Mobile Social Application [Yang+ KDD 2018]
    
    • SnapϢʔβʔͷνϟʔϯϨʔτͷ༧ଌ
    • ϢʔβʔΛΫϥελϦϯάͰηάϝϯτΘ͚ͯ͠཭୤཰͕ߴ͍Ϣʔ
    βʔΛಛఆ͠ɼAttention͖ͭLSTMͰνϟʔϯϨʔτΛ༧ଌ
    

    View Slide

30. 30
    ࢀߟ
    

    View Slide

31. 31
    ࢀߟ
    

    View Slide

32. ࠓ೔ͷྲྀΕ
    32
    ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ
    

    View Slide

33. จ຺͖ͭਪન
    33
    ௨ৗͷਪન จ຺͖ͭਪન
    User
    Item
    User
    Item
    Context
    

    View Slide

34. CTR༧ଌ
    34
    ɾ$53ΫϦοΫ਺ɹɹ޿ࠂදࣔճ਺
    ɾ6TFSº*UFNº5JNFΛݩʹΫϦοΫ͢Δ͔Λ൑அ͢Δೋ஋෼ྨ
    [࣮຿ʢ$53༧ଌʣͱػցֶशίϯϖͷൺֱ :BIPP
    ]
    

    View Slide

35. Ϣʔβʔͷ࣭໰ͷ༧ଌ
    35
    • Reinforcement Learning for User Intent Prediction in Customer
    Service Bots [Chen+ SIGIR 2019]
    
    • νϟοτϘοτͷͨΊͷਪનΞϧΰϦζϜʢAnt financialʣ
    • ࣭໰͞ΕΔલʹߦಈཤྺ͔Β࣭໰Λ༧ଌ͠ɼ࣭໰ީิΛఏҊ
    ʢUser Intent Predictionʣ
    

    View Slide

36. ઃఆ
    36
    • 6TFSJOUFOUQSFEJDUJPOΛϚϧίϑܾఆաఔͱͯ͠ఆࣜԽ͠3-Ͱֶश
    • ࣭໰จͷఏҊΛUPQ/ਪન໰୊ͱͯ͠ఆࣜԽ
    

    View Slide

37. ख๏
    37
    • CTR༧ଌ
    • ϢʔβʔͷΫϦοΫܥྻͱϢʔβʔ৘ใΛೖྗͱͯ͠࢖༻
    • લऀ͸CNNɼޙऀ͸FCNNͰಛ௃ྔΛ࡞੒͠ɼͦΕΒΛFCNN΁ೖྗ
    • Question popularity
    
    • աڈҰఆظؒʹ࣭໰͕ΫϦοΫ͞Εͨઈର਺ͱදࣔ͞Εͨճ਺͔Βɼ
    ࠓΑ͘ฉ͔Ε͍ͯΔ࣭໰Λࢉग़
    • Question Diversity
    
    • ݱࡏͷ࣭໰ͱੲͷ࣭໰ͷྨࣅ౓͔Β࣭໰ͷଟ༷ੑΛܭࢉ
    • ͜ΕΒͷείΞΛ࢖ͬͯtop-Nͷ࣭໰ީิΛϥϯΫ͚ͮ
    

    View Slide

38. ݁Ռ
    38
    • ΦϑϥΠϯͷධՁͰϕʔεϥΠϯΛ૬ରతʹվળ
    • AntͷαʔϏεϘοτͷ೔ͷABςετͰCTRΛ૬ରతʹվળ
    

    View Slide

39. ࣮ࡍʹγεςϜͱͯ͠ӡ༻͞Ε͍ͯΔ
    39
    • AntProphet: an Intention Mining
    System behind Alipay’s Intelligent
    Customer Service Bot [Chen+ IJCAI
    2020 (demo)]
    
    • Ϣʔβʔͷ࣭໰ͷʹରԠ
    • ଟ͘ͷ࣭໰͕ϢʔβʔߦಈͷΈ͔Β
    ਪଌͰ͖Δ͜ͱΛ֬ೝ
    

    View Slide

40. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ
    40
    • Deep Interest Network for Click-Through Rate Prediction [Zhou +
    KDD 2018]
    
    • AttentionΛ༻͍Δ͜ͱͰϢʔβʔͷΞΠςϜ΁ͷؔ৺Λදݱͨ͠
    CTR༧ଌʢAlibabaʣ
    

    View Slide

41. ಛ௃ྔͷ࡞੒
    41
    • ಛ௃ྔͷཁૉຖͷຒΊࠐΈͷ݁߹ΛϓʔϦϯάͯ͠ߦಈಛ௃ྔΛ࡞੒
    • ͦͷଞͷಛ௃ྔͱ݁߹ͯ͠ฏ׈Խͨ͠΋ͷΛ//ʹೖྗ
    

    View Slide

42. Deep Interest Network
    42
    • ީิͷ޿ࠂͱϢʔβʔߦಈʹ͍ͭͯͷattentionΛಋೖ
    

    View Slide

43. ࣮ݧͱ݁Ռ
    43
    • N-1εςοϓ·Ͱͷߦಈσʔλ͔Β/εςοϓͷߦಈΛ༧ଌ
    • ABςετͷ݁ՌɼCTR͕ɼΠϯϓϨογϣϯऩӹ͕վળ
    

    View Slide

44. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ
    44
    • User Behavior Retrieval for Click-Through Rate Prediction [Qin +
    SIGIR 2020]
    
    • ௕͍ϢʔβʔߦಈܥྻΛ׆༻͢ΔCTR༧ଌʢAlibabaʣ
    • AttentionϕʔεͷωοτϫʔΫͰੲͷߦಈཤྺ͔Βݕࡧ
    • ͋Δจ຺Ͱ࠷΋͋Γ͏ΔUser-ItemͷϖΞΛ༧ଌ
    

    View Slide

45. ख๏
    45
    • ߦಈܥྻʹΫΤϦΛ౤͛ͯ࠷΋ؔ࿈͢ΔϢʔβʔߦಈΛऔಘ
    • ϢʔβʔߦಈΛdocumentɼ֤ಛ௃Λtermͱͯ͠දݱ͔ͯͦ͜͠Β#.
    ͰϢʔβʔߦಈΛݕࡧɼ͜ΕΛREINFORCEͰֶश
    • attentionϕʔεͷNNΛ༻͍ͨର਺໬౓࠷େԽͰ༧ଌ
    

    View Slide

46. ࣮ݧͱ݁Ռ
    46
    • TmallɼTaobaoɼAlipayͷϢʔβߦಈΛ༧ଌ
    • શମͷͷߦಈΛ࢖͏ͱAlipayͰAUC͕ɼlog loss͕վળ
    • શߦಈΛ࢖͏ͱɼఏҊ๏͸ͦͷ͏͔ͪ͠࢖ΘͣɼAUC͸େࠩͳ͍
    ͕ɼlog loss͸େ͖͘ݮͬͨ
    

    View Slide

47. ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ
    47
    • Practice on Long Sequential User Behavior Modeling for Click-
    Through Rate Prediction [Pi+ KDD 2019]
    
    • ௕͍ϢʔβʔߦಈܥྻΛར༻ͨ͠ΦϯϥΠϯ޿ࠂCTR༧ଌͷվળ
    ʢAlibabaʣ
    
    • Neural Turing MachineΛར༻͠storageͷ੍໿ͱlatencyͷ੍໿ʹରԠ
    

    View Slide

48. ϞσϧͷΠϝʔδਤ
    48
    w ੜͷߦಈͱ/5.ʹهԱ͞Εͨ৘ใΛ྆ํอ࣋ͯ͠ؼೲతਪ࿦Λߦ͏
    

    View Slide

49. ݁Ռ
    49
    • AlibabaͷΦϯϥΠϯσΟεϓϨΠ޿ࠂͷσʔλΛ࢖༻ʢΠϯϓϨογ
    ϣϯͷϩάͱɼΫϦοΫ͔ͨ͠൱͔ͷσʔλʣ
    
    • 49೔෼ͷσʔλ͔Β࣍ͷ೔ͷ΋ͷΛ༧ଌ
    
    • 2019-03-30͔Β2019-05-10·ͰͷظؒͰABςετΛ࣮ࢪ
    
    • warm upͷͨΊʹɼ120೔ؒΦϑϥΠϯͰֶश
    
    • ABςετͰCTR͕7.5%૿Ճ͠ΠϯϓϨογϣϯऩӹ͕6%૿Ճ
    

    View Slide

50. ద੾ͳλΠϛϯάͰద੾ͳ΋ͷΛਪન
    50
    • Temporal-Contextual Recommendation in Real-Time [Ma+ KDD 2020]
    
    • ֊૚తͳRNNΛ࢖͏͜ͱͰҙਤͷมԽΛଊ͑ͯɼద੾ͳλΠϛϯάͰద
    ੾ͳ΋ͷΛਪન͢ΔϞσϧʢAmazonʣ
    

    View Slide

51. શମͷγεςϜ
    51
    • ηογϣϯ಺ͷRNNͱηογϣϯؒͷRNNΛཅʹ۠ผ͠ɼηογϣϯ
    ಺ͷRNN͸౎౓Ϧηοτ͢Δ
    

    View Slide

52. ࣮ݧ݁Ռ
    52
    • ϢʔβʔߦಈΛ࢖ΘͣҰͭલͷΞΠςϜ͚ͩͰ΋͋Δఔ౓͏·͍͘͘
    
    • Ϟσϧ͕Ͱ͔͍ͱੑೳࣗମʹվળ͸ͳ͍͕ૣ͞Ͱେ͖͘উΔ
    
    • ίʔϧυελʔτʹͦͦ͜͜ରԠͰ͖Δ
    
    • ΞΠςϜ਺͕ଟ͍࣌ʹ͸ॏ఺αϯϓϦϯά͸༗ޮ
    

    View Slide

53. ϢʔβʔͷߪങߦಈΛཅʹϞσϦϯά
    53
    • Opportunity Models for E-commerce Recommendation: Right
    Product, Right Time [Wang & Zhang SIGIR 2013]
    
    • ੜଘ࣌ؒ෼ੳʹΑΓ͋Δ࣌ؒͰ͋Δ঎඼ΛϢʔβʔ͕ങ͏֬཰Λදݱ
    
    • shop.comͰͷߪങߦಈͷ༧ଌʹ੒ޭ͠CVRͱϢʔβʔຬ଍౓΋޲্
    

    View Slide

54. ύʔνΣεϑΝωϧΛҙࣝͨ͠Ϟσϧ
    54
    • Understanding Consumer Journey using Attention based Recurrent
    Neural Networks [Zhou + KDD 2019]
    
    • ύʔνΣεϑΝωϧͷͲ͜ʹ͍Δ͔Λֶश͢Δattention͖ͭRNNʢYahooʣ
    

    View Slide

55. ϩδεςΟοΫճؼʁ
    55
    ɾSimple and Scalable Response Prediction for Display Advertising
    [Chapelle+ 2014] (Criteo)
    ɾAd Click Prediction: a View from the Trenches [McMahan + KDD
    2013] (Google)
    ɾϚΠΫϩΞυʹ͓͚ΔCTR༧ଌ΁ͷऔΓ૊Έ
    

    View Slide

56. ࣮຿ʹ͓͚ΔCTR༧ଌͱKaggle
    56
    • KaggleͰΑ͘༻͍ΒΕΔ౷ܭྔ΍௚લͷϩάΛ࢖༻ͨ͠ಛ௃ྔ͸ɼਫ਼
    ౓վળʹͭͳ͕Δ͕ΦϯϥΠϯਪ࿦؀ڥ੔උ͕େม
    
    • Target EncodingͰ࣮੷CTRΛ༻͍Δͷ͕ɼਫ਼౓ʹͱͯͭ΋ͳ͘ޮ͘
    [࣮຿ʢ$53༧ଌʣͱػցֶशίϯϖͷൺֱ :BIPP
    ]
    

    View Slide

57. CTRͷಛ௃ྔΤϯδχΞϦϯά
    57
    • ϋογϡؔ਺Λ༻͍ͯΧςΰϦม਺Λ௿࣍ݩʹຒΊࠐΉ
    
    • ࣍ݩ͕ΧςΰϦ਺ΑΓ௿͍ͱিಥ͕ൃੜ
    [ϚΠΫϩΞυʹ͓͚Δ$53༧ଌ΁ͷऔΓ૊Έ MicroAd)]
    Feature Hashing
    

    View Slide

58. CTRʹ͓͚ΔෆۉߧσʔλͷऔΓѻ͍
    58
    • ΠϯϓϨογϣϯʹର͢ΔΫϦοΫ਺ͷׂ߹͸গͳ͍
    
    • ෛྫͷΞϯμʔαϯϓϦϯά͕ඞਢ
    
    • ҎԼͷࣜͰิਖ਼
    [ϚΠΫϩΞυʹ͓͚Δ$53༧ଌ΁ͷऔΓ૊Έ MicroAd) Pozzolo+ 2015]
    

    View Slide

59. ࠓ೔ͷྲྀΕ
    59
    ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ
    ̎ɽϢʔβʔߦಈ
    ̏ɽϢʔβʔΤϯήʔδϝϯτ
    ̐ɽจ຺͖ͭਪનɾ$53༧ଌ
    ̑ɽ·ͱΊɾͦͷଞ
    

    View Slide

60. ײ૝
    60
    • Ϣʔβʔߦಈ͸ϢʔβʔҙਤΛ൓ө͢Δ͕͹Β͖͕ͭେ͖ͦ͏
    ˠɹػցֶश͸Մೳ͕ͩސ٬ຖͷࡉ͔͍ௐ੔͕ॏཁ
    • ػցֶशʹΑΔվળͷ༨஍͸͋Δͱࢥ͏
    

    View Slide

61. ͦͷଞ
    61
    • Time-Aware Prospective Modeling of Users for Online Display
    Advertising [Gligorijevic + AdKDD 2019] (Yahoo)
    
    • Latent Cross: Making Use of Context in Recurrent Recommender
    Systems [Beutel + WSDM 2019] (Google)
    
    • Contextual Sequence Modeling for Recommendation with
    Recurrent Neural Networks [Smirnova + RecSys 2017] (Criteo)
    
    • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona?
    Recommendation in a Two-Sided Travel Marketplace [Wu + SIGIR
    2017] (Airbnb)
    

    View Slide

62. اۀͷڭ܇
    62
    • Ad Click Prediction: a View from the Trenches [McMahan + KDD 2013]
    (Google)
    
    • Practical Lessons from Predicting Clicks on Ads at Facebookɹ[He +
    AdKDD 2014] (Facebook)
    
    • 150 Successful Machine Learning Models: 6 Lessons Learned at
    Booking.comɹ[Bernandi + KDD 2019] (Booking.com)
    
    • Applying Deep Learning for Airbnb Search [Halder+ KDD 2019]
    (Airbnb)
    

    View Slide

63. ࢿྉͷ঺հʢਪનʣ
    63
    • Recommender Systems Handbook
    
    • RecSys 2020 Tutorial: Feature Engineering for Recommender Systems
    
    • ʮΦϯϥΠϯ޿ࠂؔ࿈ͷ࿦จΛຊ͘Β͍ࡶʹ঺հ͢ΔAdKDDฤʯ
    • ʮΦϯϥΠϯ޿ࠂʹ͓͚Δ$53$73ਪఆؔ܎ͷ࿦จΛຊ͘Β͍ࡶʹ঺
    հ͢Δʯ
    • CyberAgent Developers Blog
    
    • AWS Recommendation Engine Seminar ࢀՃϨϙʔτʢલ൒ʣ
    • DeepCTR-Torch
    
    

    View Slide

64. ࢿྉͷ঺հʢςετʣ
    64
    • Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing
    
    • ʮւ֎ͷ༗໊*5اۀͷ"#ςετϒϩά·ͱΊʯ
    • Innovating Faster on Personalization Algorithms at Netflix Using Interleaving
    
    • A Dirty Dozen: Twelve Common Metric Interpretation Pitfalls in Online
    Controlled Experiments
    
    • Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained
    
    • Challenges, Best Practices and Pitfalls in Evaluating Results of Online
    Controlled Experiments
    

    View Slide

65. ࢀߟจݙɾࢀߟࢿྉ
    65
    • Maximizing the Engagement: Exploring New Signals of Implicit Feedback in Music Recommendations
    
    • Intention Nets: Psychology-Inspired User Choice Behavior Modeling for Next-Basket Prediction
    
    • Shopper intent prediction from clickstream e‑commerce data with minimal browsing information
    
    • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona? Recommendation in a Two-Sided Travel Marketplace
    
    • A better clickthrough rate: How Pinterest upgraded everyone’s favorite engagement metric
    
    • Real-time User Signal Serving for Feature Engineering
    
    • Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction
    
    • Deep Interest Network for Click-Through Rate Prediction
    
    • Opportunity model for e-commerce recommendation: right product; right time
    
    • Temporal-Contextual Recommendation in Real-Time
    
    • Learning Efficient Representations of Mouse Movements to Predict User Attention
    
    • Time-Aware Prospective Modeling of Users for Online Display Advertising
    
    • Understanding Consumer Journey using Attention based Recurrent Neural Networks
    
    • User Response Prediction in Online Advertising
    
    • Predicting Shopping Behavior with Mixture of RNNs
    
    • Beyond Dwell Time: Estimating Document Relevance from Cursor Movements and other Post-click Searcher Behavior
    
    • AliMe Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience
    
    • Reinforcement Learning for User Intent Prediction in Customer Service Bots
    
    • AntProphet: an Intention Mining System behind Alipay’s Intelligent Customer Service Bot
    
    • Predicting Intent Using Activity Logs: How Goal Specificity and Temporal Range Affect User Behavior
    
    

    View Slide