user-behaviour-vol1 - Speaker Deck

Slide 1

Slide 1 text

Ϣʔβʔߦಈ༧ଌʹؔ͢ΔݚڀͷαʔϕΠ ߴ໦ࢤ࿠ 1

Slide 2

Slide 2 text

͜ͷαʔϕΠͰ΍Δ͜ͱ 2 w Ϣʔβʔߦಈܥྻ͔ΒϢʔβʔߦಈ༧ଌʹؔ͢Δݚڀͷ঺հ w Ϣʔβʔߦಈ༧ଌͱͦͷࢦඪͷ঺հ w Ϣʔβʔߦಈͱͯ͠ͲͷΑ͏ͳ΋ͷ͕࢖ΘΕ͍ͯΔ͔঺հ w ϢʔβʔΤϯήʔδϝϯτʹؔ͢Δݚڀͷྫͷ঺հ঺հ w Ϣʔβʔ͕͍ͭԿΛ͢Δ͔Λ༧ଌ͢Δݚڀͷྫͷ঺հ *Ҿ༻͕ͳ͍΋ͷ͸֘౰࿦จ͔ΒҾ༻

Slide 3

Slide 3 text

ࠓ೔ͷྲྀΕ 3 ̍ɽϢʔβʔߦಈ༧ଌͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

Slide 4

Slide 4 text

ࠓ೔ͷྲྀΕ 4 ̍ɽϢʔβʔߦಈ༧ଌͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

Slide 5

Slide 5 text

໨త͋Γ͖ͷϢʔβʔߦಈ༧ଌ 5 ໨త Ϣʔβʔߦಈ

Slide 6

Slide 6 text

Ͳ͏΍ͬͯଌΔͷ͔ʁ 6 ܭଌ ʢྫɿΫϦοΫ਺ʣ ج४ ʢྫɿ$53ʣ ,1* ʢྫɿ$73ʣ [Tutorial on Online User Engagement KDD 2020]

Slide 7

Slide 7 text

ྑ͍ධՁࢦඪͱ͸ʁ 7 • SensitivityɿվળΛͪΌΜͱݕ஌Ͱ͖Δ͔ʁ • Trustworthinessɿ݁Ռ͸৴པͰ͖Δ͔ʁ • Efficiencyɿܭࢉɾܭଌͷίετ͕௿͍͔ʁ • Debuggability and ActionabilityɿࢦඪͷมԽͷཧ༝ͷಛఆ͕Մೳ͔ʁ • Interpretability and DirectionalityɿࢦඪͷมԽ͸໨తΛୡ੒͢Δ͔ʁ [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]

Slide 8

Slide 8 text

Overall Evaluation Criteria (OEC) 8 • ࠷ऴతͳ໨తͱͳΔࢦඪ • DirectivityͱSensitivity͕ॏཁ • ͜ΕΒ2ͭΛຬͨ͢ͷ͸ࠔ೉ • ྫɿ1Ϣʔβ͋ͨΓͷฏۉऩӹ [Dmitriev+ KDD 2017] [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]

Slide 9

Slide 9 text

ͦͷଞͷࢦඪ 9 • Guardrail Metricsɿ੍໿৚݅ • Operational Metricsɿσόοά༻ͷࢦඪ • Data Quality Metricsɿσʔλ͕৴པͰ͖Δ͜ͱΛอূ͢Δࢦඪ [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]

Slide 10

Slide 10 text

ࠓ೔ͷྲྀΕ 10 ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

Slide 11

Slide 11 text

ΫϦοΫ 11 • ඇৗʹΑ͘࢖ΘΕΔϢʔβʔߦಈ • ޡΫϦοΫͷՄೳੑ͕͋ΔʢಛʹεϚϗʣ • $MJDL%FQUI • ϢʔβʔΤϯήʔδϝϯτɹPSɹ໎͍ͬͯΔ • $MJDL5ISPVHI3BUF • ޿ࠂͷධՁࢦඪͱͯ͠࢖ΘΕΔ [Tolomei + 2018]

Slide 12

Slide 12 text

Ϛ΢ευϥοάɾϖʔδεΫϩʔϧ 12 ɾυϥοάͱεΫϩʔϧʹҙਤ͕ग़Δ ɾεϚϗͰ͸ಘΒΕͳ͍ ୅ΘΓʹεϫΠϓ ॎυϥοά ԣυϥοά υϥοά͕஗͍ ؔ࿈ੑ × ○ ○ εΫϩʔϧྔ εΫϩʔϧස౓ εΫϩʔϧ଎౓ ؔ࿈ੑ × ○ ○ [Guo & Agichtein WWW 2012, Arapakis & Leiva SIGIR 2020] [Guo+ SIGIR 2013, Lagun+ SIGIR 2014] ݕࡧ݁Ռͷจॻ͕ཉ͍͠৘ใͱؔ܎ͯ͠ ͍Δ͔͕Χʔιϧͷಈ͖͔ΒΘ͔Δ

Slide 13

Slide 13 text

଺ࡏ࣌ؒʢDwell Timeʣ 13 [Yi+ RecSys 2014], [Lu+ WWW 2019] • ଺ࡏ࣌ؒ͸ϢʔβʔҙਤΛΑ͘൓ө͍ͯ͠Δ • UIʹґଘ͢Δ • ΦϑϥΠϯͷ܇࿅࣌ʹ͔͠࢖͑ͳ͍ [Lalmas + KDD 2015] [Ouyang+ KDD 2019] %XFMM5JNF 8FCQBHF [Yom-Tovi+ BigData 2013], [Yi+ RecSys 2014] • ϖʔδ΍σόΠε͝ͱʹ͹Βͭ͘

Slide 14

Slide 14 text

஫ҙ఺·ͱΊ 14 • Ϣʔβʔߦಈ͸ϢʔβʔɾσόΠεɾϖʔδຖʹ͹Βͭ͘ ˠ֤ཁૉຖʹਖ਼نԽͨ͠ྔΛ࢖༻͢Δͷ͕ॏཁ • ΦϑϥΠϯͱΦϯϥΠϯͰ؀ڥ͕ҟͳΔ ˠ؀ڥมಈΛߟྀͨ͠ΦϑϥΠϯͰͷֶश͕ॏཁ

Slide 15

Slide 15 text

Ϣʔβʔߦಈ͔Βͷࢦඪͷ࡞੒ 15 ̍ɽૉ๿ͳ؍࡯͔ΒԾઆΛཱͯΔ ྫɿ଺ࡏ͕࣌ؒ௕͍͜ͱ͸Ϣʔβʔ͕ຬ଍ͯ͠Δ͜ͱΛද͢ ̎ɽ࣮ݧΛܭը͠ɼԾઆΛཱͯɼԾઆΛݕূ͢Δ ྫɿຬ଍౓ͷαʔϕΠͱ଺ࡏ࣌ؒͷܭଌΛߦ͏ ̏ɽΦϯϥΠϯͷࢦඪΛઃܭ͠ɼͦͷଥ౰ੑΛݕূ͢Δ ྫɿ଺ࡏ͕࣌ؒ30ඵҎ্ΛϢʔβʔ͕ຬ଍͍ͯ͠Δͱ൑அ͢Δ [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments, KDD 2019]

Slide 16

Slide 16 text

͍͔ͭ͘ͷࢦඪΛ૊Έ߹ΘͤΔ 16 ࠶ݕࡧ͞Ε͔ͨʁ ଺ࡏ͕࣌ؒ௕͍͔ʁ :&4 [Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments KDD 2019]

Slide 17

Slide 17 text

ࠓ೔ͷྲྀΕ 17 ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

Slide 18

Slide 18 text

཭୤ͦ͠͏ͳϢʔβʔͷಛ௃ 18 ཭୤ͦ͠͏ͳϢʔβʔ • αʔϏεΛ࢖͏ͭ΋Γ͕͕͋ͬͨఘΊͨʢ୹ظʣ • αʔϏεΛ࢖͍͕ͬͯͨ࢖Θͳ͘ͳͬͨʢ௕ظʣ ˠɹԿ͔͠ΒͷཁҼ͕αʔϏεͷັྗΛଛͳΘͤͨ ˠɹϢʔβʔΤϯήʔδϝϯτͷ໰୊

Slide 19

Slide 19 text

ϢʔβʔΤϯήʔδϝϯτͷࢦඪ 19 • Click Depth IndexɿϖʔδϏϡʔ • Duration IndexɿαΠτ଺ࡏ࣌ؒ • Interaction IndexɿߪೖɾΞοϓϩʔυ • Recency Indexɿස౓ • Loyalty Indexɿ௕ظؒ [Tutorial on Online User Engagement KDD 2020]

Slide 20

Slide 20 text

ηογϣϯຖͷࢦඪ 20 intra-session metrics 7JTJU 7JTJU 7JTJU inter-session metrics long-term value metrics [Tutorial on Online User Engagement KDD 2020]

Slide 21

Slide 21 text

Ϣʔβʔຬ଍౓༧ଌ 21 • Jointly Leveraging Intent and Interaction Signals to Predict User Satisfaction with Slate Recommendations [Mehrotra+ WWW 2019] • 16ͷϢʔβʔߦಈ͔ΒϢʔβʔຬ଍౓Λ༧ଌʢSpotifyʣ • ΠϯλϏϡʔɼαʔϕΠɼϊϯύϥϕΠζͰϢʔβʔͷҙਤΛಉఆ • ϢʔβʔҙਤΛߟྀͨ͠΄͏͕ΑΓΑ͘Ϣʔβʔຬ଍౓Λ༧ଌͰ͖ͨ • Ϣʔβʔҙਤ͝ͱʹॏཁͱͳΔϢʔβʔߦಈ͕͹Βͭ͘

Slide 22

Slide 22 text

22 ࢀߟ

Slide 23

Slide 23 text

Ϣʔβʔͷߪങ཭୤༧ଌ 23 • Predicting Shopping Behavior with Mixture of RNNs [Toh+ SIGIR 2017] • ߪങ͢Δ͔൱͔ɼ·ͨ͸Ӿཡ͍ͯ͠Δ͚͔ͩΛ༧ଌʢָఱʣ • ΫϦοΫετϦʔϜσʔλɹ→ɹ଺ࡏ࣌ؒͱϖʔδλΠϓͷϖΞͷྻ • RNNͰߴ͍ਫ਼౓Ͱ༧ଌͰ͖Δ͜ͱΛ֬ೝ • ߪങ͢Δ͔൱͔ͷ൑அ͸Ӿཡ͍ͯ͠Δ͔൱͔ͷ൑அΑΓ೉͍͠

Slide 24

Slide 24 text

ྫɿCart Abandonment Rate Statistics 24 [44 Cart Abandonment Rate Statistics by Baymard Institute]

Slide 25

Slide 25 text

25 ࢀߟ

Slide 26

Slide 26 text

26 ࢀߟ

Slide 27

Slide 27 text

໨తࢥߟͳϢʔβʔͷ෼ੳɾ༧ଌ 27 • Predicting Intent Using Activity Logs: How Goal Specificity and Temporal Range Affect User Behavior [Cheng+ WWW 2017] • PinterestϢʔβʔ͕໨తࢥߟ͔ɼͦΕ͕ߦಈ͔Β༧ଌͰ͖Δ͔ • ໨తࢤ޲ͷ৔߹ɼݕࡧ͕ଟ͘ෳࡶͰɼݕࡧʹ͔͔Δ·Ͱ͕ૣ͘ɼݟΔ ίϯςϯπ͸গͳ͍͕ΑΓ۩ମతͰɼը૾ͷιʔεʹඈͿ܏޲͕͋Δ • Ӿཡɼ֦େɼΫϦοΫɼݕࡧͳͲͷߦಈ͔ΒϥϯμϜϑΥϨετͰ Ϣʔβʔ͕໨తࢥߟ͔Λ༧ଌ • ݕࡧ͕༧ଌʹॏཁͰɼ࠷ॳͷ෼ͷߦಈͰ໨తࢥߟ͔͕༧ଌͰ͖ͨ

Slide 28

Slide 28 text

28 ࢀߟ

Slide 29

Slide 29 text

νϟʔϯϨʔτ༧ଌ 29 • I Know You’ll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application [Yang+ KDD 2018] • SnapϢʔβʔͷνϟʔϯϨʔτͷ༧ଌ • ϢʔβʔΛΫϥελϦϯάͰηάϝϯτΘ͚ͯ͠཭୤཰͕ߴ͍Ϣʔ βʔΛಛఆ͠ɼAttention͖ͭLSTMͰνϟʔϯϨʔτΛ༧ଌ

Slide 30

Slide 30 text

30 ࢀߟ

Slide 31

Slide 31 text

31 ࢀߟ

Slide 32

Slide 32 text

ࠓ೔ͷྲྀΕ 32 ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

Slide 33

Slide 33 text

จ຺͖ͭਪન 33 ௨ৗͷਪન จ຺͖ͭਪન User Item User Item Context

Slide 34

Slide 34 text

CTR༧ଌ 34 ɾ$53ΫϦοΫ਺ɹɹ޿ࠂදࣔճ਺ ɾ6TFSº*UFNº5JNFΛݩʹΫϦοΫ͢Δ͔Λ൑அ͢Δೋ஋෼ྨ [࣮຿ʢ$53༧ଌʣͱػցֶशίϯϖͷൺֱ :BIPP ]

Slide 35

Slide 35 text

Ϣʔβʔͷ࣭໰ͷ༧ଌ 35 • Reinforcement Learning for User Intent Prediction in Customer Service Bots [Chen+ SIGIR 2019] • νϟοτϘοτͷͨΊͷਪનΞϧΰϦζϜʢAnt financialʣ • ࣭໰͞ΕΔલʹߦಈཤྺ͔Β࣭໰Λ༧ଌ͠ɼ࣭໰ީิΛఏҊ ʢUser Intent Predictionʣ

Slide 36

Slide 36 text

ઃఆ 36 • 6TFSJOUFOUQSFEJDUJPOΛϚϧίϑܾఆաఔͱͯ͠ఆࣜԽ͠3-Ͱֶश • ࣭໰จͷఏҊΛUPQ/ਪન໰୊ͱͯ͠ఆࣜԽ

Slide 37

Slide 37 text

ख๏ 37 • CTR༧ଌ • ϢʔβʔͷΫϦοΫܥྻͱϢʔβʔ৘ใΛೖྗͱͯ͠࢖༻ • લऀ͸CNNɼޙऀ͸FCNNͰಛ௃ྔΛ࡞੒͠ɼͦΕΒΛFCNN΁ೖྗ • Question popularity • աڈҰఆظؒʹ࣭໰͕ΫϦοΫ͞Εͨઈର਺ͱදࣔ͞Εͨճ਺͔Βɼ ࠓΑ͘ฉ͔Ε͍ͯΔ࣭໰Λࢉग़ • Question Diversity • ݱࡏͷ࣭໰ͱੲͷ࣭໰ͷྨࣅ౓͔Β࣭໰ͷଟ༷ੑΛܭࢉ • ͜ΕΒͷείΞΛ࢖ͬͯtop-Nͷ࣭໰ީิΛϥϯΫ͚ͮ

Slide 38

Slide 38 text

݁Ռ 38 • ΦϑϥΠϯͷධՁͰϕʔεϥΠϯΛ૬ରతʹվળ • AntͷαʔϏεϘοτͷ೔ͷABςετͰCTRΛ૬ରతʹվળ

Slide 39

Slide 39 text

࣮ࡍʹγεςϜͱͯ͠ӡ༻͞Ε͍ͯΔ 39 • AntProphet: an Intention Mining System behind Alipay’s Intelligent Customer Service Bot [Chen+ IJCAI 2020 (demo)] • Ϣʔβʔͷ࣭໰ͷʹରԠ • ଟ͘ͷ࣭໰͕ϢʔβʔߦಈͷΈ͔Β ਪଌͰ͖Δ͜ͱΛ֬ೝ

Slide 40

Slide 40 text

ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ 40 • Deep Interest Network for Click-Through Rate Prediction [Zhou + KDD 2018] • AttentionΛ༻͍Δ͜ͱͰϢʔβʔͷΞΠςϜ΁ͷؔ৺Λදݱͨ͠ CTR༧ଌʢAlibabaʣ

Slide 41

Slide 41 text

ಛ௃ྔͷ࡞੒ 41 • ಛ௃ྔͷཁૉຖͷຒΊࠐΈͷ݁߹ΛϓʔϦϯάͯ͠ߦಈಛ௃ྔΛ࡞੒ • ͦͷଞͷಛ௃ྔͱ݁߹ͯ͠ฏ׈Խͨ͠΋ͷΛ//ʹೖྗ

Slide 42

Slide 42 text

Deep Interest Network 42 • ީิͷ޿ࠂͱϢʔβʔߦಈʹ͍ͭͯͷattentionΛಋೖ

Slide 43

Slide 43 text

࣮ݧͱ݁Ռ 43 • N-1εςοϓ·Ͱͷߦಈσʔλ͔Β/εςοϓͷߦಈΛ༧ଌ • ABςετͷ݁ՌɼCTR͕ɼΠϯϓϨογϣϯऩӹ͕վળ

Slide 44

Slide 44 text

ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ 44 • User Behavior Retrieval for Click-Through Rate Prediction [Qin + SIGIR 2020] • ௕͍ϢʔβʔߦಈܥྻΛ׆༻͢ΔCTR༧ଌʢAlibabaʣ • AttentionϕʔεͷωοτϫʔΫͰੲͷߦಈཤྺ͔Βݕࡧ • ͋Δจ຺Ͱ࠷΋͋Γ͏ΔUser-ItemͷϖΞΛ༧ଌ

Slide 45

Slide 45 text

ख๏ 45 • ߦಈܥྻʹΫΤϦΛ౤͛ͯ࠷΋ؔ࿈͢ΔϢʔβʔߦಈΛऔಘ • ϢʔβʔߦಈΛdocumentɼ֤ಛ௃Λtermͱͯ͠දݱ͔ͯͦ͜͠Β#. ͰϢʔβʔߦಈΛݕࡧɼ͜ΕΛREINFORCEͰֶश • attentionϕʔεͷNNΛ༻͍ͨର਺໬౓࠷େԽͰ༧ଌ

Slide 46

Slide 46 text

࣮ݧͱ݁Ռ 46 • TmallɼTaobaoɼAlipayͷϢʔβߦಈΛ༧ଌ • શମͷͷߦಈΛ࢖͏ͱAlipayͰAUC͕ɼlog loss͕վળ • શߦಈΛ࢖͏ͱɼఏҊ๏͸ͦͷ͏͔ͪ͠࢖ΘͣɼAUC͸େࠩͳ͍ ͕ɼlog loss͸େ͖͘ݮͬͨ

Slide 47

Slide 47 text

ϢʔβʔߦಈܥྻΛ׆༻ͨ͠CTR༧ଌ 47 • Practice on Long Sequential User Behavior Modeling for Click- Through Rate Prediction [Pi+ KDD 2019] • ௕͍ϢʔβʔߦಈܥྻΛར༻ͨ͠ΦϯϥΠϯ޿ࠂCTR༧ଌͷվળ ʢAlibabaʣ • Neural Turing MachineΛར༻͠storageͷ੍໿ͱlatencyͷ੍໿ʹରԠ

Slide 48

Slide 48 text

ϞσϧͷΠϝʔδਤ 48 w ੜͷߦಈͱ/5.ʹهԱ͞Εͨ৘ใΛ྆ํอ࣋ͯ͠ؼೲతਪ࿦Λߦ͏

Slide 49

Slide 49 text

݁Ռ 49 • AlibabaͷΦϯϥΠϯσΟεϓϨΠ޿ࠂͷσʔλΛ࢖༻ʢΠϯϓϨογ ϣϯͷϩάͱɼΫϦοΫ͔ͨ͠൱͔ͷσʔλʣ • 49೔෼ͷσʔλ͔Β࣍ͷ೔ͷ΋ͷΛ༧ଌ • 2019-03-30͔Β2019-05-10·ͰͷظؒͰABςετΛ࣮ࢪ • warm upͷͨΊʹɼ120೔ؒΦϑϥΠϯͰֶश • ABςετͰCTR͕7.5%૿Ճ͠ΠϯϓϨογϣϯऩӹ͕6%૿Ճ

Slide 50

Slide 50 text

ద੾ͳλΠϛϯάͰద੾ͳ΋ͷΛਪન 50 • Temporal-Contextual Recommendation in Real-Time [Ma+ KDD 2020] • ֊૚తͳRNNΛ࢖͏͜ͱͰҙਤͷมԽΛଊ͑ͯɼద੾ͳλΠϛϯάͰద ੾ͳ΋ͷΛਪન͢ΔϞσϧʢAmazonʣ

Slide 51

Slide 51 text

શମͷγεςϜ 51 • ηογϣϯ಺ͷRNNͱηογϣϯؒͷRNNΛཅʹ۠ผ͠ɼηογϣϯ ಺ͷRNN͸౎౓Ϧηοτ͢Δ

Slide 52

Slide 52 text

࣮ݧ݁Ռ 52 • ϢʔβʔߦಈΛ࢖ΘͣҰͭલͷΞΠςϜ͚ͩͰ΋͋Δఔ౓͏·͍͘͘ • Ϟσϧ͕Ͱ͔͍ͱੑೳࣗମʹվળ͸ͳ͍͕ૣ͞Ͱେ͖͘উΔ • ίʔϧυελʔτʹͦͦ͜͜ରԠͰ͖Δ • ΞΠςϜ਺͕ଟ͍࣌ʹ͸ॏ఺αϯϓϦϯά͸༗ޮ

Slide 53

Slide 53 text

ϢʔβʔͷߪങߦಈΛཅʹϞσϦϯά 53 • Opportunity Models for E-commerce Recommendation: Right Product, Right Time [Wang & Zhang SIGIR 2013] • ੜଘ࣌ؒ෼ੳʹΑΓ͋Δ࣌ؒͰ͋Δ঎඼ΛϢʔβʔ͕ങ͏֬཰Λදݱ • shop.comͰͷߪങߦಈͷ༧ଌʹ੒ޭ͠CVRͱϢʔβʔຬ଍౓΋޲্

Slide 54

Slide 54 text

ύʔνΣεϑΝωϧΛҙࣝͨ͠Ϟσϧ 54 • Understanding Consumer Journey using Attention based Recurrent Neural Networks [Zhou + KDD 2019] • ύʔνΣεϑΝωϧͷͲ͜ʹ͍Δ͔Λֶश͢Δattention͖ͭRNNʢYahooʣ

Slide 55

Slide 55 text

ϩδεςΟοΫճؼʁ 55 ɾSimple and Scalable Response Prediction for Display Advertising [Chapelle+ 2014] (Criteo) ɾAd Click Prediction: a View from the Trenches [McMahan + KDD 2013] (Google) ɾϚΠΫϩΞυʹ͓͚ΔCTR༧ଌ΁ͷऔΓ૊Έ

Slide 56

Slide 56 text

࣮຿ʹ͓͚ΔCTR༧ଌͱKaggle 56 • KaggleͰΑ͘༻͍ΒΕΔ౷ܭྔ΍௚લͷϩάΛ࢖༻ͨ͠ಛ௃ྔ͸ɼਫ਼ ౓վળʹͭͳ͕Δ͕ΦϯϥΠϯਪ࿦؀ڥ੔උ͕େม • Target EncodingͰ࣮੷CTRΛ༻͍Δͷ͕ɼਫ਼౓ʹͱͯͭ΋ͳ͘ޮ͘ [࣮຿ʢ$53༧ଌʣͱػցֶशίϯϖͷൺֱ :BIPP ]

Slide 57

Slide 57 text

CTRͷಛ௃ྔΤϯδχΞϦϯά 57 • ϋογϡؔ਺Λ༻͍ͯΧςΰϦม਺Λ௿࣍ݩʹຒΊࠐΉ • ࣍ݩ͕ΧςΰϦ਺ΑΓ௿͍ͱিಥ͕ൃੜ [ϚΠΫϩΞυʹ͓͚Δ$53༧ଌ΁ͷऔΓ૊Έ MicroAd)] Feature Hashing

Slide 58

Slide 58 text

CTRʹ͓͚ΔෆۉߧσʔλͷऔΓѻ͍ 58 • ΠϯϓϨογϣϯʹର͢ΔΫϦοΫ਺ͷׂ߹͸গͳ͍ • ෛྫͷΞϯμʔαϯϓϦϯά͕ඞਢ • ҎԼͷࣜͰิਖ਼ [ϚΠΫϩΞυʹ͓͚Δ$53༧ଌ΁ͷऔΓ૊Έ MicroAd) Pozzolo+ 2015]

Slide 59

Slide 59 text

ࠓ೔ͷྲྀΕ 59 ̍ɽϢʔβʔߦಈ༧ଌͷ֓ཁͱͦͷࢦඪ ̎ɽϢʔβʔߦಈ ̏ɽϢʔβʔΤϯήʔδϝϯτ ̐ɽจ຺͖ͭਪનɾ$53༧ଌ ̑ɽ·ͱΊɾͦͷଞ

Slide 60

Slide 60 text

ײ૝ 60 • Ϣʔβʔߦಈ͸ϢʔβʔҙਤΛ൓ө͢Δ͕͹Β͖͕ͭେ͖ͦ͏ ˠɹػցֶश͸Մೳ͕ͩސ٬ຖͷࡉ͔͍ௐ੔͕ॏཁ • ػցֶशʹΑΔվળͷ༨஍͸͋Δͱࢥ͏

Slide 61

Slide 61 text

ͦͷଞ 61 • Time-Aware Prospective Modeling of Users for Online Display Advertising [Gligorijevic + AdKDD 2019] (Yahoo) • Latent Cross: Making Use of Context in Recurrent Recommender Systems [Beutel + WSDM 2019] (Google) • Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks [Smirnova + RecSys 2017] (Criteo) • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona? Recommendation in a Two-Sided Travel Marketplace [Wu + SIGIR 2017] (Airbnb)

Slide 62

Slide 62 text

اۀͷڭ܇ 62 • Ad Click Prediction: a View from the Trenches [McMahan + KDD 2013] (Google) • Practical Lessons from Predicting Clicks on Ads at Facebookɹ[He + AdKDD 2014] (Facebook) • 150 Successful Machine Learning Models: 6 Lessons Learned at Booking.comɹ[Bernandi + KDD 2019] (Booking.com) • Applying Deep Learning for Airbnb Search [Halder+ KDD 2019] (Airbnb)

Slide 63

Slide 63 text

ࢿྉͷ঺հʢਪનʣ 63 • Recommender Systems Handbook • RecSys 2020 Tutorial: Feature Engineering for Recommender Systems • ʮΦϯϥΠϯ޿ࠂؔ࿈ͷ࿦จΛຊ͘Β͍ࡶʹ঺հ͢ΔAdKDDฤʯ • ʮΦϯϥΠϯ޿ࠂʹ͓͚Δ$53$73ਪఆؔ܎ͷ࿦จΛຊ͘Β͍ࡶʹ঺ հ͢Δʯ • CyberAgent Developers Blog • AWS Recommendation Engine Seminar ࢀՃϨϙʔτʢલ൒ʣ • DeepCTR-Torch

Slide 64

Slide 64 text

ࢿྉͷ঺հʢςετʣ 64 • Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing • ʮւ֎ͷ༗໊*5اۀͷ"#ςετϒϩά·ͱΊʯ • Innovating Faster on Personalization Algorithms at Netflix Using Interleaving • A Dirty Dozen: Twelve Common Metric Interpretation Pitfalls in Online Controlled Experiments • Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained • Challenges, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments

Slide 65

Slide 65 text

ࢀߟจݙɾࢀߟࢿྉ 65 • Maximizing the Engagement: Exploring New Signals of Implicit Feedback in Music Recommendations • Intention Nets: Psychology-Inspired User Choice Behavior Modeling for Next-Basket Prediction • Shopper intent prediction from clickstream e‑commerce data with minimal browsing information • How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona? Recommendation in a Two-Sided Travel Marketplace • A better clickthrough rate: How Pinterest upgraded everyone’s favorite engagement metric • Real-time User Signal Serving for Feature Engineering • Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction • Deep Interest Network for Click-Through Rate Prediction • Opportunity model for e-commerce recommendation: right product; right time • Temporal-Contextual Recommendation in Real-Time • Learning Efficient Representations of Mouse Movements to Predict User Attention • Time-Aware Prospective Modeling of Users for Online Display Advertising • Understanding Consumer Journey using Attention based Recurrent Neural Networks • User Response Prediction in Online Advertising • Predicting Shopping Behavior with Mixture of RNNs • Beyond Dwell Time: Estimating Document Relevance from Cursor Movements and other Post-click Searcher Behavior • AliMe Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience • Reinforcement Learning for User Intent Prediction in Customer Service Bots • AntProphet: an Intention Mining System behind Alipay’s Intelligent Customer Service Bot • Predicting Intent Using Activity Logs: How Goal Specificity and Temporal Range Affect User Behavior