KDD 2020 Marketplace Tutorial Rishabh Part 1

https://sites.google.com/view/kdd20-marketplace-autorecsys/

1. KDD 2020 Tutorial: Advances in Recommender Systems Part A: Recommendations in a Marketplace [Introduction] Rishabh Mehrotra Ben Carterette Senior Research Scientist, Senior Research Manager, Spotify, London Spotify, New York rishabhm@spotify.com benjaminc@spotify.com 23rd August 2020 @erishabh @BenCarterette https://sites.google.com/view/kdd20-marketplace-autorecsys/
2. Introductions Rishabh Mehrotra Sr Research Scientist, Spotify, London Past: • PhD in ML/IR from University College London • BE (Hons) Computer Science, MSc (Hons) Mathematics, BITS Pilani, India • Visiting researcher & intern, Microsoft Research, NYC/Redmond (2015-16) • Co-founder, UserContext.AI • Goldman Sachs Ben Carterette Sr Research Manager, Spotify, New York Past: • Associate Professor, University of Delaware (2008 onwards) • Visiting Researcher, CMU (2010) • Visiting Researcher, Yahoo Labs (2009) • PhD from University of Massachusetts Amherst (2008)
3. Outline 1. Introduction to Marketplaces 2. Optimization Objectives in a Marketplace 3. Methods for Multi-Objective Ranking & Recommendations 4. Leveraging Consumer, Supplier & Content Understanding 5. Industrial Applications
4. Outline 1. Introduction: a. (Quick) Overview of traditional RecSys approaches b. Introduction to Marketplace c. Types & examples of marketplaces d. Recommendation in a marketplace e. Segway into the rest of the tutorial → walk-through of the upcoming sections → linking it all together 2. Optimization Objectives in a Marketplace 3. Methods for Multi-Objective Ranking & Recommendations 4. Leveraging Consumer, Supplier & Content Understanding 5. Industrial Applications
5. Outline 1. Introduction to Marketplaces 2. Optimization Objectives in a Marketplace a. Case studies I - VII b. Families of objectives c. Interplay between Objectives 3. Methods for Multi-Objective Ranking & Recommendations 4. Leveraging Consumer, Supplier & Content Understanding 5. Industrial Applications
6. Outline 1. Introduction to Marketplaces 2. Optimization Objectives in a Marketplace 3. Methods for Multi-Objective Ranking & Recommendations a. Pareto optimality b. Multi-objective models i. Scalarization ii. Multi-task Learning iii. Multi-objective bandits iv. Multi-objective RL 4. Leveraging Consumer, Supplier & Content Understanding 5. Industrial Applications
7. Outline 1. Introduction to Marketplaces 2. Optimization Objectives in a Marketplace 3. Methods for Multi-Objective Ranking & Recommendations 4. Leveraging Consumer, Supplier & Content Understanding a. Consumer Understanding: i. Consumption diversity of users ii. Quantifying and estimating user tolerance iii. Leveraging User intents b. Supplier Understanding: i. Diversity across suppliers c. Content Understanding: i. Personalizing Reward function ii. Query Understanding 5. Industrial Applications
8. Outline 1. Introduction to Marketplaces 2. Optimization Objectives in a Marketplace 3. Methods for Multi-Objective Ranking & Recommendations 4. Leveraging Consumer, Supplier & Content Understanding 5. Industrial Applications a. Recommendation applications b. Search applications
9. Schedule 08:00 - 08:10: Welcome + Introduction 08:10 - 08:30: Part I: Introduction to Marketplaces 08:30 - 09:00: Part II: Optimization Objectives in a Marketplace 09:00 - 09:30: Part III: Methods for Multi-Objective Recommendations 09:30 - 10:00: Break 10:00 - 10:30: Part III: Methods for Multi-Objective Recommendations 10:30 - 11:10: Part IV: Leveraging Consumer, Supplier & Content Understanding 11:10 - 11:40: Part V: Industrial Applications 11:40 - 11:50: Questions & Discussions
10. Outline 1. Introduction: a. (Quick) Overview of traditional RecSys approaches b. Introduction to Marketplace c. Types & examples of marketplaces d. Recommendation in a marketplace
11. Traditional RecSys Approaches
12. Approaches for RecSys Collaborative Filtering, i.e. matrix factorization
13. Approaches for RecSys Collaborative Filtering -- extended, i.e. Tensor factorization AAAI 2010: Collaborative Filtering Meets Mobile Recommendation: A User-Centered Approach
14. Approaches for RecSys Latent variable models RecSys 2015: A probabilistic model for using social networks in personalized item recommendation
15. Approaches for RecSys Neural Embeddings User Embedding
16. Approaches for RecSys Neural Embeddings User Embedding … with Side Information RecSys 2016: Meta-Prod2Vec - Product Embeddings Using Side-Information for Recommendation
17. Approaches for RecSys Neural Embeddings User Embedding … with Side Information Joint User-Item Embedding WSDM 2017: Joint Deep Modeling of Users and Items Using Reviews for Recommendation
18. Approaches for RecSys Neural Collaborative Ranking WWW 2017: Neural Collaborative Filtering
19. Approaches for RecSys Variants of Recommendation Styles: - Short vs long term - Cold start or cohort based - Multi-view & multi-interest models - Mult-task recommendation SIGIR 2012: Modeling the Impact of Short- and Long-Term Behavior on Search Personalization
20. Approaches for RecSys Variants of Recommendation Styles: - Short vs long term - Cold start & cohort based - Multi-view & multi-interest models - Mult-task recommendation SIGIR 2014: Cohort Modeling for Enhanced Personalized Search
21. Approaches for RecSys Variants of Recommendation Styles: - Short vs long term - Cold start or cohort based - Multi-view & multi-interest models - Mult-task recommendation RecSys 2013: Nonlinear Latent Factorization by Embedding Multiple User Interests
22. Approaches for RecSys Variants of Recommendation Styles: - Short vs long term - Cold start or cohort based - Multi-view & multi-interest models - Multi-task recommendation KDD 2018: Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks
23. Approaches for RecSys
24. Approaches for RecSys What do they have in common?
25. Approaches for RecSys What do they have in common? User centric focus
26. Traditional RecSys: User Centric • User centric nature of systems: ◦ Recommendations models catered to users: ▪ user needs ▪ user interests ▪ user behavior & interactions ▪ personalization ◦ Evaluation approaches for user satisfaction ▪ Measuring user engagement ▪ Optimizing for user satisfaction ▪ User centric metrics *WSDM 2018 Tutorial on metrics of user engagement; Mounia Lalmas, et al [link]
27. Is caring about the user enough?
28. Is caring about the user enough? Stakeholders
29. User centric methods is not sufficient to serve recommendations in such scenarios!
30. Introduction to Marketplaces
31. Marketplace Marketplace: Intermediaries that help facilitate economic interaction between two or more sets of agents
32. Marketplaces are everywhere! Marketplaces have existed since humans first began to engage in trade - Village markets: Vegetable & fruits & flowers - Shopping Malls - Stock markets
33. Modern Marketplace
34. Modern Marketplace
35. Modern Marketplace
36. Modern Marketplace
37. Modern Marketplace
38. Modern Marketplace
39. Modern Marketplace
40. Evolution of Marketplaces https://hackernoon.com/not-all-marketplaces-are-created-equal-tales-of-a-marketplace-founder-9fc0fb802706
41. Outline 1. Introduction: a. (Quick) Overview of traditional RecSys approaches b. Introduction to Marketplace c. Types & examples of marketplaces d. Recommendations in a marketplace
42. Marketplaces: Types + Examples
43. Marketplace Types Marketplaces can be differentiated based on many fronts: 1. Need/customer coverage (Horizontal vs Vertical) 2. Participants: who to whom? (B2B, C2C, etc) 3. Offerings classification (Information, Goods, Services, Funds) 4. Classification based on Control + Functionality (Closed, Open, Focused, Censored) 5. Management approach (Unmanaged, lightly managed, fully managed) 6. Transaction regularity (One-off, repeat transactions)
44. Marketplace Types: Need Coverage Horizontal platforms: • facilitate exchanges in multiple different categories ◦ e.g services, buying and selling stuff, etc Vertical platforms: • focus on one problem / thing ◦ E.g. Uber (driving), Airbnb (accommodation)
45. Marketplace Types: Participants Who to Whom? Participants on the online platform Platform Type Platform Use 1. B2B transaction of products or services between businesses 2. C2C transaction of products or services between customers 3. B2C transaction of products or services from business to customers 4. Crowdfunding lets people post projects and raise money through campaigns 5. eCommerce platform for two parties, e.g. seller - shopper, startup owner - investor 6. Peer-to-peer brings together users offering offline services 7. auction platforms seller lists a product; buyer with the highest bid gets the item https://syndicode.com/2017/06/28/types-of-online-marketplaces/
46. Marketplace Types Platform Type Platform Use 1. B2B transaction of products or services between businesses 2. C2C transaction of products or services between customers 3. B2C transaction of products or services from business to customers 4. Crowdfunding lets people post projects and raise money through campaigns 5. eCommerce platform for two parties, e.g. seller - shopper, startup owner - investor 6. Peer-to-peer brings together users offering offline services 7. auction platforms seller lists a product; buyer with the highest bid gets the item https://syndicode.com/2017/06/28/types-of-online-marketplaces/
47. Marketplace Types: Offerings http://wiki.rademade.com/4-typology-of-marketplaces
48. Marketplace Types: Control + Functionality Classification based on control + functionality: 1. Closed 3. Censored 2. Focused 4. Open http://wrap.warwick.ac.uk/76183/1/WRAP_JIT.r3_final.pdf
49. Marketplace Types: Management Type https://rubygarage.org/blog/types-of-online-marketplaces • P2P transactions • Reviews & ratings → Trust • Lowest transaction fees
50. Marketplace Types: Management Type https://rubygarage.org/blog/types-of-online-marketplaces • Protection for users • Accurate content • Technology tools
51. Marketplace Types: Management Type https://rubygarage.org/blog/types-of-online-marketplaces • Operator mediates transactions • Whole process managed • Superior customer experience
52. Marketplace Types: Transaction Regularity
53. Success of marketplace ~ Matching consumers & suppliers
54. Outline 1. Introduction: a. (Quick) Overview of traditional RecSys approaches b. Introduction to Marketplace c. Types & examples of marketplaces d. Recommendations in a marketplace
55. Recommendations in a Marketplace
56. Recommendation in multi-sided Marketplace
57. Recommendation in multi-sided Marketplace Stakeholder(s) User Drivers Advertisers Campaign Hosts Platform provider
58. Recommendation in 2-sided Marketplace Stakeholder(s) User Drivers Advertisers Campaign(s) Platform provider Metrics Streams Engagement levels Reach / Depth / Retention Downstreams Other proxies of user satisfaction Exposure Audience growth Revenue LTV Diversity
59. Select an arm (i.e. card) Recommendation Strategy
60. Select an arm (i.e. card) Recommendation Strategy
61. Select an arm (i.e. card) Recommendation Strategy user-centric
62. User centric ML model is not meant to optimize for different objectives
63. Recommendation Strategy f( 1 , 2 , 3 , 4 ) Recommendation strategy = ??
64. Recommendation Strategy f( 1 , 2 , 3 , 4 )
65. Recommendation Strategy Select an arm (i.e. card) user-centric
66. user-centric supplier-centric platform economics Recommendation Strategy Select an arm (i.e. card)
67. user-centric artist-centric Spotify economics Recommendation Strategy Solution: find optimal recommendations which satisfy multiple objectives!
68. Marketplace powered Recommendations
69. Recommendations in a Marketplace SAT objectives User Understanding User Expectations
70. Recommendations in a Marketplace SAT objectives User Understanding User Expectations Supplier Exposure Supplier Goals
71. Recommendations in a Marketplace SAT objectives User Understanding User Expectations Supplier Exposure Supplier Goals Platform objectives Long Term Value
72. Recommendations in a Marketplace SAT objectives User Understanding User Expectations Supplier Exposure Supplier Goals Platform objectives Long Term Value Algorithmic balancing
73. Recommendations in a Marketplace SAT objectives User Understanding User Expectations Supplier Exposure Supplier Goals Platform objectives Long Term Value Algorithmic balancing User Understanding … Content Understanding Supplier Understanding
74. Recommendations in a Marketplace SAT objectives User Understanding User Expectations Supplier Exposure Supplier Goals Platform objectives Long Term Value Algorithmic balancing Evaluation Recommendations User Understanding … Content Understanding Supplier Understanding
75. Recommendations in a Marketplace Optimization Objectives What do different stakeholders care about? - User centric objectives - Supplier centric objectives - Platform goals & objectives
76. Recommendations in a Marketplace Multi-Objective Methods How do we make multiple considerations when recommending content? - Multiple predictions - Multi-objective models
77. Recommendations in a Marketplace Leveraging User & Content Understanding Consider user traits + stakeholder objectives to personalize: - Is the user open to diverse content? - Can we surface a tail supplier here?
78. Recommendations in a Marketplace Industrial Applications How different industries are developing their systems for multi-stakeholder needs?
79. Recommendations in a Marketplace ❏ Stakeholders & Objectives
80. Recommendations in a Marketplace ❏ Stakeholders & Objectives ❏ Methods for Multi-Objective Recommendations
81. Recommendations in a Marketplace ❏ Stakeholders & Objectives ❏ Methods for Multi-Objective Recommendations ❏ Leveraging User & Supplier Understanding
82. Recommendations in a Marketplace ❏ Stakeholders & Objectives ❏ Methods for Multi-Objective Recommendations ❏ Leveraging User & Supplier Understanding ❏ Industrial Applications
83. Part I: Introduction to Marketplaces 1. What is a task & why are they important? 2. Characterizing Tasks across interfaces: 1. desktop search 2. digital assistants 3. voice-only assistants 3. Understanding User Tasks in Web Search a. Extracting Query Intents b. Queries → Sessions → Tasks c. Search Task Understanding a. Task extraction b. Subtask extraction c. Hierarchies of tasks & subtasks d. Evaluating task extraction algorithms 5. Recommendation Systems a. Case study: Pinterest b. Case study: Spotify Take-home messages: • Traditional RecSys methods are majorly user-centric • Need to explicitly consider other stakeholders • Different types of marketplaces & examples • Components: – Multi-objective methods – User & Content understanding
84. Schedule 08:00 - 08:10: Welcome + Introduction 08:10 - 08:30: Part I: Introduction to Marketplaces 08:30 - 09:00: Part II: Optimization Objectives in a Marketplace 09:00 - 09:30: Part III: Methods for Multi-Objective Recommendations 09:30 - 10:00: Break 10:00 - 10:30: Part III: Methods for Multi-Objective Recommendations 10:30 - 11:10: Part IV: Leveraging Consumer, Supplier & Content Understanding 11:10 - 11:40: Part V: Industrial Applications 11:40 - 11:50: Questions & Discussions