Recommendations as a Conversation with the User

This 2011 ACM Conference on Recommender Systems (RecSys) presentation explores recommendations as part of a conversation between users and systems. A conversational approach should provide transparency, control, and guidance. Transparency means that users understand why systems offer particular recommendations. Control means that users can explicitly manipulate the behavior of recommender systems based on personal needs and preferences. Guidance means that systems offers plausible and predictable next steps rather than requiring users to guess the consequences of their interactions. Finally, there are psychological factors -- in particular, the faith that users place in the recommender system's effectiveness. Since users develop mental models of recommender systems, the system should become more predictable with repeated use.

Transcript

1. 1 Recruiting Solutions Recruiting Solutions Recruiting Solutions Recommendations as a Conversation with the User Daniel Tunkelang Principal Data Scientist at LinkedIn Daniel
2. 2 Introductions
3. 3 Let’s talk about how we talk with machines…
4. 4 Clifford Nass’s secret: 1) Find a conclusion by a social science researcher. 2) Change “People do X when interacting with other people.” to “People do X when interacting with a computer.” 3) Profit!
5. 5 Let’s work on our relationship.
6. 6 Core Message Recommendations are a conversation with the user. 1) Consider asking vs. guessing. 2) Ask good questions. 3) It's ok to make mistakes… if you have a good explanation and adapt to feedback.
7. 7 Our goal: http://www.wilsoninfo.com/computerclipart.shtml
8. 8 Overview 1) Theory 2) Examples 3) Action Items
9. 9 1) Theory
10. 10 Pragmatics: the Study of Conversation Paul Grice
11. 11 Grice’s Maxims of Conversation Maxim 1: Quality Maxim 2: Quantity Maxim 3: Relation Maxim 4: Manner H. P. Grice, "Logic and conversation” [1975]
12. 12 Maxim 1: Quality
13. 13 Quality: Above All, the Truth Xiao, Bo and Benbasat, Izak. 2011. "Product-Related Deception in E-Commerce: A Theoretical Perspective," MIS Quarterly, (35: 1) pp.169-195.
14. 14 Don’t Lie 1) Don’t use “recommended” when you really mean “sponsored” or “excess inventory”. 2) Optimize for the user’s utility. 3) Apply a standard of evidence (quality, quantity) that you believe in.
15. 15 Maxim 1: Quantity
16. 16 Right Amount of Information 1) Exchange small units of information. 2) If recommendations supplement other content, consider overall cognitive load. 3) Provide short, meaningful explanations.
17. 17 Maxim 3: Relation
18. 18 Relevant to the User 1) Offer value to the user. 2) Respect task context. 3) Don’t be obnoxious.
19. 19 Maxim 4: Manner
20. 20 Relevant to the User 1) Eschew obfuscation. 2) Avoid ambiguity. 3) Be brief. 4) Be orderly.
21. 21 Another Perspective Gary Marchionini
22. 22 Human-Computer Information Retrieval Empower people to explore large-scale information but demand that people also take responsibility for this control by expending cognitive and physical energy. Marchionini, G., “Toward Human-Computer Information Retrieval” [2006]
23. 23 Principles of HCIR 1) Do more than deliver relevant information: facilitate sensemaking. 2) Increase user responsibility and control: require and reward effort. 3) Adapt to increasingly knowledgeable users over time. 4) Be engaging and fun to use!
24. 24 Facilitate Sensemaking
25. 25 Require and Reward Effort http://www.posterenvy.com/catalog/ask_why.jpg
26. 26 Adapt to User Knowledge
27. 27 Be Engaging! http://bluenile.com/
28. 28 Applying the theory to… 1) Personalized Recommendations 2) Social Recommendations 3) Item Recommendations
29. 29 Personalized Recommendations 1) Be transparent about model so users gain insight. 2) Allow users to modify models to correct mistakes. 3) Solicit just enough information to provide value.
30. 30 Social Recommendations 1) Identify the right set of similar users. 2) Allow users to manipulate the social lens. 3) Accommodate users who break your model.
31. 31 Item Recommendations 1) Explain recommendations to users. 2) Watch out for non-sequiturs (e.g., diapers -> beer). 3) Play well with user-controlled filtering and sorting.
32. 32 2) Examples
34. 34 Initial User Experience
35. 35 “It just takes 2 minutes…”
36. 36 Asking Before Guessing
37. 37 Let’s try some answers:
38. 38 Uh oh…
39. 39 Expressing my gustibus…
40. 40 New Star Trek = Yes; New Star Wars = No
41. 41 Testing my patience…
42. 42 Bring on the quality!
43. 43 And continue the conversation.
44. 44 Learning from Netflix 1) Ask the user for help up front. But not too much help. 2) Pay attention to what the user tells you! 3) Give users value early and often. 75% of Netflix views result from recommendations
46. 46 Initial User Experience
47. 47 Seed with an artist…
48. 48 Or track or genre.
49. 49 Goo Goo G'joob!
50. 50 Ease user into recommendation space…
51. 51 And go wild!
52. 52 Shared Product: Personalized Stream
53. 53 Positive and Negative Feedback
54. 54 Learning from Pandora 1) Get meaningful input from user in one step. 2) Explain recommendations to users. 3) Solicit feedback and act on it immediately.
56. 56 My home page…
57. 57 Explanations and Humility
58. 58 Explain What and Why
59. 59 Recommendations as a Starting Point
60. 60 Learning from Amazon 1) Show the factors that drive your conclusions. 2) Distinguish different kinds of recommendations. 3) Combine recommendations with user control. Amazon: 35% of sales result from recommendations
61. 61 3) Action Items
62. 62 Increase explainability. Explanations can be even more important than the recommendations themselves. Herlocker et al., “Explaining collaborative filtering recommendations” [2000] Sinha and Swearingen, “The role of transparency in recommender systems” [2002] Tintarev and Masthoff, “Effective explanations of recommendations: User- centered design” [2007] (via Òscar Celma’s book, Music Recommendation and Discovery: The Long Tail, Long Fail, and Long Play in the Digital Music Space)
63. 63 Some models more explainable than others. 1) Consider decision trees and rule-based systems. 2) Avoid using latent, unlabeled features. 3) If the model is opaque, use examples as surrogates.
64. 64 Make a good first impression. Your user’s first experience is critical. Use popularity as a default if it makes sense. Solicit one valuable piece of information as quickly and painlessly as possible. “Do you like the taste of beer?” http://blog.okcupid.com/index.php/the-best-questions-for-first-dates/
65. 65 Design feedback into your system. You can make mistakes, if users can easily fix them. Challenging if models use offline computation. Respond instantly; generalize as quickly as possible. Agarwal and Chen, “Machine Learning for Large Scale Recommender Systems” [ICML 2011 Tutorial]
66. 66 Integrate recommendations with search. Recommend next steps, not just items. In a task context, recommendations are just another source of information scent. Be careful in integrating offline recommendations with online features like search and navigation. Pirolli, Information Foraging Theory: Adaptive Interaction with Information [2007]
67. 67 Summary Recommendations are a conversation with the user. 1) Consider asking vs. guessing. 2) Ask good questions. 3) It's ok to make mistakes… if you have a good explanation and adapt to feedback.
68. 68 Questions? Contact: dtunkelang@linkedin.com We’re Hiring! http://engineering.linkedin.com/ Thank You!