Personalised Recommendations

Transcript

1. Personalised Recommendations Saturday 9 August 14
2. About me • Ed Tsech • Clojure, JavaScript developer • @edtsech on twitter, github Saturday 9 August 14
3. Content • Collaborative filtering • User based • Item based • Content based / knowledge based recommendations • Mahout • Movie Recommender Example Saturday 9 August 14
4. Collaborative Filtering • “Collaborative filtering is a method of making automatic predictions (filtering) about the interests of a user by collecting preferences or taste information from many users (collaborating).” Saturday 9 August 14
5. Collaborative Filtering • Last.fm, Twitter, Amazon • Pros • Relatively precise, ability to recommend items from different categories • Cons • Cold start problem Saturday 9 August 14
6. User-based Saturday 9 August 14
7. Saturday 9 August 14
8. Saturday 9 August 14
9. Saturday 9 August 14
10. Saturday 9 August 14
11. Saturday 9 August 14
12. Saturday 9 August 14
13. Item-based Saturday 9 August 14
14. Saturday 9 August 14
15. Saturday 9 August 14
16. Saturday 9 August 14
17. Saturday 9 August 14
18. Saturday 9 August 14
19. Algorithms • Euclidean distance • Pearson Correlation • Tanimoto Coefficient • ... Saturday 9 August 14
20. Euclidean distance Saturday 9 August 14
21. Pearson Correlation Saturday 9 August 14
22. Pearson vs Euclidean Saturday 9 August 14
23. Tanimoto Coefficient Saturday 9 August 14
24. Other Algorithms • Log-likelihood • Slope one • Singular value decomposition • K nearest neighbors • Cluster-based Saturday 9 August 14
25. Content Based • Prismatic • Pros • No cold start problem, ability to recommender new items • Cons • Harder to implement, not so precise, sometimes stupid. Saturday 9 August 14
26. Hybrid Systems • Netflix • Mix collaborative filtering & content-based recommendations • Knowledge-based • Add domain information Saturday 9 August 14
27. Mahout • Scalable machine learning library • User based recommenders • Item based recommenders • Various algorithms • Evaluation & rescoring features • Hadoop integration Saturday 9 August 14
28. Reca • Thin Clojure wrapper for Mahout’s single- machine recommendation algorithms • https://github.com/edtsech/reca Saturday 9 August 14
29. Movie App Demo • 8400000 ratings • 1.7 Gb database • 162 037 users • 82 715 movies Saturday 9 August 14
30. Rescoring • Add application logic to the recommender • Add domain specific information • Helps to make a hybrid recommender Saturday 9 August 14
31. Evaluation Evaluation of user based algorithm based on 3% of whole ratings (y axis - average difference) Saturday 9 August 14
32. Evaluation Evaluation of item based algorithm based on 33% of whole ratings (y axis - average difference) Saturday 9 August 14
33. Performance • 1.5Gb of memory • 250 msecs for user based recommender • 60-90 secs for item based recommender • 0.1 msecs after caching Saturday 9 August 14
34. Links • Mahout in Action [book] • Collective intelligence [book] • http://mahout.apache.org/ • http://blog.comsysto.com/2013/04/03/ background-of-collaborative-filtering-with- mahout/ Saturday 9 August 14