Introduction to Recommendation Systems

Transcript

1. The Long Tail Phenomenon Introduction to Recommender Systems Jaidev Deshpande Cube26 Software Pvt Ltd November 28, 2015 Jaidev Deshpande Introduction to Recommender Systems
2. The Long Tail Phenomenon Examples Jaidev Deshpande Introduction to Recommender Systems
3. The Long Tail Phenomenon Examples Jaidev Deshpande Introduction to Recommender Systems
4. The Long Tail Phenomenon Credits: Chris Anderson (http://thelongtail.com) Brick-and-mortar vs online marketplaces: hundreds of thousands vs tens of millions of choices Tailoring to individual responses Discovery of interests Jaidev Deshpande Introduction to Recommender Systems
5. The Long Tail Phenomenon Introduction Approaches to Recommendation Systems: Content-based: Use properties of items being recommended. Collaborative Filtering: Users ”collaborate” to get the best recommendations Jaidev Deshpande Introduction to Recommender Systems
6. The Long Tail Phenomenon The Model The Utility Matrix Airframe To Kill A Mockingbird Seabiscuit Beloved users 9 0 0 0 6 14 0 0 0 0 16 9 0 0 0 17 0 0 0 0 26 0 10 9 0 Jaidev Deshpande Introduction to Recommender Systems
7. The Long Tail Phenomenon Populating the Utility Matrix Explicit: Asking users to rate items: Ola, Uber :) Flipkart, Amazon :( Inferred: Quantify user behaviour. Jaidev Deshpande Introduction to Recommender Systems
8. The Long Tail Phenomenon Content-based Recommendations Collect attributes of books? Perform text processing on the content Generally: Mining for metadata (Google Search APIs, etc) Look for web APIs of other search services Data-specific feature extraction (Signal processing, text processing) Look for scikit-learn, pandas, nltk, etc. Jaidev Deshpande Introduction to Recommender Systems
9. The Long Tail Phenomenon Collaborative Filtering Concept of similarity Similarity measures: sklearn.metrics.pairwise Clustering / grouping similar profiles Calculate ”projected” ratings. Jaidev Deshpande Introduction to Recommender Systems
10. The Long Tail Phenomenon Practical Considerations S = f (U) (1) where f is the similarity function and U ∈ Zmxn S ∈ R+ mxm U is sparse, and S is massive S is symmetrical, sparsity ≤ 50% Nature of the data affects f Distributed execution: Workers need access to S Need to maintain designated copies of U Look for scipy.[sparse, spatial, dist], sklearn.metrics Jaidev Deshpande Introduction to Recommender Systems
11. The Long Tail Phenomenon Popular Implementations Graphlab: graph based, not free Crab: NumPy based, FOSS Spark MLlib: Collaborative filtering Jaidev Deshpande Introduction to Recommender Systems