Learning Fast - Experimentation and Open Research at Spotify

Talk given at WeAreDevelopers AI Congress Vienna, Dec 4th 2018.

Spotify is the world's leading music streaming service, with over 191 million users in 78 markets. But no two users are exactly alike, so in order to serve each listener the content they will uniquely enjoy, Spotify must personalize every aspect of the user experience. From exclusive playlists like Discover Weekly and Your Daily Mix, to personalized Concert recommendations, Spotify utilizes Machine Learning technologies to learn from each user's unique listening habits and tastes, and surface the most relevant content for them.

An important aspect of building Machine Learning-based systems is experimenting quickly and effectively, as well as fostering a culture and community around Machine Learning research, both inside and outside of the company.

The first part of this talk will give an overview of Home screen personalization at Spotify: how the team uses Machine Learning to show users the most relevant content on the default screen of the Spotify app, and more importantly, how they experiment rapidly using infrastructure and tools for offline and online evaluation of Machine Learning models.

The second part of this talk will introduce the Recommended Tracks feature for playlist creation on Spotify, and showcase the results of the RecSys Challenge 2018, which was an open data science challenge launched to promote the study of automatic playlist continuation in the greater research community.

Transcript

1. 1 Learning Fast Experimentation and
 Open Research at Spotify Ching-Wei Chen @cweichen Engineering Lead, Home Personalization
2. WeAreDevelopers AI Congress Vienna, Dec 2018 Learning Fast What do I mean when I say “fast”?
3. WeAreDevelopers AI Congress Vienna, Dec 2018 Learning Fast Given a clearly defined concept of value, we want to test and evaluate hypotheses for creating value as quickly and cheaply as possible, so that we can learn, generate new hypotheses, and deliver incremental value as soon as possible. Similar philosophies are essential to: • Lean Principles • Agile Methodology • Microservice Architecture • Continuous Improvement/Delivery
4. WeAreDevelopers AI Congress Vienna, Dec 2018 Speed at Spotify Spotify is organized to optimize for speed of delivery and innovation. Small, autonomous teams with independent, but aligned, missions explore diverse problem spaces rapidly, and deliver value to the user as quickly as possible.
5. WeAreDevelopers AI Congress Vienna, Dec 2018 Speed at Spotify Spotify is organized to optimize for speed of delivery and innovation. Small, autonomous teams with independent, but aligned, missions explore diverse problem spaces rapidly, and deliver value to the user as quickly as possible. Spotify Home
6. 6 Home Home is the default screen of the mobile app for all 191M of our users worldwide. It surfaces the best of what Spotify has to offer, including music and podcasts for every situation, personalized playlists, new releases, old favorites, and undiscovered gems. Value to the user here means helping them find something they’re going to enjoy listening to, quickly.
7. 7 Home But with our vast collection of playlists, artists, albums, stations and podcasts, the choice can be paralyzing. Also, people don’t scroll (much). To help our users quickly find what they’re most likely to enjoy, we use Machine Learning to learn from their tastes and listening habits so that we can put the best content where they’re most likely to see it.
8. 8 The Task
9. 9 The Task For Context x,
10. 10 Shelf Shelf Card Card Card Shelf The Task For Context x, Card c, Shelf s,
11. 11 Shelf Shelf Card Card Card Shelf The Task For Context x, Card c, Shelf s, Predict Reward r
12. 12 The Task For Context x, Card c, Shelf s, Predict Reward r We can use different supervised Machine Learning techniques to model the reward (clicks, plays, follows, …) However, these predictions may suffer from uncertainties due to data sparsity and presentation bias. To reduce the uncertainty of the reward model, as well as to avoid the “filter bubble”, we use a contextual bandits* approach to continuously learn to optimize the model in an online setting. *Read “Explore, Exploit, and Explain: Personalizing Explainable Recommendations with Bandits” by J. McInerney et al, from ACM RecSys 2018, for an in-depth description.
13. 13 Contextual Bandits Multi-Armed Bandit approaches attempt to maximize rewards by balancing the exploitation of actions with a high confidence of a positive reward with the exploration of actions where the reward is less known. Contextual Bandit approaches take into account contextual information that could affect the reward. Policy: π e, r (A) Exploration strategy Reward function Action set
14. WeAreDevelopers AI Congress Vienna, Dec 2018 Policy: π e, r (A) Exploration strategy Reward function Action set e.g epsilon greedy, 
 Thompson sampling e.g logistic regression, factorization machine Shelves and 
 cards Lots of variables requires lots of experimentation. Reward r: Context x, Card c, Shelf s, User profile, 
 interaction history, 
 taste profile, … Time of day, 
 day of week, 
 location, … Card features (artists, genres, type, language…) Image, Description,…
15. WeAreDevelopers AI Congress Vienna, Dec 2018 Experimentation process Research Idea Online 
 A/B Test
16. WeAreDevelopers AI Congress Vienna, Dec 2018 Experimentation process Research Idea Online 
 A/B Test Policy: π e, r (A) Exploration strategy Reward function Action set Develop
17. WeAreDevelopers AI Congress Vienna, Dec 2018 Experimentation process Depending on test requirements, 
 A/B tests may take ~days to ~weeks. Together with deployment time, this limits the number of tests we can run in a given sprint. Research Idea Online 
 A/B Test Deploy
18. WeAreDevelopers AI Congress Vienna, Dec 2018 Experimentation process Research Idea Online 
 A/B Test
19. WeAreDevelopers AI Congress Vienna, Dec 2018 Experimentation process Research Idea Online 
 A/B Test Offline 
 Eval Offline evaluation speeds up experimentation, allowing hundreds of tests to be run every week
20. WeAreDevelopers AI Congress Vienna, Dec 2018 ML Lab An offline evaluation framework to launch, evaluate and archive machine learning studies, ensuring reproducibility and allowing sharing across teams. Experiments let you define a process (e.g. model under test), and the evaluation used. Evaluations define the dataset, ground truth, and evaluation metric calculations. ML Lab can do grid search and hyper parameter optimization for you. Developed through niland.io acquisition (2017)
21. WeAreDevelopers AI Congress Vienna, Dec 2018 ML Lab Designed to integrate into a researcher’s development flow. Consists of: • REST API • Command line interface • Dashboard interface
22. WeAreDevelopers AI Congress Vienna, Dec 2018 ML Lab Offline and online metrics must be correlated regularly to ensure we are testing the right things.
23. WeAreDevelopers AI Congress Vienna, Dec 2018 Experimentation process Research Idea Online 
 A/B Test Offline 
 Eval
24. WeAreDevelopers AI Congress Vienna, Dec 2018 Speed: 
 Inside 
 and 
 Out
25. WeAreDevelopers AI Congress Vienna, Dec 2018 Speed: 
 Inside 
 and 
 Out Academia Industry Developers Hobbyists
26. WeAreDevelopers AI Congress Vienna, Dec 2018 Open Research By releasing open datasets and hosting Data Science challenges, we hope to engage and contribute to the research and developer community all around the world. When we propose problems that are interesting to us (e.g. music recommendation), we hope the community will experiment with novel ideas and techniques that will advance the field of knowledge for everyone. E.g. ImageNet, CIFAR, MovieLens, …
27. WeAreDevelopers AI Congress Vienna, Dec 2018 RecSys Challenge 2018 Earlier in 2018 we hosted the RecSys Challenge on Automatic Playlist Continuation, together with researchers from JKU Linz and UMass Amherst. The dataset was 1 million user-created playlists from Spotify. The challenge was to predict tracks that would complete a given playlist. This is similar to the Recommended Songs feature on Spotify.
28. WeAreDevelopers AI Congress Vienna, Dec 2018 Participation 1791 participants from over 20 countries. 410 teams with 1497 submissions Vielen dank!
29. WeAreDevelopers AI Congress Vienna, Dec 2018 Approaches A great variety of techniques were submitted, including: • Nearest-neighbor methods • Random walk • Matrix factorization • Content features • Autoencoders • Sequence modeling (RNN, LSTM) • Multi-stage ensembles with boosting Papers and open-sourced code at 2018.recsyschallenge.com LSTM Network - Monti, et al Autoencoder - Yang, et al Title clustering - Zhao, et al Random walk - van Niedek, et al Winning system (multi-stage ensemble) - Volkovs, et al
30. WeAreDevelopers AI Congress Vienna, Dec 2018 WSDM Cup 2019 We are currently running the WSDM Cup 2019 challenge on Sequential Skip Prediction. The dataset is 130 million listening sessions on Spotify, along with associated interactions. The challenge is to predict which tracks in a session will be skipped. The challenge is running from now until Jan Sign up today! bit.ly/spotify-wsdm-cup-2019
31. WeAreDevelopers AI Congress Vienna, Dec 2018 Learning Fast Machine learning helps us learn what our listeners will enjoy at every moment. Experimentation tools and offline evaluation help us learn even faster. Sharing datasets and open research help the community learn and advance the field of knowledge for everyone. Learning isn’t a race: there’s no speed limit, but we want to make sure we are always accelerating.
32. WeAreDevelopers AI Congress Vienna, Dec 2018 Learn with us Join the band at spotifyjobs.com
33. WeAreDevelopers AI Congress Vienna, Dec 2018 Thank you! spotifyjobs.com @cweichen