Eventbrite’s Search-based Approach to Recommendations

Eventbrite is building an Elasticsearch-powered, content- and behavior-based recommendation system to match users with events they are sure to enjoy. Come learn about their approach, successes, failures, and future work at the intersection of recommendation and search.

Transcript

2. Recommendations Recommendations Search Search and and for good search the for good recommendations ~OR~ ~OR~
3. Hi! I'm John Berryman Hi! I'm John Berryman Former: Aerospace Engineer Former: Search Tech Consultant @JnBrymn Current: Recommendation Engineer at Eventbrite Current: Facial Hair Artist Future: Author?
4. Introducing Introducing
5. Exciting Times at Exciting Times at We have been very successful focusing on the organizer! Hundreds of thousands of new events every month. Only now starting to focus on the consumer. Audacious Goal: Become the Marketplace for Upcoming Events A marketplace built on Search and Recommendations.
6. Talking Points Talking Points Search - the user's interface for discovery Recommendation - how it all works Recommendation served from the Search Engine Recommendation eats Search The Brite Future
7. Search Search
8. Search at Search at
9. Sometime get complaints from organizers that their events are not easy to find. We don't seem to get many complaints from consumers... No personalization, no understanding of the context of the user. Search is Good Search is Good ...but not perfect
10. Recommendations Recommendations
11. Collaborative Filtering Collaborative Filtering Based upon user interactions (event views/purchases) Answers "What might you like based upon what others with similar interactions like?" Methods: Matrix Decomposition Count Cooccurrence
12. Matrix Decomposition Matrix Decomposition Benefits: Remove "noise" from data Introduces the idea of a genre Can compare event-to-event, user-to- user, and event-to-user Drawback: Computationally INTENSE! Why would we go to the trouble of creating a less accurate representation? Singular Value Decomposition Alternating Least Squares
13. Count Cooccurrence Count Cooccurrence First find event to event affinity Next find user to event affinity
14. Collaborative Filtering Collaborative Filtering Output: recommended events related events recommended users "Exotic" Output: user-category affinity category-terminology affinity User123 → {Event345: 9.2, Event456: 7.2,... Event678 → {Event345: 8.8, Event456: 3.4,... Event789 → {User123: 7.3, User234: 6.1, ...} User123 → {Music: 6.4, Beer: 5.1, Dance: 4.9,... Wine → {barrel: 5.4, oak: 4.8, bouquet: 1.2,...
15. The Cold Start Problem The Cold Start Problem Collaborative-filtering is based upon interactions. New events have no interactions. New events have no recommendations. (Note: this is not a problem for Amazon or Netflix)
16. Content-Based Content-Based Recommendation Recommendation recommending related events is easy given some event X collect "important" terms from X's fields search for other events with X's "important" terms filter by location and category of X return remaining events sorted by relevance ... in other words Elasticsearch "MoreLikeThis" query
17. Content-Based Content-Based Recommendation Recommendation recommending events to users is harder you must find content first scoop up user searches collect content from events that users interact with ask users to tell us about them! (user profiles!) to make the recommendation just search! (no fancy pants Machine Learning) ...
18. Searchamendations Searchamendations ... it's a thing
19. Why? Why? Why would we want to combine search and recommendations? First we must understand search: Search is like a one big giant table of stuff. Search is good at finding matching docs. especially "find a doc that contains X" Search is good a scoring and sorting docs.
20. Why? Why? Search allows recommendation methods to be combined. Search can incorporate business logic like location filtering. And you can still do search, but now informed by knowledge of the user making search. Why would we want to combine search and recommendations? We call this personalized search
21. Personalized Search Personalized Search Natalie Downe Barrett Cook "Beer Event" (non-personalized)
22. Personalized Personalized Search Search Natalie Downe Barrett Cook searches for "beer event" searches for "beer event"
23. Content-Based Content-Based Recommendation in Search Recommendation in Search Already saw this. It's trivial. Related Events = MoreLikeThis Event recommendation = Perform a search with user's "content"
24. Collaborative-Filtering Collaborative-Filtering Recommendation in Search Recommendation in Search Approach 1: for every user get recommended events and boost by those specific id curl -XGET http://eventbrite_search:9200/events -d '{ "query": { "bool": { "must": { "multi_match": { "fields": ["title","description"], "query": "hello" } }, "should": { "terms": { "id": ["Event456","Event567","Event678",...] }}}}} User123 → {Event456: 6.4, Event567: 5.1, Event678: 4.9,... "should": { "terms": { "id": ["Event456","Event567","Event678",...]
25. Collaborative-Filtering Collaborative-Filtering Recommendation in Search Recommendation in Search Approach 2: find documents with this user in the user "users_who_might_like" field curl -XGET http://eventbrite_search:9200/events -d '{ "query": { "bool": { "must": { "multi_match": { "fields": ["title","description"], "query": "hello" } }, "should": { "term": { "users_who_might_like": "User123" }}}}} User123 → {Event456: 6.4, Event567: 5.1, Event678: 4.9, ...} "should": { "terms": { "users_who_might_like": "User123"
26. Collaborative-Filtering Collaborative-Filtering Recommendation in Search Recommendation in Search Approach 3: index a "related_events" field and search with the events the user actually attended curl -XGET http://eventbrite_search:9200/events -d '{ "query": { "bool": { "must": { "multi_match": { "fields": ["title","description"], "query": "hello" } }, "should": { "term": { "related_events": ["Event456","Event567","Event678"] }}}}} Event789 → {Event456: 6.4, Event567: 5.1, Event678: 4.9, ...} "should": { "terms": { "related_events": ["Event456","Event567","Event678"]
27. Easy Peasy Easy Peasy Recommendation in Search Recommendation in Search Given any user preference data, index it and then query for it using the same pattern. curl -XGET http://eventbrite_search:9200/events -d '{ "query": { "bool": { "must": { "multi_match": { "fields": ["title","description"], "query": "hello" } }, "should": { "terms": { "affinity_tags": ["20s","male","sporty","deal_seeker] }}}}} "should": { "terms": { "affinity_tags": ["20s","male","sporty","deal_seeker]
28. Everything Everything is is Recommendation Recommendation
29. Recommendation Recommendation Reimagined Reimagined The goal of recommendation is to provide the best events possible based upon the best information available.
30. Recommendation Recommendation Reimagined Reimagined "Best information available" has two parts knowledge of events primary data: title, desc, category, date/time, location metadata: total ticket sales, tickets left, recent sales knowledge of user location previous event views previous event attendance previous searches profile information day of week/time preference cost preference or nothing at all
31. Recommendation Recommendation Reimagined Reimagined But what happened to Search? It's still there. It's part of the knowledge about the user. It's the most important part. It's what the user wants right now!
32. The Brite Future The Brite Future
33. Challenges Challenges Possibilities Possibilities Personalized Search Related Events Newsletters Browse/Explore/Discovery Some sparsity (smaller locales, semantic content in events) How to combine so many signals Chicken:Egg :: Users:Recommendations
34. Questions? Questions?