Addressing Cold-Start in App Recommendation: Latent User Models Constructed from Twitter Followers

Slide 1

Slide 1 text

Addressing Cold-Start in App Recommendations: Jovian Lin / Kazunari Sugiyama / Min-Yen Kan / Tat-Seng Chua National University of Singapore Latent User Models Constructed from Twitter Followers

Slide 2

Slide 2 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion

Slide 3

Slide 3 text

Many, many apps. Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion No. of apps Time 1/32

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

INFORMATION OVERLOAD !

Slide 6

Slide 6 text

Content-based Filtering Recommender Systems Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Collaborative Filtering 2/32

Slide 7

Slide 7 text

Content-based Filtering Recommender Systems Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Collaborative Filtering 4 1 4 5 4 ? 1 ? 5 ? 5 ? 1 ? 4 items 2 1 3 1 2 3 4 5 users ? ? ? ? ? ? ? ? ? ? 4 5 new items 3/32

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Collaborative Filtering Content-based Filtering Recommender Systems Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion •  Recommends items based on similar content. (e.g., genres, textual descriptions) •  Con: Lack of diversified recommendations. Example: a user who has downloaded a weather app will receive weather-related app-recommendations. 4/32

Slide 10

Slide 10 text

Meanwhile… Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 5/32

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

“Can we merge information mined from social networks to enhance (app) recommendations?” Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 6/32

Slide 14

Slide 14 text

“Can we address the cold-start in recommender systems by using nascent signals in social networks?” Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 7/32

Slide 15

Slide 15 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 8/32

Slide 16

Slide 16 text

Slide 17

Slide 17 text

We made two observations: 1.  Apps contain references to their Twitter accounts. Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 9/32

Slide 18

Slide 18 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 9/32

Slide 19

Slide 19 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 9/32

Slide 20

Slide 20 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Follow @angrybirds on Twitter 9/32

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

Slide 23

Slide 23 text

We made two observations: 1.  Apps contain references to their Twitter accounts. 2.  Early signals about apps can be present in social networks, even before ratings are received. Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Evernote iOS app Release Date: 8 May 2012 May 2012 Jun 2012 Jul 2012 Dec 2012 … 0 ratings 0 ratings First few ratings start coming in 118,827 ratings Has an account on Twitter since Feb 2008 By May 2012, Evernote’s Twitter account already had 120,000 followers and 1,300 tweets. 10/32

Slide 24

Slide 24 text

We made two observations: 1.  Apps contain references to their Twitter accounts. 2.  Early signals about apps can be present in social networks, even before ratings are received. Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion We *CAN* address the cold-start in recommender systems by using nascent signals in social networks. 11/32

Slide 25

Slide 25 text

We want to estimate the probability that “a target user u will like an app a.” Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion “like” app user p( + | a, u ) 12/32

Slide 26

Slide 26 text

We want to estimate the probability that “a target user u will like an app a.” Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion p( + | a, u ) p( + | t, u) p( t | a) ∑ = “like” app user Twitter-follower “Pseudo-documents” & “Pseudo-words” Uniform distribution over the various Twitter-followers (t) following app a. t∈T(a) 13/32

Slide 27

Slide 27 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion downloaded/consumed 14/32

Slide 28

Slide 28 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion User u downloads 14/32

Slide 29

Slide 29 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion User u downloads twitter.com/angrybirds twitterID_31230 twitterID_2289 twitterID_999 twitterID_50401 …… followers twitterID_2 twitterID_3142439 twitterID_111031 14/32

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion User u downloads twitterID_31230 twitterID_2289 twitterID_999 twitterID_50401 …… followers twitterID_2 twitterID_3142439 twitterID_111031 14/32

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion User u twitterID_31230 twitterID_2289 twitterID_999 twitterID_50401 …… twitterID_2 twitterID_3142439 twitterID_111031 14/32

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Pseudo-document u (twitterID10 , DISLIKED) (twitterID12 , DISLIKED) (twitterID10 , LIKED) (twitterID12 , LIKED) (twitterID29 , LIKED) (twitterID29 , LIKED) (twitterID31 , LIKED) User u disliked liked liked Followed by: •  twitterID10 •  twitterID12 Followed by: •  twitterID10 •  twitterID12 •  twitterID29 App a App b App c Twitter-follower ID Preference indicator Followed by: •  twitterID29 •  twitterID31 16/32

Slide 37

Slide 37 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Pseudo-document u (twitterID10 , DISLIKED) (twitterID12 , DISLIKED) (twitterID10 , LIKED) (twitterID12 , LIKED) (twitterID29 , LIKED) (twitterID29 , LIKED) (twitterID31 , LIKED) Twitter-follower ID Preference indicator The concept of “pseudo-documents” and “pseudo-words” *does not* apply exclusively to Twitter followers. 17/32

Slide 38

Slide 38 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Pseudo-document u (genreID10 , DISLIKED) (genreID12 , DISLIKED) (genreID10 , LIKED) (genreID12 , LIKED) (genreID29 , LIKED) (genreID29 , LIKED) (genreID31 , LIKED) Genre ID Preference indicator The concept of “pseudo-documents” and “pseudo-words” *does not* apply exclusively to Twitter followers. 17/32

Slide 39

Slide 39 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Pseudo-document u (wordID10 , DISLIKED) (wordID12 , DISLIKED) (wordID10 , LIKED) (wordID12 , LIKED) (wordID29 , LIKED) (wordID29 , LIKED) (wordID31 , LIKED) Word ID Preference indicator The concept of “pseudo-documents” and “pseudo-words” *does not* apply exclusively to Twitter followers. 17/32

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Constructing Latent Groups (twitterID10 , DISLIKED) (twitterID12 , DISLIKED) (twitterID10 , LIKED) (twitterID12 , LIKED) (twitterID29 , LIKED) (twitterID29 , LIKED) (twitterID31 , LIKED) Pseudo-documents LDA Per-document topic distribution Per-topic word distribution 19/32

Slide 43

Slide 43 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Constructing Latent Groups (twitterID10 , DISLIKED) (twitterID12 , DISLIKED) (twitterID10 , LIKED) (twitterID12 , LIKED) (twitterID29 , LIKED) (twitterID29 , LIKED) (twitterID31 , LIKED) Pseudo-documents LDA = ∑ p( +, t | z) p( z | u) p( + | t, u) Per-document topic distribution Per-topic word distribution z∈Z 19/32

Slide 44

Slide 44 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Constructing Latent Groups (twitterID10 , DISLIKED) (twitterID12 , DISLIKED) (twitterID10 , LIKED) (twitterID12 , LIKED) (twitterID29 , LIKED) (twitterID29 , LIKED) (twitterID31 , LIKED) Pseudo-documents LDA = ∑ p( +, t | z) p( z | u) p( + | t, u) Per-document topic distribution Per-topic word distribution Per-topic word distribution Per-document topic distribution z∈Z 19/32

Slide 45

Slide 45 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion Constructing Latent Groups (twitterID10 , DISLIKED) (twitterID12 , DISLIKED) (twitterID10 , LIKED) (twitterID12 , LIKED) (twitterID29 , LIKED) (twitterID29 , LIKED) (twitterID31 , LIKED) Pseudo-documents LDA = ∑ p( +, t | z) p( z | u) p( + | t, u) Per-document topic distribution Per-topic word distribution Probability that the presence of Twitter-follower t indicates that it is “liked” by user u. z∈Z 19/32

Slide 46

Slide 46 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion p( + | t, u) p( t | a) 20/32

Slide 47

Slide 47 text

We want to estimate the probability that “a target user u will like an app a.” Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion p( + | a, u ) p( + | t, u) p( t | a) ∑ = “like” app user Uniform distribution over the various Twitter-followers (t) following app a. Probability that the presence of Twitter-follower t indicates that it is “liked” by user u. Derived from Pseudo-Documents and Pseudo-Words. t∈T(a) 21/32 Stated earlier in Slide 13/32

Slide 48

Slide 48 text

Dataset •  We collected data from the Apple iTunes Store and Twitter during September to December 2012. •  Stats: •  1,289,668 ratings •  7,116 apps (with Twitter accounts) •  10,133 users. •  Restrictions: •  Each user must give at least 10 ratings for apps. •  Each Twitter ID is related to at least 5 apps. Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 22/32

Slide 49

Slide 49 text

Simulating Cold-Start •  10-fold cross validation. •  Selected 10% of the apps to be the held out set for all users. •  Each user has the same within-fold apps. •  Guarantee that none of these apps are in the training set of any user. Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 23/32

Slide 50

Slide 50 text

Evaluation Metric •  Our system outputs M apps for each user, sorted by their probability of liking the apps. •  Recall@M •  Zero ratings are uncertain – it is difficult to accurately compute precision. •  Since the ratings are true positives, recall is a more pertinent measure – it only considers the positively rated apps. Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 24/32

Slide 51

Slide 51 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion 3 Research Questions (RQ) 25/32

Slide 52

Slide 52 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ1: How does the performance of Twitter-followers feature compare with other features? RQ2: How does our method compare with other techniques? RQ3: Do the latent groups make any sense? What can we learn from them? 25/32

Slide 53

Slide 53 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ1: How does the performance of Twitter-followers feature compare with other features? 25/32

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ1: How does the performance of Twitter-followers feature compare with other features? 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 20 40 60 80 100 120 140 160 180 200 Recall Number of recommended apps (M) Pseudo-Docs (W) Pseudo-Docs (D) Pseudo-Docs (G) Pseudo-Docs (T) Pseudo-Docs (All) •  Words (W) •  Developers (D) •  Genres (G) •  Twitter-followers (T) •  All features (All) All = T = G = D = W = All features Twitter-followers Genres Developers Words 26/32

Slide 57

Slide 57 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ1: How does the performance of Twitter-followers feature compare with other features? Feature R@100 All features (TGDW) * 0.513 All, excluding Twitter-followers (GDW) 0.452 All, excluding Genres (TDW) 0.491 All, excluding Developers (TGW) 0.498 All, excluding Words (TGD) 0.507 Twitter-followers (T) 0.478 Genres (G) 0.435 Developers (D) 0.395 Words (W) 0.373 27/32

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ1: How does the performance of Twitter-followers feature compare with other features? 28/32

Slide 63

Slide 63 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ2: How does our method compare with other techniques? 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 20 40 60 80 100 120 140 160 180 200 Recall Number of recommended apps (M) Full Dataset VSM (Words) VSM (Twitter) LDA CTR Pseudo-Docs (Twitter)* Pseudo-Docs (All)** •  VSM (Words) •  VSM (Twitter) •  LDA •  Collaborative Topic Regression •  Pseudo-Docs (Twitter) •  Pseudo-Docs (All) Pseudo-Docs (All) Pseudo-Docs (Twitter) CTR LDA VSM (Twitter) VSM (Words) 29/32

Slide 64

Slide 64 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ2: How does our method compare with other techniques? 0 0.1 0.2 0.3 0.4 0.5 20 40 60 80 100 120 140 160 180 200 Recall Number of recommended apps (M) Limited to 15 apps per user (Sparse Dataset) VSM (Words) VSM (Twitter) LDA CTR Pseudo-Docs (Twitter)* Pseudo-Docs (All)* Pseudo-Docs (All) Pseudo-Docs (Twitter) CTR LDA VSM (Twitter) VSM (Words) 30/32

Slide 65

Slide 65 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? 31/32

Slide 66

Slide 66 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? 31/32

Slide 67

Slide 67 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? 31/32

Slide 68

Slide 68 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? 31/32

Slide 69

Slide 69 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? 31/32

Slide 70

Slide 70 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? “The Cat in the Hat” (Book) “Christmas Cutie” (Book) “Books” / “Education” / “Games” 31/32

Slide 71

Slide 71 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? “Friendly Shapes” (Education) “There’s No Place Like Space” (Education) “Books” / “Education” / “Games” 31/32

Slide 72

Slide 72 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? “Pasta Crazy Chef” (Games) “Gingerbread Dress” (Games) “Books” / “Education” / “Games” 31/32

Slide 73

Slide 73 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? Top 5 Twitter Profiles in Latent Group 1. “Pasta Crazy Chef” (Games) “Gingerbread Dress” (Games) “Books” / “Education” / “Games” 31/32

Slide 74

Slide 74 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? Nosy Crow Apps Nosy Crow creates children’s books and apps. You may know our 3-D Fairytale apps, The Three Little Pigs & Cinderella. “Pasta Crazy Chef” (Games) “Gingerbread Dress” (Games) “Books” / “Education” / “Games” 31/32

Slide 75

Slide 75 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? The iMums Four mums dedicated to reviewing apps and technology products for children to help educate their parents about the variety available. “Pasta Crazy Chef” (Games) “Gingerbread Dress” (Games) “Books” / “Education” / “Games” 31/32

Slide 76

Slide 76 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? Mums with Apps Supporting family-friendly developers seeking to promote quality apps for kids and families. “Pasta Crazy Chef” (Games) “Gingerbread Dress” (Games) “Books” / “Education” / “Games” 31/32

Slide 77

Slide 77 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? Charly James Div. Mom of 2 with varying SN & medical d/x. dandelion moms. “Pasta Crazy Chef” (Games) “Gingerbread Dress” (Games) “Books” / “Education” / “Games” 31/32

Slide 78

Slide 78 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? Next is Great We create and develop brain teasing educational iOS apps for kids and teenagers. Check out Pirate Trio Academy and Geek Kids. “Pasta Crazy Chef” (Games) “Gingerbread Dress” (Games) “Books” / “Education” / “Games” 31/32

Slide 79

Slide 79 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? 31/32

Slide 80

Slide 80 text

Slide 81

Slide 81 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? 31/32 “BeatStudio”, “AmpKit+”, “GuitarStudio”, “Everyday Looper”, “Mixr DJ”, etc. “Music” Top 5 Twitter Profiles in Latent Group 2.

Slide 82

Slide 82 text

Slide 83

Slide 83 text

Slide 84

Slide 84 text

Slide 85

Slide 85 text

Slide 86

Slide 86 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? 31/32 “Paper Monsters”, “Stickman Cliff Diving”, “Lili”, “Snoopy’s Street Fair”, “Gizmonauts”, etc. “Games”, “Photo & Video” “Video games warrior, lover of life, eternal student of the universe…” “I’m a Multimedia developer working at Kent State Uni! I also do art services for the game industry…” “Agalog Games is an independent iOS game studio…” “Hi! Samadhi Games LLC is an Indie Developer of iOS, Android, etc”

Slide 87

Slide 87 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion RQ3: Do the latent groups make any sense? What can we learn from them? 31/32

Slide 88

Slide 88 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion •  Alleviate the cold-start in app-recommendation by using Twitter profiles of apps + Twitter followers. •  By using the feature of Twitter-followers to generate latent groups, our method works well – especially in a domain with unreliable textual features. •  Allows us to map users from the App Store to users in Twitter. •  Future work: •  Explore second-degree relationships on Twitter. •  Explore the use of our approach in other domains, such as music-recommendation. 32/32

Slide 89

Slide 89 text

Thank you

Slide 90

Slide 90 text

Introduction ›❯ Our Approach ›❯ Experiments ›❯ Results ›❯ Conclusion •  Ratings lag behind Twitter followers. •  Because it takes more effort to post a rating/review than to follow a Twitter account. •  Monitored a few new apps: •  Average # of new ratings/reviews in a week: 4.2 •  Average # of new Twitter followers in a week: 21.4 •  We want to recommend ASAP – even if it’s 1 day faster. #extra