Amanda Ford

insight week 5B

Transcript

1. By Amanda Ford •  People value reviews by others like them. •  Can you identify parents without having to ask?
2. •  Yelp’s Data Challenge: 1.5+ million reviews from 300,000+ users across 6 U.S. cities (~2Gb) •  How to identify parents? 2   Words: ‘kids’, as in “my wife and kids” ‘baby’, as in “our baby loved it!” Categories: Pediatricians: “Dr. L is the best!” Preschools: “We love St. Marks!” Signal Words: ‘kids’, as in “way too many loud kids” ‘baby’, as in “Oh baby that was hot!” Categories (cross-listed) General Practice: “Dr. L is the best urologist’ Religious Organizations: “We love St. Marks Sunday worship services ” Noise Amanda  Ford   The Data
3. •  Supervised learning, k-means clustering, word frequency analysis all suffered from: –  Noise –  Lack of clear class of non-parents •  Simple bigram formula worked best –  Searched for possessive word(s) + kid word –  “My kid”, “Our son”, “My wife and baby” •  “Parent” tag propagated across all reviews by that user 3   Amanda  Ford   Algorithm
4. •  > 20% of Yelp reviews written by parents •  Parents rate differently 4   Amanda  Ford   Data Story
5. 5   Amanda  Ford   Parent Ratings
6. 6   Amanda Ford Amanda  Ford  
7. BACKUPS 7   Amanda  Ford  
8. Amanda  Ford   8  
9. Unsupervised Learning? Plot frequency of broader range of kid words (“my kids” AND “kids”, “babies”, “stroller” etc) vs. how often someone reviews a business in a “kid category”. 1.  ALL users: BLUE 2.  Maybe parents (said “my kid” or similar, OR reviewed a kid business such as a daycare) 3.  Definitely parents (said “my kid” or similar)     9   Amanda  Ford  
10. Search for Distinguishing Bigrams •  Bigrams with biggest difference between parents and others are not obviously parent-related •  Parents less likely to say: –  ‘have been’ –  ‘when I’ –  ‘they were’ –  ‘have to’ •  Parents more likely to say: –  ‘out of’ –  ‘a bit’ –  ‘very good’ –  ‘that I’ Amanda  Ford   10