Application of natural language processing

Application of natural language processing to Comcast reviews.

Transcript

1. Case study: Comcast Consumer reviews Olabanji Shonibare and Melissa Davidson March 1, 2017
2. Outline • Introduction • Word co-occurrences • Topic modeling • Sentiment analysis
3. Introduction
4. Comcast Consumer Complaints • comcast_consumeraffairs_complaints.csv • comcast_fcc_complaints_2015.csv Raw complaint data about Comcast television and internet published at consumeraffairs.com between 04/08 and 09/16. Raw complaints made to the FCC about Comcast between 04/15 and 06/15.
5. ConsumerAffairs
6. Number of complaints (CA) AK BC NE ON PE WY HI ID ND KS NV ME NC WI OH VT MO AR KY WV AZ LA NY MS NM DC SC DE NH AL UT CT OR MN IN CO MA VA WA MD TN MI TX NJ PA IL GA CA FL 0 200 400 600 800 Count Location
7. AL AZ AR CA CO CT DE FL GA ID IL IN IA KS KY LA ME MD MA MI MN MS MO MT NE NV NH NJ NM NY NC ND OH OK OR PA RI SC SD TN TX UT VT VA WA WV WI WY Percent [0.00272 to 0.00387) [0.00387 to 0.00443) [0.00443 to 0.00481) [0.00481 to 0.00571) [0.00571 to 0.00705) [0.00705 to 0.00931) [0.00931 to 0.27972] NA CA Percentage for Areas with Comcast Complaints by percentage (CA)
8. FCC
9. Complaints by percentage (FCC) AL AZ AR CA CO CT DE FL GA ID IL IN IA KS KY LA ME MD MA MI MN MS MO MT NE NV NH NJ NM NY NC ND OH OK OR PA RI SC SD TN TX UT VT VA WA WV WI WY Percent [0.000663 to 0.001310) [0.001315 to 0.001580) [0.001577 to 0.001680) [0.001682 to 0.002090) [0.002093 to 0.003100) [0.003103 to 0.005270) [0.005271 to 0.303950] NA FCC Percentage for Areas with Comcast
10. Complaints by zip code (FCC) 30 40 50 −120 −100 −80 longitude latitude 1000 1500 2000 rank(freq.x) FCC Comments by Zip Code
11. CA and FCC
12. Complaints for FCC and CA alaska iowa montana nebraska rhode island wyoming hawaii idaho north dakota kansas nevada wisconsin north carolina maine ohio vermont missouri arkansas kentucky west virginia new york louisiana arizona new mexico delaware new hampshire district of columbia south carolina mississippi alabama utah connecticut minnesota oregon indiana massachusetts colorado virginia washington maryland texas new jersey michigan tennessee pennsylvania illinois georgia california florida oklahoma south dakota 0 250 500 750 1000 Count Location
13. Complaints by percentage (FCC and CA) AL AZ AR CA CO CT DE FL GA ID IL IN IA KS KY LA ME MD MA MI MN MS MO MT NE NV NH NJ NM NY NC ND OH OK OR PA RI SC SD TN TX UT VT VA WA WV WI WY Percent [0.00340 to 0.00522) [0.00522 to 0.00623) [0.00623 to 0.00690) [0.00690 to 0.00799) [0.00799 to 0.01037) [0.01037 to 0.01295) [0.01295 to 0.30395] NA Both Percentage for Areas with Comcast
14. Word co-occurrences
15. Co-occurrence network in Comcast dataset texts (CA) • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • call customer service Internet month bill day told time hour charge pay week cable tv supervisor technician rep bad receive finally tech phone home fee company account issue cancel box speak people wait credit minute fix n 1000 1500 2000 2500 3000
16. • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • service data customer caps speed billing practices issues unfair slow speeds poor internet phone throttling xfinity switch connection tv complaint cap horrible bill services business failure pricing issue charges terrible price rental fraudulent 300gb bad usage cable bait overage modem n 25 50 75 100 Co-occurrence network in Comcast dataset titles (FCC)
17. • • • • • • • • • • • • • • • • • • • • • • • • • service told month called bill time call customer phone cable pay speed services account issue paying times home months tv day modem charge internet received n 300 500 700 900 Co-occurrence network in Comcast dataset text (FCC)
18. Topic modeling
19. Topic modeling Topic modeling is a method for unsupervised classification of documents, similar to “clustering” on numerical data. Topic model: Latent Dirichlet Allocation (LDA)
20. 1 2 3 4 0.00 0.02 0.04 0.06 0.00 0.02 0.04 0.06 0.00 0.01 0.02 0.03 0.00 0.02 0.04 0.06 channel time pay charge cable day service phone bill call charge cable hour service pay month Internet time phone customer day cable pay customer time bill Internet month call service Internet cable receive technician customer home bill box call service β Top 10 terms in each LDA topic (CA)
21. 1 2 3 4 0.00 0.01 0.02 0.03 0.04 0.00 0.01 0.02 0.03 0.000.010.020.030.040.05 0.00 0.02 0.04 0.06 pay tv rate package price cable bill month Internet service told issue customer account day Internet phone time service call time issue bill month pay charge modem service Internet speed customer stream pay limit usage month service cap Internet datum β Top 10 terms in each LDA topic (FCC)
22. Probability distribution for each topic (CA) 3 4 1 2 0.2 0.4 0.6 0.2 0.4 0.6 0 1000 2000 3000 4000 5000 0 1000 2000 3000 4000 5000 gamma Number of documents Topic 1 2 3 4
23. Probability distribution for each topic (FCC) 3 4 1 2 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0 500 1000 0 500 1000 gamma Number of documents Topic 1 2 3 4
24. Sentiment analysis
25. Sentiment analysis It is a computational approach to identify people’s opinion towards an entity Classification techniques • Machine learning • Lexicon-based
26. Sentiment analysis Classification techniques • Machine learning • Lexicon-based ## # A tibble: 13,901 × 2
 ## word sentiment
 ## <chr> <chr>
 ## 1 abacus trust
 ## 2 abandon fear
 ## 3 abandon negative
 ## 4 abandon sadness
 ## 5 abandoned anger
 ## 6 abandoned fear
 ## 7 abandoned negative
 ## 8 abandoned sadness
 ## 9 abandonment anger
 ## 10 abandonment fear
 ## # ... with 13,891 more rows ## # A tibble: 6,788 × 2
 ## word sentiment
 ## <chr> <chr>
 ## 1 2-faced negative
 ## 2 2-faces negative
 ## 3 a+ positive
 ## 4 abnormal negative
 ## 5 abolish negative
 ## 6 abominable negative
 ## 7 abominably negative
 ## 8 abominate negative
 ## 9 abomination negative
 ## 10 abort negative
 ## # ... with 6,778 more rows ## # A tibble: 2,476 × 2
 ## word score
 ## <chr> <int>
 ## 1 abandon -2
 ## 2 abandoned -2
 ## 3 abandons -2
 ## 4 abducted -2
 ## 5 abduction -2
 ## 6 abductions -2
 ## 7 abhor -3
 ## 8 abhorred -3
 ## 9 abhorrent -3
 ## 10 abhors -3
 ## # ... with 2,466 more rows NRC BING AFINN
27. AFINN lexicon • “ I’m not happy and I don’t like it ” 2 3 Sentiment score = 5 • This is utterly excellent! 3 Sentiment score = 3 • "I continue to receive unwanted calls from Comcast despite my instructions." Sentiment score = -2 -2
28. Alaska North Carolina Hawaii North Dakota Maine Wyoming New York Idaho Missouri Alabama Nevada Utah New Mexico Pennsylvania Massachusetts Texas Nebraska Arkansas Maryland Illinois New Jersey Colorado Connecticut Georgia Vermont Minnesota Florida Arizona Michigan California Tennessee Washington Ohio New Hampshire Virginia South Carolina Louisiana Delaware Oregon Indiana Kansas Kentucky Wisconsin Mississippi West Virginia −3 −2 −1 0 Average sentiment score state sentiment negative Average AFINN score for reviews within each state (CA)
29. Average AFINN score for reviews within each state (CA) AL AZ AR CA CO CT DE FL GA ID IL IN IA KS KY LA ME MD MA MI MN MS MO MT NE NV NH NJ NM NY NC ND OH OK OR PA RI SC SD TN TX UT VT VA WA WV WI WY Sentiment [−3.000 to −0.833) [−0.833 to −0.735) [−0.735 to −0.683) [−0.683 to −0.616) [−0.616 to −0.549) [−0.549 to −0.516) [−0.516 to −0.282] NA
30. Average AFINN score for reviews within each state (FCC) Iowa North Carolina Arkansas Kansas West Virginia Maine Vermont Arizona Washington New York Florida South Carolina Illinois Pennsylvania Montana Michigan Tennessee Minnesota Alabama California Georgia Colorado Missouri Kentucky District Of Columbia Mississippi Texas New Jersey Oregon Indiana New Hampshire Maryland Massachusetts Delaware Virginia New Mexico Utah Connecticut Nevada Louisiana Ohio Rhode Island −1 0 1 Average sentiment score State sentiment negative positive
31. pay bad cancel charged wrong refuse horrible poor complain terrible mistake worst ridiculous error lost leave hate free happy hope nice fine support resolve care −4000 −2000 0 Contribution to sentiment word sentiment negative positive Words with the highest contribution to sentiment scores (CA)
32. pay charged cancel refuse bad complain wrong ridiculous poor drop lack terrible error mistake unfair agree save hope promise fine resolved care increase support resolve −1500 −1000 −500 0 500 Contribution to sentiment word sentiment negative positive Words with the highest contribution to sentiment scores (FCC)
33. without won't no not can't don't −40 −30 −20 −10 0 0 5 10 15 −200 −100 0 100 0 100 0 20 40 60 −50 0 50 100 150 200 250 worry pay bother honor trust recommend appreciate like want care pay worth like resolved recommend true happy help want care pay bore fight stop waste agree accept allow like recommend support honor help pay blame cancel awful punish effectively fulfill improve recommend save thank guarantee trust honor win help problem problems warning better solution success matter good luck help warning penalty fail interruption losing problems killing loss luck resolve success Sentiment score * number of occurrences Words preceded by a negation Negations that contributed the most to sentiment (CA)
34. without won't no not can't don't −30 −20 −10 0 10 −2 −1 0 1 2 −40 −20 0 20 0 20 40 60 80 0 5 10 15 0 20 40 60 pay complain allow appreciate help honor support trust like care want helpful like fair true allow care honor help resolved want pay allow extend restore help honor cancel avoid leave pay honor help problem warning problems charges progress resolve thanks improvement luck success matter good help warning fail penalty losing worrying fear interrupted problems restriction trouble approval resolving success Sentiment score * number of occurrences Words preceded by a negation Negations that contributed the most to sentiment (FCC)
35. Thank you Questions or comments?
36. Next step • Machine learning • Other packages: sentimentr, algorithmia, …