Talks #6 - Mihai Pricochi - Opinion mining

Opinion mining, cunoscută original și ca sentiment analysis este o "ramură" a procesării limbajului natural. Principalul scop este de a determina informațiile de natură subiectivă dintr-un text, de aici și opinie/sentiment din titlu, ca de exemplu polaritatea unui document/review, parte pe care mă voi concentra eu.

Transcript

1. Opinion mining (Sentiment analysis)
2. What is opinion mining? • Application of natural language processing, computational linguistics and text analytics • Aims to determine the attitude of a speaker/writer, or the polarity of a document
3. What can it do? • polarity classification • “beyond polarity” classification such as emotion states (angry, sad, happy) • subjectivity/objectivity identification • feature/aspect-based sentiment analysis
4. How tools does it use? • Scaling systems • Latent semantic analysis • Support vector machines • “bag of words”
5. Polarity classification, sounds simple? • How would you do it?
6. Contrast with standard fact-based textual analysis • Different from text categorization, which can have many classes; Opinion mining has relatively few classes (positive/negative, 3/5 stars). • The extracted information is different. • The templates for opinion-interested information extraction often generalize well across different domains • Compared to topic, sentiment can often be expressed in a more subtle manner
7. What makes opinion mining difficult? • Sarcasm • Irony • Generally, linguistic subtleties
8. Degrees of positivity • Let’s talk about phones
9. Degrees of positivity • Let’s talk about phones • Does “long battery life” sound positive?
10. Degrees of positivity • Let’s talk about phones • Does “long battery life” sound positive? • How about “the battery lasts 8 hours”?
11. Degrees of positivity • Let’s talk about phones • Does “long battery life” sound positive? • How about “the battery lasts 8 hours”? • How about “the battery lasts only 8 hours”?
12. Degrees of positivity • Let’s talk about phones • Does “long battery life” sound positive? • How about “the battery lasts 8 hours”? • How about “the battery lasts only 8 hours”? • Is only the key word here?
13. Degrees of positivity • Let’s talk about phones • Does “long battery life” sound positive? • How about “the battery lasts 8 hours”? • How about “the battery lasts only 8 hours”? • Is only the key word here? • How about “the phone weights 125 grams” versus “the phone weights only 125 grams”?
14. Parts of speech • Adjectives seem very important. Are they the most important?
15. Parts of speech • Adjectives seem very important. Are they the most important? • Nouns are important too (“this movie is a gem”)
16. Parts of speech • Adjectives seem very important. Are they the most important? • Nouns are important too (“this movie is a gem”) • Let’s not forget verbs (“I love this movie”)
17. Negation • “I like this book” and “I don’t like this book” may sound very similar to a computer • But negation doesn’t necessary mean reversal of polarity, for example “No wonder this is considered one of the best”
18. Text structure • It’s been noticed, most of the times, in reviews, the last phrase serves as a conclusion to the entire text • This may completely change the polarity of the review in some cases • “The lighting was bad, the music was out of sync with the events… But in the end I think it was a very good movie”
19. Dataset • Polarity dataset v2.0 – Movie Review Data • 1000 positive and 1000 negative processed reviews. Introduced in Pang/Lee ACL 2004. Released June 2004.
20. How good is good enough? • How does 70% accuracy sound?
21. How good is good enough? • How does 70% accuracy sound? • The inter-rater reliability is around 79% • So even 100% accuracy with the test datasets would mean raters/reviewers would disagree with the results about 20% of the times in real life situations
22. Best result until now? • 90.2% • Using: • Bag of Words – relative frequencies of all words in the text • Appraisal Group by Attitude & Orientation -Total frequency of appraisal groups with each possible combination of Attitude and Orientation, normalized by total number of appraisal groups in the text.
23. Sounds complicated? How about something simpler? • How about a system that requires little to no pre-processing?
24. Sounds complicated? How about something simpler? • How about a system that requires little to no pre-processing? • How about a system that is language independent?
25. Sounds complicated? How about something simpler? • How about a system that requires little to no pre-processing? • How about a system that is language independent? • How about a character-based analysis system?
26. How would it work? (short version) • We find all character n-grams (groups of n characters) in all the texts • We compare all texts with each other based on the n-grams they have in common and we get a score for each one • We then apply a string kernel on the results and see if it works
27. Does it work?
28. Does it work? • If 87% in cross-validation sounds enough, then yes, it does
29. Does it work? • If 87% in cross-validation sounds enough, then yes, it does • And according to what was said earlier it should be more than enough • It’s not the best system, but with 87% accuracy it’s one of the best • And compared to other systems it’s much easier to implement and should work just as well on other training datasets, including ones written in other languages (something most systems don’t support without prior data collection)
30. But why does it work? • The top score was obtained for 9-grams, that’s groups of 9 characters (including spaces, punctuation) • 9 characters means groups of about 2 words (in English), and these seem to be enough to extract information related to polarity in movie reviews • 7, 8, 10 n-grams also proved over 86% accuracy, and different n’s should work better or worse on different languages, mostly based on the average number of characters in a word in each language
31. I don’t really have an ending • But thanks for listening 