pycon_delhi_lightening

Lightening talk delivered at PyCon India 2016

Transcript

1. News classification with Gensim Devashish Deshpande Undergraduate student RaRe Technologies Incubator Program Github: dsquareindia Blogs: https://rare-technologies.com/blog/
2. Gensim: Topic modeling in python
3. Problem of News (mis)classification
4. Screenshots from play newsstand
5. Topic-word coloring with LDA Image taken from LDA paper by David Blei
6. What is a good LDA model? • Come up with good topics • Infer topic distribution (United topic): mourinho, red_devils, old_trafford, bad_team... (Arsenal topic): wenger, henry, invincibles,.... (City topic): aguero, etihad, england, premier_league (Chelsea topic): blues, football, roman, bridge,... Football LDA model
7. Evaluating topic models • Manually – Look at the topics. See if they are interpretable. – Comparing different topic models Qualititative
9. Topic Coherence • Quantitave
10. Topic Coherence • Assign a number to the human interpretability! Comparing topic models becomes much easier
11. Topic Coherence • Better LDA -> Better topics -> Better classification Topics from topic modeling tutorial on Lee corpus
12. Join the community! • Pick up issues from: https://github.com/RaRe-Technologies/gensim • Come for the sprint!