POPmine: public opinion tracking using social media & online news

Presentated by Pedro Saleiro (pedrosaleiro at gmail.com) at the Oporto MUG #2 (Dec 17, 2013)

Transcript

1. POPmine: public opinion tracking using social media & online news 16th December 2013 Pedro Saleiro pedrosaleiro@gmail.com
2. context POPSTAR • FCT funded R&D project • explores social media & online news • creates indicators of frequency & opinion of mentions to political actors • trend analysis and comparison to traditional survey based indicators
3. context POPSTAR
4. objectives POPSTAR • Collect tweets, facebook posts, blog posts & online news • Opinion classification of retrieved messages • Authority and influence detection • Data aggregation (by date, user, topic, personality, opinion score, …)
5. requirements POPSTAR • Modularity & Scalability • Flexibility ◦ reusable modules, well defined interfaces ◦ new features = reduced development effort • Batch + real time processing • Easiness to replicate, install & low maintenance
6. architecture POPmine
11. POPmine • mongoDB rocks! • Features were not closed! Even now... • We wanted to create time series (not relational) • We needed fast prototyping
12. Tweets Tweets mentions Hourly aggregation Daily aggregation Custom aggregation Targets
13. POPmine: 1. get data (insert) 2. classify data (find + update) 3. aggregate data (map_reduce)
14. • POPmine ◦ don’t need to know any data model ◦ get tweets, blog posts and news ◦ for each in response: insert(each)
18. • Buzz share between targets • Sentiment aggregators:
24. et voilá www.popstar.pt
25. join, invite, learn, share @mongodb_opo