Sentimental Headlines

These are the slides for my Devbootcamp final project. Repo is here: https://github.com/kelmerp/headline_sentiment_rating

Transcript

1. MO DATA, MO PROBLEMS Sunday, October 13, 13
2. HEADLINES SAY A LOT Sunday, October 13, 13
3. THE SCRIPT unless blacklist.include?(a.text) headlines << a.text a.text.strip! h1 a Sunday, October 13, 13
4. THE SCRIPT unless blacklist.include?(a.text) headlines << a.text a.text.strip! h1 a h4 a h2 a h3 a a Sunday, October 13, 13
5. THE BLACKLIST Blacklist Text [“home”, “log in”, “Sports section”, “t.co/blah/junk”, ...] Blacklist Sunday, October 13, 13
6. THE BLACKLIST Previous day Current day Blacklist [“home”, “log in”, “Sports section”, “t.co/blah/junk”, ...] Blacklist Match? [“Headline”, “Headline”, “Weather”, “Headline”, “Headline”, ... ] “Weather” [“Headline”, “Headline”, “Weather”, “Headline”, “Headline”, ... ] Sunday, October 13, 13
7. OK... BUT HOW? Internet Archives Sentiment Analysis Engine Sunday, October 13, 13
8. One day’s headlines 1. Headline 2. Headline 3. Headline ... 20. Headline -.23 0.12 -.45 (-1..1) 0.0 Content: Alchemy Score: Average: -.142 PLOTTING (CNN) -1 1 Sunday, October 13, 13
9. One day’s headlines Average: -.142 PLOTTING (CNN) -1 1 Next day’s headlines 1. Headline 2. Headline 3. Headline ... 20. Headline -.1 0.32 -.05 (-1..1) 0.0 Content: (CNN) Average: 0.12 1. Headline 2. Headline 3. Headline ... 20. Headline -.23 0.12 -.45 (-1..1) 0.0 Content: Alchemy Score: Alchemy Score: Sunday, October 13, 13
10. PLOTTING -1 1 Jan Feb Mar Sunday, October 13, 13
11. PLOTTING -1 1 Jan Feb Mar Jan. Average Feb. Average Sunday, October 13, 13
12. PLOTTING -1 1 J F M A M J J A S Sunday, October 13, 13
13. THE PROCESS Text 3 Hour sprints Feature branches Version commits Days 1-3: Days 4-6: Days 7-9: Set-up database Built Script Scraped first 50,000 headlines Fed headlines through Alchemy Graphed D3 scatterplots Scraped next 50,000 headlines Accomplished calendar view Decreased load time to < 3 secs Polished front-end Sunday, October 13, 13
14. UNIQUE CHALLENGES Big data = cumbersome D.B. optimization matters Statistical analysis is HARD others... Sunday, October 13, 13
15. Intro/Idea (30 seconds) The tools (1 min) wayback machine, Alchemy API The script (2 min) wayback gem, nokogiri logic/key points The scatter plot (2 min) math/logic behind it D3 show calendar..? or wait till demo Demo (1 min) have key dates picked out The Process (1 -2 min) 3 hour sprints feature branches commits Unique challenges (1-2mins) Big data is cumbersome DB optimization Sunday, October 13, 13