Tweet Emotion: Realtime Tweet Pseudo-Analysis with PubNub Data Stream

creating a cartographic visualization using D3.js, Twitter data and the PubNub API.

Transcript

1. Realtime Tweet Analysis with PubNub Data Stream Tomomi Imura
2. "How is America feeling now?" Visualized cartographic data of the emotional "state" of each state at any given moment 2
3. http://pubnub.github.io/tweet-emotion/ 3
4. Pseudo-analysis Positive or negative sentiments are loosely determined based on an analysis of keywords used in tweets p o s i t i v e W o r d s = [ ' e x c e l l e n t ' , ' a m a z i n g ' , ' m a r v e l o u s ' , ' m a g n i f i c e n t ' , ' f a b u l o u s ' , ' a s t o n i s h i n g ' , ' f a n t a s t i c ' , ' p e a c e f u l ' , ' b r i l l i a n t ' , ' g l o r i o u s ' , ' c h e e r f u l ' , ' g r a c i o u s ' , . . . ] ; 4
5. Process 1.  using TopoJSON to display a US state map using d3.js 2.  using PubNub JavaScript API to consume Twitter stream 3.  colorizing the map by moods 5
6. GeoJSON •  Open standard format for encoding geographical features •  Written in JavaScript Object Notation •  Typically large file size 6
7. TopoJSON •  Extension of GeoJSON that encodes geospatial topology •  Compressed / Smaller size •  Tailored for only what you need 7
8. TopoJSON 8
9. Render TopoJSON with D3.js v a r s v g = d 3 . s e l e c t ( ' # m a p ' ) . a p p e n d ( ' s v g ' ) . a t t r ( ' w i d t h ' , 9 0 0 ) . a t t r ( ' h e i g h t ' , 5 5 0 ) ; v a r p a t h = d 3 . g e o . p a t h ( ) . p r o j e c t i o n ( d 3 . g e o . a l b e r s U s a ( ) ) ; 9
10. Render TopoJSON with D3.js v a r g = s v g . a p p e n d ( ' g ' ) ; d 3 . j s o n ( ' u s - s t a t e s . j s o n ' , f u n c t i o n ( e r r o r , t o p o l o g y ) { g . s e l e c t A l l ( ' p a t h ' ) . d a t a ( t o p o j s o n . f e a t u r e ( t o p o l o g y , t o p o l o g y . o b j e c t s . u s S t a t e s ) . f e a t u r e s ) . e n t e r ( ) . a p p e n d ( ' p a t h ' ) . a t t r ( ' c l a s s ' , f u n c t i o n ( d ) { r e t u r n ' s t a t e s ' + d . p r o p e r t i e s . S T A T E _ A B B R ; } ) . a t t r ( ' d ' , p a t h ) . a t t r ( ' f i l l ' , ' g r a y ' ) } ) ; 10
11. Getting Realtime Tweets from PubNub Stream 11
12. Getting Started with PubNub APIs < s c r i p t s r c = " / / c d n . p u b n u b . c o m / p u b n u b . m i n . j s " > < / s c r i p t > v a r c h a n n e l = ' p u b n u b - t w i t t e r ' ; v a r p u b n u b = P U B N U B . i n i t ( { s u b s c r i b e _ k e y : ' s u b - c - 7 8 8 0 6 d d 4 - 4 2 a 6 - 1 1 e 4 - a e d 8 - 0 2 e e 2 d d a b 7 f e ' } ) ; 0 1 . 0 2 . 12
13. Receiving Live JSON Data To retrieve the live tweets, you simply use PubNub s u b s c r i b e ( ) API. p u b n u b . s u b s c r i b e ( { c h a n n e l : c h a n n e l , c a l l b a c k : p r o c e s s D a t a } ) ; 13
14. Pseudoanalyze Tweets Create a mini dictionary that describes moods v a r p o s i t i v e W o r d s = [ ' e x c e l l e n t ' , ' a m a z i n g ' , ' h a p p y ' , ' m a g n i f i c e n t ' , ' : ) ' , . . . ] ; v a r n e g a t i v e W o r d s = [ ' u n h a p p y ' , ' s o r r y ' , ' a n n o y e d ' , ' d i s l i k e ' , ' a n x i o u s ' , ' : ( ' , . . . ] ; 14
15. Pseudoanalyze Tweets See if these words are used in each tweet, in the callback function f u n c t i o n p r o c e s s D a t a ( d a t a ) { / / l o o k i n g f o r U S d a t a o n l y f o r t h i s d e m o i f ( d a t a . p l a c e . c o u n t r y _ c o d e ! = = ' U S ' ) r e t u r n ; / / C h e c k i f e m o t i o n a l w o r d s a r e u s e d i n t h e t w e e t i f ( p o s i t i v e W o r d s . s o m e ( f u n c t i o n ( v ) { r e t u r n d a t a . t e x t . t o L o w e r C a s e ( ) . i n d e x O f ( v ) > 0 ; } ) ) { d i s p l a y D a t a ( d a t a , p o s i t i v e ) ; } e l s e i f . . . } 15
16. Pseudoanalyze Tweets Create a mini dictionary that describes moods v a r p o s i t i v e W o r d s = [ ' e x c e l l e n t ' , ' a m a z i n g ' , ' h a p p y ' , ' m a g n i f i c e n t ' , ' : ) ' , . . . ] ; v a r n e g a t i v e W o r d s = [ ' u n h a p p y ' , ' s o r r y ' , ' a n n o y e d ' , ' d i s l i k e ' , ' a n x i o u s ' , ' : ( ' , . . . ] ; 16
17. Display Data on Map f u n c t i o n d i s p l a y D a t a ( d a t a , e m o t i o n ) { v a r s t a t e = d a t a . p l a c e . f u l l _ n a m e . s u b s t r i n g ( c i t y . l a s t I n d e x O f ( ' , ' ) + 1 ) . t r i m ( ) ; i f ( d o c u m e n t . q u e r y S e l e c t o r ( ' . ' + s t a t e ) ) { v a r s t a t e E l = d o c u m e n t . q u e r y S e l e c t o r ( ' . ' + s t a t e ) ; / / c o u n t t a l l y [ s t a t e ] = ( t a l l y [ s t a t e ] | | { p o s i t i v e : 0 , n e g a t i v e : 0 } ) ; t a l l y [ s t a t e ] [ e m o t i o n ] = ( t a l l y [ s t a t e ] [ e m o t i o n ] | | 0 ) + 1 ; . . . 17
18. ...cont'd / / c o l o r i z e s v g p a t h s t a t e E l . s t y l e . f i l l = ( t a l l y [ s t a t e ] . p o s i t i v e > t a l l y [ s t a t e ] . n e g a t i v e ) ? ' p i n k ' : ( ( t a l l y [ s t a t e ] . p o s i t i v e < t a l l y [ s t a t e ] . n e g a t i v e ) ? ' t u r q u o i s e ' : ' b e i g e ' ) ; } } } 18
20. Full Article  blog.twitter.com 20
21. Thank you! Tomomi Imura @ PubNub  pubnub.com  @pubnub  @girlie_mac  github.com/pubnub 21
22. Photo Credit •  Cover: https://flic.kr/p/9id7d9 by Kate Ter Haar b 22