Realtime Tweet Analysis with PubNub Data Stream Tomomi Imura 

"How is America feeling now?" Visualized cartographic data of the emotional "state" of each state at any given moment 2 

http://pubnub.github.io/tweet-emotion/ 3 

Pseudo-analysis Positive or negative sentiments are loosely determined based on an analysis of keywords used in tweets p o s i t i v e W o r d s = [ ' e x c e l l e n t ' , ' a m a z i n g ' , ' m a r v e l o u s ' , ' m a g n i f i c e n t ' , ' f a b u l o u s ' , ' a s t o n i s h i n g ' , ' f a n t a s t i c ' , ' p e a c e f u l ' , ' b r i l l i a n t ' , ' g l o r i o u s ' , ' c h e e r f u l ' , ' g r a c i o u s ' , . . . ] ; 4 

Process 1.  using TopoJSON to display a US state map using d3.js 2.  using PubNub JavaScript API to consume Twitter stream 3.  colorizing the map by moods 5 

GeoJSON •  Open standard format for encoding geographical features •  Written in JavaScript Object Notation •  Typically large file size 6 

TopoJSON •  Extension of GeoJSON that encodes geospatial topology •  Compressed / Smaller size •  Tailored for only what you need 7 

TopoJSON 8 

Render TopoJSON with D3.js v a r s v g = d 3 . s e l e c t ( ' # m a p ' ) . a p p e n d ( ' s v g ' ) . a t t r ( ' w i d t h ' , 9 0 0 ) . a t t r ( ' h e i g h t ' , 5 5 0 ) ; v a r p a t h = d 3 . g e o . p a t h ( ) . p r o j e c t i o n ( d 3 . g e o . a l b e r s U s a ( ) ) ; 9 

Render TopoJSON with D3.js v a r g = s v g . a p p e n d ( ' g ' ) ; d 3 . j s o n ( ' u s - s t a t e s . j s o n ' , f u n c t i o n ( e r r o r , t o p o l o g y ) { g . s e l e c t A l l ( ' p a t h ' ) . d a t a ( t o p o j s o n . f e a t u r e ( t o p o l o g y , t o p o l o g y . o b j e c t s . u s S t a t e s ) . f e a t u r e s ) . e n t e r ( ) . a p p e n d ( ' p a t h ' ) . a t t r ( ' c l a s s ' , f u n c t i o n ( d ) { r e t u r n ' s t a t e s ' + d . p r o p e r t i e s . S T A T E _ A B B R ; } ) . a t t r ( ' d ' , p a t h ) . a t t r ( ' f i l l ' , ' g r a y ' ) } ) ; 10 

Getting Realtime Tweets from PubNub Stream 11 

Getting Started with PubNub APIs < s c r i p t s r c = " / / c d n . p u b n u b . c o m / p u b n u b . m i n . j s " > < / s c r i p t > v a r c h a n n e l = ' p u b n u b - t w i t t e r ' ; v a r p u b n u b = P U B N U B . i n i t ( { s u b s c r i b e _ k e y : ' s u b - c - 7 8 8 0 6 d d 4 - 4 2 a 6 - 1 1 e 4 - a e d 8 - 0 2 e e 2 d d a b 7 f e ' } ) ; 0 1 . 0 2 . 12 

Receiving Live JSON Data To retrieve the live tweets, you simply use PubNub s u b s c r i b e ( ) API. p u b n u b . s u b s c r i b e ( { c h a n n e l : c h a n n e l , c a l l b a c k : p r o c e s s D a t a } ) ; 13 

Pseudoanalyze Tweets Create a mini dictionary that describes moods v a r p o s i t i v e W o r d s = [ ' e x c e l l e n t ' , ' a m a z i n g ' , ' h a p p y ' , ' m a g n i f i c e n t ' , ' : ) ' , . . . ] ; v a r n e g a t i v e W o r d s = [ ' u n h a p p y ' , ' s o r r y ' , ' a n n o y e d ' , ' d i s l i k e ' , ' a n x i o u s ' , ' : ( ' , . . . ] ; 14 

Pseudoanalyze Tweets See if these words are used in each tweet, in the callback function f u n c t i o n p r o c e s s D a t a ( d a t a ) { / / l o o k i n g f o r U S d a t a o n l y f o r t h i s d e m o i f ( d a t a . p l a c e . c o u n t r y _ c o d e ! = = ' U S ' ) r e t u r n ; / / C h e c k i f e m o t i o n a l w o r d s a r e u s e d i n t h e t w e e t i f ( p o s i t i v e W o r d s . s o m e ( f u n c t i o n ( v ) { r e t u r n d a t a . t e x t . t o L o w e r C a s e ( ) . i n d e x O f ( v ) > 0 ; } ) ) { d i s p l a y D a t a ( d a t a , p o s i t i v e ) ; } e l s e i f . . . } 15 

Pseudoanalyze Tweets Create a mini dictionary that describes moods v a r p o s i t i v e W o r d s = [ ' e x c e l l e n t ' , ' a m a z i n g ' , ' h a p p y ' , ' m a g n i f i c e n t ' , ' : ) ' , . . . ] ; v a r n e g a t i v e W o r d s = [ ' u n h a p p y ' , ' s o r r y ' , ' a n n o y e d ' , ' d i s l i k e ' , ' a n x i o u s ' , ' : ( ' , . . . ] ; 16 

Display Data on Map f u n c t i o n d i s p l a y D a t a ( d a t a , e m o t i o n ) { v a r s t a t e = d a t a . p l a c e . f u l l _ n a m e . s u b s t r i n g ( c i t y . l a s t I n d e x O f ( ' , ' ) + 1 ) . t r i m ( ) ; i f ( d o c u m e n t . q u e r y S e l e c t o r ( ' . ' + s t a t e ) ) { v a r s t a t e E l = d o c u m e n t . q u e r y S e l e c t o r ( ' . ' + s t a t e ) ; / / c o u n t t a l l y [ s t a t e ] = ( t a l l y [ s t a t e ] | | { p o s i t i v e : 0 , n e g a t i v e : 0 } ) ; t a l l y [ s t a t e ] [ e m o t i o n ] = ( t a l l y [ s t a t e ] [ e m o t i o n ] | | 0 ) + 1 ; . . . 17 

...cont'd / / c o l o r i z e s v g p a t h s t a t e E l . s t y l e . f i l l = ( t a l l y [ s t a t e ] . p o s i t i v e > t a l l y [ s t a t e ] . n e g a t i v e ) ? ' p i n k ' : ( ( t a l l y [ s t a t e ] . p o s i t i v e < t a l l y [ s t a t e ] . n e g a t i v e ) ? ' t u r q u o i s e ' : ' b e i g e ' ) ; } } } 18 

19 

Full Article  blog.twitter.com 20 

Thank you! Tomomi Imura @ PubNub  pubnub.com  @pubnub  @girlie_mac  github.com/pubnub 21 

Photo Credit •  Cover: https://flic.kr/p/9id7d9 by Kate Ter Haar b 22