Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Data Mining 250 million Tweets a Day

Data Mining 250 million Tweets a Day

Presentation by Paul Gesiak, Data Scientist @DataSift at Data Science London

Data Science London

July 03, 2012
Tweet

More Decks by Data Science London

Other Decks in Technology

Transcript

  1. 5,0&(2VUHVLJQDWLRQ 5,0VRFLDOPHGLDFRQWHQW KRXUWLPHIUDPH -LP%DOVLOOLHDQG 0LNH/D]DULGLV WKHFR&(2VRI 5HVHDUFKLQ0RWLRQ DQQRXQFHGWKHLUUHVLJQDWLRQ 5,0KDVDQQRXQFHGRQ WKHLUZHEVLWHWKDWLWV

    &(2VDUHUHVLJQLQJ7KH QHZVEURNHRQ6XQGD\ HYHQLQJDQGJHQHUDWHG ODUJHVSLNHRIVRFLDOPHGLD DFWLYLW\ RYHUSRVWVDQ KRXU HYHU\VHF  7KHILUVWUHDFWLRQKDVEHHQ RYHUZKHOPLQJO\QHJDWLYH VKDUSVSLNHRIQHJDWLYH VHQWLPHQW UHGOLQH  
  2. *UDSKUHSUHVHQWLQJ SDWWHUQRIVRFLDOPHGLD DFWLYLW\JHQHUDWHG DURXQG5,0/WG    7LPHKRXU  

    1HZVEUHDNVRXW ,QIRUPDWLRQVSUHDGV WURXJKWKHQHWZRUN 5,0&(2VUHVLJQDWLRQ QHZVHYHQWLQVRFLDOPHGLDQHWZRUN
  3. *UDSKUHSUHVHQWLQJ SDWWHUQRIVRFLDOPHGLD DFWLYLW\JHQHUDWHG DURXQG5,0/WG    7LPHKRXU  

     &RPPHQWVDQG UHDFWLRQVWRWKHQHZV 5,0&(2VUHVLJQDWLRQ QHZVHYHQWLQVRFLDOPHGLDQHWZRUN
  4. 5,0&(2VUHVLJQDWLRQ 5,0VRFLDOPHGLDFRQWHQW PLQWLPHIUDPH %\LQFUHDVLQJWKH WHPSRUDOUHVROXWLRQ IURPKWRPLQWLPH IUDPHZHFDQVHHPRUH GHWDLOVRIWKHHYHQW 7KHWUDIILFSLFNHGXS MXVWDIWHU5,0

    DQQRXQFHGWKHQHZV RQWKHSP6XQGD\ (&7 ZKLFKLVDP 0RQGD\8&7DV VKRZQRQWKHSORW 7KH KLJKOHYHORILQWHUHVW ODVWVIRUDQKRXURUVR WKHQGHFOLQHVRYHU QLJKWWRLQFUHDVHLQ 1HZ<RUN6WRFN ([FKDQJHSUHPDUNHW KRXUVRQ0RQGD\ PRUQLQJ
  5. 5,0&(2VUHVLJQDWLRQ 5,0VRFLDOPHGLDFRQWHQW PLQWLPHIUDPH ,QFUHDVLQJWLPHLQWHUYDOVWR PLQIUDPHV 7KH5,0VWRFNRSHQVRQ 0RQGD\VLJQLILFDQWO\EHORZ WKH)ULGD\FORVHSULFHOHYHO DQGNHHSIDOOLQJLQWKHILUVW PLQ,WLVUDWKHU

    SUHGLFWDEOHFRQVLGHULQJWKH QDWXUHRIWKHQHZVDQGWKH VHQWLPHQWRIFRPPHQWV IURPSUHYLRXVHYHQLQJ +RZHYHUWKHVHQWLPHQWRI VRFLDOPHGLDFRQWHQW VZLWFKHVWRSRVLWLYHYDOXH DURXQG (&7  GHVSLWHSULFHVWLOOIDOOLQJ PLQODWWHUWKHVWRFNSULFH UHDFKHVWKHVXSSRUWOHYHODW 7KLVOHYHOLVWHVWHG DJDLQDIWHUDQG KROGVDVWKHVHQWLPHQWRI VRFLDOPHGLDFRQWHQW UHPDLQSRVLWLYH 
  6. 5,0&(2VUHVLJQDWLRQ 'DWDDJJUHJDWLRQ &RQFOXVLRQ  %\DQDO\VLQJDIORZRIWKHVRFLDOPHGLDFRQWHQWZHJDLQWZRLQWHUHVWLQJSLHFHVRILQIRUPDWLRQ  :HFDQVHHLPPHGLDWHO\ZKDWLVWKHFURZGUHDFWLRQIRUWKHQHZV  D 7KHVLJQLILFDQFHRIWKHQHZVHYHQWLVSURSRUWLRQDOWRWKHVSLNHRIWUDIILFJHQHUDWHG

     E 7KHGLIIHUHQFHEHWZHHQSRVLWLYHDQGQHJDWLYHVHQWLPHQWLQGLFDWHVLIWKHUHDFWLRQWRWKHQHZVLV SRVLWLYHQHJDWLYHRULQGLIIHUHQW  $QLQFUHDVHRISRVLWLYHVHQWLPHQWSUHFHGHGWKHPRPHQWZKHQDVWRFNSULFHKDGVWRSSHGIDOOLQJDQGUHDFKHG DVXSSRUWOHYHO  D 'LGWKHPDUNHWGHFLGHDWWKLVSRLQWWKDWWKHSULFHKDVEHHQGLVFRXQWHGHQRXJK"&HUWDLQO\WKH VHQWLPHQWRIWKHVRFLDOPHGLDFRQWHQWFRXOGVXJJHVWVR  E /RRNLQJDWWKH0RQGD\PRUQLQJFKDUWZHFDQVHHWKDWWKHVHQWLPHQWOHYHORIVRFLDOPHGLDFRQWHQW EHLQJDQLQWHUHVWLQJLQGLFDWRUVKRZLQJHPRWLRQVRIWKHFURZG)XUWKHUPRUHWKLVLQIRUPDWLRQFDQEH WUDFHGXVLQJYHU\VKRUWWLPHIUDPHV HYHQPLQ