Upgrade to Pro — share decks privately, control downloads, hide ads and more …

InsightDemo_3

CJ
February 26, 2015

 InsightDemo_3

CJ

February 26, 2015
Tweet

Other Decks in Technology

Transcript

  1. Motivations • Tool to monitor occurrence and location of extreme

    weather events in real time • Unified platform to integrate streaming data from very large number of sensors • Store global meteorological information - integrate historical data - model testing • API to help local agencies in planning for the future
  2. Data aggregation 0 10 20 30 40 50 60 70

    80 90 100 0 10 20 30 40 50 60 70 80 90 100 Temperature Monthly average temperatures San Francisco, CA Pittsburgh, PA
  3. Measuring extremes -140 -120 -100 -80 -60 -40 -20 0

    20 40 Lowest Temperature (F) Sensors in Antarctica Data must be put in proper context
  4. Lessons learned • Pipeline is can be adopted for other

    situations • HBase is resource hungry • Other alternatives might be more appropriate for the time series nature of data such as druid, opentsdb • Apache Storm is awesome • Weather in San Francisco is great. Definitely move here from Pittsburgh!