Upgrade to Pro — share decks privately, control downloads, hide ads and more …

IoE: Using Elasticsearch to Unlock an Analytical Goldmine

Dd9d954997353b37b4c2684f478192d3?s=47 Elastic Co
March 10, 2015

IoE: Using Elasticsearch to Unlock an Analytical Goldmine

This talk was presented at the inaugural Elastic{ON} conference, http://elasticon.com

Session Abstract:

The true value of IoE starts to be realized not when you connect things, but when you are best able to use the data generated by those things. Not only is the final impact of a device event important, but the ability to analyze the cascading effects felt throughout the device ecosystem and different layers of the business is paramount. The Cisco Analytics Incubation Center developed a real time, data-driven, and rules-based framework to identify these cause-effect relationships. This talk will showcase how Elasticsearch has helped make it easy to utilize IoE data in real-time, and how critical it is to do so given the flood of IoE device data expected in the future.

Presented by Dr. Connie Brett & Vikram Kalyanaraman, Cisco

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

March 10, 2015
Tweet

More Decks by Elastic Co

Other Decks in Technology

Transcript

  1. IoE: Using Elasticsearch to Unlock an Analytical Goldmine Dr. Connie

    Brett and Vikram Kalyanaraman
  2. { } CC-BY-ND 4.0 2   Source: Cisco Consulting Services

    primary research, 2013 $6.4T Of  IoE  Opportunity  over  the  Next  10  Years  will  be  Realized  Through  Data  &  AnalyDcs   Data Monetization Customer Engagement Optimized Operations Threat Minimization Organizational Effectiveness
  3. { } CC-BY-ND 4.0 3   Mike  O'Callaghan–Pat  Tillman  Memorial

     Bridge   Business   Enablers   things people process data
  4. { } CC-BY-ND 4.0 4   { } CC-BY-ND 4.0

    What’s the state of my empire? ATM not working Sorry for the inconvenience All good Fine Umm … ATMs are down in SF Why are they down? Not sure Day 1
  5. { } CC-BY-ND 4.0 5   { } CC-BY-ND 4.0

    Let’s use IoE We need to know sooner We’ll build a dashboard Here’s some data from sensors We’ll crunch it offline and aggregate results Day 2
  6. { } CC-BY-ND 4.0 6   { } CC-BY-ND 4.0

    What’s the state of my empire? How does that data help? We built this dashboard Not sure Are any ATMs down? The average uptime for our network is 247,365 mins Day 3
  7. { } CC-BY-ND 4.0 7   Mike  O'Callaghan–Pat  Tillman  Memorial

     Bridge   Business   Enablers   things people process data
  8. { } CC-BY-ND 4.0 GOALS 8   Build  a  mechanism

     to  connect  relevant  data  from   devices  and  things  to  business  outcomes   1 Model  and  visualize  cause-­‐effect  rela3onships   across  IT  &  Business  layers  in  near  real-­‐Dme   2 Enable  rapid  root  cause  analysis  of  IT  issues   that  affect  the  business   3
  9. { } CC-BY-ND 4.0 9   Development Scenario: A large

    retailer with 100s of stores globally
  10. { } CC-BY-ND 4.0 Challenges 10   { } CC-BY-ND

    4.0         Volume   Types   Status   Data  Variety   Data  Velocity   Business  Value??     S E N S O R S   DATA   Sta3c  &  Highly   Aggregated  Dashboards   Delayed  Insights  from   Tradi3onal  Processing   VISUALIZATION  
  11. { } CC-BY-ND 4.0 Data & Workflow 11   {

    } CC-BY-ND 4.0 AggregaDons   AnalyDcs   Business  Rule  ApplicaDons   Filtering   Continual Analysis Engine Alerts & Events x x
  12. { } CC-BY-ND 4.0 Data & Workflow 12   {

    } CC-BY-ND 4.0 AggregaDons   AnalyDcs   Business  Rule  ApplicaDons   Filtering   Continual Analysis Engine Alerts & Events data  driven   visualiza3on   mongo  connector  
  13. { } CC-BY-ND 4.0 Visualization Business   Layers   Device

      Data   Business Rules
  14. { } CC-BY-ND 4.0 Cause-Effect Visualization RULES  ENGINE   connects

        CONCRETE  EVENTS   with     BUSINESS  HIERARCHIES   Ingest  Events   &  Alerts   Run   Rules   Propagate  effect  through  tree  using   Color  to  indicate  severity  of  effect   >   >  
  15. { } CC-BY-ND 4.0 15   LIVE DEMO

  16. { } CC-BY-ND 4.0 16   END OF DEMO

  17. { } CC-BY-ND 4.0 17   { } CC-BY-ND 4.0

    ATM not working Sorry for the inconvenience. We are aware of the issue and are working to resolve it. In the meanwhile, the nearest working ATM is just one block from your current location. Day 4 >   ATM 1 DOWN for 3 mins ATM 5 UP (nearest) What’s the state of my empire? We have current data on an outage of a few ATMs in SF. Customers are being redirected to nearby ATMs. We know the issue is localized and therefore impact to business is minimal.
  18. { } CC-BY-ND 4.0 The Causality Framework 18   LIGHTWEIGHT

      EASILY  CUSTOMIZED   LOW  MAINTENANCE   We  are  embedding  the  Causality   Framework  in  our  Connected   AnalyDcs  Products  and  SoluDons  
  19. { } CC-BY-ND 4.0 19   Q & A

  20. { } Special Thanks To: Dr. Connie Brett cbrett @cisco.com

    @cbdsabc Vikram Kalyanaraman vkalyana @cisco.com @vikalytics
  21. { } CC-BY-ND 4.0 This work is licensed under the

    Creative Commons Attribution-NoDerivatives 4.0 International License. To view a copy of this license, visit: http://creativecommons.org/licenses/by-nd/4.0/ or send a letter to: Creative Commons PO Box 1866 Mountain View, CA 94042 USA { 21