Making sense of your data to give you actionable insights

Presented at Findability Day 2014, Copenhagen, Sept. 11, 2014

The capabilities of Elasticsearch go beyond pure search. With the ELK stack (Elasticsearch, Logstash & Kibana), you have a cutting-edge technology for data ingestion, indexing and visualization. ELK is widely deployed by companies to help them understand their data. Yellow Brand Protection is a great example of how Elasticsearch is used to prevent brand counterfeit (which is a session that will be presented by Klas Karlsson). To compliment this session we'll use some real-life data and demonstrate the analytics you can perform displayed in a Kibana dashboard. By the end of this session you'll leave with ideas and inspiration for performing meaningful analytics for your own projects.

Transcript

1. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Making sense of your data 
 to give you actionable insights Presented by: Dimitri Marx, Sales Engineer ! Team here today: Sejal Korenromp, Arie Chapman
2. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Copyright  Elasticsearch  2014.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited What about Elasticsearch the Company? ! • Drive the ELK Projects forward, great things to come! • Commercial products: Marvel to monitor and manage ELK • Support 100s of Companies in Production environments • Training Developers and Ops around the world on ELK
3. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Copyright  Elasticsearch  2014.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Copyright  Elasticsearch  2014.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Who’s using Elasticsearch?
4. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited A quick history of Search Who doesn’t like a history lesson :-) ! ! ! ! ! Disclaimer: includes fire
5. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Life before Elasticsearch!
6. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Why Search?
7. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited How else would you find this?
8. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Or this?
9. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Or this? Eyjafjallajökull
10. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Kinds of Search
11. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited People search
12. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Geographic Search
13. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Product Search
14. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Knowledge Search
15. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Thanks for listening!
16. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Something else?
17. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited USS Car Carrier
18. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Search is more than… ! ?q=what+is+the+difference+between
 +holland+and+netherlands" ! Don’t know? Watch this:" https://www.youtube.com/watch?v=eE_IUPInEuc
19. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited What if we could ask questions like…
20. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited What searches are trending right now? Where are my customers coming from right now? Are there any anomalies in our data right now? How long is it taking for my customers to get 
 through our service process? Are we the victims of a DDOS attack right now?
21. The logging stack: ES + LS + KB
22. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Copyright  Elasticsearch  2014.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited Data From Any Source Instantly Analyze Actionable Insights The ELK Stack Logstash Elasticsearch Kibana
23. Clean & enrich before storage Mobile Devices Financial Web Trends Data Game Data CRM News Articles Log Data e-commerce Social Data Internet of things Enrich: Geo code IP address, aggregate fields, etc. Clean: Filter out junk, mask, normalize into JSON Logstash 
 Collect From Any Source
24. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch real time, 
 search and 
 analytics engine distributed
25. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch real time, 
 search and 
 analytics engine distributed scales
 massively
26. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch real time, 
 search and 
 analytics engine distributed scales
 massively high
 availability
27. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch real time, 
 search and 
 analytics engine distributed scales
 massively high
 availability RESTful
 API
28. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch real time, 
 search and 
 analytics engine distributed scales
 massively high
 availability RESTful
 API JSON 
 over HTTP
29. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch real time, 
 search and 
 analytics engine distributed scales
 massively high
 availability RESTful
 API JSON 
 over HTTP schema
 free
30. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch real time, 
 search and 
 analytics engine distributed scales
 massively high
 availability RESTful
 API JSON 
 over HTTP schema
 free multi
 tenancy
31. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch real time, 
 search and 
 analytics engine open-source distributed scales
 massively high
 availability RESTful
 API JSON 
 over HTTP schema
 free multi
 tenancy
32. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch real time, 
 search and 
 analytics engine open-source Lucene
 based distributed scales
 massively high
 availability RESTful
 API JSON 
 over HTTP schema
 free multi
 tenancy
33. Kibana: Easily Explore Data • See the Bigger “Picture” from your data • Explore from 50K feet down to the ground • Bring together a variety of Machine and User Data, New Correlations • Constant query - not a point in a time - live stream
34. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Combine Search and Analytics on structured and unstructured data in a scalable way
35. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited You can always go back to this
36. Copyright  Elasticsearch  2013.  Copying,  publishing  and/or  distributing  without  written  permission  is  strictly  prohibited It’s showtime!