Why we built Elasticsearch

In this talk Shay Banon outlines the reason why we built Elasticsearch, and how and why it’s being used by companies like Foursquare, GitHub, Facebook, SoundCloud and many more.

Transcript

1. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Data Zen or why we built elasticsearch Shay Banon - @kimchy
2. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited just 140 characters? Tweet
3. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited just 140 characters? Tweet
4. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited just a message? Log I’m broken. Please show this to someone who can fix can fix
5. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited just a message? Log I’m broken. Please show this to someone who can fix can fix timestamp code location hostname ip address process parameter values request id who committed this?!!!
6. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited just a number? Metric URL: https://download.elasticsearch.org/ elasticsearch/elasticsearch-0.90.1.zip
7. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited just a number? Metric URL: https://download.elasticsearch.org/ elasticsearch/elasticsearch-0.90.1.zip timestamp remote ip geo location host name package format product
8. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited just code? Code
9. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited just a lat/lon? Location
10. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited just a lat/lon? Location venue rating reviews name city country neighborhood
11. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask show me the tweets that mention obama
12. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask show me the tweets that mention obama unstructured
13. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask show me the tweets that mention obama unstructured in ohio
14. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask show me the tweets that mention obama unstructured in ohio structure
15. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask show me the tweets that mention obama unstructured in ohio structure in the past month
16. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask show me the tweets that mention obama unstructured in ohio structure in the past month moar structure
17. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask show me the tweets that mention obama in ohio in the past month total: 255010294
18. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask show me the tweets that mention obama in ohio in the past month broken by day analytics
19. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask Anything show me the tweets that mention romney in ohio in the past month broken by day analytics
20. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask Anything show me the tweets that mention romney in california in the past month broken by day analytics
21. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask Anything show me the tweets that mention romney in california in the past year broken by day analytics
22. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask Anything show me the tweets that mention romney in california in the past year broken by month analytics
23. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited and you shall be answered Ask Anything with as little data munging as possible!
24. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Data Triangulation text meta data analytics
25. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Data Triangulation unstructured structure aggregation
26. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited realtime is the only time Fresh!
27. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited what is realtime? Fresh! how quickly can we get results?
28. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited what is realtime? Fresh! how quickly can we get results? took: 70ms
29. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited what is realtime? Fresh! how quickly can we see new data?
30. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited what is realtime? Fresh! how quickly can we see new data? milliseconds!
31. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited what is realtime? Fresh! how big is the data?
32. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited what is realtime? Fresh! how big is the data? irrelevant (but make sure enough HW)
33. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited collocation Data Fight Club
34. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited collocation Data Fight Club the first rule of distributed systems collocation
35. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited collocation Data Fight Club the second rule of distributed systems collocation
36. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited collocation Data Fight Club in order to achieve data triangulation a system should provide all of them
37. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Final Words elasticsearch!