Introduction to Elasticsearch

Introduction to Elasticsearch

New England Search Technologies (NEST) Group Meetup

Ba4e53e23b4b96623fdbcd0eae2af923?s=128

Igor Motov

March 25, 2015
Tweet

Transcript

  1. Introduction to Elasticsearch Igor Motov Software Engineer 03/25/2015

  2. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 2 Elastic • Founded in 2012 • By the people behind the Elasticsearch and Apache Lucene • Renamed to Elastic in 2015 • Headquarters: Amsterdam and Mountain View, CA • We provide • Training (public & onsite) • Development support • Production support • Hosted elasticsearch
  3. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 3 Enough about me, let’s talk about you for a bit!
  4. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 4 Have you used Elasticsearch recently?
  5. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 5 Have you used elasticsearch recently?
  6. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 6 Some Elasticsearch users
  7. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 7 What’s Elasticsearch • Real time search and analytics engine • JSON-oriented, Apache Lucene-based • Automatic Schema Detection • With full control of schema when needed • Distributed • Scales Up+Out, Highly Available • Multi-tenancy • Dynamically create/delete indices • API centric • Most functionality is exposed through an API
  8. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 8 Elasticsearch Documents • Identified by index/type/id • JSON Document: { "author": "imotov", "title": "Introduction to Elasticsearch", "post_date": "2015-03-25", "size": 30 }
  9. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 9 Elasticsearch Download
  10. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 10 Elasticsearch distribution . ├── LICENSE.txt ├── NOTICE.txt ├── README.textile ├── bin │ ├── elasticsearch │ ├── elasticsearch.in.sh │ └── plugin ├── config │ ├── elasticsearch.yml │ └── logging.yml ├── data │ └── elasticsearch ├── lib │ ├── elasticsearch-x.y.z.jar │ ├── ... │ └── └── logs ├── elasticsearch.log └── elasticsearch_index_search_slowlog.log
  11. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 11 DEMO
  12. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 12 Communicating with Elasticsearch • REST Protocol • Binary Protocol
  13. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 13 Communicating with Elasticsearch • REST API • JavaScript • Perl • PHP • Python • Ruby • .NET • Binary Protocol • Java • Groovy
  14. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 14 It’s not just search
  15. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 15 Aggregations
  16. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 16 Aggregations
  17. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 17 Aggregations • Group by • values • ranges • histogram • time intervals (years/months/days…) • IP ranges • geo location • …
  18. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 18 • Calculate • counts • max • min • avg • sum • standard deviation (with bounds) • variance • sum of squares • percentile rank
  19. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 19 DEMO
  20. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 20 Aggregations
  21. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 21 On a large scale!
  22. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 22 Let’s talk about scale and high availability.
  23. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 23 Scale and High Availability node1 0 1 2 3 4 hash(id)%5 M
  24. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 24 High Availability node1 0 1 2 3 4 node2 M
  25. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 25 High Availability node1 0 1 2 3 4 node2 0 1 2 3 4 M
  26. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 26 High Availability node1 0 1 2 3 4 hash(id%5) node1 0 1 2 3 4 M
  27. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 27 Scale node1 0 2 4 node2 1 2 3 node3 0 1 3 4 M
  28. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 28 Scale node1 0 2 4 node2 1 2 3 node3 0 1 3 4 M
  29. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 29 Scale node1 0 2 node2 1 2 3 node3 1 3 4 0 4 M
  30. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 30 Scale node1 0 2 node2 1 2 3 node3 1 3 4 0 4 M
  31. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 31 Master is gone node2 1 2 3 node3 3 4 0 4
  32. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 32 Master is elected node2 1 2 3 node3 3 4 0 4 M
  33. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 33 Primary are promoted node2 1 2 3 node3 3 4 0 4 M
  34. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 34 High Availability node2 1 2 3 node3 3 4 0 4 M
  35. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 35 High Availability is restored node2 1 2 3 node3 3 4 0 4 M 1 2 0
  36. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 36 Elasticsearch ecosystem
  37. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 37 Elasticsearch Ecosystem
  38. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 38 Elasticsearch Ecosystem logstash
  39. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 39 Elasticsearch Ecosystem logstash
  40. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited Shield 40 Elasticsearch Ecosystem logstash
  41. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 41 Elasticsearch Resources
  42. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 42 Elasticsearch - The Definitive Guide http://www.elastic.co/guide/en/elasticsearch/guide/current/index.html
  43. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 43 Elasticsearch Meetups
  44. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 44 http://www.meetup.com/Elasticsearch-Boston/ Boston Elasticsearch Meetup
  45. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 45
  46. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 46 Elasticsearch Training http://purchases.elastic.co/
  47. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 47 Questions?
  48. Thank you! www.elastic.co igor@elastic.co @imotov github.com/imotov