Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Whats new in Elasticsearch?

Whats new in Elasticsearch?

This presentation covers the most interesting features of Elasticsearch 1.0 and 1.1 and gives a short forecast on the most interesting features of the upcoming release.

Elasticsearch Inc

May 06, 2014
Tweet

More Decks by Elasticsearch Inc

Other Decks in Technology

Transcript

  1. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Whats new in Elasticsearch? Covering 1.0, 1.1, 1.2 & 1.x Alexander Reelsen @spinscale [email protected]
  2. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Agenda • Introduction • Elasticsearch • 1.0: Aggregations, Snapshot/Restore, Percolator, cat API • 1.1: More Aggregations, Search template, Recovery • 1.2: More Aggregations, Context Suggester, global ordinals • 1.3: More Aggregations, Benchmark API • Q & A
  3. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited about • Me Interested in metrics, ops and the web Likes the JVM Working with elasticsearch since 2011 • Elasticsearch, founded in 2012 Products: Elasticsearch, Logstash, Kibana, Marvel Professional services: Support & development subscriptions Trainings
  4. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Introduction
  5. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited http://www.elasticsearch.org/blog/elasticsearch-definitive-guide/
  6. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Unstructured search
  7. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Structured search
  8. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Enrichment
  9. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Sorting
  10. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Pagination
  11. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregation
  12. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Suggestions
  13. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch in 10 seconds • Schema-free, REST & JSON based distributed document store • Open Source: Apache License 2.0 • Zero configuration • Written in Java, extensible
  14. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
  15. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Installation & first steps
  16. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Zero configuration $ wget https://download.elasticsearch.org/... $ tar -xf elasticsearch-1.2.1.tar.gz $ ./elasticsearch-1.2.1/bin/elasticsearch ... [2014-01-19 14:53:11,508][INFO ][node] [Scanner] started ...
  17. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Is it alive? » curl localhost:9200 { "status" : 200, "name" : "Scanner", "version" : { "number" : “1.2.1", "build_hash" : "e018cda7e7a32643d59e0ac3cdb412ccc239af04", "build_timestamp" : "2014-03-25T15:11:47Z", "build_snapshot" : true, "lucene_version" : “4.8.1" }, "tagline" : "You Know, for Search" }
  18. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited » curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : "Clinton Gormley", "started" : "2013-02-04", "pages" : 230 }' Create…
  19. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited » curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : "2013-02-04", "pages" : 230 }' Update…
  20. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Delete… » curl -X DELETE localhost:9200/books/book/1 Realtime GET… » curl —X GET localhost:9200/books/book/1 » curl —X GET localhost:9200/books/book/1/_source
  21. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Search » curl -XGET localhost:9200/books/_search?q=elasticsearch { "took" : 2, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.076713204, "hits" : [ { "_index" : “books", "_type" : “book", "_id" : "1", "_score" : 0.076713204, "_source" : { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : “2013-02-04", "pages" : 230 } } ] } }
  22. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited » curl -XGET ‘localhost:9200/books/book/_search' -d '{ "query": { "filtered" : { "query" : { "match": { "text" : { "query" : “To Be Or Not To Be", "cutoff_frequency" : 0.01 } } }, "filter" : { "range": { "price": { "gte": 20.0 "lte": 50.0 ... } }' Search - Query DSL
  23. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Scalability
  24. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Distributed & scalable • Replication Read scalability Removing SPOF • Sharding Split logical data over several machines Write scalability Control data flows
  25. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Distributed & scalable node 1 (m) orders products 1 4 1 2 2 2 curl  -­‐X  PUT  localhost:9200/orders  -­‐d  '{      "settings.index.number_of_shards"  :  4      "settings.index.number_of_replicas"  :  1   }' curl  -­‐X  PUT  localhost:9200/products  -­‐d  '{      "settings.index.number_of_shards"  :  2      "settings.index.number_of_replicas"  :  0   }'
  26. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Distributed and scalable node 1 (m) orders products 2 1 4 1 node 2 orders products 2 2 3 3 4 1
  27. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Distributed & scalable node 1 (m) orders products 2 1 4 1 node 2 orders products 2 2 node 3 orders products 3 4 1 3
  28. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited A request under the hood REST Event Loop Transport Event Loop Action Event Loop Request Response
  29. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Think async! • Enforces event driven architecture • Support for non-blocking model • Enforce loose coupling • Prefers push over pull • Callback based concurrency • Helps to avoid contention on resources / threads
  30. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch 1.0 - 1.x
  31. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch • Aggregations • Snapshot/Restore • Distributed percolator • Cat API • ... and more • Cardinality Agg • Percentiles Agg • Significant Terms Agg • Search Templates • Cross fields search 1.0 1.1
  32. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch • Java 7 required • Context suggester • reverse_nested Agg • Indexing/Merging • Benchmark API • Geo bounds agg • Lucene checksums 1.2 1.x
  33. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Release map • 0.4.0 - 2010-02 • 0.5.0 - 2010-03 • … • 0.19.0 - 2012-03 • 0.20.0 - 2012-12 • 0.90.0 - 2013-04 • v1.0.0 - 2014-02 • v1.1.0 - 2014-03 • v1.2.0 - 2014-05 • v1.3.0 - 2014-0?
  34. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch 1.0
  35. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Aggregations
  36. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Aggregations • Aggregation of information • Facets are one dimensional Categories/brands/material of all results of this query • Questions are multidimensional Average revenue per category id per day • What is the average shopping cart size per order per hour?
  37. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Aggregations Documents
  38. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Aggregations Documents Query
  39. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Aggregations Documents Query Buckets
  40. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Aggregations Documents Query Buckets
  41. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Aggregations Documents Query Buckets Metrics 123 123 243 185
  42. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited bucket aggregators • global • filter • missing • terms • range • date range • ip range • histogram • date histogram • geo distance • nested
  43. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited metrics aggregators • count • stats • extended stats • avg • max • min • sum
  44. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Order average » curl -XGET 'localhost:9200/orders/order/_search' -d ' { "aggs": { "average_order_size" : { "avg" : { "field" : "total" } } } } ' ... "aggregations": { "average_order_size" : { "value" : 658.369 } } ...
  45. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Order average - filters { "aggs": { “average_order_size_january" : { "filter" : { "range" : { "created_at" : { "gte" : "2014-01-01", "lt" : "2014-02-01" } } }, "aggs" : { "avg" : { "field" : "total" } } } } } ... "aggregations": { "average_order_size_january" : { "doc_count" : 8, "value" : 540.89754 } } ...
  46. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Order average - by day { "aggs": { "by_day" : { "filter" : { "range" : { "created_at" : { "gte" : "2014-01-01", "lt" : "2014-02-01" } } }, "aggs" : { "daily_filter" : { "date_histogram" : { "field" : "created_at", "interval" : "day", "format" : "yyyy-MM-dd" }, "aggs" : { "average_order_size" : { "avg" : { "field" : "total" } } } } } } } }
  47. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Order average - by day ... "aggregations": { "by_day" : { "doc_count" : 32422, "daily_filter" : [ { "key_as_string" : "2014-01-01", "key" : 1388534400000 "doc_count" : 423, "average_order_size" : { "value" : 380.0 } }, { "key_as_string" : "2014-01-02", "key" : 1388534400000 "doc_count" : 543, "average_order_size" : { "value" : 323.432 } }, { ... ] ...
  48. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Order average - by hour { "aggs": { "by_day" : { "filter" : { "range" : { "created_at" : { "gte" : "2014-01-01", "lt" : "2014-02-01" } } }, "aggs" : { "hourly_filter" : { "histogram" : { "script" : "doc[\0027created_at\0027].date.hourOfDay", "interval" : 1 }, "aggs" : { "average_order_size" : { "avg" : { "field" : "total" } } } } } } } }
  49. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Order average - by hour ... "aggregations": { "by_day" : { "doc_count" : 32422, "daily_filter" : [ { "key" : "11", "doc_count" : 1534, "average_order_size" : { "value" : 380.0 } }, { "key" : "18", "doc_count" : 8923, "average_order_size" : { "value" : 485.4323 } }, { ... ] ...
  50. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Snapshot/Restore http://www.elasticsearch.org/blog/introducing-snapshot-restore/
  51. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Backup made easy • Several shell commands + login were needed for pre 1.0 backups, but not via API $ curl -XPUT "localhost:9200/_snapshot/my_backup" -d '{! "type": "fs", ! "settings": {! "location":"/mnt/es-test-repo"! }! }' location repository repository type
  52. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Start snapshot $ curl -XPUT "localhost:9200/_snapshot/my_backup/snapshot_20131010" -d '{! "indices":"+test_*,-test_4"! }' snapshot name repository index list (optional)
  53. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Restore snapshot $ curl -XPOST "localhost:9200/test_*/_close" snapshot name close all indices that start with test_ $ curl -XPOST "localhost:9200/_snapshot/my_backup/snapshot_20131010" -d '{! "indices":"test_*"! }' repository name index list
  54. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Distributed & scalable Percolator http://www.elasticsearch.org/blog/percolator-redesign-blog-post/
  55. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited registering percolator in 1.0 $ curl -XPUT “localhost:9200/some_index/.percolator/es-tweets” -d ‘{! “query”: {! “match”: { “body”: “elasticsearch” }! }! }’! reserved percolator type query id any index with as many shards as you need
  56. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Percolate $ curl -XGET “localhost:9200/some_index/some-type/_percolate” -d ‘{! “doc”: {! “body”: “#elasticsearch is awesome”! “nick”: “@imotov”! “name”: “Igor Motov”! “date”: “2013-11-03” ! }! }’ document to be percolated {! "took": 38,! "_shards": { "total": 5, "successful": 5, "failed": 0 },! "total": 1,! "matches": [! { "_index": “some_index", "_id": “es-tweets" }! ]! }
  57. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited other features • percolation of existing document • percolate count api • filter support (in addition to queries in 0.90) • highlighting, scoring • multi-index, aliases support • multi percolate (bulk percolation)
  58. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Cat API http://www.elasticsearch.org/blog/introducing-cat-api/
  59. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Helping sysadmins • Elasticsearch is full of monitoring APIs Everything is returned as JSON • Humans are not the world’s best JSON parsers • What if elasticsearch had an easy to use interface from the commandline?
  60. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Which one is the master? $ curl "localhost:9200/_cluster/state?pretty&filter_metadata=true&! filter_routing_table=true"! {! "cluster_name" : "elasticsearch",! "master_node" : "GNf0hEXlTfaBvQXKBF300A",! "blocks" : { },! "nodes" : {! "ObdRqLHGQ6CMI5rOEstA5A" : {! "name" : "Triton",! "transport_address" : “inet[/10.0.1.11:9300]”,! "attributes" : { }! },! "4C7pKbfhTvu0slcSy_G4_w" : {! "name" : "Kid Colt",! "transport_address" : "inet[/10.0.1.12:9300]",! "attributes" : { }! },! "GNf0hEXlTfaBvQXKBF300A" : {! "name" : "Lang, Steven",! "transport_address" : "inet[/10.0.1.13:9300]",! "attributes" : { }! }! }! }
  61. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited $ curl "localhost:9200/_cluster/state? pretty&filter_metadata=true&filter_routing_table=true"! {! "cluster_name" : "elasticsearch",! "master_node" : "GNf0hEXlTfaBvQXKBF300A",! "blocks" : { },! "nodes" : {! "ObdRqLHGQ6CMI5rOEstA5A" : {! "name" : "Triton",! "transport_address" : “inet[/10.0.1.11:9300]”,! "attributes" : { }! },! "4C7pKbfhTvu0slcSy_G4_w" : {! "name" : "Kid Colt",! "transport_address" : "inet[/10.0.1.12:9300]",! "attributes" : { }! },! "GNf0hEXlTfaBvQXKBF300A" : {! "name" : "Lang, Steven",! "transport_address" : "inet[/10.0.1.13:9300]",! "attributes" : { }! }! }! } Which one is the master? (v0.90)
  62. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Which one is the master? (v1.0) $ curl localhost:9200/_cat/master GNf0hEXlTfaBvQXKBF300A 10.0.1.13 Lang, Steven
  63. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited /cat/* » curl 'localhost:9200/_cat?v'! =^.^=! /_cat/allocation! /_cat/shards! /_cat/shards/{index}! /_cat/master! /_cat/nodes! /_cat/indices! /_cat/indices/{index}! /_cat/segments! /_cat/segments/{index}! /_cat/count! /_cat/count/{index}! /_cat/recovery! /_cat/recovery/{index}! /_cat/health! /_cat/pending_tasks! /_cat/aliases! /_cat/aliases/{alias}! /_cat/thread_pool! /_cat/plugins
  64. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited /cat/indices » curl 'localhost:9200/_cat/indices?help'! health | h | current health status! index | i,idx | index name! ...! ! ! » curl 'localhost:9200/_cat/indices?h=i,p,dc,fm'! project 5 8 0b! .marvel-2014.05.06 1 1069 0b! some_index 5 1 8.4kb! ! ! » curl 'localhost:9200/_cat/indices?h=i,p,dc,fm&v'! i p dc fm ! project 5 8 0b ! .marvel-2014.05.06 1 1099 0b ! some_index 5 1 8.4kb !
  65. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited And more…
  66. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited And more… • Disk-based fielddata http://www.elasticsearch.org/blog/disk-based-field-data-a-k-a-doc-values/ • Fielddata circuit breaker • Federated search http://www.elasticsearch.org/blog/tribe-node/
  67. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch 1.1
  68. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch 1.1 • Aggregations Percentile http://www.elasticsearch.org/blog/averages-can-dangerous-use-percentile/ Significant terms http://www.elasticsearch.org/blog/significant-terms-aggregation/ Cardinality http://www.elasticsearch.org/blog/count-elasticsearch/ • Improved multi field search http://www.elasticsearch.org/blog/multi-field-search-just-got- better/
  69. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch 1.1 • Search Templates http://www.elasticsearch.org/blog/dry-keeping-your-queries- short/ • Create index/template: Support for aliases http://www.elasticsearch.org/blog/aliases-ftw/ • Recovery API http://www.elasticsearch.org/blog/aliases-ftw/
  70. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Significant Terms • Finding the uncommonly common http://www.infoq.com/presentations/elasticsearch-revealing- uncommonly-common • Use cases Finding mis-categorized content using the “like this but not this” pattern Training a classifier Root cause analysis Geographic anomalies Detecting credit card fraud
  71. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Significant Terms Source https://xkcd.com/1138/
  72. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Template Query GET /_search! {! "query": {! "template": {! "query": {"match_{{template}}": {}},! "params" : {! "template" : "all"! }! }! }! }
  73. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Template Query GET /_search! {! "query": {! "template": {! "query": “storedTemplate",! "params" : {! "template" : "all"! }! }! }! } # cat config/scripts/storedTemplate.mustache! ! {"match_{{template}}": {}}
  74. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Search Templates GET /_search/template! {! "template" : {! "query": { ! "match" : { "{{my_field}}" : "{{my_value}}" }! },! "size" : "{{my_size}}"! },! "params" : {! "my_field" : "user",! "my_value" : "alr",! "my_size" : 50! }! }
  75. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Search Templates GET /_search/template! {! "template": "storedTemplate",! "params": {! “my_field": “user”,! “my_value": “alr”,! “my_size": 50,! }! } # cat config/scripts/storedTemplate.mustache! ! {! "query": { ! "match" : { "{{my_field}}" : "{{my_value}}" }! },! "size" : "{{my_size}}"! }
  76. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch 1.2
  77. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch 1.2 • Aggregations: global ordinals, reverse_nested • Context Suggester • Java7 required
  78. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Context suggester POST services/_suggest! {! "suggest" : {! "text" : "m",! "completion" : {! "field" : "suggest_field",! "size": 10,! "context": {! "location": {! "lat": 51,! "lon": 0! }! }! }! }! }
  79. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch 1.3
  80. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Elasticsearch 1.x • Resiliency & Stability Checksums in Lucene http://www.elasticsearch.org/blog/investing-apache-lucene/ ! http://www.elasticsearch.org/blog/resiliency-elasticsearch/ • Benchmark API • Geo bounds Agg • Lucene checksums
  81. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Thanks for listening
  82. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission

    is strictly prohibited Q & A Alexander Reelsen @spinscale [email protected] P.S. We’re hiring http://elasticsearch.com/about/jobs http://elasticsearch.com/support