What's coming in Elasticsearch 5.0

Talking at the Berlin Search UG on 26.05.2016 about some new features in the coming 5.0 release of Elasticsearch

Transcript

1. ‹#› Search UG Meetup Berlin 26.05.2016 Christoph Büscher @dalatangi What's coming in Elasticsearch 5.0
2. Why is ES going from 2.x to 5.0.0 2 5.0.0-alpha1 released April 5th 2016
3. 3 Versions across the stack es kibana ls beats Nov 5, 2014
 1.4 May 23, 2015
 1.5 Jun 9, 2015
 1.6 Jul 16, 2015
 1.7 Feb 19, 2015
 4.0 Jun 10, 2015
 4.1 May 14, 2015
 1.5 May 27, 2015
 1.0 Beta 1 Jul 13, 2015
 1.0 Beta 2 Sep 4, 2015
 1.0 Beta 3
4. 4 Aligning releases with 2.x
5. 5 Now, just take five…
6. ‹#› So whats new?
 2.3 5.0
7. ‹#› 7 POST _reindex { "src": { "index": "old_index", "query": { "match": { "user": "twitter" } } }, "dest": { "index": "new_index" } } Reindex (2.3)
8. ‹#› Supports • Cancellation • Throttling • Ignoring version conflicts 8 Reindex (2.3)
9. ‹#› 9 Update by Query (2.3) POST users/_update_by_query { "query": { "match": { "user": "twitter" } }, "script": "...." } 9
10. ‹#› • GET /_tasks • monitoring of long running tasks • control, e.g. cancellation 10 Task Management (2.3)
11. ‹#› 11 Query Profiler (2.3)
13. ‹#› 13 Ingest Node (5.0) Putting the power of Logstash filters inside an Elasticsearch node
14. 14 Ingest Node
15. ‹#› 15 • enabled by default • you define pipelines • as series of processors Ingest Node (5.0)
16. ‹#› 16 Painless Scripting (5.0) • fast • secure • single function only • groovy-like syntax • dynamic & static typing
17. ‹#› • decouple server/client • minimize dependencies • first version targeted for 5.0 release 17 Java HTTP Client (5.0)
18. ‹#› 18 Some other improvements
19. ‹#› 19 • improves indexing of numeric field • faster indexing • less memory at search time Lucene 6 Multi-dimensional points

20. NumericField vs PointField 20 0% 25% 50% 75% 100% Index Size Index Time Search Time Search Time Heap Usage 15% 76% 49% 49% 100% 100% 100% 100% NumericField PointField (Michael McCandless, https://www.elastic.co/blog/lucene-points-6.0)
21. ‹#› String Mappings { "type": "string", "index: "not_analyzed" } 21 Text vs. Keyword
22. ‹#› String Mappings { "type": "text"} Full text field Full analysis chain Full text relevance 22 Text vs. Keyword
23. ‹#› String Mappings { "type": "keyword"} Concrete string value Doc values by default 23 Text vs. Keyword
24. ‹#› • stricter parsing • improved caching • allows for internal rewrites 24 Search Refactoring
25. ‹#› • strict validation • settings updates are atomic • reset to default • retrieve defaults 25 Settings
 Validation
26. ‹#› • document oriented • respects deletes • boosting by: • context • geolocation 26 Completion
 Suggester v2
27. 27 elasticsearch-5.0.0-alpha2 Out soon!
28. ‹#› Pioneer Program https://www.elastic.co/blog/ elastic-pioneer-program
29. 29 Further information • https://www.elastic.co/blog • Reindex: https://www.elastic.co/blog/reindex-is-coming • Multi-dimensional Points: https://www.elastic.co/blog/lucene-points-6.0 • Ingest node: https://www.elastic.co/elasticon/conf/2016/sf/ingest-node- enriching-documents-within-elasticsearch • Profile API https://www.elastic.co/guide/en/elasticsearch/reference/master/ search-profile.html • Task Management https://www.elastic.co/guide/en/elasticsearch/reference/ master/tasks.html • Pioneer Program https://www.elastic.co/blog/elastic-pioneer-program