Slide 1

Slide 1 text

‹#› Luca Cavanna Software Developer @javanna Elasticsearch: what’s cooking

Slide 2

Slide 2 text

Agenda 2 About Elastic Elasticsearch 2.0 Elasticsearch 2.1 Elasticsearch 2.2 Elasticsearch 3.0 1 2 3 4 5

Slide 3

Slide 3 text

About Elastic • Elastic the company gets founded • Elasticsearch 0.19 • Series A investment • Commercial offerings: support and trainings 3 2012

Slide 4

Slide 4 text

About Elastic • Series B investment • Kibana joins Elastic • Elasticsearch for Apache Hadoop gets released • Logstash joins Elastic • Language clients 4 2012 2013

Slide 5

Slide 5 text

About Elastic • Series C investment • Elasticsearch 1.0 released • Marvel gets released 5 2012 2013 2014

Slide 6

Slide 6 text

About Elastic • Shield gets released • Elastic{ON}15: first user conference • Found gets acquired • Packetbeat gets acquired • Watcher gets released • Elasticsearch 2.0 gets released 6 2012 2013 2014 2015

Slide 7

Slide 7 text

About Elastic • > 300 employees worldwide • We are hiring: https://www.elastic.co/about/careers • Elastic{ON}16: February 17-19, 2016 | San Francisco • Elasticsearch 3.0? 7 2012 2013 2014 2015 2016

Slide 8

Slide 8 text

About me • Joined in July 2013 • Working on Elasticsearch • Development, trainings, support 8 2012 2013 2014 2015 2016

Slide 9

Slide 9 text

‹#› Elasticsearch 2.0 October 2015

Slide 10

Slide 10 text

Features • Pipeline aggregations • Better query execution • Structured, readable exceptions 10

Slide 11

Slide 11 text

Better query execution https://www.elastic.co/blog/better-query-execution-coming-elasticsearch-2-0 11 { “query”: { “filtered”: { “query”: { //query definition }, “filter”: { //filter definition } } } } { “query”: { “bool”: { “must”: { //query definition }, “filter”: { //filter definition } } } }

Slide 12

Slide 12 text

‹#› Leniency is the root of all evil. Robert Muir (@Muir_Says)

Slide 13

Slide 13 text

The Great Mapping Refactoring 13 Fields Types • Names without dots • Full name references only • No type name prefix • Mappings at index level • Cannot be deleted • Names cannot start with a dot https://www.elastic.co/blog/great-mapping-refactoring

Slide 14

Slide 14 text

Resiliency • Cluster state diffs • Doc values by default for non analyzed fields • Durable writes by default • More checksums to detect corruption • Java classpath validation (aka JarHell) 14

Slide 15

Slide 15 text

Security • Java security manager enabled by default • Unicast discovery • Bind to local interfaces only by default 15

Slide 16

Slide 16 text

Cleanups 16 Removed Moved to plugins • Facets • Rivers • Shutdown api • MVEL scripting language • Delete by query • Multicast discovery • _size metadata field • murmur3 type

Slide 17

Slide 17 text

Migration plugin 17

Slide 18

Slide 18 text

Sense is back 18

Slide 19

Slide 19 text

Commercial plugins 19 Shield Watcher Marvel • Document level security • Field level security • Realms api • Hipchat/slack actions • Activate/deactivate api • Less but more useful metrics • Free in production too!

Slide 20

Slide 20 text

‹#› Elasticsearch 2.1 November 2015

Slide 21

Slide 21 text

Main features • Additional pipeline aggregations • Geo-centroid aggregation • Faster recovery and shard allocation • As always, many bugs fixed! 21

Slide 22

Slide 22 text

‹#› Elasticsearch 2.2 February 2016

Slide 23

Slide 23 text

Main improvements 23 Profile api Geo point rewrite Scripting security • Detailed query profiling • More to come • 50% faster indexing • 20-50% faster geo queries • Half disk and heap usage • Permissions lock-down • Whitelist of classes to load

Slide 24

Slide 24 text

‹#› Elasticsearch 3.0 ? 2016

Slide 25

Slide 25 text

Highlights 25 Ingest node Scripting language Task management • Processing pipeline execution within a node • Simulate api • Dynamic • More than lucene expressions • Safe • Will unblock new features • Reindex api

Slide 26

Slide 26 text

More highlights 26 Settings Percolator • Strict settings validation • Ability to unset a setting • Search request parsing on the coordinating node • Unblocks request caching for time based indices • Query terms get indexed • Percolate api becomes a query Search internals

Slide 27

Slide 27 text

‹#› Thank you! Q & A