Elasticsearch: what's cooking

Elasticsearch: what's cooking

Presentation given at the Milan Elasticsearch meetup on February 5th 2016

660d1a296a8badddc4c44fb2c7eef011?s=128

Luca Cavanna

February 05, 2016
Tweet

Transcript

  1. ‹#› Luca Cavanna Software Developer @javanna Elasticsearch: what’s cooking

  2. Agenda 2 About Elastic Elasticsearch 2.0 Elasticsearch 2.1 Elasticsearch 2.2

    Elasticsearch 3.0 1 2 3 4 5
  3. About Elastic • Elastic the company gets founded • Elasticsearch

    0.19 • Series A investment • Commercial offerings: support and trainings 3 2012
  4. About Elastic • Series B investment • Kibana joins Elastic

    • Elasticsearch for Apache Hadoop gets released • Logstash joins Elastic • Language clients 4 2012 2013
  5. About Elastic • Series C investment • Elasticsearch 1.0 released

    • Marvel gets released 5 2012 2013 2014
  6. About Elastic • Shield gets released • Elastic{ON}15: first user

    conference • Found gets acquired • Packetbeat gets acquired • Watcher gets released • Elasticsearch 2.0 gets released 6 2012 2013 2014 2015
  7. About Elastic • > 300 employees worldwide • We are

    hiring: https://www.elastic.co/about/careers • Elastic{ON}16: February 17-19, 2016 | San Francisco • Elasticsearch 3.0? 7 2012 2013 2014 2015 2016
  8. About me • Joined in July 2013 • Working on

    Elasticsearch • Development, trainings, support 8 2012 2013 2014 2015 2016
  9. ‹#› Elasticsearch 2.0 October 2015

  10. Features • Pipeline aggregations • Better query execution • Structured,

    readable exceptions 10
  11. Better query execution https://www.elastic.co/blog/better-query-execution-coming-elasticsearch-2-0 11 { “query”: { “filtered”: {

    “query”: { //query definition }, “filter”: { //filter definition } } } } { “query”: { “bool”: { “must”: { //query definition }, “filter”: { //filter definition } } } }
  12. ‹#› Leniency is the root of all evil. Robert Muir

    (@Muir_Says)
  13. The Great Mapping Refactoring 13 Fields Types • Names without

    dots • Full name references only • No type name prefix • Mappings at index level • Cannot be deleted • Names cannot start with a dot https://www.elastic.co/blog/great-mapping-refactoring
  14. Resiliency • Cluster state diffs • Doc values by default

    for non analyzed fields • Durable writes by default • More checksums to detect corruption • Java classpath validation (aka JarHell) 14
  15. Security • Java security manager enabled by default • Unicast

    discovery • Bind to local interfaces only by default 15
  16. Cleanups 16 Removed Moved to plugins • Facets • Rivers

    • Shutdown api • MVEL scripting language • Delete by query • Multicast discovery • _size metadata field • murmur3 type
  17. Migration plugin 17

  18. Sense is back 18

  19. Commercial plugins 19 Shield Watcher Marvel • Document level security

    • Field level security • Realms api • Hipchat/slack actions • Activate/deactivate api • Less but more useful metrics • Free in production too!
  20. ‹#› Elasticsearch 2.1 November 2015

  21. Main features • Additional pipeline aggregations • Geo-centroid aggregation •

    Faster recovery and shard allocation • As always, many bugs fixed! 21
  22. ‹#› Elasticsearch 2.2 February 2016

  23. Main improvements 23 Profile api Geo point rewrite Scripting security

    • Detailed query profiling • More to come • 50% faster indexing • 20-50% faster geo queries • Half disk and heap usage • Permissions lock-down • Whitelist of classes to load
  24. ‹#› Elasticsearch 3.0 ? 2016

  25. Highlights 25 Ingest node Scripting language Task management • Processing

    pipeline execution within a node • Simulate api • Dynamic • More than lucene expressions • Safe • Will unblock new features • Reindex api
  26. More highlights 26 Settings Percolator • Strict settings validation •

    Ability to unset a setting • Search request parsing on the coordinating node • Unblocks request caching for time based indices • Query terms get indexed • Percolate api becomes a query Search internals
  27. ‹#› Thank you! Q & A