Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic{ON} 2018 - Managing the Elastic Stack in Production

Elastic Co
March 01, 2018

Elastic{ON} 2018 - Managing the Elastic Stack in Production

There are many things to consider and many tools and techniques at your disposal when you begin running and managing the Elastic Stack in production. This talk highlights management features in Kibana and covers the most important elements for running and managing the Elastic Stack in production.

Steve Kearns| Director of Product Management | Elastic
Chris Earle | Monitoring Lead | Elastic
Chris Roberson | Senior Javascript Engineer | Elastic

Elastic Co

March 01, 2018
Tweet

More Decks by Elastic Co

Other Decks in Technology

Transcript

  1. Product Management @skearns64 Managing the Elastic Stack In Production Steve

    Kearns Stack Monitoring Lead @pickypg Chris Earle Senior UI Engineer @chrisronline Chris Roberson
  2. What do your tenants need? 7 • Dedicated Indexes /

    Custom Mappings? • Dedicated Data Lifecycles? • What are Their Usage Patterns? • Service Level Agreement (hard with noisy neighbors!) • Upgrade and Maintenance • Chargeback / Showback • Security & Compliance requirements?
  3. Cluster Architecture Master Data Coordinating Ingest Machine Learning Master Data

    Coordinating Ingest Machine Learning Master Data Coordinating Ingest Machine Learning
  4. Indexing Strategy - Time Series Data Pro Tips • Set

    shard count based on expected ingest rate • For efficient search, fewer shards are better • Roll indexes based on target size, rather than daily • Shrink Indexes to reduce shard count
  5. Index Lifecycle Rollover every day/size 1 2 3 4 Re-allocate

    indices Shrink to 1 shard Forcemerge Change # of replicas Re-allocate indices Change # of replicas Delete Hot Warm Cold Delete
  6. Indexing Strategy - Non-Time Series Data Pro Tips • Plan

    shard count based on total docs & insert/update rate • Scale up search throughput with additional replicas • Set number_of_routing_shards to use Shard Splitting in 6.1+
  7. In production What’s next 16 Use Case 1 2 3

    4 5 Cluster Level Operational Level Data Level Data Discovery
  8. 18 Cluster Alerts Opening Cluster Alerts Diagnosing and Fixing •

    Growing list • Watches under the hood • Uses monitoring data • Cluster alerts will be opened to allow more customization and collaboration • Unknown unknowns • What’s happening in Monitoring outside of alerts • What does it mean X-Pack monitoring
  9. Except where otherwise noted, this work is licensed under http://creativecommons.org/licenses/by-nd/4.0/

    Creative Commons and the double C in a circle are registered trademarks of Creative Commons in the United States and other countries. Third party marks and brands are the property of their respective holders. 21 Please attribute Elastic with a link to elastic.co