Product Management @skearns64 Managing the Elastic Stack In Production Steve Kearns Stack Monitoring Lead @pickypg Chris Earle Senior UI Engineer @chrisronline Chris Roberson

Agenda 2 1 Planning for Production 2 Detecting Problems 3 Fixing Problems

Planning for Production

Are you Single or Multi-Tenant? 4

Are you Single or Multi-Tenant? 5

Are you Single or Multi-Tenant? 6

What do your tenants need? 7 • Dedicated Indexes / Custom Mappings? • Dedicated Data Lifecycles? • What are Their Usage Patterns? • Service Level Agreement (hard with noisy neighbors!) • Upgrade and Maintenance • Chargeback / Showback • Security & Compliance requirements?

Multi-Cluster, Perhaps? 8

Cluster Architecture Master Data Coordinating Ingest Machine Learning Master Data Coordinating Ingest Machine Learning Master Data Coordinating Ingest Machine Learning

Cluster Architecture Master Data Coordinating Ingest Machine Learning

Cluster Architecture Master Data Coordinating Ingest Machine Learning Hot Hot Hot Warm Warm Warm

Indexing Strategy - Time Series Data Pro Tips • Set shard count based on expected ingest rate • For efficient search, fewer shards are better • Roll indexes based on target size, rather than daily • Shrink Indexes to reduce shard count

Index Lifecycle Rollover every day/size 1 2 3 4 Re-allocate indices Shrink to 1 shard Forcemerge Change # of replicas Re-allocate indices Change # of replicas Delete Hot Warm Cold Delete

Indexing Strategy - Non-Time Series Data Pro Tips • Plan shard count based on total docs & insert/update rate • Scale up search throughput with additional replicas • Set number_of_routing_shards to use Shard Splitting in 6.1+

Detecting Problems

In production What’s next 16 Use Case 1 2 3 4 5 Cluster Level Operational Level Data Level Data Discovery

Fixing Problems

18 Cluster Alerts Opening Cluster Alerts Diagnosing and Fixing • Growing list • Watches under the hood • Uses monitoring data • Cluster alerts will be opened to allow more customization and collaboration • Unknown unknowns • What’s happening in Monitoring outside of alerts • What does it mean X-Pack monitoring

19 More Questions? Visit us at the AMA

www.elastic.c o

Except where otherwise noted, this work is licensed under http://creativecommons.org/licenses/by-nd/4.0/ Creative Commons and the double C in a circle are registered trademarks of Creative Commons in the United States and other countries. Third party marks and brands are the property of their respective holders. 21 Please attribute Elastic with a link to elastic.co