Slide 1

Slide 1 text

Product Management @skearns64 Managing the Elastic Stack In Production Steve Kearns Stack Monitoring Lead @pickypg Chris Earle Senior UI Engineer @chrisronline Chris Roberson

Slide 2

Slide 2 text

Agenda 2 1 Planning for Production 2 Detecting Problems 3 Fixing Problems

Slide 3

Slide 3 text

Planning for Production

Slide 4

Slide 4 text

Are you Single or Multi-Tenant? 4

Slide 5

Slide 5 text

Are you Single or Multi-Tenant? 5

Slide 6

Slide 6 text

Are you Single or Multi-Tenant? 6

Slide 7

Slide 7 text

What do your tenants need? 7 • Dedicated Indexes / Custom Mappings? • Dedicated Data Lifecycles? • What are Their Usage Patterns? • Service Level Agreement (hard with noisy neighbors!) • Upgrade and Maintenance • Chargeback / Showback • Security & Compliance requirements?

Slide 8

Slide 8 text

Multi-Cluster, Perhaps? 8

Slide 9

Slide 9 text

Cluster Architecture Master Data Coordinating Ingest Machine Learning Master Data Coordinating Ingest Machine Learning Master Data Coordinating Ingest Machine Learning

Slide 10

Slide 10 text

Cluster Architecture Master Data Coordinating Ingest Machine Learning

Slide 11

Slide 11 text

Cluster Architecture Master Data Coordinating Ingest Machine Learning Hot Hot Hot Warm Warm Warm

Slide 12

Slide 12 text

Indexing Strategy - Time Series Data Pro Tips • Set shard count based on expected ingest rate • For efficient search, fewer shards are better • Roll indexes based on target size, rather than daily • Shrink Indexes to reduce shard count

Slide 13

Slide 13 text

Index Lifecycle Rollover every day/size 1 2 3 4 Re-allocate indices Shrink to 1 shard Forcemerge Change # of replicas Re-allocate indices Change # of replicas Delete Hot Warm Cold Delete

Slide 14

Slide 14 text

Indexing Strategy - Non-Time Series Data Pro Tips • Plan shard count based on total docs & insert/update rate • Scale up search throughput with additional replicas • Set number_of_routing_shards to use Shard Splitting in 6.1+

Slide 15

Slide 15 text

Detecting Problems

Slide 16

Slide 16 text

In production What’s next 16 Use Case 1 2 3 4 5 Cluster Level Operational Level Data Level Data Discovery

Slide 17

Slide 17 text

Fixing Problems

Slide 18

Slide 18 text

18 Cluster Alerts Opening Cluster Alerts Diagnosing and Fixing • Growing list • Watches under the hood • Uses monitoring data • Cluster alerts will be opened to allow more customization and collaboration • Unknown unknowns • What’s happening in Monitoring outside of alerts • What does it mean X-Pack monitoring

Slide 19

Slide 19 text

19 More Questions? Visit us at the AMA

Slide 20

Slide 20 text

www.elastic.c o

Slide 21

Slide 21 text

Except where otherwise noted, this work is licensed under http://creativecommons.org/licenses/by-nd/4.0/ Creative Commons and the double C in a circle are registered trademarks of Creative Commons in the United States and other countries. Third party marks and brands are the property of their respective holders. 21 Please attribute Elastic with a link to elastic.co