Elastic{ON}Tour Tokyo 2017 - Elastic Cloud Deep Dive

Elastic Cloud Enterpriseをお好きな環境で運用し、セキュアにElasticsearchクラスターを展開・管理する方法をご紹介します。

Mat Schaffer | Site Reliability Engineer | Elastic

Transcript

1. 1 Mat Schaffer ϚοτɹγΣΠϑΝʔ αΠτϨϥΠΞϏϦςΟΤϯδχΞ, Elastic Cloud Elastic Cloud Enterprise
2. 2 2 I’m an engineer
3. 3 How many clusters do you run? Just one? About 10? 100 or more?
4. 4 Clusters more every minute Providers AWS GCP more soon on Elastic Cloud Regions 8 + 4 more soon 2 12 1000s
5. 5 1000s? How? Elastic Cloud
6. 6 6 Cloud UI Cloud API director Admin API Admin UI allocators proxies coordinator Apps Apps director Admin API Admin UI allocators proxies c director Admin API Admin UI allocators proxies c Apps Apps director Admin API Admin UI allocators proxies c
7. 7 director Admin API Apps Admin UI allocators proxies c
8. 8 Provisioning, orchestration, and management of multiple clusters Deployed on-premise, in your private cloud … or wherever you want Automates frequent tasks such as snapshot/restore, upgrade and scale
9. 9 9 When do you need it?
10. 10 Elasticsearch Adoption Pattern One cluster One data type One app 1 2 3 4 One cluster Many data types One app One cluster Many data types Many apps Many clusters Many data types Many apps
11. 11 Multiple applications, multiple clusters
12. 12 Multiple Environments - Dev, Test, QA, Prod...
13. 13 Challenges with many clusters Upgrades Security Backups •  Many versions •  Upgrade path “gotchas” •  Handling upgrade failures •  Controlled access •  SSL •  Repository •  Schedule •  Rotation
14. 14 14 Architecture
15. 15 High Level Architecture Allocator Cloud UI Admin API ZooKeeper + Director Constructor Allocator ES Admin Cluster Allocator ES Logging Cluster Allocator Load Balancer (User supplied) Proxy Proxy Proxy + HTTP + HTTPS TLS Over Stunnel HTTPS Admin User HTTPS Allocator HTTPS HTTPS Runner Client Forwarder Beats
16. 16 16 Demo
17. 17 17 SRE’s Notes
18. 18 Preparing for ECE reliability •  First: Docs ‒  https://www.elastic.co/guide/en/cloud-enterprise/1.1/ece-limitations.html ‒  https://www.elastic.co/guide/en/cloud-enterprise/1.1/ece-issues.html •  Support, of course •  Some others...
19. 19 System overview Memory Disk Network CPU
20. 20 Allocator memory Some changes create new nodes Instances stay in- zone Tiebreakers & Kibanas too
21. 21 Allocator failure readiness Are disks persistent? Single instances might fall back to snapshots Overloaded clusters might have trouble
22. 22 Connection counts Get 20 per proxy per cluster Recommend HTTP Load Balancing Will improve in future
23. 23 Network and disk performance Better when faster Too slow can cause disconnects fio & esrally are great tools
24. 24 Thanks! elastic.co/downloads/enterprise github.com/elastic/examples/tree/master/CloudEnterprise