Save 37% off PRO during our Black Friday Sale! »

Elastic{ON}Tour Tokyo 2017 - Elastic Cloud Deep Dive

Dd9d954997353b37b4c2684f478192d3?s=47 Elastic Co
December 14, 2017

Elastic{ON}Tour Tokyo 2017 - Elastic Cloud Deep Dive

Elastic Cloud Enterpriseをお好きな環境で運用し、セキュアにElasticsearchクラスターを展開・管理する方法をご紹介します。

Mat Schaffer | Site Reliability Engineer | Elastic

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

December 14, 2017
Tweet

Transcript

  1. 1 Mat Schaffer ϚοτɹγΣΠϑΝʔ αΠτϨϥΠΞϏϦςΟΤϯδχΞ, Elastic Cloud Elastic Cloud Enterprise

  2. 2 2 I’m an engineer

  3. 3 How many clusters do you run? Just one? About

    10? 100 or more?
  4. 4 Clusters more every minute Providers AWS GCP more soon

    on Elastic Cloud Regions 8 + 4 more soon 2 12 1000s
  5. 5 1000s? How? Elastic Cloud

  6. 6 6 Cloud UI Cloud API director Admin API Admin

    UI allocators proxies coordinator Apps Apps director Admin API Admin UI allocators proxies c director Admin API Admin UI allocators proxies c Apps Apps director Admin API Admin UI allocators proxies c
  7. 7 director Admin API Apps Admin UI allocators proxies c

  8. 8 Provisioning, orchestration, and management of multiple clusters Deployed on-premise,

    in your private cloud … or wherever you want Automates frequent tasks such as snapshot/restore, upgrade and scale
  9. 9 9 When do you need it?

  10. 10 Elasticsearch Adoption Pattern One cluster One data type One

    app 1 2 3 4 One cluster Many data types One app One cluster Many data types Many apps Many clusters Many data types Many apps
  11. 11 Multiple applications, multiple clusters

  12. 12 Multiple Environments - Dev, Test, QA, Prod...

  13. 13 Challenges with many clusters Upgrades Security Backups •  Many

    versions •  Upgrade path “gotchas” •  Handling upgrade failures •  Controlled access •  SSL •  Repository •  Schedule •  Rotation
  14. 14 14 Architecture

  15. 15 High Level Architecture Allocator Cloud UI Admin API ZooKeeper

    + Director Constructor Allocator ES Admin Cluster Allocator ES Logging Cluster Allocator Load Balancer (User supplied) Proxy Proxy Proxy + HTTP + HTTPS TLS Over Stunnel HTTPS Admin User HTTPS Allocator HTTPS HTTPS Runner Client Forwarder Beats
  16. 16 16 Demo

  17. 17 17 SRE’s Notes

  18. 18 Preparing for ECE reliability •  First: Docs ‒  https://www.elastic.co/guide/en/cloud-enterprise/1.1/ece-limitations.html

    ‒  https://www.elastic.co/guide/en/cloud-enterprise/1.1/ece-issues.html •  Support, of course •  Some others...
  19. 19 System overview Memory Disk Network CPU

  20. 20 Allocator memory Some changes create new nodes Instances stay

    in- zone Tiebreakers & Kibanas too
  21. 21 Allocator failure readiness Are disks persistent? Single instances might

    fall back to snapshots Overloaded clusters might have trouble
  22. 22 Connection counts Get 20 per proxy per cluster Recommend

    HTTP Load Balancing Will improve in future
  23. 23 Network and disk performance Better when faster Too slow

    can cause disconnects fio & esrally are great tools
  24. 24 Thanks! elastic.co/downloads/enterprise github.com/elastic/examples/tree/master/CloudEnterprise