Elastic{ON} 2018 - Elastic Cloud.(next)

Transcript

1. 2018-03-01 Elastic Cloud: What's next? Alex Brasetvik and Chris Overton Cloud tech and team leads at Elastic
2. 2018-03-01 Elastic Cloud: What's next? Alex Brasetvik and Chris Overton Cloud tech and team leads at Elastic
3. Elastic Cloud is the official hosted Elastic Stack service.
4. Elastic Cloud Enterprise is Elastic Cloud on your premises.
9. • Different importance, versions, SLAs, workloads, backup policies • Noisy neighbours • Upgrades and maintenance • Reducing blast radius
15. Version Management • Manage multiple versions • One Click upgrades • Same day Stack releases
16. Backups • Automatic backups • Every 30 minutes • 48 hours retention • Single click restore
17. • Isolating clusters. • Different teams have different needs. • Availability vs. resource usage on cluster change. • Over-provisioning.
46. + with every
47. • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
48. • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
49. • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
50. • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
51. • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
52. • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
53. • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
56. ECE 1.0: Turning a SaaS into a product • Elastic Cloud Enterprise 1.0.0 launched May 2017. • Focus: Easy installation and maintenance.
57. host-A $ ./elastic-cloud-enterprise.sh install [ progress info, … eventually: ] Elastic Cloud Enterprise installation completed successfully [ details on how to log in + token to add new servers ] Then, on another host: host-B $ ./elastic-cloud-enterprise.sh install \ --coordinator-host host-A \ --roles 'roles_for_new_host' --roles-token 'token' \
58. host-A $ ./elastic-cloud-enterprise.sh install [ progress info, … eventually: ] Elastic Cloud Enterprise installation completed successfully [ details on how to log in + token to add new servers ] Then, on another host: host-B $ ./elastic-cloud-enterprise.sh install \ --coordinator-host host-A \ --roles 'roles_for_new_host' --roles-token 'token' \
59. host-A $ ./elastic-cloud-enterprise.sh install [ progress info, … eventually: ] Elastic Cloud Enterprise installation completed successfully [ details on how to log in + token to add new servers ] Then, on another host: host-B $ ./elastic-cloud-enterprise.sh install \ --coordinator-host host-A \ --roles 'roles_for_new_host' --roles-token 'token' \ --roles 'roles_for_new_host' --roles-token 'token' \ Optional, but let's you automate deployment (We do that on our SaaS)
67. 100% of its Elasticsearch workloads on ECE. 50 nodes on 10 hosts. See use case for more details.
68. "We were quite happy with Elastic Cloud, but Elastic Cloud Enterprise gave us all of those same features in a way that also allowed us to have full management control over the entire server and underlying software." – Edwin Law Data Engineering and Database Operations Lead, Grab
69. ECE 1.1: Upgradability and 6.0 • X-pack Security and 6.0: • Encryption required between nodes. • Certificates managed behind the scenes. • Upgrade all the things with little impact.
75. • ECE 1.0: Launch, installation and maintenance. • ECE 1.1: Upgrades and 6.0 • ECE 1.2: Get the most out of your hardware. • ECE 1.3: ( described soon too )
76. Different is different • SSDs: Great for hot data, but (still) expensive. • Dense storage: Cheaper storage for archiving. • High CPU: Ingest, machine learning, high query rates • Lots of memory: Analytics, "instant search", ... • Cheap memory: Kibana, master nodes, ...
77. At different times • SSDs: Today's data. Maybe last week's. • Dense storage: Last 90 days data. • High CPU: • Burst-y usage vs. sustained. • Overcommitting can be acceptable.
78. For different products • Elasticsearch wants memory, CPU, storage, and bandwidth: lots and fast. • Kibana / APM: A little bit of memory and some CPU. • ML: CPU and memory.
79. Kibana Elasticsearch Logstash Beats / APM
80. Kibana Elasticsearch Logstash Beats / APM
81. Kibana Elasticsearch Logstash Beats / APM Elastic Cloud Enterprise will package and manage Elasticsearch, Kibana, Logstash, Beats, APM, Swiftype's ( site + app + enterprise ) search
82. Kibana Elasticsearch Logstash Beats / APM Elastic Cloud Enterprise will package and manage Elasticsearch, Kibana, Logstash, Beats, APM, Swiftype's ( site + app + enterprise ) search
83. Elastic Cloud Enterprise will package and manage Elasticsearch, Kibana, Logstash, Beats, APM, Swiftype's ( site + app + enterprise ) search
85. • Last 7 days' data on hot nodes. • Last 90 days' data on cold nodes. • … in the same cluster and dashboards. In the same cluster
92. Demo Reticulating splines… Testing for perfection… Challenging everything…
93. Demo summary • View and manage infrastructure. • Easily create hot/warm deployments. • Machine Learning + APM coming to Cloud. Soon.
94. Demo summary • View and manage infrastructure. • Easily create hot/warm deployments. • Machine Learning + APM coming to Cloud. Soon.
95. Talk at 2:30 today. Covers this in more detail. Stretching the Cloud: Flexibility in Cloud Deployments Jonathan Halterman Cloud Engineer, Elastic Uri Cohen Director of Product Management, Elastic
96. Tagging. Got it. What's the big deal?
97. Scheduling stateful nodes • Changing Kibana is simple as it's stateless. • Changing Elasticsearch … • … takes >50 steps … • … with many different strategies.
98. Elasticsearch on Kubernetes • Kubernetes with stateful sets? • Helps, but it's a small piece of the puzzle. • ECE complements Kubernetes.
100. Elastic Cloud Enterprise alongside Kubernetes … integrated with e.g. Kubernetes operator
101. Elastic Cloud Enterprise on Kubernetes
102. ECE 1.3: Role Based Access Control • 1.2 lets you slice and dice your infrastructure. • 1.3's focus is Role Based Access Control: • SAML, LDAP, … • Tagging + RBAC = private infrastructure
103. ECE 1.3 • Usage metering/auditing • Track which user runs what. • Cross Cluster Search • Search many clusters as if they were one.
104. And More • Index lifecycle management. • Audit logging. • Quality of life improvements. • More snapshot targets.
105. Meet us at the AMA booth. Thanks! Talks by Cloud team members: 1:30: Logging and Metrics in Elastic Cloud 2:30: Stretching the Cloud
106. Meet us at the AMA booth. Thanks! Talks by Cloud team members: 1:30: Logging and Metrics in Elastic Cloud 2:30: Stretching the Cloud
107. Except where otherwise noted, this work is licensed under http://creativecommons.org/licenses/by-nd/4.0/ Creative Commons and the double C in a circle are registered trademarks of Creative Commons in the United States and other countries. Third party marks and brands are the property of their respective holders. 107 Please attribute Elastic with a link to elastic.co