Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic{ON} 2018 - Elastic Cloud.(next)

Elastic Co
March 01, 2018

Elastic{ON} 2018 - Elastic Cloud.(next)

Elastic Co

March 01, 2018
Tweet

More Decks by Elastic Co

Other Decks in Technology

Transcript

  1. 2018-03-01
    Elastic Cloud: What's
    next?
    Alex Brasetvik and Chris Overton
    Cloud tech and team leads at Elastic

    View full-size slide

  2. 2018-03-01
    Elastic Cloud: What's
    next?
    Alex Brasetvik and Chris Overton
    Cloud tech and team leads at Elastic

    View full-size slide

  3. Elastic Cloud is the official hosted Elastic Stack service.

    View full-size slide

  4. Elastic Cloud Enterprise is Elastic Cloud on your premises.

    View full-size slide

  5. • Different importance, versions,
    SLAs, workloads, backup policies
    • Noisy neighbours
    • Upgrades and maintenance
    • Reducing blast radius

    View full-size slide

  6. Version Management
    • Manage multiple versions
    • One Click upgrades
    • Same day Stack releases

    View full-size slide

  7. Backups
    • Automatic backups
    • Every 30 minutes
    • 48 hours retention
    • Single click restore

    View full-size slide

  8. • Isolating clusters.
    • Different teams have different needs.
    • Availability vs. resource usage on cluster change.
    • Over-provisioning.

    View full-size slide

  9. + with every

    View full-size slide

  10. • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress
    • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress

    View full-size slide

  11. • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress
    • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress

    View full-size slide

  12. • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress
    • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress

    View full-size slide

  13. • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress
    • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress

    View full-size slide

  14. • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress
    • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress

    View full-size slide

  15. • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress
    • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress

    View full-size slide

  16. • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress
    • Available just
    milliseconds away in
    • North Virginia
    • Oregon
    • North California
    • Iowa
    • Ireland
    • Frankfurt
    • Belgium
    • São Paulo
    • Sydney
    • Tokyo
    • Singapore
    • + 5 more regions in 2018
    • AWS and GCP
    • Azure in progress

    View full-size slide

  17. ECE 1.0: Turning a SaaS into a product
    • Elastic Cloud Enterprise 1.0.0 launched
    May 2017.
    • Focus: Easy installation and maintenance.

    View full-size slide

  18. host-A $ ./elastic-cloud-enterprise.sh install
    [ progress info, … eventually: ]
    Elastic Cloud Enterprise installation completed successfully
    [ details on how to log in + token to add new servers ]
    Then, on another host:
    host-B $ ./elastic-cloud-enterprise.sh install \
    --coordinator-host host-A \
    --roles 'roles_for_new_host'
    --roles-token 'token' \

    View full-size slide

  19. host-A $ ./elastic-cloud-enterprise.sh install
    [ progress info, … eventually: ]
    Elastic Cloud Enterprise installation completed successfully
    [ details on how to log in + token to add new servers ]
    Then, on another host:
    host-B $ ./elastic-cloud-enterprise.sh install \
    --coordinator-host host-A \
    --roles 'roles_for_new_host'
    --roles-token 'token' \

    View full-size slide

  20. host-A $ ./elastic-cloud-enterprise.sh install
    [ progress info, … eventually: ]
    Elastic Cloud Enterprise installation completed successfully
    [ details on how to log in + token to add new servers ]
    Then, on another host:
    host-B $ ./elastic-cloud-enterprise.sh install \
    --coordinator-host host-A \
    --roles 'roles_for_new_host'
    --roles-token 'token' \
    --roles 'roles_for_new_host'
    --roles-token 'token' \
    Optional, but let's you automate deployment
    (We do that on our SaaS)

    View full-size slide

  21. 100% of its Elasticsearch workloads on ECE.
    50 nodes on 10 hosts.
    See use case for more details.

    View full-size slide

  22. "We were quite happy with Elastic Cloud, but
    Elastic Cloud Enterprise gave us all of those
    same features in a way that also allowed us
    to have full management control over the
    entire server and underlying software."
    – Edwin Law
    Data Engineering and Database Operations Lead, Grab

    View full-size slide

  23. ECE 1.1: Upgradability and 6.0
    • X-pack Security and 6.0:
    • Encryption required between nodes.
    • Certificates managed behind the scenes.
    • Upgrade all the things with little impact.

    View full-size slide

  24. • ECE 1.0: Launch, installation and maintenance.
    • ECE 1.1: Upgrades and 6.0
    • ECE 1.2: Get the most out of your hardware.
    • ECE 1.3: ( described soon too )

    View full-size slide

  25. Different is different
    • SSDs: Great for hot data, but (still) expensive.
    • Dense storage: Cheaper storage for archiving.
    • High CPU: Ingest, machine learning, high query
    rates
    • Lots of memory: Analytics, "instant search", ...
    • Cheap memory: Kibana, master nodes, ...

    View full-size slide

  26. At different times
    • SSDs: Today's data. Maybe last week's.
    • Dense storage: Last 90 days data.
    • High CPU:
    • Burst-y usage vs. sustained.
    • Overcommitting can be acceptable.

    View full-size slide

  27. For different products
    • Elasticsearch wants memory, CPU, storage, and
    bandwidth: lots and fast.
    • Kibana / APM: A little bit of memory and some CPU.
    • ML: CPU and memory.

    View full-size slide

  28. Kibana
    Elasticsearch
    Logstash
    Beats / APM

    View full-size slide

  29. Kibana
    Elasticsearch
    Logstash
    Beats / APM

    View full-size slide

  30. Kibana
    Elasticsearch
    Logstash
    Beats / APM
    Elastic Cloud Enterprise will package and manage
    Elasticsearch, Kibana, Logstash, Beats, APM,
    Swiftype's ( site + app + enterprise ) search

    View full-size slide

  31. Kibana
    Elasticsearch
    Logstash
    Beats / APM
    Elastic Cloud Enterprise will package and manage
    Elasticsearch, Kibana, Logstash, Beats, APM,
    Swiftype's ( site + app + enterprise ) search

    View full-size slide

  32. Elastic Cloud Enterprise will package and manage
    Elasticsearch, Kibana, Logstash, Beats, APM,
    Swiftype's ( site + app + enterprise ) search

    View full-size slide

  33. • Last 7 days' data on hot nodes.
    • Last 90 days' data on cold nodes.
    • … in the same cluster and dashboards.
    In the same cluster

    View full-size slide

  34. Demo
    Reticulating splines…
    Testing for perfection…
    Challenging everything…

    View full-size slide

  35. Demo summary
    • View and manage infrastructure.
    • Easily create hot/warm deployments.
    • Machine Learning + APM coming to Cloud.
    Soon.

    View full-size slide

  36. Demo summary
    • View and manage infrastructure.
    • Easily create hot/warm deployments.
    • Machine Learning + APM coming to Cloud.
    Soon.

    View full-size slide

  37. Talk at 2:30 today. Covers this in more detail.
    Stretching the Cloud:
    Flexibility in Cloud Deployments
    Jonathan Halterman
    Cloud Engineer, Elastic
    Uri Cohen
    Director of Product
    Management, Elastic

    View full-size slide

  38. Tagging. Got it.
    What's the big deal?

    View full-size slide

  39. Scheduling stateful nodes
    • Changing Kibana is simple as it's stateless.
    • Changing Elasticsearch …
    • … takes >50 steps …
    • … with many different strategies.

    View full-size slide

  40. Elasticsearch on Kubernetes
    • Kubernetes with stateful sets?
    • Helps, but it's a small piece of the puzzle.
    • ECE complements Kubernetes.

    View full-size slide

  41. Elastic Cloud Enterprise
    alongside Kubernetes
    … integrated with e.g.
    Kubernetes operator

    View full-size slide

  42. Elastic Cloud Enterprise
    on Kubernetes

    View full-size slide

  43. ECE 1.3: Role Based Access Control
    • 1.2 lets you slice and dice your infrastructure.
    • 1.3's focus is Role Based Access Control:
    • SAML, LDAP, …
    • Tagging + RBAC = private infrastructure

    View full-size slide

  44. ECE 1.3
    • Usage metering/auditing
    • Track which user runs what.
    • Cross Cluster Search
    • Search many clusters as if they were one.

    View full-size slide

  45. And More
    • Index lifecycle management.
    • Audit logging.
    • Quality of life improvements.
    • More snapshot targets.

    View full-size slide

  46. Meet us at the AMA booth.
    Thanks!
    Talks by Cloud team members:
    1:30: Logging and Metrics in Elastic Cloud
    2:30: Stretching the Cloud

    View full-size slide

  47. Meet us at the AMA booth.
    Thanks!
    Talks by Cloud team members:
    1:30: Logging and Metrics in Elastic Cloud
    2:30: Stretching the Cloud

    View full-size slide

  48. Except where otherwise noted, this work is licensed under
    http://creativecommons.org/licenses/by-nd/4.0/
    Creative Commons and the double C in a circle are
    registered trademarks of Creative Commons in the United States and other countries.
    Third party marks and brands are the property of their respective holders.
    107
    Please attribute Elastic with a link to elastic.co

    View full-size slide