Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic{ON} 2018 - Elastic Cloud.(next)

Dd9d954997353b37b4c2684f478192d3?s=47 Elastic Co
March 01, 2018

Elastic{ON} 2018 - Elastic Cloud.(next)

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

March 01, 2018
Tweet

Transcript

  1. 2018-03-01 Elastic Cloud: What's next? Alex Brasetvik and Chris Overton

    Cloud tech and team leads at Elastic
  2. 2018-03-01 Elastic Cloud: What's next? Alex Brasetvik and Chris Overton

    Cloud tech and team leads at Elastic
  3. Elastic Cloud is the official hosted Elastic Stack service.

  4. Elastic Cloud Enterprise is Elastic Cloud on your premises.

  5. None
  6. None
  7. None
  8. None
  9. • Different importance, versions, SLAs, workloads, backup policies • Noisy

    neighbours • Upgrades and maintenance • Reducing blast radius
  10. None
  11. None
  12. None
  13. None
  14. None
  15. Version Management • Manage multiple versions • One Click upgrades

    • Same day Stack releases
  16. Backups • Automatic backups • Every 30 minutes • 48

    hours retention • Single click restore
  17. • Isolating clusters. • Different teams have different needs. •

    Availability vs. resource usage on cluster change. • Over-provisioning.
  18. None
  19. None
  20. None
  21. None
  22. None
  23. None
  24. None
  25. None
  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None
  38. None
  39. None
  40. None
  41. None
  42. None
  43. None
  44. None
  45. None
  46. + with every

  47. • Available just milliseconds away in • North Virginia •

    Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
  48. • Available just milliseconds away in • North Virginia •

    Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
  49. • Available just milliseconds away in • North Virginia •

    Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
  50. • Available just milliseconds away in • North Virginia •

    Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
  51. • Available just milliseconds away in • North Virginia •

    Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
  52. • Available just milliseconds away in • North Virginia •

    Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
  53. • Available just milliseconds away in • North Virginia •

    Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress • Available just milliseconds away in • North Virginia • Oregon • North California • Iowa • Ireland • Frankfurt • Belgium • São Paulo • Sydney • Tokyo • Singapore • + 5 more regions in 2018 • AWS and GCP • Azure in progress
  54. None
  55. None
  56. ECE 1.0: Turning a SaaS into a product • Elastic

    Cloud Enterprise 1.0.0 launched May 2017. • Focus: Easy installation and maintenance.
  57. host-A $ ./elastic-cloud-enterprise.sh install [ progress info, … eventually: ]

    Elastic Cloud Enterprise installation completed successfully [ details on how to log in + token to add new servers ] Then, on another host: host-B $ ./elastic-cloud-enterprise.sh install \ --coordinator-host host-A \ --roles 'roles_for_new_host' --roles-token 'token' \
  58. host-A $ ./elastic-cloud-enterprise.sh install [ progress info, … eventually: ]

    Elastic Cloud Enterprise installation completed successfully [ details on how to log in + token to add new servers ] Then, on another host: host-B $ ./elastic-cloud-enterprise.sh install \ --coordinator-host host-A \ --roles 'roles_for_new_host' --roles-token 'token' \
  59. host-A $ ./elastic-cloud-enterprise.sh install [ progress info, … eventually: ]

    Elastic Cloud Enterprise installation completed successfully [ details on how to log in + token to add new servers ] Then, on another host: host-B $ ./elastic-cloud-enterprise.sh install \ --coordinator-host host-A \ --roles 'roles_for_new_host' --roles-token 'token' \ --roles 'roles_for_new_host' --roles-token 'token' \ Optional, but let's you automate deployment (We do that on our SaaS)
  60. None
  61. None
  62. None
  63. None
  64. None
  65. None
  66. None
  67. 100% of its Elasticsearch workloads on ECE. 50 nodes on

    10 hosts. See use case for more details.
  68. "We were quite happy with Elastic Cloud, but Elastic Cloud

    Enterprise gave us all of those same features in a way that also allowed us to have full management control over the entire server and underlying software." – Edwin Law Data Engineering and Database Operations Lead, Grab
  69. ECE 1.1: Upgradability and 6.0 • X-pack Security and 6.0:

    • Encryption required between nodes. • Certificates managed behind the scenes. • Upgrade all the things with little impact.
  70. None
  71. None
  72. None
  73. None
  74. None
  75. • ECE 1.0: Launch, installation and maintenance. • ECE 1.1:

    Upgrades and 6.0 • ECE 1.2: Get the most out of your hardware. • ECE 1.3: ( described soon too )
  76. Different is different • SSDs: Great for hot data, but

    (still) expensive. • Dense storage: Cheaper storage for archiving. • High CPU: Ingest, machine learning, high query rates • Lots of memory: Analytics, "instant search", ... • Cheap memory: Kibana, master nodes, ...
  77. At different times • SSDs: Today's data. Maybe last week's.

    • Dense storage: Last 90 days data. • High CPU: • Burst-y usage vs. sustained. • Overcommitting can be acceptable.
  78. For different products • Elasticsearch wants memory, CPU, storage, and

    bandwidth: lots and fast. • Kibana / APM: A little bit of memory and some CPU. • ML: CPU and memory.
  79. Kibana Elasticsearch Logstash Beats / APM

  80. Kibana Elasticsearch Logstash Beats / APM

  81. Kibana Elasticsearch Logstash Beats / APM Elastic Cloud Enterprise will

    package and manage Elasticsearch, Kibana, Logstash, Beats, APM, Swiftype's ( site + app + enterprise ) search
  82. Kibana Elasticsearch Logstash Beats / APM Elastic Cloud Enterprise will

    package and manage Elasticsearch, Kibana, Logstash, Beats, APM, Swiftype's ( site + app + enterprise ) search
  83. Elastic Cloud Enterprise will package and manage Elasticsearch, Kibana, Logstash,

    Beats, APM, Swiftype's ( site + app + enterprise ) search
  84. None
  85. • Last 7 days' data on hot nodes. • Last

    90 days' data on cold nodes. • … in the same cluster and dashboards. In the same cluster
  86. None
  87. None
  88. None
  89. None
  90. None
  91. None
  92. Demo Reticulating splines… Testing for perfection… Challenging everything…

  93. Demo summary • View and manage infrastructure. • Easily create

    hot/warm deployments. • Machine Learning + APM coming to Cloud. Soon.
  94. Demo summary • View and manage infrastructure. • Easily create

    hot/warm deployments. • Machine Learning + APM coming to Cloud. Soon.
  95. Talk at 2:30 today. Covers this in more detail. Stretching

    the Cloud: Flexibility in Cloud Deployments Jonathan Halterman Cloud Engineer, Elastic Uri Cohen Director of Product Management, Elastic
  96. Tagging. Got it. What's the big deal?

  97. Scheduling stateful nodes • Changing Kibana is simple as it's

    stateless. • Changing Elasticsearch … • … takes >50 steps … • … with many different strategies.
  98. Elasticsearch on Kubernetes • Kubernetes with stateful sets? • Helps,

    but it's a small piece of the puzzle. • ECE complements Kubernetes.
  99. None
  100. Elastic Cloud Enterprise alongside Kubernetes … integrated with e.g. Kubernetes

    operator
  101. Elastic Cloud Enterprise on Kubernetes

  102. ECE 1.3: Role Based Access Control • 1.2 lets you

    slice and dice your infrastructure. • 1.3's focus is Role Based Access Control: • SAML, LDAP, … • Tagging + RBAC = private infrastructure
  103. ECE 1.3 • Usage metering/auditing • Track which user runs

    what. • Cross Cluster Search • Search many clusters as if they were one.
  104. And More • Index lifecycle management. • Audit logging. •

    Quality of life improvements. • More snapshot targets.
  105. Meet us at the AMA booth. Thanks! Talks by Cloud

    team members: 1:30: Logging and Metrics in Elastic Cloud 2:30: Stretching the Cloud
  106. Meet us at the AMA booth. Thanks! Talks by Cloud

    team members: 1:30: Logging and Metrics in Elastic Cloud 2:30: Stretching the Cloud
  107. Except where otherwise noted, this work is licensed under http://creativecommons.org/licenses/by-nd/4.0/

    Creative Commons and the double C in a circle are registered trademarks of Creative Commons in the United States and other countries. Third party marks and brands are the property of their respective holders. 107 Please attribute Elastic with a link to elastic.co