Share
Embed
Databases On Kubernetes: Why You should care
Developers always expected databases to work out-of-the-box, but historically it is the exact opposite.
With the rise of Kubernetes StatefulSets and CRDs, we started thinking about running databases on it. But why should I do that in the first place? How hard is it? Which are the challenges? Is it production already? All those questions will be answered during a live demo where we will deploy a database, deploy an operator, fail nodes, scale up and down with almost no manual intervention.
deniswsrosa
June 06, 2020
Transcript
35 slides
-
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. DATABASES ON KUBERNETES Denis Rosa | Developer Advocate @deniswsrosa @deniswsrosa WHY YOU SHOULD CARE -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 2 2016 -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. @deniswsrosa 2016 Containers are designed to be stateless -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. @deniswsrosa Fast forward to 2020… -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 5 2020 -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. @deniswsrosa What have changed? -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. @deniswsrosa • Production-Grade Images • Easier to setup and configure • Separate storage from compute • Kubernetes and StatefulSets … -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. @deniswsrosa Databases typically have numerous tuning parameters, a new immutable container image for every possible database configuration can quickly result in image sprawl. -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. @deniswsrosa Using configuration management tools can help to solve part of the problem, but shouldn’t it be part of the solution? -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. DEMO1 – A K8S TODO LIST -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 11 K8 CRDs -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 12 CRDs + K8s Operators -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 13 Java Operator - Example Source: https://developers.redhat.com/blog/2019/10/07/write-a-simple-kubernetes-operator-in-java-using-the-fabric8-kubernetes-client/ -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 14 Quick Operator Revision Operator Knowledge from app experts Deployments Autoscaling StatefulSets Configs -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 15 K8s Database Operators Provide a cloud agnostic Database-as-a-Service solution -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. DEMO 2 – LET’S PLAY WITH A DATABASE ON KUBERNETES -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 17 Other Possibilities Automated Backup XDCR Prometheus Metrics -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 18 Standard Deployment -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 19 MySQL Operator https://github.com/presslabs/mysql-operator -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 20 DBaaS vs Databases on K8S • Easy; • Low TCO for small/medium workloads*; • Lacks flexibility*; • Average performance; DBaaS DB on K8S • Still requires some K8s knowledge; • Same flexibility as bare metal; • Operators can automate most of the hard work; • One of the best options for databases with large number of nodes; • Potentially the default option in the near future; -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 21 DBaaS -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 22 Kubernetes Storage https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads Ephemeral Remote Persistent Local Persistent Easiest one, but you lose all data if the pod gets killed. Latency limits the performance of applications with intensive read/write profiles or distributed datastores Not a general purpose storage solution -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 23 Local Persistent Storage https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 24 Local Persistent Storage - Trade offs https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads • Inflexible placements • Lower availability • Lower data durability (not replicated) • Requires Node/disk preparation -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 25 Local Persistent Storage - Trade offs https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads It is ok for databases as they already have data replication internally -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 26 Host Path vs Local Persistent Storage https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads Allows your pod to specify any path in your disk Not portable Not disk accountable Hard to use at scale Host Path -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 27 Host Path vs Local Persistent Storage https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads Leverages the persistent volumes interface Security: Only adms can create/define the content of the PVs Portability Disk accountability StatefulSets Local Persistent Storage -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 28 Bare Metal Vs Containers https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads Yahoo! Cloud Serving Benchmark YCSB includes a set of core workloads that define a basic benchmark for cloud systems -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 29 Workload A https://blog.couchbase.com/performance-certification-of-couchbase-server-5-5-on-kubernetes-platform/ Workload A Update heavy workload: Mix of 50/50 reads and writes Workload E Short ranges: Short ranges of records are queried, instead of individual records. ~0% No substantial difference <10% Performance penalty -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 30 Workload A https://blog.couchbase.com/performance-certification-of-couchbase-server-5-5-on-kubernetes-platform/ -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 31 Workload E https://blog.couchbase.com/performance-certification-of-couchbase-server-5-5-on-kubernetes-platform/ -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 32 Bare Metal Vs Containers https://blog.couchbase.com/performance-certification-of-couchbase-server-5-5-on-kubernetes-platform/ Worst Case: Add an extra node to compensate for potential performance penalties -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 33 Other Operators @deniswsrosa -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. 34 Other Operators https://www.youtube.com/watch?v=J7h0F34iBx0 https://www.youtube.com/watch?v=Zn1vd7sQ_bc @deniswsrosa -
Confidential and Proprietary. Do not distribute without Couchbase consent. ©
Couchbase 2017. All rights reserved. THANKS! @DENISWSROSA