Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Crafty Coders Meetup: Databases On Kubernetes: ...

Crafty Coders Meetup: Databases On Kubernetes: Why You should care

deniswsrosa

August 25, 2020
Tweet

More Decks by deniswsrosa

Other Decks in Programming

Transcript

  1. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. DATABASES ON KUBERNETES Denis Rosa | Developer Advocate @deniswsrosa @deniswsrosa WHY YOU SHOULD CARE
  2. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. • +15 years as a Developer • NoSQL & Java Specialist • OSS contributor Denis Rosa deniswsrosa @deniswsrosa [email protected]
  3. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa 2016 Containers are designed to be stateless
  4. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa Fast forward to 2020…
  5. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa What have changed?
  6. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa • Production-Grade Images • Easier to setup and configure • Separate storage from compute • Kubernetes
  7. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 9 Default Deployment
  8. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa K8S Worker1 MySQL Volume MYSQL DEPLOYMENT
  9. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa K8S Worker1 MySQL Volume DEALING WITH FAILURES
  10. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa K8S Worker1 MySQL Volume MySQL Volume DEALING WITH FAILURES
  11. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa K8S Worker1 MySQL Volume MySQL Volume DEALING WITH FAILURES
  12. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa K8S Worker1 MySQL Volume K8S Worker2 MySQL Volume DEALING WITH FAILURES
  13. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa K8S Worker1 MySQL Volume K8S Worker2 MySQL Volume DEALING WITH FAILURES
  14. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa K8S Worker1 MySQL Volume K8S Worker3 MySQL Volume (empty) DEALING WITH FAILURES ?
  15. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa Looks like something is missing…
  16. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. DEMO1 – A K8S TODO LIST
  17. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 20 CRDs + K8s Operators
  18. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 21 Java Operator - Example Source: https://developers.redhat.com/blog/2019/10/07/write-a-simple-kubernetes-operator-in-java-using-the-fabric8-kubernetes-client/
  19. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. @deniswsrosa K8S Worker1 Couchbase Volume K8S Worker2 Couchbase Volume OPERATORS + KUBERNETES Operator (stateless)
  20. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 23 Quick Operator Revision Operator Knowledge from app experts Deployments Autoscaling StatefulSets Configs
  21. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 24 K8s Database Operators Provide a cloud agnostic Database-as-a-Service solution
  22. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. DEMO 2 – LET’S PLAY WITH A DATABASE ON KUBERNETES
  23. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 26 Other Possibilities Automated Backup XDCR Prometheus Metrics
  24. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 27 Standard Deployment
  25. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 28 DBaaS vs Databases on K8S • Easy; • Low TCO for small/medium workloads*; • Lacks flexibility*; • Average performance; DBaaS DB on K8S • Still requires some K8s knowledge; • Same flexibility as bare metal; • Operators can automate most of the hard work; • One of the best options for databases with large number of nodes; • Potentially the default option in the near future;
  26. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 30 Kubernetes Storage https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads Ephemeral Remote Persistent Local Persistent Easiest one, but you lose all data if the pod gets killed. Latency limits the performance of applications with intensive read/write profiles or distributed datastores Not a general purpose storage solution
  27. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 31 Local vs Remote Persistent Storage
  28. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 32 Local Persistent Storage - Trade offs https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads • Inflexible placements • Lower availability • Lower data durability (not replicated) • Requires Node/disk preparation
  29. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 33 Local Persistent Storage - Trade offs https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads It is ok for databases as they already have data replication internally
  30. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 34 Host Path vs Local Persistent Storage https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads Allows your pod to specify any path in your disk Not portable Not disk accountable Hard to use at scale Host Path
  31. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 35 Host Path vs Local Persistent Storage https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads Leverages the persistent volumes interface Security: Only adms can create/define the content of the PVs Portability Disk accountability StatefulSets Local Persistent Storage
  32. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 36 Bare Metal Vs Containers https://github.com/brianfrankcooper/YCSB/wiki/Core-Workloads Yahoo! Cloud Serving Benchmark YCSB includes a set of core workloads that define a basic benchmark for cloud systems
  33. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 37 Results https://blog.couchbase.com/performance-certification-of-couchbase-server-5-5-on-kubernetes-platform/ Workload A Update heavy workload: Mix of 50/50 reads and writes Workload E Short ranges: Short ranges of records are queried, instead of individual records. ~0% No substantial difference <10%* Performance penalty * This is more than one year old test, since then some improvements have been made to docker
  34. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 38 Workload A https://blog.couchbase.com/performance-certification-of-couchbase-server-5-5-on-kubernetes-platform/
  35. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 39 Workload E https://blog.couchbase.com/performance-certification-of-couchbase-server-5-5-on-kubernetes-platform/
  36. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. 40 Other Operators @deniswsrosa
  37. Confidential and Proprietary. Do not distribute without Couchbase consent. ©

    Couchbase 2017. All rights reserved. THANKS! @DENISWSROSA REGISTER ON