Why you should be deploying Postgres primarily on Kubernetes

Running a Postgres installation, with or without containers, is trivial. However, setting up a production environment is a whole different matter.

Postgres is not by itself a production-ready software: it requires a set of side tools to complement its functionality: connection pooling, monitoring, backup tools, high availability software, you name it. This is called the “Stack Problem”.
Join this brief talk to discuss the Stack Problem, understand how Kubernetes is the platform that best solves it, and what are the main advantages (and disadvantages!) of running Postgres on Kubernetes.

Transcript

1. #Dokc2021 Why you should be deploying Postgres primarily on Kubernetes Alvaro Hernandez DoKC Day Kubecon EU 2021
2. Deploying Postgres without Kubernetes DoKC Day Kubecon EU 2021
3. DoKC Day Kubecon EU 2021 apt-get install postgresql # well... How to deploy Postgres
4. DoKC Day Kubecon EU 2021 OK, we need to tune the database https://postgresqlco.nf 2-8h Postgres DBA
5. DoKC Day Kubecon EU 2021 We need to add connection pooling pg_bench, scale 2000, m4.large (2 vCPU, 8GB RAM, 1k IOPS) 4-16h DevOps / pgDBA
6. DoKC Day Kubecon EU 2021 And High Availability! 8-24h DevOps / pgDBA • HA software (e.g. Patroni) • Distributed configuration • Entrypoint: ◦ DNS? ◦ Virtual IP? ◦ External discovery service (e.g. Consul)?
7. DoKC Day Kubecon EU 2021 Do you backup your data? 4-16h DevOps • Backup software (e.g. WAL-G, pgBackRest) • Backup Storage • Backups lifecycle management • Backup testing / restoration
8. DoKC Day Kubecon EU 2021 You wouldn’t deploy Postgres without monitoring, would you? 8-24h DevOps / pgDBA
9. DoKC Day Kubecon EU 2021 Do you leave Postgres logs on each server? 4-48h DevOps • Configure CSV logging • Add a logging agent (e.g. FluentBit) to export logs • Add a logging collector (e.g. Fluentd) to collect logs, write code to store it and manage lifecycle. • Or use a paid logs-as-a-Service service
10. DoKC Day Kubecon EU 2021 For advanced users: proxy Postgres traffic with Envoy 8-16h DevOps • Exports additional network metrics to Prometheus • Offloads Postgres SSL https://www.cncf.io/blog/2020/08/13/envoy-1-15-in troduces-a-new-postgres-extension-with-monitoring- support/
11. DoKC Day Kubecon EU 2021 Install cluster management software ?h DevOps ??????????????
12. DoKC Day Kubecon EU 2021 IaC: Infrastructure as Code 48-96h DevOps
13. DoKC Day Kubecon EU 2021 This is the whole stack you need to deploy Postgres
14. Deploying Postgres with Kubernetes DoKC Day Kubecon EU 2021
15. DoKC Day Kubecon EU 2021 Deploy a simple cluster with Kubernetes (w/ StackGres) 1h CKA apiVersion: stackgres.io/v1 kind: SGCluster metadata: name: simple spec: instances: 2 postgresVersion: 'latest' pods: persistentVolume: size: '100Gi'
16. DoKC Day Kubecon EU 2021 Deploy an advanced cluster with Kubernetes (w/ StackGres) 4-16h CKA • Create YAMLs for several CRDs • Create Ingress if needed • Expose Web Console (Ingress/LB) • Integrate with GitOps
17. Total time to deploy whole Postgres Stack Non Kubernetes Kubernetes Postgres Configuration 2-8h (DBA) Connection pooling 4-16h (DevOps + DBA) High Availability 8-24h (DevOps + DBA) Backups 4-16h (DevOps) Monitoring 8-24h (DevOps + DBA) Distributed Logs 4-48h (DevOps) Envoy proxy 8-16h (DevOps) Cluster Management ???? IaC 48-96h (DevOps) Total: 86-248h (DevOps + DBA) 4-16h CKA
18. Day 2 operations DoKC Day Kubecon EU 2021
19. DoKC Day Kubecon EU 2021 • Kubernetes also allows to automate Day 2 operations • CKA is enough, mostly no Postgres expertise needed • E.g. Day 2 operations implemented in StackGres: ◦ Repack ◦ Vacuum ◦ Repack ◦ Minor version upgrade ◦ Major version upgrade ◦ Controlled restart ◦ Benchmark Automating Day 2 operations
20. When NOT TO deploy Postgres on K8s DoKC Day Kubecon EU 2021
21. DoKC Day Kubecon EU 2021 • You want to consume a fully managed service (e.g. RDS) and pay for it. • You need specific OS-level software/tuning not available in your K8s (e.g. ZFS). • If you still believe that the Earth is flat, vaccines are bad, Kubernetes is not a solid platform for running you database; and don’t want to get out of your comfort zone. When NOT TO deploy Postgres on Kubernetes
22. DoKC Day Kubecon EU 2021