Why you should be deploying Postgres primarily on Kubernetes

Running a Postgres installation, with or without containers, is trivial. However, setting up a production environment is a whole different matter.

Postgres is not by itself a production-ready software: it requires a set of side tools to complement its functionality: connection pooling, monitoring, backup tools, high availability software, you name it. This is called the “Stack Problem”.
Join this brief talk to discuss the Stack Problem, understand how Kubernetes is the platform that best solves it, and what are the main advantages (and disadvantages!) of running Postgres on Kubernetes.

Transcript

1. #Dokc2021
   Why you should be
   deploying Postgres
   primarily on
   Kubernetes
   Alvaro Hernandez
   DoKC Day Kubecon EU 2021
   

   View Slide

2. Deploying Postgres
   without Kubernetes
   DoKC Day Kubecon EU 2021
   

   View Slide

3. DoKC Day Kubecon EU 2021
   apt-get install postgresql
   # well...
   How to deploy Postgres
   

   View Slide

4. DoKC Day Kubecon EU 2021
   OK, we need to tune the database
   https://postgresqlco.nf
   2-8h
   Postgres DBA
   

   View Slide

5. DoKC Day Kubecon EU 2021
   We need to add connection pooling
   pg_bench, scale 2000, m4.large
   (2 vCPU, 8GB RAM, 1k IOPS)
   4-16h
   DevOps / pgDBA
   

   View Slide

6. DoKC Day Kubecon EU 2021
   And High Availability!
   8-24h
   DevOps / pgDBA
   ● HA software (e.g. Patroni)
   ● Distributed configuration
   ● Entrypoint:
   ○ DNS?
   ○ Virtual IP?
   ○ External discovery service (e.g. Consul)?
   

   View Slide

7. DoKC Day Kubecon EU 2021
   Do you backup your data?
   4-16h
   DevOps
   ● Backup software (e.g. WAL-G, pgBackRest)
   ● Backup Storage
   ● Backups lifecycle management
   ● Backup testing / restoration
   

   View Slide

8. DoKC Day Kubecon EU 2021
   You wouldn’t deploy Postgres without monitoring, would you?
   8-24h
   DevOps / pgDBA
   

   View Slide

9. DoKC Day Kubecon EU 2021
   Do you leave Postgres logs on each server?
   4-48h
   DevOps
   ● Configure CSV logging
   ● Add a logging agent (e.g. FluentBit) to export
   logs
   ● Add a logging collector (e.g. Fluentd) to collect
   logs, write code to store it and manage lifecycle.
   ● Or use a paid logs-as-a-Service service
   

   View Slide

10. DoKC Day Kubecon EU 2021
    For advanced users: proxy Postgres traffic with Envoy
    8-16h
    DevOps
    ● Exports additional network metrics to
    Prometheus
    ● Offloads Postgres SSL
    https://www.cncf.io/blog/2020/08/13/envoy-1-15-in
    troduces-a-new-postgres-extension-with-monitoring-
    support/
    

    View Slide

11. DoKC Day Kubecon EU 2021
    Install cluster management software
    ?h
    DevOps
    ??????????????
    

    View Slide

12. DoKC Day Kubecon EU 2021
    IaC: Infrastructure as Code
    48-96h
    DevOps
    

    View Slide

13. DoKC Day Kubecon EU 2021
    This is the whole stack you need to deploy Postgres
    

    View Slide

14. Deploying Postgres
    with Kubernetes
    DoKC Day Kubecon EU 2021
    

    View Slide

15. DoKC Day Kubecon EU 2021
    Deploy a simple cluster with Kubernetes (w/ StackGres)
    1h
    CKA
    apiVersion: stackgres.io/v1
    kind: SGCluster
    metadata:
    name: simple
    spec:
    instances: 2
    postgresVersion: 'latest'
    pods:
    persistentVolume:
    size: '100Gi'
    

    View Slide

16. DoKC Day Kubecon EU 2021
    Deploy an advanced cluster with Kubernetes (w/ StackGres)
    4-16h
    CKA
    ● Create YAMLs for several CRDs
    ● Create Ingress if needed
    ● Expose Web Console (Ingress/LB)
    ● Integrate with GitOps
    

    View Slide

17. Total time to deploy whole Postgres Stack
    Non Kubernetes Kubernetes
    Postgres Configuration 2-8h (DBA)
    Connection pooling 4-16h (DevOps + DBA)
    High Availability 8-24h (DevOps + DBA)
    Backups 4-16h (DevOps)
    Monitoring 8-24h (DevOps + DBA)
    Distributed Logs 4-48h (DevOps)
    Envoy proxy 8-16h (DevOps)
    Cluster Management ????
    IaC 48-96h (DevOps)
    Total: 86-248h (DevOps + DBA) 4-16h CKA
    

    View Slide

18. Day 2 operations
    DoKC Day Kubecon EU 2021
    

    View Slide

19. DoKC Day Kubecon EU 2021
    ● Kubernetes also allows to automate Day 2 operations
    ● CKA is enough, mostly no Postgres expertise needed
    ● E.g. Day 2 operations implemented in StackGres:
    ○ Repack
    ○ Vacuum
    ○ Repack
    ○ Minor version upgrade
    ○ Major version upgrade
    ○ Controlled restart
    ○ Benchmark
    Automating Day 2 operations
    

    View Slide

20. When NOT TO deploy
    Postgres on K8s
    DoKC Day Kubecon EU 2021
    

    View Slide

21. DoKC Day Kubecon EU 2021
    ● You want to consume a fully managed service (e.g. RDS) and pay for it.
    ● You need specific OS-level software/tuning not available in your K8s (e.g. ZFS).
    ● If you still believe that the Earth is flat,
    vaccines are bad,
    Kubernetes is not a solid platform for running you database;
    and don’t want to get out of your comfort zone.
    When NOT TO deploy Postgres on Kubernetes
    

    View Slide

22. DoKC Day Kubecon EU 2021
    

    View Slide