Why you should be running Postgres on Kubernetes

Installing Postgres may be trivial. But setting up a production environment is NOT. Postgres is not batteries included: it requires a set of side tools for production deployments: pooling, monitoring, backup tools, high availability, etc.
Join this talk to discuss this so-called “Stack Problem”, understand how Kubernetes is the platform that best solves it, and what are the main advantages (and disadvantages!) of running Postgres on K8s.

There will be a deep, live demo using the open source StackGres.io project.

https://www.meetup.com/Montreal-Postgres/events/289745522/

Transcript

1. Why you should be running Postgres on Kubernetes
   Why you should be
   running Postgres
   on Kubernetes
   Alvaro Hernandez
   @ahachete
   

   View Slide

2. Why you should be running Postgres on Kubernetes
   ` whoami `
   Alvaro Hernandez
   
   aht.es
   ● Founder & CEO, OnGres
   ● 20+ years Postgres user and DBA
   ● Mostly doing R&D to create new, innovative
   software on Postgres
   ● More than 100 tech talks, most about Postgres
   ● Founder and President of the NPO Fundación
   PostgreSQL
   ● AWS Data Hero
   

   View Slide

3. Why you should be running Postgres on Kubernetes
   Deploying Postgres
   without Kubernetes
   

   View Slide

4. Why you should be running Postgres on Kubernetes
   apt-get install postgresql
   # well...
   How to deploy Postgres
   

   View Slide

5. Why you should be running Postgres on Kubernetes
   OK, we need to tune the database
   2-8h
   Postgres DBA
   

   View Slide

6. Why you should be running Postgres on Kubernetes
   We need to add connection pooling
   pg_bench, scale 2000, m4.large
   (2 vCPU, 8GB RAM, 1k IOPS)
   4-16h
   DevOps / pgDBA
   

   View Slide

7. Why you should be running Postgres on Kubernetes
   And High Availability!
   8-24h
   DevOps / pgDBA
   ● HA software (e.g. Patroni)
   ● Distributed configuration
   ● Entrypoint:
   ○ DNS?
   ○ Virtual IP?
   ○ External discovery service (e.g. Consul)?
   

   View Slide

8. Why you should be running Postgres on Kubernetes
   Do you backup your data?
   4-16h
   DevOps
   ● Backup software (e.g. WAL-G, pgBackRest)
   ● Backup Storage
   ● Backups lifecycle management
   ● Backup testing / restoration
   

   View Slide

9. Why you should be running Postgres on Kubernetes
   You wouldn’t deploy Postgres without monitoring, would you?
   8-24h
   DevOps / pgDBA
   

   View Slide

10. Why you should be running Postgres on Kubernetes
    Do you leave Postgres logs on each server?
    4-48h
    DevOps
    ● Configure CSV logging
    ● Add a logging agent (e.g. FluentBit) to export
    logs
    ● Add a logging collector (e.g. Fluentd) to collect
    logs, write code to store it and manage lifecycle.
    ● Or use a paid logs-as-a-Service
    

    View Slide

11. Why you should be running Postgres on Kubernetes
    For advanced users: proxy Postgres traffic with Envoy
    8-16h
    DevOps
    ● Exports additional network metrics to
    Prometheus
    ● Offloads Postgres SSL
    https://www.cncf.io/blog/2020/08/13/envoy-1-15-in
    troduces-a-new-postgres-extension-with-monitoring-
    support/
    

    View Slide

12. Why you should be running Postgres on Kubernetes
    Install cluster management software
    ?h
    DevOps
    ??????????????
    

    View Slide

13. Why you should be running Postgres on Kubernetes
    IaC: Infrastructure as Code
    48-96h
    DevOps
    

    View Slide

14. Why you should be running Postgres on Kubernetes
    This is the whole stack you need to deploy Postgres
    

    View Slide

15. Why you should be running Postgres on Kubernetes
    Deploying Postgres
    with Kubernetes
    

    View Slide

16. Why you should be running Postgres on Kubernetes
    Deploy a simple cluster with Kubernetes (w/ StackGres)
    1h
    CKA
    apiVersion: stackgres.io/v1
    kind: SGCluster
    metadata:
    name: simple
    spec:
    instances: 2
    postgres:
    version: 'latest'
    pods:
    persistentVolume:
    size: '100Gi'
    

    View Slide

17. Why you should be running Postgres on Kubernetes
    Deploy an advanced cluster with Kubernetes (w/ StackGres)
    4-16h
    CKA
    ● Create YAMLs for several CRDs
    ● Create Ingress if needed
    ● Expose Web Console (Ingress/LB)
    ● Integrate with GitOps
    

    View Slide

18. Why you should be running Postgres on Kubernetes
    Total time to deploy whole Postgres Stack
    Without Kubernetes Kubernetes
    Postgres Configuration 2-8h (DBA)
    Connection pooling 4-16h (DevOps + DBA)
    High Availability 8-24h (DevOps + DBA)
    Backups 4-16h (DevOps)
    Monitoring 8-24h (DevOps + DBA)
    Distributed Logs 4-48h (DevOps)
    Envoy proxy 8-16h (DevOps)
    Cluster Management ????
    IaC 48-96h (DevOps)
    Total: 86-248h (DevOps + DBA) 4-16h CKA
    

    View Slide

19. Why you should be running Postgres on Kubernetes
    Day 2 Operations
    

    View Slide

20. Why you should be running Postgres on Kubernetes
    ● Kubernetes also allows to automate Day 2 operations
    ● CKA is enough, mostly no Postgres expertise needed
    ● E.g. Day 2 operations implemented in StackGres:
    ○ Repack
    ○ Vacuum
    ○ Repack
    ○ Minor version upgrade
    ○ Major version upgrade
    ○ Controlled restart
    ○ Benchmark
    Automating Day 2 operations
    

    View Slide

21. Why you should be running Postgres on Kubernetes
    When not to
    deploy Postgres on
    Kubernetes
    

    View Slide

22. Why you should be running Postgres on Kubernetes
    ● You want to consume a fully managed service (e.g. RDS) and pay for it.
    ● You need specific OS-level software/tuning not available in your K8s (e.g. ZFS).
    ● If you still believe that the Earth is flat,
    vaccines are bad,
    Kubernetes is not a solid platform for running you database;
    and don’t want to get out of your comfort zone.
    When NOT TO deploy Postgres on Kubernetes
    

    View Slide

23. Why you should be running Postgres on Kubernetes
    Demo!!!
    Follow the tutorial: https://stackgres.io/doc/latest/tutorial/
    Get help, share feedback: https://slack.stackgres.io
    

    View Slide

24. Why you should be running Postgres on Kubernetes
    Q & A
    Alvaro Hernandez
    @ahachete
    

    View Slide