Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Why you should be running Postgres on Kubernetes

OnGres
November 22, 2022

Why you should be running Postgres on Kubernetes

Installing Postgres may be trivial. But setting up a production environment is NOT. Postgres is not batteries included: it requires a set of side tools for production deployments: pooling, monitoring, backup tools, high availability, etc.
Join this talk to discuss this so-called “Stack Problem”, understand how Kubernetes is the platform that best solves it, and what are the main advantages (and disadvantages!) of running Postgres on K8s.

There will be a deep, live demo using the open source StackGres.io project.

https://www.meetup.com/Montreal-Postgres/events/289745522/

OnGres

November 22, 2022
Tweet

More Decks by OnGres

Other Decks in Technology

Transcript

  1. Why you should be running Postgres on Kubernetes
    Why you should be
    running Postgres
    on Kubernetes
    Alvaro Hernandez
    @ahachete

    View Slide

  2. Why you should be running Postgres on Kubernetes
    ` whoami `
    Alvaro Hernandez

    aht.es
    ● Founder & CEO, OnGres
    ● 20+ years Postgres user and DBA
    ● Mostly doing R&D to create new, innovative
    software on Postgres
    ● More than 100 tech talks, most about Postgres
    ● Founder and President of the NPO Fundación
    PostgreSQL
    ● AWS Data Hero

    View Slide

  3. Why you should be running Postgres on Kubernetes
    Deploying Postgres
    without Kubernetes

    View Slide

  4. Why you should be running Postgres on Kubernetes
    apt-get install postgresql
    # well...
    How to deploy Postgres

    View Slide

  5. Why you should be running Postgres on Kubernetes
    OK, we need to tune the database
    2-8h
    Postgres DBA

    View Slide

  6. Why you should be running Postgres on Kubernetes
    We need to add connection pooling
    pg_bench, scale 2000, m4.large
    (2 vCPU, 8GB RAM, 1k IOPS)
    4-16h
    DevOps / pgDBA

    View Slide

  7. Why you should be running Postgres on Kubernetes
    And High Availability!
    8-24h
    DevOps / pgDBA
    ● HA software (e.g. Patroni)
    ● Distributed configuration
    ● Entrypoint:
    ○ DNS?
    ○ Virtual IP?
    ○ External discovery service (e.g. Consul)?

    View Slide

  8. Why you should be running Postgres on Kubernetes
    Do you backup your data?
    4-16h
    DevOps
    ● Backup software (e.g. WAL-G, pgBackRest)
    ● Backup Storage
    ● Backups lifecycle management
    ● Backup testing / restoration

    View Slide

  9. Why you should be running Postgres on Kubernetes
    You wouldn’t deploy Postgres without monitoring, would you?
    8-24h
    DevOps / pgDBA

    View Slide

  10. Why you should be running Postgres on Kubernetes
    Do you leave Postgres logs on each server?
    4-48h
    DevOps
    ● Configure CSV logging
    ● Add a logging agent (e.g. FluentBit) to export
    logs
    ● Add a logging collector (e.g. Fluentd) to collect
    logs, write code to store it and manage lifecycle.
    ● Or use a paid logs-as-a-Service

    View Slide

  11. Why you should be running Postgres on Kubernetes
    For advanced users: proxy Postgres traffic with Envoy
    8-16h
    DevOps
    ● Exports additional network metrics to
    Prometheus
    ● Offloads Postgres SSL
    https://www.cncf.io/blog/2020/08/13/envoy-1-15-in
    troduces-a-new-postgres-extension-with-monitoring-
    support/

    View Slide

  12. Why you should be running Postgres on Kubernetes
    Install cluster management software
    ?h
    DevOps
    ??????????????

    View Slide

  13. Why you should be running Postgres on Kubernetes
    IaC: Infrastructure as Code
    48-96h
    DevOps

    View Slide

  14. Why you should be running Postgres on Kubernetes
    This is the whole stack you need to deploy Postgres

    View Slide

  15. Why you should be running Postgres on Kubernetes
    Deploying Postgres
    with Kubernetes

    View Slide

  16. Why you should be running Postgres on Kubernetes
    Deploy a simple cluster with Kubernetes (w/ StackGres)
    1h
    CKA
    apiVersion: stackgres.io/v1
    kind: SGCluster
    metadata:
    name: simple
    spec:
    instances: 2
    postgres:
    version: 'latest'
    pods:
    persistentVolume:
    size: '100Gi'

    View Slide

  17. Why you should be running Postgres on Kubernetes
    Deploy an advanced cluster with Kubernetes (w/ StackGres)
    4-16h
    CKA
    ● Create YAMLs for several CRDs
    ● Create Ingress if needed
    ● Expose Web Console (Ingress/LB)
    ● Integrate with GitOps

    View Slide

  18. Why you should be running Postgres on Kubernetes
    Total time to deploy whole Postgres Stack
    Without Kubernetes Kubernetes
    Postgres Configuration 2-8h (DBA)
    Connection pooling 4-16h (DevOps + DBA)
    High Availability 8-24h (DevOps + DBA)
    Backups 4-16h (DevOps)
    Monitoring 8-24h (DevOps + DBA)
    Distributed Logs 4-48h (DevOps)
    Envoy proxy 8-16h (DevOps)
    Cluster Management ????
    IaC 48-96h (DevOps)
    Total: 86-248h (DevOps + DBA) 4-16h CKA

    View Slide

  19. Why you should be running Postgres on Kubernetes
    Day 2 Operations

    View Slide

  20. Why you should be running Postgres on Kubernetes
    ● Kubernetes also allows to automate Day 2 operations
    ● CKA is enough, mostly no Postgres expertise needed
    ● E.g. Day 2 operations implemented in StackGres:
    ○ Repack
    ○ Vacuum
    ○ Repack
    ○ Minor version upgrade
    ○ Major version upgrade
    ○ Controlled restart
    ○ Benchmark
    Automating Day 2 operations

    View Slide

  21. Why you should be running Postgres on Kubernetes
    When not to
    deploy Postgres on
    Kubernetes

    View Slide

  22. Why you should be running Postgres on Kubernetes
    ● You want to consume a fully managed service (e.g. RDS) and pay for it.
    ● You need specific OS-level software/tuning not available in your K8s (e.g. ZFS).
    ● If you still believe that the Earth is flat,
    vaccines are bad,
    Kubernetes is not a solid platform for running you database;
    and don’t want to get out of your comfort zone.
    When NOT TO deploy Postgres on Kubernetes

    View Slide

  23. Why you should be running Postgres on Kubernetes
    Demo!!!
    Follow the tutorial: https://stackgres.io/doc/latest/tutorial/
    Get help, share feedback: https://slack.stackgres.io

    View Slide

  24. Why you should be running Postgres on Kubernetes
    Q & A
    Alvaro Hernandez
    @ahachete

    View Slide