Agenda for this Citus & Patroni talk
@ Citus Con: An Event for Postgres 2023
▪ What (business)problems do we solve?
▪ High Availability with Patroni
▪ Horizontal scalability with Citus
▪ PostgreSQL + Citus + Patroni for the win
▪ Live demo!
▪ Conclusion
1
2
3
4
5
6
Slide 3
Slide 3 text
As an application developer I want...
• A database
3
Slide 4
Slide 4 text
As an application developer I want...
• A database
• A highly available database
4
Slide 5
Slide 5 text
As an application developer I want...
• A database
• A highly available database
• A highly available database that scales
5
Slide 6
Slide 6 text
As an application developer I want...
• A database
• A highly available database
• A highly available database that scales
• A highly available database that scales horizontally!
6
Slide 7
Slide 7 text
As an application developer I want...
• A database
• A highly available database
• A highly available database that scales
• A highly available database that scales horizontally!
• Highly available database that scales horizontally for free!
7
Slide 8
Slide 8 text
PostgreSQL: The World's
Most Advanced Open
Source Relational Database
8
Slide 9
Slide 9 text
High availability with Patroni
9
Primary
Standby
Etcd cluster
Application Level Sharding
12
Shard 1
Shard 2
Shard 3
Shard 4
Application
SELECT * FROM u WHERE id=4
SELECT * FROM u WHERE id=5
SELECT * FROM u WHERE id=6
SELECT * FROM u WHERE id=7
ID%4 == 0
ID%4 == 1
ID%4 == 2
ID%4 == 3
Slide 13
Slide 13 text
Disadvantages of Application Level Sharding
• Must be implemented in the application
• Separate connection (pool) for every shard
• Must be introduced from day 1
• Depending on implementation adding new shards is either
impossible or hard
• If new shards are added, they are heavily disbalanced
• Rebalancing (without downtime) is a challenge
• Schema changes must be deployed on all shards
13
Slide 14
Slide 14 text
Sharding with Citus
14
Worker 1
Worker 2
Worker 3
Worker 4
Application
Coordinator
Slide 15
Slide 15 text
Sharding with Citus: Pros & Cons
Pros
• Transparent sharding
• A single connection (pool)
• Query from any node!
• Schema changes via coordinator
• Start with small and scale-out
• Rebalancing without downtime!
15
Cons
• Increased query latency
Slide 16
Slide 16 text
Citus HA with Patroni 3.0
16
primary
standby
Citus worker 1
primary
standby
Citus coordinator
primary
standby
Citus worker 2
Slide 17
Slide 17 text
Enabling Citus support in Patroni 3.0
scope: my-citus-cluster # must be the same on all nodes
citus:
group: X # 0 for coordinator and 1, 2, 3, etc for workers
database: citus # must be the same on all nodes
17
Slide 18
Slide 18 text
Adding new workers to the Citus cluster
1. Adjust patroni.yaml, citus.group parameter
2. Start Patroni on the new node
3. SELECT * FROM citus_rebalance_start(); -- rebalance
shards
4. SELECT * FROM citus_rebalance_status(); -- check
rebalance progress
18
Slide 19
Slide 19 text
DEMO
19
Slide 20
Slide 20 text
20
Conclusion
Slide 21
Slide 21 text
Conclusion
• Enterprise-grade sharding with Citus
• Enterprise-grade HA with Patroni
• Everything is open-source!
• Start with small and scale
• Vertically
• Horizontally
21
Slide 22
Slide 22 text
Thank you!
• Citus open source repo on
GitHub aka.ms/citus
• Patroni repo on GitHub
github.com/zalando/patroni
• Alexander on Twitter
@cyberdemn