Slide 1
Slide 1 text
The relational model in the modern
development age
Shlomi Noach
PlanetScale
FOSDEM 2022
Towards an improved operational paradigm,
illustrated by Vitess
Slide 2
Slide 2 text
Agenda
- Relational databases as production systems
- Modern expectations
- Schema deployment friction
- Deployment sheduling
- Deployment conflicts
- Undeploying
- Redeploying, idempotency & version control
Slide 3
Slide 3 text
About me
Engineer at PlanetScale
Author of open source projects orchestrator, gh-ost,
freno and others
Maintainer for Vitess
Blog at http://openark.org
github.com/shlomi-noach
@ShlomiNoach
Slide 4
Slide 4 text
Founded Feb. 2018 by co-creators of Vitess, ~85
employees, HQ San Francisco, remote team
MySQL-compatible serverless database platform, built
for developers
Built on top of Vitess

Slide 5
Slide 5 text
Vitess
A database clustering system for horizontal scaling of
MySQL
● CNCF graduated project
● Open source, Apache 2.0 licence
● Contributors from around the community
Slide 6
Slide 6 text
The Relational Model
Data changes vs. schema changes, and where relational
databases fell short
Slide 7
Slide 7 text
Deploying: size and time
Deployments can be measured in minutes, hours or days
Slide 8
Slide 8 text
Deploying: submission & runtime
Scheduling migrations
Slide 9
Slide 9 text
Scheduling: offered paradigm
Asynchronous migrations
Database scheduler
Slide 10
Slide 10 text
Vitess: migration scheduler
Migrations run asynchronous, decoupled from caller
Flexible scheduling:
Single pending migration at a time
Single running migration at a time
Concurrent migrations
Slide 11
Slide 11 text
Deployment conflicts
Choose between a required deployment or a required
operation
● Server restart/reboot
● Promotion/failover
Slide 12
Slide 12 text
Deployment conflicts
Interrupt due to conflicting operations
Slide 13
Slide 13 text
Deployment conflicts
Withhold due to conflicting operations
Slide 14
Slide 14 text
Deployment conflicts
Pursue and withhold operation
Slide 15
Slide 15 text
Deployment conflicts
Deployment is coupled with the running database server
process
Slide 16
Slide 16 text
Deployment conflicts: offered
paradigm
Process, server & failure agnostic deployments
Slide 17
Slide 17 text
Vitess: failure agnostic migrations
mysql> SET @@ddl_strategy=’online’;
mysql> ALTER TABLE my_table
ADD INDEX name_idx(name(24)), DROP COLUMN c;
Auto-resumes after restart
Auto resumes on promoted replica
Slide 18
Slide 18 text
Undeploying
Rolling back a deployment
Slide 19
Slide 19 text
Undeploying
Blast radius
Slide 20
Slide 20 text
Undeploying
Feasibility
Slide 21
Slide 21 text
Undeploying
Feasibility: constructing the reverse ALTER statement
mysql> ALTER TABLE my_table DROP INDEX state_idx;
mysql> ALTER TABLE my_table ADD INDEX
state_idx(state, token(32));
Slide 22
Slide 22 text
Undeploying
Feasibility: dropped data
mysql> ALTER TABLE my_table DROP COLUMN my_col;
mysql> DROP TABLE my_table;
Slide 23
Slide 23 text
Undeploying
Time
Slide 24
Slide 24 text
Undeploying: offered paradigm
Revert as first class citizen
Slide 25
Slide 25 text
Vitess: REVERT
mysql> SET @@ddl_strategy=’online’;
mysql> REVERT VITESS_MIGRATION
‘a1dac193_4b86_11ec_a827_0a43f95f28a3’;
Restore dropped tables, populated with data
Restore dropped columns, populated with data
Quick operation, proportional to time since migration
completion, not to table size.
Slide 26
Slide 26 text
Vitess: REVERT
Near instant REVERT made possible by:
mysql> SET @@ddl_strategy=’online
-allow-concurrent -postpone-completion’;
mysql> REVERT VITESS_MIGRATION
‘A1dac193_4b86_11ec_a827_0a43f95f28a3’;
+--------------------------------------+
| uuid |
+--------------------------------------+
| bf4598ab_8d55_11eb_815f_f875a4d24e90 |
+--------------------------------------+
Runs, but does not complete, a reverting migration
Slide 27
Slide 27 text
Vitess: REVERT
Either choose to actually revert in case of trouble, or
abort the revert if all goes well:
mysql> ALTER VITESS_MIGRATION
'bf4598ab_8d55_11eb_815f_f875a4d24e90' COMPLETE;
mysql> ALTER VITESS_MIGRATION
'bf4598ab_8d55_11eb_815f_f875a4d24e90' CANCEL;
Slide 28
Slide 28 text
Redeploying
Painfully untrivial
Slide 29
Slide 29 text
Redeploying
Relational databases and imperative changes
Slide 30
Slide 30 text
Redeploying
Analyzing errors
mysql> ALTER TABLE my_table ADD COLUMN my_column
INT NOT NULL DEFAULT 0;
Slide 31
Slide 31 text
Redeploying
Lack of errors, undesired operation
mysql> ALTER TABLE my_table ADD INDEX (status);
mysql> ALTER TABLE my_table
PARTITION BY HASH(id) PARTITIONS 4;
Slide 32
Slide 32 text
Redeploying
What is your current schema?
Slide 33
Slide 33 text
Redeploying
What is your current schema?
db_changes/0001_initial
db_changes/0002_add_products
db_changes/0003_key_on_user
db_changes/0004_refactor_address_to_another_table
Slide 34
Slide 34 text
Redeploying
So what is your current schema?
Slide 35
Slide 35 text
Redeploying
Effective source of truth?
Slide 36
Slide 36 text
Redeploying
Would you agree to read code by series of patches?
Slide 37
Slide 37 text
Redeploying
Captive in decades old flow
Slide 38
Slide 38 text
Redeploying: offered paradigm
Declarative schema changes
Slide 39
Slide 39 text
Vitess: declarative changes
mysql> SET @@ddl_strategy='online -declarative';
-- The following migration creates a new table, as
the table does not exist:
mysql> CREATE TABLE decl_table(
id INT PRIMARY KEY
);
+--------------------------------------+
| uuid |
+--------------------------------------+
| b06475e5_8a74_11eb_badd_f875a4d24e90 |
+--------------------------------------+
Slide 40
Slide 40 text
Vitess: declarative changes
mysql> SET @@ddl_strategy='online -declarative';
-- The next migration will implicitly ALTER the
table decl_table into desired state:
mysql> CREATE TABLE decl_table(
id INT PRIMARY KEY,
ts TIMESTAMP NOT NULL
);
+--------------------------------------+
| uuid |
+--------------------------------------+
| b7d6e6fb_8a74_11eb_badd_f875a4d24e90 |
+--------------------------------------+
Slide 41
Slide 41 text
Vitess: declarative changes
mysql> SET @@ddl_strategy='online -declarative';
-- Next migration does not change table structure,
hence is a noop and implicitly successful:
mysql> CREATE TABLE decl_table(
id INT PRIMARY KEY,
ts TIMESTAMP NOT NULL
);
+--------------------------------------+
| uuid |
+--------------------------------------+
| 110574b1_8a75_11eb_badd_f875a4d24e90 |
+--------------------------------------+
Slide 42
Slide 42 text
Vitess: declarative changes
A hybrid approach: per-table declarative
Exploration continues
Slide 43
Slide 43 text
Fear of deployment
“Risky” schema deployments
Human-handled large deployments
Avoid dealing with the database
Slide 44
Slide 44 text
Existing paradigm: conclusion
We are caught in a paradigm that has evolved an
elaborate ecosystem, yet one that does not meet modern
expectations
Outdated paradigms go beyond schema deployments
Slide 45
Slide 45 text
The relational developer paradigm
Asynchronous migrations
Scheduled migrations
Migrations decoupled from server, survive reboot and
failover
Revertible migrations as first class citizens
Retriable migrations
Support declarative, idempotent migrations
Slide 46
Slide 46 text
Links
Vitess project home page and docs: https://vitess.io/
Vitess repo: https://github.com/vitessio/vitess
Vitess Slack workspace: https://vitess.io/slack
PlanetScale docs: https://docs.planetscale.com
Slide 47
Slide 47 text
Thank you!
Questions?
github.com/shlomi-noach
@ShlomiNoach