Postgres Is the New Default

Slide 1

Slide 1 text

Postgres is the new default How we transitioned our platform and why you should too

Slide 2

Slide 2 text

I work with Data!

Slide 3

Slide 3 text

Agenda • Where we were • Our relationship with Postgres • Challenges • Our future together

Slide 4

Slide 4 text

We live in The Cloud!

Slide 5

Slide 5 text

Engine Yard • Languages • Amazing Support • Team DBAs & me • 2 Products *

Slide 6

Slide 6 text

> 1 Petabyte

Slide 7

Slide 7 text

No multi-tenancy.

Slide 8

Slide 8 text

...A year (or so) ago

Slide 9

Slide 9 text

Engine Yard Cloud • Only MySQL 5.0 • 8.3 in perpetual Alpha • No Slaves, No Docs • Increasing interest • Stack Expansion - DIRTT

Slide 10

Slide 10 text

Our Relationship with Postgres

Slide 11

Slide 11 text

Infatuation Stage 1

Slide 12

Slide 12 text

Why love Postgres? • Flexible Replication: streaming & hot standby • Features & Extensions • Versatile Data Types • FDWs

Slide 13

Slide 13 text

Why love Postgres? • All of the Indexes! • B-tree, GiST, GIN, SP-Gist* • Procedural Languages (PL) • Full-Text Search

Slide 14

Slide 14 text

Why love Postgres? • Feature parity with Managed • Outstanding community & commercial support • Let’s make it our default!

Slide 15

Slide 15 text

Understanding Stage 2

Slide 16

Slide 16 text

Re-Tool • MySQL Shop • DBA Training: DBAs & Engineers • OSS to help us gain velocity

Slide 17

Slide 17 text

Customer Education • Customer Facing Docs focused on MySQL • Technical Writers FTW!

Slide 18

Slide 18 text

Disturbances Stage 3

Slide 19

Slide 19 text

Roadmap issues • Shiny new process for DB releases • Alpha, Beta, GA • Surprise we got it WRONG!

Slide 20

Slide 20 text

Multiple Architectures • EBS Volumes & 32-bit instance sizes • Dump & Restore only way to vertically scale

Slide 21

Slide 21 text

Bad Assumptions • Product codebase tied to old assumptions • MySQL everywhere • Refactor!

Slide 22

Slide 22 text

Moulding Stage 5

Slide 23

Slide 23 text

Company-Wide Effort

Slide 24

Slide 24 text

Installation • Engine Yard Cloud • Chef > EBS • Engine Yard Managed • Puppet > Storage

Slide 25

Slide 25 text

Configuration • $PGDATA on EBS • We don’t RAID • Snapshots to S3 • pg_xlog same volume

Slide 26

Slide 26 text

Replication • Archiving on db_master • hot_standby slaves • All slaves <> AZs • Smaller slaves

Slide 27

Slide 27 text

Monitoring • Bucardo’s check_postgres • Collectd & Nagios • Process health, replication, checkpoints - MORE

Slide 28

Slide 28 text

Backups • Internal tool wraps around pg_dump • Snapshots & dumps to S3 • Only backup from slave (if present)

Slide 29

Slide 29 text

Allow Extensions • Dedicated instances • Customizations OK! • Ship recipes for ~30 extensions • Curated & tested

Slide 30

Slide 30 text

Architectures • New instance types: 64-bit small, mediums • Standardized instances architecture 64-bit

Slide 31

Slide 31 text

Happiness & Trust Stage 9

Slide 32

Slide 32 text

Future Directions • 9.2 Support • AWS Optimizations • Provisioned IOPS • EBS optimized instances • SSD-backed instances

Slide 33

Slide 33 text

Future Directions • Painless upgrades • Better extensions • Ugh to cookbooks! • HA/DR

Slide 34

Slide 34 text

Future Directions • Customer education still a problem • Low adoption of slaves • ~146 year ago • ~165 now