How PostgreSQL is tested

Slide 1

Slide 1 text

How PostgreSQL is tested Peter Eisentraut [email protected] @petereisentraut

Slide 2

Slide 2 text

POSTGRES 4.2

Slide 3

Slide 3 text

src/test/regress/

Slide 4

Slide 4 text

sql/ results/ expected/

Slide 5

Slide 5 text

make installcheck

Slide 6

Slide 6 text

make check 7.0 install before test parallel

Slide 7

Slide 7 text

pg_regress

Slide 8

Slide 8 text

Problems can only test SQL-level things must be deterministic plan differences platform differences configuration differences changes in external dependencies inverts TDD approach

Slide 9

Slide 9 text

Module tests 6.5: mb tests 7.1: intarray, cube, seg ...

Slide 10

Slide 10 text

ECPG tests 8.2

Slide 11

Slide 11 text

Build farm http://buildfarm.postgresql.org/ 8.0 ... 8.3 ...

Slide 12

Slide 12 text

Why test?

Slide 13

Slide 13 text

The perfect patch code documentation tests commit message

Slide 14

Slide 14 text

make check-world 9.0

Slide 15

Slide 15 text

Isolation tests 9.1

Slide 16

Slide 16 text

pg_upgrade tests 9.2

Slide 17

Slide 17 text

libpq (URL) tests 9.2

Slide 18

Slide 18 text

TAP/Perl tests 9.4: frontends 9.5: SSL 9.6: replication

Slide 19

Slide 19 text

Testing zoo pg_regress ecpg isolationtester pg_upgrade tests libpq tests TAP/Perl

Slide 20

Slide 20 text

Other testing Debian packaging QA teams users

Slide 21

Slide 21 text

Coverity, scan-build, Valgrind, ... see PGCon 2014

Slide 22

Slide 22 text

Test coverage make coverage-clean make check make coverage-html http://coverage.postgresql.org/ http://blog.2ndquadrant.com/code-coverage/ http://pgci.eisentraut.org/jenkins/job/postgresql_master_coverage/

Slide 23

Slide 23 text

Gaps pg_dump pg_upgrade more details pg_upgrade cross-version other clients/tools indexes vacuum crash recovery

Slide 24

Slide 24 text

More gaps configurations protocol access control replication archiving

Slide 25

Slide 25 text

Even more gaps performance packaging extensions

Slide 26

Slide 26 text

Test infrastructure goals test discovery test result reporting test performance running tests partially tests that don't run everywhere