Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Efficient mutation analysis of relational datab...

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Gregory Kapfhammer
March 18, 2013
Research
310
2

Efficient mutation analysis of relational database structure using mutant schemata and parallelisation

Interested in learning more about this topic? Visit this web site to read the paper: https://www.gregorykapfhammer.com/research/papers/Wright2013/

Gregory Kapfhammer

March 18, 2013

Transcript

209 slides

  1. Efficient Mutation Analysis of Relational Database Structure Using Mutant Schemata

    and Parallelisation Chris J. Wright Gregory M. Kapfhammer Phil McMinn Mutation 2013

  2. Chris J. Wright - [email protected] Relational Database Management Systems (RDBMS)

  3. Chris J. Wright - [email protected] Relational Database Management Systems (RDBMS)

  4. Chris J. Wright - [email protected]

  5. Chris J. Wright - [email protected] Many different RDBMSs...

  6. Chris J. Wright - [email protected] Many different RDBMSs... ...same specification

    of structure

  7. Chris J. Wright - [email protected] 1 CREATE TABLE T (

    2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); Database Schema

  8. Chris J. Wright - [email protected] 1 CREATE TABLE T (

    2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); Database Schema Tables

  9. Chris J. Wright - [email protected] 1 CREATE TABLE T (

    2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); Database Schema Columns

  10. Chris J. Wright - [email protected] 1 CREATE TABLE T (

    2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); Database Schema Constraints

  11. Chris J. Wright - [email protected] 1 CREATE TABLE T (

    2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); Database Schema How do we know this is correct?

  12. Chris J. Wright - [email protected] Why Test Database Structure?

  13. Chris J. Wright - [email protected] Database Schema Why Test Database

    Structure?

  14. Chris J. Wright - [email protected] DBMS Database Schema Why Test

    Database Structure?

  15. Chris J. Wright - [email protected] DBMS Database Schema Why Test

    Database Structure?

  16. Chris J. Wright - [email protected] DBMS Application Database Schema Why

    Test Database Structure?

  17. Chris J. Wright - [email protected] DBMS Application Web Server Database

    Schema Why Test Database Structure?

  18. Chris J. Wright - [email protected] DBMS Application Web Server Third

    Party Database Schema Why Test Database Structure?

  19. Chris J. Wright - [email protected] DBMS Application Web Server Third

    Party Database Schema ✗ Why Test Database Structure?

  20. Chris J. Wright - [email protected] DBMS Application Web Server Third

    Party Database Schema ✗ ✗ Why Test Database Structure?

  21. Chris J. Wright - [email protected] DBMS Application Web Server Third

    Party Database Schema ✗ ✗ ✗ Why Test Database Structure?

  22. Chris J. Wright - [email protected] DBMS Application Web Server Third

    Party Database Schema ✗ ✗ ✗ ✗ Why Test Database Structure?

  23. Chris J. Wright - [email protected] Mutation Analysis

  24. Chris J. Wright - [email protected] Mutation Analysis Test Suite Application

  25. Chris J. Wright - [email protected] Mutation Analysis Database Test Suite

  26. Chris J. Wright - [email protected] Mutation Analysis Database Insert Statements

  27. Chris J. Wright - [email protected] Mutation Analysis Database Insert Statements

  28. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); Database Insert Statements

  29. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); Database Insert Statements

  30. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); Database Insert Statements

  31. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ Database Insert Statements

  32. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ Database Insert Statements

  33. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); Database Insert Statements

  34. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ Database Insert Statements

  35. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ Database Insert Statements

  36. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); Database Insert Statements

  37. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database Insert Statements

  38. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database Insert Statements

  39. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database Insert Statements INSERT INTO S(X, Y, Z) VALUES('a', 'b', 'a');

  40. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database Insert Statements INSERT INTO S(X, Y, Z) VALUES('a', 'b', 'a'); ✗

  41. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database Insert Statements INSERT INTO S(X, Y, Z) VALUES('a', 'b', 'a'); ✗

  42. Chris J. Wright - [email protected] Mutating the Structure 1 CREATE

    TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, ) 9 REFERENCES T (A, B) 10 ); Y Z

  43. Chris J. Wright - [email protected] Mutating the Structure 1 CREATE

    TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, ) 9 REFERENCES T (A, B) 10 ); Y Z

  44. Chris J. Wright - [email protected] Mutating the Structure 1 CREATE

    TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, ) 9 REFERENCES T (A, B) 10 ); Y Z

  45. Chris J. Wright - [email protected] Mutating the Structure 1 CREATE

    TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, ) 9 REFERENCES T (A, B) 10 ); Y Z

  46. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 );

  47. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 );

  48. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); Database (mutated)

  49. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); Database (mutated) Insert Statements

  50. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); Database (mutated) Insert Statements

  51. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ Database (mutated) Insert Statements

  52. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ Database (mutated) Insert Statements

  53. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); Database (mutated) Insert Statements

  54. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ Database (mutated) Insert Statements

  55. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ Database (mutated) Insert Statements

  56. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); Database (mutated) Insert Statements

  57. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database (mutated) Insert Statements

  58. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database (mutated) Insert Statements

  59. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database (mutated) Insert Statements INSERT INTO S(X, Y, Z) VALUES('a', 'b', 'a');

  60. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database (mutated) Insert Statements INSERT INTO S(X, Y, Z) VALUES('a', 'b', 'a'); ✓

  61. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database (mutated) Insert Statements INSERT INTO S(X, Y, Z) VALUES('a', 'b', 'a'); ✓

  62. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database (mutated) Insert Statements INSERT INTO S(X, Y, Z) VALUES('a', 'b', 'a'); ✓ Results are different

  63. Chris J. Wright - [email protected] Mutation Analysis 1 CREATE TABLE

    T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✓ INSERT INTO T(A, B, C) VALUES('a', 'a', 'a'); ✗ INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'b'); ✓ Database (mutated) Insert Statements INSERT INTO S(X, Y, Z) VALUES('a', 'b', 'a'); ✓ Mutant is killed

  64. Chris J. Wright - [email protected] Mutation Operators

  65. Chris J. Wright - [email protected] Mutation Operators Primary Key

  66. Chris J. Wright - [email protected] Mutation Operators Primary Key Not

    Null

  67. Chris J. Wright - [email protected] Mutation Operators Check Primary Key

    Not Null

  68. Chris J. Wright - [email protected] Mutation Operators Check Primary Key

    Not Null Unique

  69. Chris J. Wright - [email protected] 1 CREATE TABLE T (

    2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); The Problem? 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 );

  70. Chris J. Wright - [email protected] 1 CREATE TABLE T (

    2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); The Problem? 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 );

  71. Chris J. Wright - [email protected] 1 CREATE TABLE T (

    2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); The Problem? Many mutants to analyse

  72. Chris J. Wright - [email protected] 1 CREATE TABLE T (

    2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); The Problem? A time consuming process

  73. Chris J. Wright - [email protected] The Solution?

  74. Chris J. Wright - [email protected] The Solution? Mutant Schemata

  75. Chris J. Wright - [email protected] The Solution? Mutant Schemata Combine

    mutants into a ‘meta-mutant’

  76. Chris J. Wright - [email protected] The Solution? Parallelisation Mutant Schemata

    Combine mutants into a ‘meta-mutant’

  77. Chris J. Wright - [email protected] The Solution? Parallelisation Mutant Schemata

    Combine mutants into a ‘meta-mutant’ Analyse multiple mutants simultaneously

  78. Chris J. Wright - [email protected] Mutation Analysis Approaches Mutant Representation

    Parallelisation Strategy Normal “Original”

  79. Chris J. Wright - [email protected] Mutation Analysis Approaches Mutant Representation

    Parallelisation Strategy Schemata Normal “Original” Full “Full Schemata”

  80. Chris J. Wright - [email protected] Original Approach

  81. Chris J. Wright - [email protected] Original Approach Create structure in

    database

  82. Chris J. Wright - [email protected] Original Approach Create structure in

    database Execute insert statements

  83. Chris J. Wright - [email protected] Original Approach Create structure in

    database Execute insert statements Drop structure from database

  84. Chris J. Wright - [email protected] Original Approach Create structure in

    database Execute insert statements Drop structure from database

  85. Chris J. Wright - [email protected] Mutant Schemata Approach Create structure

    in database Execute insert statements Drop structure from database

  86. Chris J. Wright - [email protected] Mutant Schemata Approach Create structure

    in database Execute insert statements Drop structure from database

  87. Chris J. Wright - [email protected] Mutant Schemata Approach Create structure

    in database Execute insert statements Drop structure from database Reduce time creating/ dropping database

  88. Chris J. Wright - [email protected] Full Schemata Approach

  89. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 );

  90. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES T (A, B) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 );

  91. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 1 CREATE TABLE mutant_2_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 4 ); 5 6 CREATE TABLE mutant_2_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES mutant_2_T (A, B) 10 );

  92. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 11 12 CREATE TABLE mutant_2_T ( 13 A CHAR, B CHAR, C CHAR, 14 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 15 ); 16 17 CREATE TABLE mutant_2_S ( 18 X CHAR, Y CHAR, Z CHAR, 19 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 20 REFERENCES mutant_2_T (A, B) 21 );

  93. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 11 12 CREATE TABLE mutant_2_T ( 13 A CHAR, B CHAR, C CHAR, 14 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 15 ); 16 17 CREATE TABLE mutant_2_S ( 18 X CHAR, Y CHAR, Z CHAR, 19 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 20 REFERENCES mutant_2_T (A, B) 21 );

  94. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 11 12 CREATE TABLE mutant_2_T ( 13 A CHAR, B CHAR, C CHAR, 14 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 15 ); 16 17 CREATE TABLE mutant_2_S ( 18 X CHAR, Y CHAR, Z CHAR, 19 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 20 REFERENCES mutant_2_T (A, B) 21 ); Database (mutated) Insert Statements

  95. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 11 12 CREATE TABLE mutant_2_T ( 13 A CHAR, B CHAR, C CHAR, 14 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 15 ); 16 17 CREATE TABLE mutant_2_S ( 18 X CHAR, Y CHAR, Z CHAR, 19 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 20 REFERENCES mutant_2_T (A, B) 21 ); Database (mutated) INSERT INTO S(X, Y, Z) VALUES('a', 'a', 'a'); Insert Statements

  96. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 11 12 CREATE TABLE mutant_2_T ( 13 A CHAR, B CHAR, C CHAR, 14 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 15 ); 16 17 CREATE TABLE mutant_2_S ( 18 X CHAR, Y CHAR, Z CHAR, 19 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 20 REFERENCES mutant_2_T (A, B) 21 ); Database (mutated) INSERT INTO mutant_1_S(X, Y, Z) VALUES('a', 'a', 'a'); Insert Statements

  97. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 11 12 CREATE TABLE mutant_2_T ( 13 A CHAR, B CHAR, C CHAR, 14 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 15 ); 16 17 CREATE TABLE mutant_2_S ( 18 X CHAR, Y CHAR, Z CHAR, 19 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 20 REFERENCES mutant_2_T (A, B) 21 ); Database (mutated) INSERT INTO mutant_1_S(X, Y, Z) VALUES('a', 'a', 'a'); ✓ Insert Statements

  98. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 11 12 CREATE TABLE mutant_2_T ( 13 A CHAR, B CHAR, C CHAR, 14 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 15 ); 16 17 CREATE TABLE mutant_2_S ( 18 X CHAR, Y CHAR, Z CHAR, 19 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 20 REFERENCES mutant_2_T (A, B) 21 ); Database (mutated) INSERT INTO mutant_1_S(X, Y, Z) VALUES('a', 'a', 'a'); ✓ Insert Statements

  99. Chris J. Wright - [email protected] Full Schemata Approach

  100. Chris J. Wright - [email protected] Create tables once... Full Schemata

    Approach

  101. Chris J. Wright - [email protected] Create tables once... ...rewrite queries

    to match Full Schemata Approach

  102. Chris J. Wright - [email protected] Mutation Analysis Approaches Mutant Representation

    Parallelisation Strategy Schemata Normal “Original” Full “Full Schemata”

  103. Chris J. Wright - [email protected] Mutation Analysis Approaches Mutant Representation

    Parallelisation Strategy Schemata Normal “Original” Full “Full Schemata” Minimal “Minimal Schemata”

  104. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 11 12 CREATE TABLE mutant_2_T ( 13 A CHAR, B CHAR, C CHAR, 14 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 15 ); 16 17 CREATE TABLE mutant_2_S ( 18 X CHAR, Y CHAR, Z CHAR, 19 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 20 REFERENCES mutant_2_T (A, B) 21 );

  105. Chris J. Wright - [email protected] Full Schemata Approach 1 CREATE

    TABLE mutant_1_T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE mutant_1_S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 9 REFERENCES mutant_1_T (A, B) 10 ); 11 12 CREATE TABLE mutant_2_T ( 13 A CHAR, B CHAR, C CHAR, 14 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 15 ); 16 17 CREATE TABLE mutant_2_S ( 18 X CHAR, Y CHAR, Z CHAR, 19 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 20 REFERENCES mutant_2_T (A, B) 21 ); Mutated Tables

  106. Chris J. Wright - [email protected] Minimal Schemata Approach 1 CREATE

    TABLE mutant_1_S ( 2 X CHAR, Y CHAR, Z CHAR, 3 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 4 REFERENCES mutant_1_T (A, B) 5 ); 6 7 CREATE TABLE mutant_2_T ( 8 A CHAR, B CHAR, C CHAR, 9 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 10 ); Mutated Tables

  107. Chris J. Wright - [email protected] Minimal Schemata Approach 1 CREATE

    TABLE mutant_1_S ( 2 X CHAR, Y CHAR, Z CHAR, 3 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 4 REFERENCES mutant_1_T (A, B) 5 ); 6 7 CREATE TABLE mutant_2_T ( 8 A CHAR, B CHAR, C CHAR, 9 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 10 );

  108. Chris J. Wright - [email protected] Minimal Schemata Approach 1 CREATE

    TABLE mutant_1_S ( 2 X CHAR, Y CHAR, Z CHAR, 3 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 4 REFERENCES mutant_1_T (A, B) 5 ); 6 7 CREATE TABLE mutant_2_T ( 8 A CHAR, B CHAR, C CHAR, 9 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 10 );

  109. Chris J. Wright - [email protected] Minimal Schemata Approach 1 CREATE

    TABLE mutant_1_S ( 2 X CHAR, Y CHAR, Z CHAR, 3 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 4 REFERENCES mutant_1_T (A, B) 5 ); 6 7 CREATE TABLE mutant_2_T ( 8 A CHAR, B CHAR, C CHAR, 9 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 10 );

  110. Chris J. Wright - [email protected] Minimal Schemata Approach 1 CREATE

    TABLE mutant_1_S ( 2 X CHAR, Y CHAR, Z CHAR, 3 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 4 REFERENCES mutant_1_T (A, B) 5 ); 6 7 CREATE TABLE mutant_2_T ( 8 A CHAR, B CHAR, C CHAR, 9 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 10 ); 1 CREATE TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 );

  111. Chris J. Wright - [email protected] Minimal Schemata Approach 1 CREATE

    TABLE T ( 2 A CHAR, B CHAR, C CHAR, 3 CONSTRAINT UniqueOnColsAandB UNIQUE (A, B) 4 ); 5 6 CREATE TABLE S ( 7 X CHAR, Y CHAR, Z CHAR, 8 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Y) 9 REFERENCES T (A, B) 10 ); 11 12 CREATE TABLE mutant_1_S ( 13 X CHAR, Y CHAR, Z CHAR, 14 CONSTRAINT RefToColsAandB FOREIGN KEY (X, Z) 15 REFERENCES T (A, B) 16 ); 17 18 CREATE TABLE mutant_2_T ( 19 A CHAR, B CHAR, C CHAR, 20 CONSTRAINT UniqueOnColsAandB UNIQUE (A, C) 21 );

  112. Chris J. Wright - [email protected] Minimal Schemata Approach

  113. Chris J. Wright - [email protected] Create tables once... Minimal Schemata

    Approach

  114. Chris J. Wright - [email protected] Create tables once... ...only mutated

    tables... Minimal Schemata Approach

  115. Chris J. Wright - [email protected] Create tables once... ...only mutated

    tables... Minimal Schemata Approach ...reduce queries executed

  116. Chris J. Wright - [email protected] Create tables once... ...only mutated

    tables... Minimal Schemata Approach ...reduce queries executed Plus one copy for foreign keys

  117. Chris J. Wright - [email protected] Mutation Analysis Approaches Mutant Representation

    Parallelisation Strategy Schemata Normal “Original” Full “Full Schemata” Minimal “Minimal Schemata”

  118. Chris J. Wright - [email protected] Mutation Analysis Approaches Mutant Representation

    Parallelisation Strategy Schemata Normal “Original” Full “Full Schemata” Minimal “Minimal Schemata” Up front “Up-Front Schemata”

  119. Chris J. Wright - [email protected] Mutation Analysis Approaches Mutant Representation

    Parallelisation Strategy Schemata Normal “Original” Full “Full Schemata” Minimal “Minimal Schemata” Up front “Up-Front Schemata” Just in time “Just-in-Time Schemata”

  120. Chris J. Wright - [email protected] Parallelisation ‘Just-in-Time Schemata’

  121. Chris J. Wright - [email protected] Parallelisation ‘Just-in-Time Schemata’ Make the

    ‘Original’ approach parallel

  122. Chris J. Wright - [email protected] Parallelisation ‘Up-Front Schemata’ ‘Just-in-Time Schemata’

    Make the ‘Original’ approach parallel

  123. Chris J. Wright - [email protected] Parallelisation ‘Up-Front Schemata’ ‘Just-in-Time Schemata’

    Make the ‘Original’ approach parallel Make the ‘Full Schemata’ approach parallel

  124. Chris J. Wright - [email protected] Original Approach Create structure in

    database Execute insert statements Drop structure from database

  125. Chris J. Wright - [email protected] ‘Just-in-Time’ Approach Create structure in

    database Execute insert statements Drop structure from database Parallel

  126. Chris J. Wright - [email protected] ‘Just-in-Time’ Approach Create structure in

    database Execute insert statements Drop structure from database Parallel

  127. Chris J. Wright - [email protected] Mutant Schemata Approach Create structure

    in database Execute insert statements Drop structure from database

  128. Chris J. Wright - [email protected] ‘Up-Front’ Approach Create structure in

    database Execute insert statements Drop structure from database Parallel

  129. Chris J. Wright - [email protected] Empirical Study

  130. Chris J. Wright - [email protected] Empirical Study Evaluation Metric Mutation

    Time Approaches 5 Case Studies 6 DBMSs 2 Repetitions 30

  131. Chris J. Wright - [email protected] Empirical Study Evaluation Metric Mutation

    Time Approaches 5 Case Studies 6 DBMSs 2 Repetitions 30

  132. Chris J. Wright - [email protected] Empirical Study Evaluation Metric Mutation

    Time Approaches 5 Case Studies 6 DBMSs 2 Repetitions 30

  133. Chris J. Wright - [email protected] Empirical Study Evaluation Metric Mutation

    Time Approaches 5 Case Studies 6 DBMSs 2 Repetitions 30

  134. Chris J. Wright - [email protected] Empirical Study Evaluation Metric Mutation

    Time Approaches 5 Case Studies 6 DBMSs 2 Repetitions 30

  135. Chris J. Wright - [email protected] Empirical Study: Approaches Mutant Representation

    Parallelisation Strategy Schemata Normal “Original” Full “Full Schemata” Minimal “Minimal Schemata” Up front “Up-Front Schemata” Just in time “Just-in-Time Schemata”

  136. Chris J. Wright - [email protected] Empirical Study: Case Studies Case

    Study Tables Columns Primary Keys Foreign Keys Unique Constraints Cloc 2 10 0 0 0 JWhoisServer 6 49 6 0 0 NistDML182 2 32 1 1 0 NistDML183 2 6 0 1 1 RiskIt 13 56 11 10 0 UnixUsage 8 32 7 7 0

  137. Chris J. Wright - [email protected] Empirical Study: Case Studies Case

    Study Tables Columns Primary Keys Foreign Keys Unique Constraints Cloc 2 10 0 0 0 JWhoisServer 6 49 6 0 0 NistDML182 2 32 1 1 0 NistDML183 2 6 0 1 1 RiskIt 13 56 11 10 0 UnixUsage 8 32 7 7 0

  138. Chris J. Wright - [email protected] Empirical Study: Case Studies Case

    Study Tables Columns Primary Keys Foreign Keys Unique Constraints Cloc 2 10 0 0 0 JWhoisServer 6 49 6 0 0 NistDML182 2 32 1 1 0 NistDML183 2 6 0 1 1 RiskIt 13 56 11 10 0 UnixUsage 8 32 7 7 0

  139. Chris J. Wright - [email protected] Empirical Study: Case Studies Case

    Study Tables Columns Primary Keys Foreign Keys Unique Constraints Cloc 2 10 0 0 0 JWhoisServer 6 49 6 0 0 NistDML182 2 32 1 1 0 NistDML183 2 6 0 1 1 RiskIt 13 56 11 10 0 UnixUsage 8 32 7 7 0

  140. Chris J. Wright - [email protected] Empirical Study: Case Studies Case

    Study Tables Columns Primary Keys Foreign Keys Unique Constraints Cloc 2 10 0 0 0 JWhoisServer 6 49 6 0 0 NistDML182 2 32 1 1 0 NistDML183 2 6 0 1 1 RiskIt 13 56 11 10 0 UnixUsage 8 32 7 7 0

  141. Chris J. Wright - [email protected] Empirical Study: Case Studies Case

    Study Tables Columns Primary Keys Foreign Keys Unique Constraints Cloc 2 10 0 0 0 JWhoisServer 6 49 6 0 0 NistDML182 2 32 1 1 0 NistDML183 2 6 0 1 1 RiskIt 13 56 11 10 0 UnixUsage 8 32 7 7 0

  142. Chris J. Wright - [email protected] Empirical Study: Case Studies Case

    Study Tables Columns Primary Keys Foreign Keys Unique Constraints Cloc 2 10 0 0 0 JWhoisServer 6 49 6 0 0 NistDML182 2 32 1 1 0 NistDML183 2 6 0 1 1 RiskIt 13 56 11 10 0 UnixUsage 8 32 7 7 0

  143. Chris J. Wright - [email protected] Empirical Study: Case Studies Case

    Study Total Constraints Total Mutants Cloc 0 30 JWhoisServer 50 184 NistDML182 2 66 NistDML183 2 18 RiskIt 36 160 UnixUsage 23 69

  144. Chris J. Wright - [email protected] Empirical Study: DBMSs

  145. Chris J. Wright - [email protected] Empirical Study: DBMSs PostgreSQL

  146. Chris J. Wright - [email protected] Empirical Study: DBMSs SQLite PostgreSQL

  147. Chris J. Wright - [email protected] Empirical Study: DBMSs SQLite PostgreSQL

    Client-Server Model

  148. Chris J. Wright - [email protected] Empirical Study: DBMSs SQLite PostgreSQL

    Client-Server Model Local Client Model

  149. Chris J. Wright - [email protected] Empirical Study: DBMSs SQLite PostgreSQL

    Client-Server Model Local Client Model Simultaneous Read/Write

  150. Chris J. Wright - [email protected] Empirical Study: DBMSs SQLite PostgreSQL

    Client-Server Model Local Client Model Simultaneous Read/Write Locking on Write

  151. Chris J. Wright - [email protected] Empirical Study: DBMSs SQLite PostgreSQL

    Client-Server Model Local Client Model Simultaneous Read/Write Locking on Write Prevents Parallel Approaches

  152. Chris J. Wright - [email protected] Results

  153. Chris J. Wright - [email protected] Results • Median of repetitions

  154. Chris J. Wright - [email protected] Results • Median of repetitions

    • Lower-is-better metric

  155. Chris J. Wright - [email protected] Results • Median of repetitions

    • Lower-is-better metric • Split by...

  156. Chris J. Wright - [email protected] Results • Median of repetitions

    • Lower-is-better metric • Split by... ...case study

  157. Chris J. Wright - [email protected] Results • Median of repetitions

    • Lower-is-better metric • Split by... ...case study ...DBMS

  158. Chris J. Wright - [email protected] Results • Median of repetitions

    • Lower-is-better metric • Split by... ...case study ...DBMS • Full details in paper (including statistics)

  159. Chris J. Wright - [email protected] 0 1000 2000 3000 Original

    Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Postgres – Cloc Original Full Minimal Up-Front Just-in-Time

  160. Chris J. Wright - [email protected] 0 1000 2000 3000 Original

    Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Postgres – Cloc ~3.27s Original Full Minimal Up-Front Just-in-Time

  161. Chris J. Wright - [email protected] 0 1000 2000 3000 Original

    Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Postgres – Cloc ~3.27s ~2.08s Original Full Minimal Up-Front Just-in-Time

  162. Chris J. Wright - [email protected] 0 50000 100000 150000 200000

    250000 Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Postgres – RiskIt Original Full Minimal Up-Front Just-in-Time

  163. Chris J. Wright - [email protected] 0 50000 100000 150000 200000

    250000 Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Postgres – RiskIt ~238s Original Full Minimal Up-Front Just-in-Time

  164. Chris J. Wright - [email protected] 0 50000 100000 150000 200000

    250000 Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Postgres – RiskIt ~238s ~225s Original Full Minimal Up-Front Just-in-Time

  165. Chris J. Wright - [email protected] 0 50000 100000 150000 200000

    250000 Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata Up−Front Schemata Just−in−Time Schemata Postgres – RiskIt ~238s ~225s ~23s Original Full Minimal Up-Front Just-in-Time

  166. Chris J. Wright - [email protected] Results – Postgres

  167. Chris J. Wright - [email protected] Results – Postgres ‘Full Schemata’

  168. Chris J. Wright - [email protected] Results – Postgres ‘Full Schemata’

    Improvement decreases with larger schemas

  169. Chris J. Wright - [email protected] Results – Postgres ‘Minimal Schemata’

    ‘Full Schemata’ Improvement decreases with larger schemas

  170. Chris J. Wright - [email protected] Results – Postgres ‘Minimal Schemata’

    ‘Full Schemata’ Improvement decreases with larger schemas Consistently faster, scales very well

  171. Chris J. Wright - [email protected] Results – Postgres ‘Just-in-Time Schemata’

    ‘Minimal Schemata’ ‘Full Schemata’ Improvement decreases with larger schemas Consistently faster, scales very well

  172. Chris J. Wright - [email protected] Results – Postgres ‘Just-in-Time Schemata’

    Consistently faster, scales very well ‘Minimal Schemata’ ‘Full Schemata’ Improvement decreases with larger schemas Consistently faster, scales very well

  173. Chris J. Wright - [email protected] Results – Postgres ‘Up-Front Schemata’

    ‘Just-in-Time Schemata’ Consistently faster, scales very well ‘Minimal Schemata’ ‘Full Schemata’ Improvement decreases with larger schemas Consistently faster, scales very well

  174. Chris J. Wright - [email protected] Results – Postgres ‘Up-Front Schemata’

    ‘Just-in-Time Schemata’ Consistently faster, scales very well Improvement decreases with larger schemas ‘Minimal Schemata’ ‘Full Schemata’ Improvement decreases with larger schemas Consistently faster, scales very well

  175. Chris J. Wright - [email protected] Results – Postgres ‘Up-Front Schemata’

    ‘Just-in-Time Schemata’ Consistently faster, scales very well Improvement decreases with larger schemas ‘Minimal Schemata’ ‘Full Schemata’ Improvement decreases with larger schemas Consistently faster, scales very well

  176. Chris J. Wright - [email protected] 0 5000 10000 15000 Original

    Full Schemata Minimal Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata SQLite – Cloc Original Full Minimal

  177. Chris J. Wright - [email protected] 0 5000 10000 15000 Original

    Full Schemata Minimal Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata SQLite – Cloc ~18.0s Original Full Minimal

  178. Chris J. Wright - [email protected] 0 5000 10000 15000 Original

    Full Schemata Minimal Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata SQLite – Cloc ~18.0s ~18.7s Original Full Minimal

  179. Chris J. Wright - [email protected] 0 5000 10000 15000 Original

    Full Schemata Minimal Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata SQLite – Cloc ~18.0s ~18.7s ~8.28s Original Full Minimal

  180. Chris J. Wright - [email protected] 0 500000 1000000 1500000 2000000

    Original Full Schemata Minimal Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata SQLite – RiskIt Original Full Minimal

  181. Chris J. Wright - [email protected] 0 500000 1000000 1500000 2000000

    Original Full Schemata Minimal Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata SQLite – RiskIt ~20.2 min Original Full Minimal

  182. Chris J. Wright - [email protected] 0 500000 1000000 1500000 2000000

    Original Full Schemata Minimal Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata SQLite – RiskIt ~20.2 min ~31.8 min Original Full Minimal

  183. Chris J. Wright - [email protected] 0 500000 1000000 1500000 2000000

    Original Full Schemata Minimal Schemata Mutation Analysis Technique Mutation Analysis Time (ms) Original Full Schemata Minimal Schemata SQLite – RiskIt ~20.2 min ~31.8 min ~2.5 min Original Full Minimal

  184. Chris J. Wright - [email protected] Results – SQLite

  185. Chris J. Wright - [email protected] Results – SQLite ‘Full Schemata’

  186. Chris J. Wright - [email protected] Results – SQLite ‘Full Schemata’

    Increasingly worsened with larger schemas

  187. Chris J. Wright - [email protected] Results – SQLite ‘Minimal Schemata’

    ‘Full Schemata’ Increasingly worsened with larger schemas

  188. Chris J. Wright - [email protected] Results – SQLite ‘Minimal Schemata’

    ‘Full Schemata’ Increasingly worsened with larger schemas Consistently faster, scales very well

  189. Chris J. Wright - [email protected] Results – SQLite ‘Minimal Schemata’

    ‘Full Schemata’ Increasingly worsened with larger schemas Consistently faster, scales very well

  190. Chris J. Wright - [email protected] Future Work & Limitations

  191. Chris J. Wright - [email protected] Future Work & Limitations Case

    Studies

  192. Chris J. Wright - [email protected] Future Work & Limitations Case

    Studies DBMSs

  193. Chris J. Wright - [email protected] Future Work & Limitations Detailed

    Timing Case Studies DBMSs

  194. Chris J. Wright - [email protected] Future Work & Limitations Approaches

    Detailed Timing Case Studies DBMSs

  195. Chris J. Wright - [email protected] Future Work & Limitations Approaches

    Detailed Timing Case Studies DBMSs Parallel Configurations

  196. Chris J. Wright - [email protected] Future Work & Limitations Test

    Suite Approaches Detailed Timing Case Studies DBMSs Parallel Configurations

  197. Chris J. Wright - [email protected] Test Suite Generation

  198. Chris J. Wright - [email protected] Test Suite Generation • SchemaAnalyst

    tool

  199. Chris J. Wright - [email protected] Test Suite Generation • SchemaAnalyst

    tool • Gregory Kapfhammer

  200. Chris J. Wright - [email protected] Test Suite Generation • SchemaAnalyst

    tool • Gregory Kapfhammer • Tuesday 11:00am, Research & Industrial Track

  201. Chris J. Wright - [email protected] Conclusions

  202. Chris J. Wright - [email protected] Conclusions • Mutant Schemata and

    Parallelisation can both reduce the cost of mutation analysis

  203. Chris J. Wright - [email protected] Conclusions • Mutant Schemata and

    Parallelisation can both reduce the cost of mutation analysis • The ‘Minimal Schemata’ approach...

  204. Chris J. Wright - [email protected] Conclusions • Mutant Schemata and

    Parallelisation can both reduce the cost of mutation analysis • The ‘Minimal Schemata’ approach... ...consistently faster than original

  205. Chris J. Wright - [email protected] Conclusions • Mutant Schemata and

    Parallelisation can both reduce the cost of mutation analysis • The ‘Minimal Schemata’ approach... ...consistently faster than original ...gives a reduction of up to 10x

  206. Chris J. Wright - [email protected] Conclusions • Mutant Schemata and

    Parallelisation can both reduce the cost of mutation analysis • The ‘Minimal Schemata’ approach... ...consistently faster than original ...gives a reduction of up to 10x ...scales very well (for our case studies)

  207. Chris J. Wright - [email protected] Conclusions • Mutant Schemata and

    Parallelisation can both reduce the cost of mutation analysis • The ‘Minimal Schemata’ approach... ...consistently faster than original ...gives a reduction of up to 10x ...scales very well (for our case studies) ...doesn’t require parallel DBMS access

  208. Chris J. Wright - [email protected] Conclusions • Mutant Schemata and

    Parallelisation can both reduce the cost of mutation analysis • The ‘Minimal Schemata’ approach... ...consistently faster than original ...gives a reduction of up to 10x ...scales very well (for our case studies) ...doesn’t require parallel DBMS access • Website: http://schemaanalyst.org/

  209. Chris J. Wright - [email protected] Conclusions • Mutant Schemata and

    Parallelisation can both reduce the cost of mutation analysis • The ‘Minimal Schemata’ approach... ...consistently faster than original ...gives a reduction of up to 10x ...scales very well (for our case studies) ...doesn’t require parallel DBMS access • Website: http://schemaanalyst.org/

More Decks by Gregory Kapfhammer

See All by Gregory Kapfhammer
Don't blame the type checker: Finding who Is right when Python type checkers disagree
gkapfham
0
26
Testing the parts you don’t see: Pseudo-tested native code in Python libraries
gkapfham
0
23
Chasten Your Python Program: Configurable Program Analysis and Linting with XPath
gkapfham
0
87
ExecExam: Streaming Python assessments with automation and personalized feedback
gkapfham
0
97
Automated and configurable programming project checking with Chasten
gkapfham
0
94
Up and running with GitHub, GitHub Classroom, and GitHub Actions
gkapfham
0
88
ExecExam: A tool to facilitate effective executable examinations in Python
gkapfham
0
94
Building and deploying course websites with Python, Quarto, and Mkdocs
gkapfham
0
150
Exploring pseudo-testedness: Empirically evaluating extreme mutation testing at the statement level
gkapfham
0
110

Other Decks in Research

See All in Research
OWASP AISVS - C7
shiell
2
860
XDPerf: A High-Performance Traffic Generator Built with WASM and eBPF
takehaya
1
390
MIRU2026 チュートリアル講演2：三次元データ処理の動向
nnchiba
8
5.6k
2025年度秋葉原ウォーカブルプロジェクト調査報告 「アキバらしいウォーカブル」とは何か
izumiyama_lab
1
300
実例から見るLLMのマンガ理解：実務VQAタスクによる長期的文脈と視覚情報の定性評価
kzmssk
0
220
幾何学的深層学習入門 ～３次元構造と対称性を学習するニューラルネットワーク～
taniai
3
1.1k
Détecter les vulnérabilités dans un environnement souverain
hellosct1
0
130
完全自律LLMエージェントでDIVER OSINT CTF 2026に挑戦してみた
analokmaus
0
820
就労支援の現場での生成AI活用
enochan
0
210
超効率化への挑戦：1bit LLMの現状と展望
yumaichikawa
0
1k
ICAITD 2026 Oral Presentation
rikkabotan7
0
150
最先端NLP 2026 論文紹介: Wait, Wait, Wait... Why Do Reasoning Models Loop? / SNLP Paper Review: Wait, Wait, Wait... Why Do Reasoning Models Loop?
tkng
0
280

Featured

See All Featured
Winning Ecommerce Organic Search in an AI Era - #searchnstuff2025
aleyda
2
2.2k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Building Flexible Design Systems
yeseniaperezcruz
330
41k
Building Better People: How to give real-time feedback that sticks.
wjessup
371
20k
Building a Scalable Design System with Sketch
lauravandoore
464
34k
Utilizing Notion as your number one productivity tool
mfonobong
4
640
Context Engineering - Making Every Token Count
addyosmani
9
1.3k
Building Applications with DynamoDB
mza
96
7.3k
Facilitating Awesome Meetings
lara
57
7.2k
Leveraging LLMs for student feedback in introductory data science courses - posit::conf(2025)
minecr
1
470
The Limits of Empathy - UXLibs8
cassininazir
2
780
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
davidcarrasco
1
300