Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

Gregory Kapfhammer
January 17, 2012
240

Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

Interested in learning more about this topic? Visit this web site to read a related paper: https://www.gregorykapfhammer.com/research/papers/Cobb2011/

Gregory Kapfhammer

January 17, 2012
Tweet

More Decks by Gregory Kapfhammer

Transcript

  1. Using Dynamic Invariant Detection
    to Support the Testing and Analysis
    of Database Applications
    Gregory M. Kapfhammer†
    Department of Computer Science
    Allegheny College
    http://www.cs.allegheny.edu/∼gkapfham/
    University of Ulm – January 17, 2012
    †Joint with Mary Jean Harrold and Jake Cobb (GA Tech), James A. Jones (UC Irvine), Jonathan Miller Kauffman (Allegheny)

    View Slide

  2. Introduction Database Applications Dynamic Invariants Conclusion
    Important Points
    Accessing the Presentation
    Scan this QR Code with your smartphone!
    ... or, visit this Web site:
    http://www.cs.allegheny.edu/˜gkapfham/ulm2012.pdf
    ... or, ask me for a USB drive!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  3. Introduction Database Applications Dynamic Invariants Conclusion
    Important Points
    Presenter Introduction: Gregory M. Kapfhammer
    test
    testing
    software
    suites
    prioritization
    components
    coverage
    empirically
    regression
    suite
    applications
    evaluating
    algorithm
    analysis
    data
    effectiveness
    empirical
    genetic
    performance
    understanding
    using
    approach
    commercialofftheshelf
    comparison
    comprehensive
    constrained
    creation
    databasecentric
    environments
    execution
    finding
    framework
    identifying
    interactive
    java
    methods
    multiplots
    mutation
    party
    prioritized
    reduction
    relational
    study
    third
    timeaware
    towards
    105
    adequacy
    approaches
    array
    automatically
    building
    call
    challenges
    chapter
    communication
    compare
    compressing
    computer
    conditional
    cost
    cots
    covering
    creating
    criteria
    database
    databaseaware
    databasedriven
    databases
    declarative
    dependable
    detection
    devices
    distributed
    distributing
    duringsearchbased
    dynamic
    efficiency
    efficient
    engineering
    environment
    evaluate
    examination
    executing
    experimental
    family
    flow
    forward
    frameworks
    free
    generation
    greedy
    gui
    hamiltonian
    handbook
    heaps
    implementation
    improve
    incorporating
    increase
    information
    initial
    intranode
    invariant
    javaspace
    javaspacebased
    kernel
    knapsack
    linux
    measurement
    memory
    method
    monitoring
    operators
    paths
    poster
    potential
    preliminary
    primitives
    prioritizations
    prioritizers
    prioritizing
    problematic
    receive
    remote
    reports
    resource
    resourceconstrained
    results
    role
    runtime
    science
    searchbased
    selection
    solutions
    solvers
    space
    studies
    studying
    supported
    synthetic
    techniques
    transmission
    transparently
    trees
    tuple
    unstructured
    wrappers
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  4. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  5. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  6. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  7. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  8. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Household
    Appliance
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  9. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Household
    Appliance
    Program
    Network
    Router
    Program
    Scientific
    Device
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  10. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Household
    Appliance
    Program
    Network
    Router
    Program
    Scientific
    Device
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Network
    Router
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  11. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Household
    Appliance
    Program
    Network
    Router
    Program
    Scientific
    Device
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Network
    Router
    Program
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  12. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Household
    Appliance
    Program
    Network
    Router
    Program
    Scientific
    Device
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Network
    Router
    Program
    Program
    Program
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  13. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Household
    Appliance
    Program
    Network
    Router
    Program
    Scientific
    Device
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Network
    Router
    Program
    Program
    Program
    Program
    Program
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  14. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Household
    Appliance
    Program
    Network
    Router
    Program
    Scientific
    Device
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Network
    Router
    Program
    Program
    Program
    Program
    Program
    Program
    Program
    Program Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  15. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Everywhere
    Computer
    Server
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Household
    Appliance
    Program
    Network
    Router
    Program
    Scientific
    Device
    Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Network
    Router
    Program
    Program
    Program
    Program
    Program
    Program
    Program
    Program Program
    Computer
    Server
    Program
    Desktop
    Computer
    Program
    Mobile
    Computer
    Program
    Household
    Appliance
    Program
    Scientific
    Device
    Program
    Network
    Router
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  16. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software Complexity and Data Enormity
    Computer
    Software
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  17. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software Complexity and Data Enormity
    Computer
    Software
    Lines of
    Code
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  18. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software Complexity and Data Enormity
    Computer
    Software
    Lines of
    Code
    Numerous
    Features
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  19. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software Complexity and Data Enormity
    Computer
    Software
    Lines of
    Code
    Numerous
    Features
    Feature
    Interactions
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  20. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software Complexity and Data Enormity
    Computer
    Software
    Lines of
    Code
    Numerous
    Features
    Feature
    Interactions
    Execution
    Environments
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  21. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software Complexity and Data Enormity
    Computer
    Software
    Lines of
    Code
    Numerous
    Features
    Feature
    Interactions
    Execution
    Environments
    Software entities are more complex for their size than per-
    haps any other human construct - Frederick P. Brooks, Jr.
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  22. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software Complexity and Data Enormity
    Computer
    Software
    Lines of
    Code
    Numerous
    Features
    Feature
    Interactions
    Execution
    Environments
    Software entities are more complex for their size than per-
    haps any other human construct - Frederick P. Brooks, Jr.
    Prediction: in 2011, 1.8 zettabytes (i.e., 1.8 trillion giga-
    bytes) of data will be created - IDC Digital Universe Study
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  23. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Evolving
    Execution
    Environment
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  24. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Evolving
    Execution
    Environment
    Program
    Execution
    Environment
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  25. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Evolving
    Execution
    Environment
    Program
    Execution
    Environment
    Program
    Program Changed because of the addition
    of a new feature or the correction of a defect
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  26. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Evolving
    Execution
    Environment
    Program
    Execution
    Environment
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  27. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Evolving
    Execution
    Environment
    Program
    Execution
    Environment
    Program
    Execution
    Environment
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  28. Introduction Database Applications Dynamic Invariants Conclusion
    Software and Data Challenges
    Software and Data are Evolving
    Execution
    Environment
    Program
    Execution
    Environment
    Program
    Execution
    Environment
    Program
    Execution Environment Changed due to modifica-
    tion of a kernel, device driver, or relational database
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  29. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Database Challenges
    An Interesting Defect Report
    Database
    Server Crashes
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  30. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Database Challenges
    An Interesting Defect Report
    Database
    Server Crashes
    When you run a complex query against Microsoft SQL
    Server 2000, the SQL Server scheduler may stop respond-
    ing. Additionally, you receive an error message that resem-
    bles the following: Date Time server Error: 17883 Sever-
    ity: 1, State: 0 Date Time server Process 52:0 (94c) ...
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  31. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Database Challenges
    An Interesting Defect Report
    Input-Dependent
    Defect
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  32. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Database Challenges
    An Interesting Defect Report
    Input-Dependent
    Defect
    This problem occurs when one or more of the following con-
    ditions are true: The query contains a UNION clause or a
    UNIONALL clause that affects many columns. The query
    contains several JOIN statements. The query has a large
    estimated cost. BUG 473858 (SQL Server 8.0)
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  33. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Database Challenges
    Real-World Defective Database Application
    The Risks Digest, Volume 22, Issue 64, 2003
    Jeppesen reports airspace boundary problems
    About 350 airspace boundaries contained in Jeppesen Nav-
    Data are incorrect, the FAA has warned. The error occurred
    at Jeppesen after a software upgrade when information was
    pulled from a database containing 20,000 airspace bound-
    aries worldwide for the March NavData update, which takes
    effect March 20.
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  34. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Database Challenges
    Real-World Defective Database Application
    The Risks Digest, Volume 22, Issue 64, 2003
    Jeppesen reports airspace boundary problems
    About 350 airspace boundaries contained in Jeppesen Nav-
    Data are incorrect, the FAA has warned. The error occurred
    at Jeppesen after a software upgrade when information was
    pulled from a database containing 20,000 airspace bound-
    aries worldwide for the March NavData update, which takes
    effect March 20.
    Practically all use of databases occurs from within applica-
    tion programs [Silberschatz et al., 2006, pg. 311]
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  35. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Structured Query Language
    The structured query language (SQL) is an established stan-
    dard and a query and manipulation language for relational
    database management systems (RDBMS)
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  36. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Structured Query Language
    The structured query language (SQL) is an established stan-
    dard and a query and manipulation language for relational
    database management systems (RDBMS)
    A schema is a collection of table definitions:
    CREATE TABLE person (
    id INT,
    name VARCHAR(100) NOT NULL,
    age INT(3),
    PRIMARY KEY (id)
    )
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  37. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Structured Query Language
    The structured query language (SQL) is an established stan-
    dard and a query and manipulation language for relational
    database management systems (RDBMS)
    The data manipulation language supports several operations:
    SELECT name FROM person WHERE age >= 30 AND age <= 40
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  38. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Structured Query Language
    The structured query language (SQL) is an established stan-
    dard and a query and manipulation language for relational
    database management systems (RDBMS)
    The data manipulation language supports several operations:
    UPDATE person SET name = Jan WHERE id = 2
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  39. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Structured Query Language
    The structured query language (SQL) is an established stan-
    dard and a query and manipulation language for relational
    database management systems (RDBMS)
    The data manipulation language supports several operations:
    INSERT INTO person (id, name, age) VALUES
    (1, John, 38)
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  40. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Structured Query Language
    The structured query language (SQL) is an established stan-
    dard and a query and manipulation language for relational
    database management systems (RDBMS)
    The data manipulation language supports several operations:
    DELETE FROM person WHERE id = 2
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  41. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Relational Database Tables
    id name age
    1 Chalker Conrad 12
    2 Abby Clulow 14
    3 David Rogan 18
    4 Stacie Reckling 32
    5 Megan Hartnup 29
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  42. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Relational Database Tables
    id name age
    1 Chalker Conrad 12
    2 Abby Clulow 14
    3 David Rogan 18
    4 Stacie Reckling 32
    5 Megan Hartnup 29
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  43. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Relational Database Tables
    id name age
    1 Chalker Conrad 12
    2 Abby Clulow 14
    3 David Rogan 18
    4 Stacie Reckling 32
    5 Megan Hartnup 29
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  44. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Relational Database Tables
    id name age
    1 Chalker Conrad 12
    2 Abby Clulow 14
    3 David Rogan 18
    4 Stacie Reckling 32
    5 Megan Hartnup 29
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  45. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Relational Database Tables
    id name age
    1 Chalker Conrad 12
    2 Abby Clulow 14
    3 David Rogan 18
    4 Stacie Reckling 32
    5 Megan Hartnup 29
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  46. Introduction Database Applications Dynamic Invariants Conclusion
    Relational Databases
    Relational Database Tables
    id name age
    1 Chalker Conrad 12
    2 Abby Clulow 14
    3 David Rogan 18
    4 Stacie Reckling 32
    5 Megan Hartnup 29
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  47. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Database Applications
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  48. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Database Applications
    Program
    Relational Database
    Management System
    Database
    State
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  49. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    select
    Database
    State
    query
    Data Manipulation Language (DML) Statements
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  50. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    update
    Database
    State
    modify
    Data Manipulation Language (DML) Statements
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  51. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Database Applications
    Program
    Relational Database
    Management System
    Data Manipulation Language (DML) Statements
    Relational Database
    Management System
    insert
    Database
    State
    modify
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  52. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Database Applications
    Program
    Relational Database
    Management System
    Data Manipulation Language (DML) Statements
    Relational Database
    Management System
    delete
    Database
    State
    modify
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  53. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    create table
    Database
    Structure
    modify
    Data Definition Language (DDL) Statements
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  54. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    create table
    Database
    Structure
    modify
    Data Definition Language (DDL) Statements
    Can we categorize the dif-
    ferent ways of implementing
    database applications?
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  55. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Categorizing Database Applications
    Database
    Applications
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  56. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Categorizing Database Applications
    Database
    Applications
    Interaction
    Approach
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  57. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Categorizing Database Applications
    Database
    Applications
    Interaction
    Approach
    Program
    Location
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  58. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Categorizing Database Applications
    Database
    Applications
    Interaction
    Approach
    Program
    Location
    Interface
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  59. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Categorizing Database Applications
    Database
    Applications
    Interaction
    Approach
    Program
    Location
    Interface Embedded
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  60. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Categorizing Database Applications
    Database
    Applications
    Interaction
    Approach
    Program
    Location
    Interface Embedded
    Outside
    RDBMS
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  61. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Categorizing Database Applications
    Database
    Applications
    Interaction
    Approach
    Program
    Location
    Interface Embedded
    Outside
    RDBMS
    Inside
    RDBMS
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  62. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Categorizing Database Applications
    Database
    Applications
    Interaction
    Approach
    Program
    Location
    Interface Embedded
    Outside
    RDBMS
    Inside
    RDBMS
    Interface
    Outside
    RDBMS
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  63. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Categorizing Database Applications
    Database
    Applications
    Interaction
    Approach
    Program
    Location
    Interface Embedded
    Outside
    RDBMS
    Inside
    RDBMS
    Interface
    Outside
    RDBMS
    Java application that submits SQL strings to HSQLDB using JDBC
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  64. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Evolution of Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    DML Command
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  65. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Evolution of Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    DML Command
    Database
    State
    Database
    Structure
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  66. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Evolution of Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    DML Command
    Database
    State
    Database
    Structure
    Only the database administrator can add new constraints to the schema!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  67. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Evolution of Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    DML Command
    Database
    State
    Database
    Structure
    The programmers encode the constraints in the program’s source code!
    Constraint Ci
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  68. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Evolution of Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    DML Command
    Database
    State
    Database
    Structure
    The programmers encode the constraints in the program’s source code!
    Constraint Ci
    Constraint Cj
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  69. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Evolution of Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    DML Command
    Database
    State
    Database
    Structure
    The programmers encode the constraints in the program’s source code!
    Constraint Ci
    Constraint Cj
    Constraint Ck
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  70. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Evolution of Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    DML Command
    Database
    State
    Database
    Structure
    Constraint Ci
    Constraint Cj
    Constraint Ck
    Constraints Ci, Cj, Ck
    should be encoded in the schema!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  71. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Evolution of Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    DML Command
    Database
    State
    Database
    Structure
    Constraint Ci
    Constraint Cj
    Constraint Ck
    Goal: extract Ci, Cj, Ck
    from the source code of the program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  72. Introduction Database Applications Dynamic Invariants Conclusion
    Programs and Databases
    Evolution of Database Applications
    Program
    Relational Database
    Management System
    Relational Database
    Management System
    DML Command
    Database
    State
    Database
    Structure
    Goal: extract Ci, Cj, Ck
    from the source code of the program
    Database
    Structure
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  73. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariants
    Definition
    An invariant is a mathematical property that holds through
    some set of transformations
    Motivating Examples
    • 0 × y = 0
    • |x| ≥ 0
    • C
    d
    = π
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  74. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariants
    Definition
    An invariant is a mathematical property that holds through
    some set of transformations
    Motivating Examples
    • 0 × y = 0
    • |x| ≥ 0
    • C
    d
    = π
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  75. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariants
    Definition
    An invariant is a mathematical property that holds through
    some set of transformations
    Motivating Examples
    • 0 × y = 0
    • |x| ≥ 0
    • C
    d
    = π
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  76. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariants
    Definition
    An invariant is a mathematical property that holds through
    some set of transformations
    Motivating Examples
    • 0 × y = 0
    • |x| ≥ 0
    • C
    d
    = π
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  77. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Program Invariants
    Invariant with respect to:
    • State
    • Input/Output
    Simple Examples
    • 0 ≤ x ≤ 10
    • 1 ≤ nextX() ≤ 11
    • nextX() =
    (x + 1) mod 11
    1 class Invariant {
    2 static int x = 0;
    3 public static int nextX() {
    4 if( ++x > 10 )
    5 x = 0;
    6 return x + 1;
    7 }
    8 }
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  78. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Program Invariants
    Invariant with respect to:
    • State
    • Input/Output
    Simple Examples
    • 0 ≤ x ≤ 10
    • 1 ≤ nextX() ≤ 11
    • nextX() =
    (x + 1) mod 11
    1 class Invariant {
    2 static int x = 0;
    3 public static int nextX() {
    4 if( ++x > 10 )
    5 x = 0;
    6 return x + 1;
    7 }
    8 }
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  79. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Program Invariants
    Invariant with respect to:
    • State
    • Input/Output
    Simple Examples
    • 0 ≤ x ≤ 10
    • 1 ≤ nextX() ≤ 11
    • nextX() =
    (x + 1) mod 11
    1 class Invariant {
    2 static int x = 0;
    3 public static int nextX() {
    4 if( ++x > 10 )
    5 x = 0;
    6 return x + 1;
    7 }
    8 }
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  80. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Program Invariants
    Invariant with respect to:
    • State
    • Input/Output
    Simple Examples
    • 0 ≤ x ≤ 10
    • 1 ≤ nextX() ≤ 11
    • nextX() =
    (x + 1) mod 11
    1 class Invariant {
    2 static int x = 0;
    3 public static int nextX() {
    4 if( ++x > 10 )
    5 x = 0;
    6 return x + 1;
    7 }
    8 }
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  81. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Dynamic Invariants
    Definition
    A dynamic invariant is a property that is observed to hold
    during a series of executions
    • Not guaranteed for all possible executions
    • May reflect property of:
    • Program
    • Inputs
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  82. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Dynamic Invariants
    Definition
    A dynamic invariant is a property that is observed to hold
    during a series of executions
    • Not guaranteed for all possible executions
    • May reflect property of:
    • Program
    • Inputs
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  83. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Daikon Invariant Detector
    Daikon [Ernst et al. 2001] is a dynamic invariant detection
    engine originally designed for traditional C and Java programs
    Detection Process
    • Collect data traces for variables at program points
    • Compare to pool of potential invariants
    • Output remaining invariants that meet confidence threshold
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  84. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Daikon Invariant Detector
    Daikon [Ernst et al. 2001] is a dynamic invariant detection
    engine originally designed for traditional C and Java programs
    Detection Process
    • Collect data traces for variables at program points
    • Compare to pool of potential invariants
    • Output remaining invariants that meet confidence threshold
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  85. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariant Detection Process
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  86. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariant Detection Process
    Program
    Instrument Program’
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  87. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariant Detection Process
    Program
    Instrument Program’ Execute
    Test Suite
    Trace File
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  88. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariant Detection Process
    Program
    Instrument Program’ Execute
    Test Suite
    Trace File Daikon
    Potential Invariants
    Dynamic Invariants
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  89. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariant Detection Process
    Program
    Instrument Program’ Execute
    Test Suite
    Trace File Daikon
    Potential Invariants
    Dynamic Invariants
    Supporting
    Techniques
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  90. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariant Detection Process
    Program
    Instrument Program’ Execute
    Test Suite
    Trace File Daikon
    Potential Invariants
    Dynamic Invariants
    Supporting
    Techniques
    Instrumentation
    Instrument
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  91. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariant Detection Process
    Program
    Instrument Program’ Execute
    Test Suite
    Trace File Daikon
    Potential Invariants
    Dynamic Invariants
    Supporting
    Techniques
    Instrumentation
    Instrument
    Test Suite Execution
    Execute
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  92. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariant Detection Process
    Program
    Instrument Program’ Execute
    Test Suite
    Trace File Daikon
    Potential Invariants
    Dynamic Invariants
    Supporting
    Techniques
    Instrumentation
    Instrument
    Test Suite Execution
    Execute
    Statistical Analysis
    Daikon
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  93. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Invariant Detection Process
    Program
    Instrument Program’ Execute
    Test Suite
    Trace File Daikon
    Potential Invariants
    Dynamic Invariants
    Supporting
    Techniques
    Instrumentation
    Instrument
    Test Suite Execution
    Execute
    Statistical Analysis
    Daikon
    Refer to [Ernst et al. 2001] for additional details about these techniques
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  94. Introduction Database Applications Dynamic Invariants Conclusion
    Traditional Invariant Detection
    Applications of Daikon
    Applications of dynamic invariants in software engineering:
    • Programmer understanding
    • Run-time checking
    • Integration testing
    • Interface discovery
    • Test-input generation
    • Test-suite reduction
    • Many additional techniques
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  95. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Structural Mapping
    Program Element Database Element
    Program Point Table
    Variable Column
    Occurence Row
    Detect invariants for:
    • Individual columns
    • Between columns in a given row
    id name age employed . . .
    1 ’John Smith’ 38 5 . . .
    2 ’Jan Downing’ 22 2 . . .
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  96. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Structural Mapping
    Program Element Database Element
    Program Point Table
    Variable Column
    Occurence Row
    Detect invariants for:
    • Individual columns
    • Between columns in a given row
    id name age employed . . .
    1 ’John Smith’ 38 5 . . .
    2 ’Jan Downing’ 22 2 . . .
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  97. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Structural Mapping
    Program Element Database Element
    Program Point Table
    Variable Column
    Occurence Row
    Detect invariants for:
    • Individual columns
    • Between columns in a given row
    id name age employed . . .
    1 ’John Smith’ 38 5 . . .
    2 ’Jan Downing’ 22 2 . . .
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  98. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Data Mapping
    Daikon Concepts
    • Representation type
    • int
    • double
    • String
    • int[]
    • Comparability
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  99. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Data Mapping
    Group Name SQL Types Java Type
    1 Text
    CHAR
    String
    VARCHAR
    TEXT
    2 Integer
    INTEGER
    int
    NUMERIC
    BIT
    3 Decimal
    FLOAT
    double
    DOUBLE
    REAL
    DECIMAL
    4 Binary BLOB
    byte[]
    BIT
    5 Text Set SET String[]
    6 Datetime DATETIME
    String
    TIMESTAMP
    7 Date DATE String
    8 Time TIME String
    9 Interval INTERVAL int
    10 Primary Key INTEGER reference
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  100. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Data Mapping
    Handling NULL Values
    • NULL is a possible value for any SQL type
    • Daikon does not accept null for primitive representation
    types such as int
    • Introduce synthetic variable for each NULL-able column
    • Representation type is hashcode (reference)
    • Value is either null or a constant
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  101. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Data Mapping
    Handling NULL Values
    • NULL is a possible value for any SQL type
    • Daikon does not accept null for primitive representation
    types such as int
    • Introduce synthetic variable for each NULL-able column
    • Representation type is hashcode (reference)
    • Value is either null or a constant
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  102. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Database-Aware Procedure
    Read Schema
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  103. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Database-Aware Procedure
    Read Schema
    Relational Database
    Management System
    Schema
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  104. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Database-Aware Procedure
    Read Schema
    Relational Database
    Management System
    Schema
    Scan Database
    State
    All Rows, Columns
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  105. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Database-Aware Procedure
    Read Schema
    Relational Database
    Management System
    Schema
    Scan Database
    State
    All Rows, Columns Instrumented
    Program
    All Modified Rows
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  106. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Database-Aware Procedure
    Read Schema
    Relational Database
    Management System
    Schema
    Scan Database
    State
    All Rows, Columns Instrumented
    Program
    All Modified Rows
    Collect Trace
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  107. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Database-Aware Procedure
    Read Schema
    Relational Database
    Management System
    Schema
    Scan Database
    State
    All Rows, Columns Instrumented
    Program
    All Modified Rows
    Collect Trace
    Infer Invariants
    Data Trace
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  108. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Database-Aware Procedure
    Read Schema
    Relational Database
    Management System
    Schema
    Scan Database
    State
    All Rows, Columns Instrumented
    Program
    All Modified Rows
    Collect Trace
    Infer Invariants
    Data Trace
    Relevant Invariants
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  109. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Database-Aware Procedure
    Read Schema
    Relational Database
    Management System
    Schema
    Scan Database
    State
    All Rows, Columns Instrumented
    Program
    All Modified Rows
    Collect Trace
    Infer Invariants
    Data Trace
    Relevant Invariants
    Present the Invariants to
    the Database Administrator
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  110. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Database-Aware Procedure
    Read Schema
    Relational Database
    Management System
    Schema
    Scan Database
    State
    All Rows, Columns Instrumented
    Program
    All Modified Rows
    Collect Trace
    Infer Invariants
    Data Trace
    Relevant Invariants
    Present the Invariants to
    the Database Administrator
    Place Ci, Cj, Ck
    into
    the database schema
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  111. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Implementation
    Trace Collector
    • Pythona program:
    • Input: Database connection information
    • Output: Daikon declarations and data trace files
    • Process:
    1 Read schema metadata to determine tables, columns, and
    data mapping
    2 Write declarations file and serialize mapping info for reuse
    3 SELECT table contents, transform data by mapping, write to
    GZip’d trace file
    • Supports various RDBMS with the SQLAlchemy toolkit
    a. . . plus a small amount of Cython
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  112. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Implementation
    Trace Collector
    • Pythona program:
    • Input: Database connection information
    • Output: Daikon declarations and data trace files
    • Process:
    1 Read schema metadata to determine tables, columns, and
    data mapping
    2 Write declarations file and serialize mapping info for reuse
    3 SELECT table contents, transform data by mapping, write to
    GZip’d trace file
    • Supports various RDBMS with the SQLAlchemy toolkit
    a. . . plus a small amount of Cython
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  113. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Implementation
    Trace Collector
    • Pythona program:
    • Input: Database connection information
    • Output: Daikon declarations and data trace files
    • Process:
    1 Read schema metadata to determine tables, columns, and
    data mapping
    2 Write declarations file and serialize mapping info for reuse
    3 SELECT table contents, transform data by mapping, write to
    GZip’d trace file
    • Supports various RDBMS with the SQLAlchemy toolkit
    a. . . plus a small amount of Cython
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  114. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Implementation
    Instrumentation Wrapper
    • Modified P6Spy JDBC driver wrapper
    • On connection, capture information and initiate initial
    metadata read and trace
    • On statement execution, append the trace file if the
    database could be modified
    • INSERT statement
    • UPDATE statement
    • Unknown (e.g., a stored procedure call)
    • Ignore others, including DELETE and TRUNCATE
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  115. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Implementation
    Instrumentation Wrapper
    • Modified P6Spy JDBC driver wrapper
    • On connection, capture information and initiate initial
    metadata read and trace
    • On statement execution, append the trace file if the
    database could be modified
    • INSERT statement
    • UPDATE statement
    • Unknown (e.g., a stored procedure call)
    • Ignore others, including DELETE and TRUNCATE
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  116. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Invariant Detection
    Implementation
    Instrumentation Wrapper
    • Modified P6Spy JDBC driver wrapper
    • On connection, capture information and initiate initial
    metadata read and trace
    • On statement execution, append the trace file if the
    database could be modified
    • INSERT statement
    • UPDATE statement
    • Unknown (e.g., a stored procedure call)
    • Ignore others, including DELETE and TRUNCATE
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  117. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Objects of Analysis
    Fixed Data Sets
    Database Tables Columns Rows
    world 3 24 5302
    sakila 23 131 50,086
    menagerie 2 10 19
    employees 6 24 3,919,015
    • MySQL sample databases commonly used for training,
    certification, and testing
    • Trace the entire dataset during invariant detection
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  118. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Objects of Analysis
    Database Applications
    Program iTrust JWhoisServer JTrac
    Tables 30 7 13
    Columns 177 57 126
    KLOC 25.5 (Java), 8.6 (JSP) 6.7 12
    Test Cases 787 67 41
    • Java applications that interact with a relational database
    • Wrap real database driver in a modified P6Spy driver
    • Execute the entire test suite during invariant detection
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  119. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Meaningful Invariants
    Invariants that capture a semantic relationship
    • dept_emp.from_date <= dept_emp.to_date
    • employees.gender one of { "F", "M" }
    • employees.birth_date < employees.hire_date
    • country.Population >= 0
    • icdcodes.Chronic one of { "no", "yes" }
    All of these invariants were automatically generated!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  120. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Meaningful Invariants
    Invariants that capture a semantic relationship
    • dept_emp.from_date <= dept_emp.to_date
    • employees.gender one of { "F", "M" }
    • employees.birth_date < employees.hire_date
    • country.Population >= 0
    • icdcodes.Chronic one of { "no", "yes" }
    All of these invariants were automatically generated!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  121. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Meaningful Invariants
    Invariants that capture a semantic relationship
    • dept_emp.from_date <= dept_emp.to_date
    • employees.gender one of { "F", "M" }
    • employees.birth_date < employees.hire_date
    • country.Population >= 0
    • icdcodes.Chronic one of { "no", "yes" }
    All of these invariants were automatically generated!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  122. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Meaningful Invariants
    Invariants that capture a semantic relationship
    • dept_emp.from_date <= dept_emp.to_date
    • employees.gender one of { "F", "M" }
    • employees.birth_date < employees.hire_date
    • country.Population >= 0
    • icdcodes.Chronic one of { "no", "yes" }
    All of these invariants were automatically generated!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  123. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Meaningful Invariants
    Invariants that capture a semantic relationship
    • dept_emp.from_date <= dept_emp.to_date
    • employees.gender one of { "F", "M" }
    • employees.birth_date < employees.hire_date
    • country.Population >= 0
    • icdcodes.Chronic one of { "no", "yes" }
    All of these invariants were automatically generated!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  124. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Meaningful Invariants
    Invariants that capture a semantic relationship
    • dept_emp.from_date <= dept_emp.to_date
    • employees.gender one of { "F", "M" }
    • employees.birth_date < employees.hire_date
    • country.Population >= 0
    • icdcodes.Chronic one of { "no", "yes" }
    All of these invariants were automatically generated!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  125. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Meaningful Invariants
    Invariants that capture a semantic relationship
    • dept_emp.from_date <= dept_emp.to_date
    • employees.gender one of { "F", "M" }
    • employees.birth_date < employees.hire_date
    • country.Population >= 0
    • icdcodes.Chronic one of { "no", "yes" }
    All of these invariants were automatically generated!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  126. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Spurious Invariants
    Spurious Invariants
    • Vacuous invariants reflect a meaningless relationship.
    • patients.phone1 <= patients.BloodType
    • patients.lastName >= patients.address1
    • cptcodes.Description != cptcodes.Attribute
    • Lack-of-data invariants result from limited data samples.
    • mntnr.login == "mntnt"
    • inetnum.changed == "2006-10-14 16:21:09"
    • person.name one of { "no name company",
    "persona non grata"}
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  127. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Spurious Invariants
    Spurious Invariants
    • Vacuous invariants reflect a meaningless relationship.
    • patients.phone1 <= patients.BloodType
    • patients.lastName >= patients.address1
    • cptcodes.Description != cptcodes.Attribute
    • Lack-of-data invariants result from limited data samples.
    • mntnr.login == "mntnt"
    • inetnum.changed == "2006-10-14 16:21:09"
    • person.name one of { "no name company",
    "persona non grata"}
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  128. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Spurious Invariants
    Spurious Invariants
    • Vacuous invariants reflect a meaningless relationship.
    • patients.phone1 <= patients.BloodType
    • patients.lastName >= patients.address1
    • cptcodes.Description != cptcodes.Attribute
    • Lack-of-data invariants result from limited data samples.
    • mntnr.login == "mntnt"
    • inetnum.changed == "2006-10-14 16:21:09"
    • person.name one of { "no name company",
    "persona non grata"}
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  129. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Number of Invariants
    employees
    world
    menagerie
    JWhoisServer
    JTrac
    sakila
    iTrust
    0 50 100 150
    Type of Invariant
    Vacuous Lack−of−data Meaningful
    The majority of detected dynamic invariants are not spurious
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  130. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Number of Invariants
    employees
    world
    menagerie
    JWhoisServer
    JTrac
    sakila
    iTrust
    0 50 100 150
    Type of Invariant
    Vacuous Lack−of−data Meaningful
    employees’ invariants are meaningful due to wealth of data
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  131. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Invariant Quality
    Number of Invariants
    employees
    world
    menagerie
    JWhoisServer
    JTrac
    sakila
    iTrust
    0 50 100 150
    Type of Invariant
    Vacuous Lack−of−data Meaningful
    JWhoisServer’s few meaningful invariants suggests poor tests
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  132. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Using Dynamic Invariants
    • Some invariants can be enforced by the schema definition
    • Schema enforcement:
    • Provides a stronger assurance of data integrity than
    application enforcement
    • Enables easy long-term maintenance of the program and
    the relational database
    • Analyze enforceable invariants:
    • Already enforced by the schema
    • Suggest modification to enforce the invariant
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  133. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Using Dynamic Invariants
    • Some invariants can be enforced by the schema definition
    • Schema enforcement:
    • Provides a stronger assurance of data integrity than
    application enforcement
    • Enables easy long-term maintenance of the program and
    the relational database
    • Analyze enforceable invariants:
    • Already enforced by the schema
    • Suggest modification to enforce the invariant
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  134. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Schema Enforced
    Invariant Schema Definition
    employees.gender one of { "F", "M" } ENUM(’F’,’M’)
    countrylanguage.IsOfficial one of { "F", "T" } ENUM(’F’,’T’)
    customer.active one of { 0, 1 } TINYINT(1)
    inventory.film id >= 1 SMALLINT(5) UNSIGNED
    spaces.guest allowed one of { 0, 1 } BIT(1)
    Reverse engineered many constraints already enforced by the schema
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  135. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Schema Enforced
    Invariant Schema Definition
    employees.gender one of { "F", "M" } ENUM(’F’,’M’)
    countrylanguage.IsOfficial one of { "F", "T" } ENUM(’F’,’T’)
    customer.active one of { 0, 1 } TINYINT(1)
    inventory.film id >= 1 SMALLINT(5) UNSIGNED
    spaces.guest allowed one of { 0, 1 } BIT(1)
    Reverse engineered many constraints already enforced by the schema
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  136. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Schema Enforceable
    Invariant Schema Modification
    isnull(message.message) != null TEXT NOT NULL
    isnull(film text.description) != null TEXT NOT NULL
    isnull(history.time stamp) != null DATETIME NOT NULL
    user space roles.user id >= 1 BIGINT(20) UNSIGNED
    pet.sex one of { "f", "m" } CHAR(1) ENUM(’m’,’f’)
    country.Population >= 0 INT(11) UNSIGNED
    isnull(titles.to date) != null DATE NOT NULL
    Many detected constraints can easily be added to the schema
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  137. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Schema Enforceable
    Invariant Schema Modification
    isnull(message.message) != null TEXT NOT NULL
    isnull(film text.description) != null TEXT NOT NULL
    isnull(history.time stamp) != null DATETIME NOT NULL
    user space roles.user id >= 1 BIGINT(20) UNSIGNED
    pet.sex one of { "f", "m" } CHAR(1) ENUM(’m’,’f’)
    country.Population >= 0 INT(11) UNSIGNED
    isnull(titles.to date) != null DATE NOT NULL
    Many detected constraints can easily be added to the schema
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  138. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Percentage of Meaningful Invariants
    JWhoisServer
    menagerie
    JTrac
    employees
    iTrust
    world
    sakila
    0.0 0.2 0.4 0.6 0.8 1.0
    Type of Meaningful Invariant
    Enforceable with Standards−Compliant Database
    Enforceable with Current Database
    Already Enforced
    0
    0
    0
    Percentages relative to the total number of non-spurious invariants
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  139. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Percentage of Meaningful Invariants
    JWhoisServer
    menagerie
    JTrac
    employees
    iTrust
    world
    sakila
    0.0 0.2 0.4 0.6 0.8 1.0
    Type of Meaningful Invariant
    Enforceable with Standards−Compliant Database
    Enforceable with Current Database
    Already Enforced
    0
    0
    0
    All constraints enforceable by a standards-compliant database
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  140. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Percentage of Meaningful Invariants
    JWhoisServer
    menagerie
    JTrac
    employees
    iTrust
    world
    sakila
    0.0 0.2 0.4 0.6 0.8 1.0
    Type of Meaningful Invariant
    Enforceable with Standards−Compliant Database
    Enforceable with Current Database
    Already Enforced
    0
    0
    0
    Three schemas can be enhanced with many new constraints
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  141. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Percentage of Meaningful Invariants
    JWhoisServer
    menagerie
    JTrac
    employees
    iTrust
    world
    sakila
    0.0 0.2 0.4 0.6 0.8 1.0
    Type of Meaningful Invariant
    Enforceable with Standards−Compliant Database
    Enforceable with Current Database
    Already Enforced
    0
    0
    0
    menagerie did not already enforce any of the meaningful invariants
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  142. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Schema Modification
    Percentage of Meaningful Invariants
    JWhoisServer
    menagerie
    JTrac
    employees
    iTrust
    world
    sakila
    0.0 0.2 0.4 0.6 0.8 1.0
    Type of Meaningful Invariant
    Enforceable with Standards−Compliant Database
    Enforceable with Current Database
    Already Enforced
    0
    0
    0
    JWhoisServer’s MySQL doesn’t support constraint enforcement
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  143. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Empirical Conclusions and Future Research
    Conclusions
    • Meaningful invariants may be mined from both relational
    databases and database applications
    • Invariant quality depends on existence of diverse data
    • Data integrity may be enhanced by using invariants for
    modification of the database’s schema
    Future Research
    • Invariants between multiple tables
    • Invariants for individual queries
    • Explore additional client applications
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  144. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Empirical Conclusions and Future Research
    Conclusions
    • Meaningful invariants may be mined from both relational
    databases and database applications
    • Invariant quality depends on existence of diverse data
    • Data integrity may be enhanced by using invariants for
    modification of the database’s schema
    Future Research
    • Invariants between multiple tables
    • Invariants for individual queries
    • Explore additional client applications
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  145. Introduction Database Applications Dynamic Invariants Conclusion
    Empirical Results
    Empirical Conclusions and Future Research
    Conclusions
    • Meaningful invariants may be mined from both relational
    databases and database applications
    • Invariant quality depends on existence of diverse data
    • Data integrity may be enhanced by using invariants for
    modification of the database’s schema
    Future Research
    • Invariants between multiple tables
    • Invariants for individual queries
    • Explore additional client applications
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  146. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  147. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Input
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  148. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Input Output
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  149. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  150. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  151. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input Output
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  152. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input Output
    Test
    Clean Up
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  153. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input Output
    Test
    Clean Up
    Test
    Oracle
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  154. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input Output
    Test
    Clean Up
    Test
    Oracle
    Expected
    Output
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  155. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input Output
    Test
    Clean Up
    Test
    Oracle
    Expected
    Output
    Test
    Verdict
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  156. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input Output
    Test
    Clean Up
    Test
    Oracle
    Expected
    Output
    Test
    Verdict
    Expected
    Output
    Output
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  157. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input Output
    Test
    Clean Up
    Test
    Oracle
    Expected
    Output
    Test
    Verdict
    Expected
    Output
    Output
    Test
    Verdict
    The test case passes and the code is correct!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  158. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input Output
    Test
    Clean Up
    Test
    Oracle
    Expected
    Output
    Test
    Verdict
    Expected
    Output
    Output
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  159. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Case?
    Method
    Under Test
    Test
    Set Up
    Input Output
    Test
    Clean Up
    Test
    Oracle
    Expected
    Output
    Test
    Verdict
    Expected
    Output
    Output
    Test
    Verdict
    The test case fails and a defect is found!
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  160. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  161. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  162. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    T3
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  163. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    T3 T4
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  164. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    T3 T4
    . . .
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  165. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    T3 T4
    . . . Tn
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  166. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    T3 T4
    . . . Tn
    Organize the Test Cases into a Test Suite
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  167. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    T3 T4
    . . . Tn
    Organize the Test Cases into a Test Suite
    Tool Support for Software Testing?
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  168. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    T3 T4
    . . . Tn
    Organize the Test Cases into a Test Suite
    Tool Support for Software Testing?
    JUnit
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  169. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    T3 T4
    . . . Tn
    Organize the Test Cases into a Test Suite
    Tool Support for Software Testing?
    JUnit Apache Ant
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  170. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    What is a Test Suite?
    T1 T2
    T3 T4
    . . . Tn
    Organize the Test Cases into a Test Suite
    Tool Support for Software Testing?
    JUnit Apache Ant Eclipse
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  171. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Organize the Test Cases into a Test Suite
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  172. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Regression Testing Technique
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  173. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Regression Testing Technique
    What if Some Test Cases are More Effective?
    T3 Tn
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  174. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Regression Testing Technique
    What if Some Test Cases are More Effective?
    T3 Tn
    Prioritization
    T3 Tn T1 T4
    . . . T2
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  175. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Regression Testing Technique
    What if Some Test Cases are More Effective?
    T3 Tn
    Prioritization
    T3 Tn T1 T4
    . . . T2
    T1 T2
    T3 T4
    . . . Tn
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  176. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Regression Testing Technique
    T3 Tn
    Prioritization
    T3 Tn T1 T4
    . . . T2
    T1 T2
    T3 T4
    . . . Tn
    What if Some Test Cases are Redundant?
    T1 T2
    T3 T4
    . . . Tn
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  177. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Regression Testing Technique
    T3 Tn
    Prioritization
    T3 Tn T1 T4
    . . . T2
    T1 T2
    T3 T4
    . . . Tn
    What if Some Test Cases are Redundant?
    T1 T2
    T3 T4
    . . . Tn
    Reduction
    T4
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  178. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Regression Testing Technique
    T3 Tn
    Prioritization
    T3 Tn T1 T4
    . . . T2
    T1 T2
    T3 T4
    . . . Tn
    What if Some Test Cases are Redundant?
    T1 T2
    T3 T4
    . . . Tn
    Reduction
    T4
    T1 T2
    T3 T4
    . . . Tn
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  179. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Regression Testing Technique
    T3 Tn
    Prioritization
    T3 Tn T1 T4
    . . . T2
    T1 T2
    T3 T4
    . . . Tn
    T1 T2
    T3 T4
    . . . Tn
    Reduction
    T4
    T1 T2
    T3 T4
    . . . Tn
    What if Only Certain Tests are Needed?
    T2 Tn
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  180. Introduction Database Applications Dynamic Invariants Conclusion
    Regression Testing
    Test Suite Management
    T1 T2
    T3 T4
    . . . Tn
    Regression Testing Technique
    T3 Tn
    Prioritization
    T3 Tn T1 T4
    . . . T2
    T1 T2
    T3 T4
    . . . Tn
    T1 T2
    T3 T4
    . . . Tn
    Reduction
    T4
    T1 T2
    T3 T4
    . . . Tn
    What if Only Certain Tests are Needed?
    T2 Tn
    Selection
    T1 T2
    T3 T4
    . . . Tn
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  181. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  182. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  183. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Program
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  184. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Program
    Test Suite
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  185. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Program
    Test Suite
    Database
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  186. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Program
    Test Suite
    Database
    Invariants
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  187. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Program
    Test Suite
    Database
    Invariants
    T1, . . . , T10
    =
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  188. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Program
    Test Suite
    Database
    Invariants
    T1, . . . , T10
    =
    T41, . . . , T50
    =
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  189. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Program
    Test Suite
    Database
    Invariants
    T1, . . . , T10
    =
    T41, . . . , T50
    =
    T21, . . . , T30
    =
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  190. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Program
    Test Suite
    Database
    Invariants
    T1, . . . , T10
    =
    T41, . . . , T50
    =
    T21, . . . , T30
    =
    Divide and Conquer
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  191. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Database-Aware Test Suite Reduction
    T1, . . . , T10
    T11, . . . , T20
    T21, . . . , T30
    T31, . . . , T40
    T41, . . . , T50
    Dynamic
    Invariant
    Detection
    Program
    Test Suite
    Database
    Invariants
    T1, . . . , T10
    =
    T41, . . . , T50
    =
    T21, . . . , T30
    Divide and Conquer
    =
    T21, . . . , T30
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  192. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Conclusion
    Conclusion
    • Databases are widely used in real-world applications
    • Database applications have complex state and structure
    • Programmers often encode constraints in program source
    • Dynamic invariant detection reverse engineers constraints
    • Detected invariants are meaningful and enforceable
    Future Work
    • Further empirical studies of dynamic invariants
    • Implement and evaluate several client applications
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  193. Introduction Database Applications Dynamic Invariants Conclusion
    Database-Aware Method
    Conclusion
    Conclusion
    • Databases are widely used in real-world applications
    • Database applications have complex state and structure
    • Programmers often encode constraints in program source
    • Dynamic invariant detection reverse engineers constraints
    • Detected invariants are meaningful and enforceable
    Future Work
    • Further empirical studies of dynamic invariants
    • Implement and evaluate several client applications
    Kapfhammer Allegheny College
    Using Dynamic Invariant Detection to Support the Testing and Analysis of Database Applications

    View Slide

  194. Using Dynamic Invariant Detection
    to Support the Testing and Analysis
    of Database Applications
    Gregory M. Kapfhammer
    Department of Computer Science
    Allegheny College
    http://www.cs.allegheny.edu/∼gkapfham/
    Thank you for your attention!
    I welcome your questions and comments.

    View Slide