Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A primer on testing database-driven applications

A primer on testing database-driven applications

Interested in learning more about this topic? Read the overview of my research to learn more: https://www.gregorykapfhammer.com/research/

Gregory Kapfhammer

May 18, 2004
Tweet

More Decks by Gregory Kapfhammer

Other Decks in Technology

Transcript

  1. A Primer on Testing
    Database-Driven Applications
    Gregory M. Kapfhammer
    Department of Computer Science
    Allegheny College
    3rd Biennial Intramural Faculty Conference
    May 18 - 19, 2004
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 1/23

    View full-size slide

  2. Outline
    Introduction and Motivation
    Testing Challenges
    Database-Driven Applications
    A Unified Representation
    Test Adequacy Criteria
    Test Suite Execution
    Test Coverage Monitoring
    Conclusions and Resources
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 2/23

    View full-size slide

  3. Invocation
    Thus spake the master, “Any program, no matter
    how small, contains bugs.”
    The novice did not believe the master’s words.
    “What if the program were so small that it
    performed a single function?”
    The master thoughtfully responded, “Such a
    program would have no meaning. But, if such a
    one existed, the operating system would fail
    eventually, producing a bug.”
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 3/23

    View full-size slide

  4. Invocation
    But the novice was not satisfied. “What if the
    operating system did not fail?”
    The master responded, “There is no operating
    system that does not fail. But if such a one existed,
    the hardware would fail eventually, producing a
    bug.”
    The novice still was not satisfied. “What if the
    hardware did not fail?”
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 4/23

    View full-size slide

  5. Invocation
    The master gave a great sigh. “There is no
    hardware that does not fail. But if such a one
    existed, the user would want the program to do
    something different, and this too is a bug.”
    A program without bugs would be an absurdity, a
    nonesuch. If there were a program without any bugs then
    the world would cease to exist.
    Geoffrey James
    The Zen of Programming
    (Adaptation)
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 5/23

    View full-size slide

  6. Life of a Program: Compilation
    P
    Source Code
    P
    Byte Code
    Compiler
    The programming language compiler produces a
    representation of a program that can be executed
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 6/23

    View full-size slide

  7. Life of a Program: Execution
    P
    Virtual
    Machine
    Byte Code
    Input
    a 5 print ... exit Final Result: 45
    Output
    The virtual machine executes the program’s byte codes
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 7/23

    View full-size slide

  8. Testing Challenges
    I shall not deny that the construction of these
    testing programs has been a major intellectual
    effort: to convince oneself that one has not
    overlooked “a relevant state” and to convince
    oneself that the testing programs generate them all
    is no simple matter. The encouraging thing is that
    (as far as we know!) it could be done.
    Edsger W. Dijkstra, 1968
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 8/23

    View full-size slide

  9. Where Bugs Live
    Machine
    P
    Virtual
    File
    System
    System
    Operating
    Database
    Byte Code
    Execution Environment
    Input
    a 5 print ... exit Final Result: 45
    Output
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 9/23

    View full-size slide

  10. Motivation
    The Risks Digest, Volume 22, Issue 64, 2003
    Jeppesen reports airspace boundary problems
    About 350 airspace boundaries contained in Jeppesen
    NavData are incorrect, the FAA has warned. The error
    occurred at Jeppesen after a software upgrade when
    information was pulled from a database containing
    20,000 airspace boundaries worldwide for the March
    NavData update, which takes effect March 20.
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 10/23

    View full-size slide

  11. More Testing Challenges
    Should consider the environment in which software
    applications execute
    Must test a program and its interaction with a database
    Database-driven application’s state space is
    well-structured, but infinite (Chays et al.)
    Need to show program does not violate database
    integrity, where integrity = consistency + validity (Motro)
    Must locate program and database coupling points that
    vary in granularity
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 11/23

    View full-size slide

  12. Database-Driven Applications
    P
    m i
    m
    j
    Dl
    Dk
    R
    R2
    1
    E F G H
    A B C D
    I
    R3
    J K L
    Program P interacts with two relational
    databases
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 12/23

    View full-size slide

  13. Database Interaction Levels
    P
    Database Level
    1
    n
    D
    D
    A program can interact with a
    database at different levels of
    granularity
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

    View full-size slide

  14. Database Interaction Levels
    UserInfo
    user_name
    4
    acct_lock
    1 Brian Zorman
    2 Robert Roos
    3
    card_number pin_number
    Geoffrey Arnold
    0
    0
    0
    0
    32142
    41601
    45322
    56471
    Marcus Bittman
    Relation Level
    P
    D1
    Dn
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

    View full-size slide

  15. Database Interaction Levels
    UserInfo
    user_name
    4
    acct_lock
    1 Brian Zorman
    2 Robert Roos
    3
    card_number pin_number
    Geoffrey Arnold
    0
    0
    0
    0
    32142
    41601
    45322
    56471
    Marcus Bittman
    Record Level
    P
    n
    D1
    D
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

    View full-size slide

  16. Database Interaction Levels
    UserInfo
    4
    acct_lock
    1 Brian Zorman
    2 Robert Roos
    3
    card_number pin_number
    0
    0
    0
    0
    32142
    41601
    45322
    56471
    user_name
    Attribute Level
    Marcus Bittman
    Geoffrey Arnold
    P
    D1
    Dn
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

    View full-size slide

  17. Database Interaction Levels
    UserInfo
    4
    acct_lock
    1 Brian Zorman
    2 Robert Roos
    3
    card_number pin_number
    Geoffrey Arnold
    0
    0
    0
    0
    32142
    41601
    45322
    56471
    user_name
    Attribute Value Level
    Marcus Bittman
    P
    D1
    n
    D
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 13/23

    View full-size slide

  18. Database Interaction Points
    select A1
    , A2
    , . . . , Aq
    from r1
    , r2
    , . . . , rm
    where Q
    delete from r
    where Q
    insert into r(A1
    , A2
    , . . . , Aq
    )
    values(v1
    , v2
    , . . . , vq
    )
    update r
    set Al
    = F(Al
    )
    where Q
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 14/23

    View full-size slide

  19. Refined Database-Driven Application
    P
    m i
    m
    j
    R
    R2
    1
    E F G H
    A B C D
    l
    D
    k
    D
    where
    set J = 500
    update
    L < 1000
    R3
    select 1
    * from R
    R2
    from )
    select
    where A < ( avg(G)
    I
    R3
    J K L
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 15/23

    View full-size slide

  20. Data Flow Information
    Interaction point: ‘‘UPDATE UserInfo SET
    acct lock=1 WHERE card number=’’ +
    card number + ‘‘;’’;
    Database Level: define(BankDB)
    Attribute Level: define(acct_lock) and
    use(card_number)
    Data flow information varies with respect to
    the granularity of the database interaction
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 16/23

    View full-size slide

  21. Test Adequacy Criteria
    all−attribute−value−DUs
    all−record−DUs all−attribute−DUs
    all−relation−DUs all−database−DUs
    Database interaction
    association (DIA) involves the
    def and use of a database
    entity
    DIAs can be located in the
    DICFG with data flow analysis
    all-database-DUs requires
    tests to exercise all DIAs for all
    of the accessed databases
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 17/23

    View full-size slide

  22. Generating Test Requirements
    Database Seeder
    Database
    (P)
    Test Adequacy Criterion
    (C)
    System Under Test
    Test Case Specification
    Relational Schema
    Requirements
    Test
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 18/23

    View full-size slide

  23. Tracking Covered DIGs and DIAs
    DB
    P DIGr
    m i
    m
    j
    DIGs
    DIG #
    1
    DEF USE
    { ... } { ... }
    q { ... } { ... }
    1
    1
    2
    2
    TEST
    { ... }
    { ... }
    COV?
    DIG Coverage Table
    DIA coverage can be tracked by recording which DIGs
    within a DICFG were executed during testing
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 19/23

    View full-size slide

  24. Types of Test Suites
    T
    T

    1
    e
    m
    1

    1
    e

    0
    m
    e
    ∆e−1
    Independent

    m
    T1 1
    0
    ∆1
    ε−1

    Tε ε
    m
    ε

    e

    e
    m
    Te
    ∆e
    Partially Independent
    T
    T

    1
    e
    m
    1

    1
    e

    0
    m
    e
    ∆e−1
    Non-restricted
    – p. 20/23

    View full-size slide

  25. Calculating Adequacy
    m
    i
    mj
    m
    i
    DIA


    COV?


    Test Requirements
    DIA COV?






    Test Requirements
    m
    j
    Tf
    cov(mi
    ) = 2
    4
    cov(mj
    ) = 4
    6
    cov(Tf
    ) = 6
    10
    – p. 21/23

    View full-size slide

  26. Conclusions
    Software testing is hard, especially when interaction
    with the application’s environment is considered
    Must test the program’s interaction with the database
    Many challenges associated with (1) unified program
    representation, (2) test adequacy criteria, (3) test
    coverage monitoring, (4) test suite execution
    Unique family of test adequacy criteria to ensure that
    test suites detect violations of database validity and
    completeness
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 22/23

    View full-size slide

  27. Resources
    Gregory M. Kapfhammer and Mary Lou Soffa. A Family of Test
    Adequacy Criteria for Database-Driven Applications. In
    Proceedings of the ACM SIGSOFT International Conference on
    the Foundations of Software Engineering, 2003.
    Gregory M. Kapfhammer. Software Testing. CRC Press
    Computer Science Handbook. June, 2004.
    http://cs.allegheny.edu/˜gkapfham/research/diatoms/
    Database drIven Application T esting tOol ModuleS, Allegheny Faculty Conference, May 18, 2004 – p. 23/23

    View full-size slide