Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Family of Test Adequacy Criteria for Database-driven Applications

A Family of Test Adequacy Criteria for Database-driven Applications

Interested in learning more about this topic? Please visit my research page for more details: https://www.gregorykapfhammer.com/research/

Gregory Kapfhammer

September 04, 2003
Tweet

More Decks by Gregory Kapfhammer

Other Decks in Science

Transcript

  1. A Family of Test Adequacy Criteria for
    Database-Driven Applications
    Gregory M. Kapfhammer
    Department of Computer Science
    University of Pittsburgh
    Department of Computer Science
    Allegheny College
    Mary Lou Soffa
    Department of Computer Science
    University of Pittsburgh
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 1/15

    View full-size slide

  2. Motivation
    The Risks Digest, Volume 22, Issue 64, 2003
    Jeppesen reports airspace boundary problems
    About 350 airspace boundaries contained in Jeppesen
    NavData are incorrect, the FAA has warned. The error
    occurred at Jeppesen after a software upgrade when
    information was pulled from a database containing
    20,000 airspace boundaries worldwide for the March
    NavData update, which takes effect March 20.
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 2/15

    View full-size slide

  3. Testing Challenges
    Should consider the environment in which software
    applications execute
    Must test a program and its interaction with a database
    Database-driven application’s state space is
    well-structured, but infinite (Chays et al.)
    Need to show program does not violate database
    integrity, where integrity = consistency + validity (Motro)
    Must locate program and database coupling points that
    vary in granularity
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 3/15

    View full-size slide

  4. Database-Driven Applications
    P
    Database Level
    1
    n
    D
    D
    A program can interact with a
    database at different levels of
    granularity
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

    View full-size slide

  5. Database-Driven Applications
    P
    D
    D
    1
    n
    UserInfo
    user_name
    4
    acct_lock
    1 Brian Zorman
    2 Robert Roos
    3
    card_number pin_number
    Geoffrey Arnold
    0
    0
    0
    0
    32142
    41601
    45322
    56471
    Marcus Bittman
    Relation Level
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

    View full-size slide

  6. Database-Driven Applications
    P
    D
    D
    1
    n
    UserInfo
    user_name
    4
    acct_lock
    1 Brian Zorman
    2 Robert Roos
    3
    card_number pin_number
    Geoffrey Arnold
    0
    0
    0
    0
    32142
    41601
    45322
    56471
    Marcus Bittman
    Record Level
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

    View full-size slide

  7. Database-Driven Applications
    P
    D
    D
    1
    n
    UserInfo
    4
    acct_lock
    1 Brian Zorman
    2 Robert Roos
    3
    card_number pin_number
    0
    0
    0
    0
    32142
    41601
    45322
    56471
    user_name
    Attribute Level
    Marcus Bittman
    Geoffrey Arnold
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

    View full-size slide

  8. Database-Driven Applications
    P
    D
    D
    1
    n
    UserInfo
    4
    acct_lock
    1 Brian Zorman
    2 Robert Roos
    3
    card_number pin_number
    Geoffrey Arnold
    0
    0
    0
    0
    32142
    41601
    45322
    56471
    user_name
    Attribute Value Level
    Marcus Bittman
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15

    View full-size slide

  9. Test Adequacy Criteria
    P violates a database Di
    ’s validity when it:
    (1-v) inserts entities into Di
    that do not reflect real
    world
    P violates a database Di
    ’s completeness when it:
    (1-c) deletes entities from Di
    that still reflect real
    world
    In order to verify (1-v) and (1-c), T must cause P to
    define and then use entities within D1
    , . . . , Dn
    !
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 5/15

    View full-size slide

  10. Data Flow Information
    Interaction point: ‘‘UPDATE UserInfo SET
    acct lock=1’’ + ‘‘WHERE
    card number=’’ + c n + ‘‘;’’;
    Database Level: define(BankDB)
    Attribute Level: define(acct_lock) and
    use(card_number)
    Data flow information varies with respect to
    the granularity of the database interaction
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 6/15

    View full-size slide

  11. Database Entities
    UserInfo user_name
    4
    acct_lock
    1 Brian Zorman
    2 Robert Roos
    3
    card_number pin_number
    Marcus Bittman
    Geoffrey Arnold
    41601
    45322
    56471
    32142
    0
    0
    0
    0
    v r
    A (I ) = { 32142 }
    1 Geoffrey Arnold 0
    , , . . . , ,
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 7/15

    View full-size slide

  12. The DICFG: A Unified Representation
    entry lockAccount
    temp1 = parameter0:c_n
    temp2 = LocalDatabaseEntity0:Bank
    temp3 = LocalDatabaseEntity1:acct_lock
    temp4 = LocalDatabaseEntity2:card_number
    “Database-enhanced”
    CFG for lockAccount
    Define temporaries to
    represent the
    program’s interaction
    at the levels of
    database and attribute
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 8/15

    View full-size slide

  13. The DICFG: A Unified Representation
    exit
    G
    G G
    G
    r
    r2
    r 2
    r1
    1
    entry entry
    exit
    lockAccount
    update_lock = m_connect.createStatement()
    if( result_lock == 1)
    completed = true
    exit
    D
    qu_lck = "UPDATE UserInfo ..." + temp1 + ";"
    use(temp4)
    result_lock = update_lock.executeUpdate(qu_lck)
    define(temp2)
    A
    Ir
    define(temp3)
    Database interaction
    graphs (DIGs) are
    placed before the
    database interaction
    point Ir
    Multiple DIGs can be
    integrated into a single
    CFG
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 8/15

    View full-size slide

  14. Test Adequacy Criteria
    all−attribute−value−DUs
    all−record−DUs all−attribute−DUs
    all−relation−DUs all−database−DUs
    Database interaction
    association (DIA) involves the
    def and use of a database
    entity
    DIAs can be located in the
    DICFG with data flow analysis
    all-database-DUs requires
    tests to exercise all DIAs for all
    of the accessed databases
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 9/15

    View full-size slide

  15. Generating Test Requirements
    Database Seeder
    Database
    (P)
    Test Adequacy Criterion
    (C)
    System Under Test
    Test Case Specification
    Relational Schema
    Requirements
    Test
    Measured time and space overhead
    when computing family of test adequacy
    criteria
    Modified ATM and mp3cd to contain
    appropriate database interaction points
    Soot 1.2.5 to calculate intraprocedural
    associations
    GNU/Linux workstation with kernel
    2.4.18-smp and dual 1 GHz Pentium III
    Xeon processors
    Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 10/15

    View full-size slide

  16. Counting Associations and Definitions
    D Rc Rl A Av
    Database Granularity
    0
    250
    500
    750
    1000
    1250
    1500
    1750
    Assoc & Def Count
    D Rc Rl A Av
    mp3cd HD
    ATM HD
    mp3cd DB
    ATM DB
    DIAs at attribute value level represent 16.8% of mp3cd’s and
    9.6% of ATM’s total number of intraprocedural associations
    – p. 11/15

    View full-size slide

  17. Measuring Time Overhead
    None D Rc Rl A Av
    Database Granularity
    22.5
    25
    27.5
    30
    32.5
    35
    37.5
    System Time sec
    None D Rc Rl A Av
    Time Overhead
    mp3cd
    ATM
    Computing DIAs at the attribute value level incurs no more
    than a 5 second time overhead
    – p. 12/15

    View full-size slide

  18. Measuring Average Space Overhead
    None D Rc Rl A Av
    Database Granularity
    16
    18
    20
    22
    24
    26
    28
    30
    Node & Edge Count
    None D Rc Rl A Av
    Emp3
    Eatm
    Nmp3
    Natm
    mp3cd shows a more marked increase in the average
    number of nodes and edges than ATM – p. 13/15

    View full-size slide

  19. Measuring Maximum Space Overhead
    None D Rc Rl A Av
    Database Granularity
    200
    400
    600
    800
    1000
    1200
    1400
    Node & Edge Count
    None D Rc Rl A Av
    Emp3
    Eatm
    Nmp3
    Natm
    mp3cd shows a significantly greater maximum space
    overhead than ATM
    – p. 14/15

    View full-size slide

  20. Conclusions
    Must test the program’s interaction with the database
    Unique family of test adequacy criteria to detect type (1) violations
    of database validity and completeness
    Intraprocedural database interactions can be computed from a
    DICFG with minimal time and space overhead
    High number of hanging definitions indicates that the scope of
    data flow analysis could be broadened
    This data flow-based test adequacy criteria can serve as the
    foundation for algorithms that measure test suite quality,
    automatically generate test cases, and support regression testing
    http://cs.allegheny.edu/˜gkapfham/research/diatoms/
    – p. 15/15

    View full-size slide