A Family of Test Adequacy Criteria for Database-driven Applications

Interested in learning more about this topic? Please visit my research page for more details: https://www.gregorykapfhammer.com/research/

Transcript

1. A Family of Test Adequacy Criteria for Database-Driven Applications Gregory M. Kapfhammer Department of Computer Science University of Pittsburgh Department of Computer Science Allegheny College Mary Lou Soffa Department of Computer Science University of Pittsburgh Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 1/15
2. Motivation The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems About 350 airspace boundaries contained in Jeppesen NavData are incorrect, the FAA has warned. The error occurred at Jeppesen after a software upgrade when information was pulled from a database containing 20,000 airspace boundaries worldwide for the March NavData update, which takes effect March 20. Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 2/15
3. Testing Challenges Should consider the environment in which software applications execute Must test a program and its interaction with a database Database-driven application’s state space is well-structured, but infinite (Chays et al.) Need to show program does not violate database integrity, where integrity = consistency + validity (Motro) Must locate program and database coupling points that vary in granularity Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 3/15
4. Database-Driven Applications P Database Level 1 n D D A program can interact with a database at different levels of granularity Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15
5. Database-Driven Applications P D D 1 n UserInfo user_name 4 acct_lock 1 Brian Zorman 2 Robert Roos 3 card_number pin_number Geoffrey Arnold 0 0 0 0 32142 41601 45322 56471 Marcus Bittman Relation Level Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15
6. Database-Driven Applications P D D 1 n UserInfo user_name 4 acct_lock 1 Brian Zorman 2 Robert Roos 3 card_number pin_number Geoffrey Arnold 0 0 0 0 32142 41601 45322 56471 Marcus Bittman Record Level Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15
7. Database-Driven Applications P D D 1 n UserInfo 4 acct_lock 1 Brian Zorman 2 Robert Roos 3 card_number pin_number 0 0 0 0 32142 41601 45322 56471 user_name Attribute Level Marcus Bittman Geoffrey Arnold Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15
8. Database-Driven Applications P D D 1 n UserInfo 4 acct_lock 1 Brian Zorman 2 Robert Roos 3 card_number pin_number Geoffrey Arnold 0 0 0 0 32142 41601 45322 56471 user_name Attribute Value Level Marcus Bittman Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 4/15
9. Test Adequacy Criteria P violates a database Di ’s validity when it: (1-v) inserts entities into Di that do not reflect real world P violates a database Di ’s completeness when it: (1-c) deletes entities from Di that still reflect real world In order to verify (1-v) and (1-c), T must cause P to define and then use entities within D1 , . . . , Dn ! Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 5/15
10. Data Flow Information Interaction point: ‘‘UPDATE UserInfo SET acct lock=1’’ + ‘‘WHERE card number=’’ + c n + ‘‘;’’; Database Level: define(BankDB) Attribute Level: define(acct_lock) and use(card_number) Data flow information varies with respect to the granularity of the database interaction Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 6/15
11. Database Entities UserInfo user_name 4 acct_lock 1 Brian Zorman 2 Robert Roos 3 card_number pin_number Marcus Bittman Geoffrey Arnold 41601 45322 56471 32142 0 0 0 0 v r A (I ) = { 32142 } 1 Geoffrey Arnold 0 , , . . . , , Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 7/15
12. The DICFG: A Unified Representation entry lockAccount temp1 = parameter0:c_n temp2 = LocalDatabaseEntity0:Bank temp3 = LocalDatabaseEntity1:acct_lock temp4 = LocalDatabaseEntity2:card_number “Database-enhanced” CFG for lockAccount Define temporaries to represent the program’s interaction at the levels of database and attribute Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 8/15
13. The DICFG: A Unified Representation exit G G G G r r2 r 2 r1 1 entry entry exit lockAccount update_lock = m_connect.createStatement() if( result_lock == 1) completed = true exit D qu_lck = "UPDATE UserInfo ..." + temp1 + ";" use(temp4) result_lock = update_lock.executeUpdate(qu_lck) define(temp2) A Ir define(temp3) Database interaction graphs (DIGs) are placed before the database interaction point Ir Multiple DIGs can be integrated into a single CFG Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 8/15
14. Test Adequacy Criteria all−attribute−value−DUs all−record−DUs all−attribute−DUs all−relation−DUs all−database−DUs Database interaction association (DIA) involves the def and use of a database entity DIAs can be located in the DICFG with data flow analysis all-database-DUs requires tests to exercise all DIAs for all of the accessed databases Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 9/15
15. Generating Test Requirements Database Seeder Database (P) Test Adequacy Criterion (C) System Under Test Test Case Specification Relational Schema Requirements Test Measured time and space overhead when computing family of test adequacy criteria Modified ATM and mp3cd to contain appropriate database interaction points Soot 1.2.5 to calculate intraprocedural associations GNU/Linux workstation with kernel 2.4.18-smp and dual 1 GHz Pentium III Xeon processors Database drIven Application T esting tOol ModuleS, ESEC/FSE 2003, Helsinki, Finland, Sept 1 - 5 – p. 10/15
16. Counting Associations and Definitions D Rc Rl A Av Database Granularity 0 250 500 750 1000 1250 1500 1750 Assoc & Def Count D Rc Rl A Av mp3cd HD ATM HD mp3cd DB ATM DB DIAs at attribute value level represent 16.8% of mp3cd’s and 9.6% of ATM’s total number of intraprocedural associations – p. 11/15
17. Measuring Time Overhead None D Rc Rl A Av Database Granularity 22.5 25 27.5 30 32.5 35 37.5 System Time sec None D Rc Rl A Av Time Overhead mp3cd ATM Computing DIAs at the attribute value level incurs no more than a 5 second time overhead – p. 12/15
18. Measuring Average Space Overhead None D Rc Rl A Av Database Granularity 16 18 20 22 24 26 28 30 Node & Edge Count None D Rc Rl A Av Emp3 Eatm Nmp3 Natm mp3cd shows a more marked increase in the average number of nodes and edges than ATM – p. 13/15
19. Measuring Maximum Space Overhead None D Rc Rl A Av Database Granularity 200 400 600 800 1000 1200 1400 Node & Edge Count None D Rc Rl A Av Emp3 Eatm Nmp3 Natm mp3cd shows a significantly greater maximum space overhead than ATM – p. 14/15
20. Conclusions Must test the program’s interaction with the database Unique family of test adequacy criteria to detect type (1) violations of database validity and completeness Intraprocedural database interactions can be computed from a DICFG with minimal time and space overhead High number of hanging definitions indicates that the scope of data flow analysis could be broadened This data flow-based test adequacy criteria can serve as the foundation for algorithms that measure test suite quality, automatically generate test cases, and support regression testing http://cs.allegheny.edu/˜gkapfham/research/diatoms/ – p. 15/15