Regression Testing Techniques for Relational Database Applications

Interested in learning more about this topic? Visit this web site to read a related post: https://www.gregorykapfhammer.com/testing/research/advice/2017/07/03/Regression-Testing-Costs/

Transcript

1. Regression Testing Techniques for Relational Database Applications Gregory M. Kapfhammer† Department of Computer Science Allegheny College http://www.cs.allegheny.edu/∼gkapfham/ University of Ulm – January 23, 2012 †Joint with Mary Lou Soffa (University of Virginia) and Jonathan Miller Kauffman (Allegheny College)
2. Introduction Database Applications Regression Testing Conclusion Important Points Presenter Introduction: Gregory M. Kapfhammer test testing software suites prioritization components coverage empirically regression suite applications evaluating algorithm analysis data effectiveness empirical genetic performance understanding using approach commercialofftheshelf comparison comprehensive constrained creation databasecentric environments execution finding framework identifying interactive java methods multiplots mutation party prioritized reduction relational study third timeaware towards 105 adequacy approaches array automatically building call challenges chapter communication compare compressing computer conditional cost cots covering creating criteria database databaseaware databasedriven databases declarative dependable detection devices distributed distributing duringsearchbased dynamic efficiency efficient engineering environment evaluate examination executing experimental family flow forward frameworks free generation greedy gui hamiltonian handbook heaps implementation improve incorporating increase information initial intranode invariant javaspace javaspacebased kernel knapsack linux measurement memory method monitoring operators paths poster potential preliminary primitives prioritizations prioritizers prioritizing problematic receive remote reports resource resourceconstrained results role runtime science searchbased selection solutions solvers space studies studying supported synthetic techniques transmission transparently trees tuple unstructured wrappers Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
3. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
4. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
5. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
6. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
7. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
8. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
9. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
10. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
11. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program Program Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
12. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program Program Program Program Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
13. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program Program Program Program Program Program Program Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
14. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Everywhere Computer Server Program Computer Server Program Desktop Computer Program Desktop Computer Program Mobile Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Household Appliance Program Network Router Program Scientific Device Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Program Program Program Program Program Program Program Program Computer Server Program Desktop Computer Program Mobile Computer Program Household Appliance Program Scientific Device Program Network Router Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
15. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software Complexity and Data Enormity Computer Software Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
16. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software Complexity and Data Enormity Computer Software Lines of Code Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
17. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software Complexity and Data Enormity Computer Software Lines of Code Numerous Features Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
18. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software Complexity and Data Enormity Computer Software Lines of Code Numerous Features Feature Interactions Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
19. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software Complexity and Data Enormity Computer Software Lines of Code Numerous Features Feature Interactions Execution Environments Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
20. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software Complexity and Data Enormity Computer Software Lines of Code Numerous Features Feature Interactions Execution Environments Software entities are more complex for their size than per- haps any other human construct - Frederick P. Brooks, Jr. Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
21. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software Complexity and Data Enormity Computer Software Lines of Code Numerous Features Feature Interactions Execution Environments Software entities are more complex for their size than per- haps any other human construct - Frederick P. Brooks, Jr. Prediction: in 2011, 1.8 zettabytes (i.e., 1.8 trillion giga- bytes) of data will be created - IDC Digital Universe Study Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
22. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Evolving Execution Environment Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
23. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Evolving Execution Environment Program Execution Environment Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
24. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Evolving Execution Environment Program Execution Environment Program Program Changed because of the addition of a new feature or the correction of a defect Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
25. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Evolving Execution Environment Program Execution Environment Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
26. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Evolving Execution Environment Program Execution Environment Program Execution Environment Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
27. Introduction Database Applications Regression Testing Conclusion Software and Data Challenges Software and Data are Evolving Execution Environment Program Execution Environment Program Execution Environment Program Execution Environment Changed due to modifica- tion of a kernel, device driver, or relational database Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
28. Introduction Database Applications Regression Testing Conclusion Relational Database Challenges An Interesting Defect Report Database Server Crashes Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
29. Introduction Database Applications Regression Testing Conclusion Relational Database Challenges An Interesting Defect Report Database Server Crashes When you run a complex query against Microsoft SQL Server 2000, the SQL Server scheduler may stop respond- ing. Additionally, you receive an error message that resem- bles the following: Date Time server Error: 17883 Sever- ity: 1, State: 0 Date Time server Process 52:0 (94c) ... Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
30. Introduction Database Applications Regression Testing Conclusion Relational Database Challenges An Interesting Defect Report Input-Dependent Defect Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
31. Introduction Database Applications Regression Testing Conclusion Relational Database Challenges An Interesting Defect Report Input-Dependent Defect This problem occurs when one or more of the following con- ditions are true: The query contains a UNION clause or a UNIONALL clause that affects many columns. The query contains several JOIN statements. The query has a large estimated cost. BUG 473858 (SQL Server 8.0) Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
32. Introduction Database Applications Regression Testing Conclusion Relational Database Challenges Real-World Defective Database Application The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems About 350 airspace boundaries contained in Jeppesen Nav- Data are incorrect, the FAA has warned. The error occurred at Jeppesen after a software upgrade when information was pulled from a database containing 20,000 airspace bound- aries worldwide for the March NavData update, which takes effect March 20. Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
33. Introduction Database Applications Regression Testing Conclusion Relational Database Challenges Real-World Defective Database Application The Risks Digest, Volume 22, Issue 64, 2003 Jeppesen reports airspace boundary problems About 350 airspace boundaries contained in Jeppesen Nav- Data are incorrect, the FAA has warned. The error occurred at Jeppesen after a software upgrade when information was pulled from a database containing 20,000 airspace bound- aries worldwide for the March NavData update, which takes effect March 20. Practically all use of databases occurs from within applica- tion programs [Silberschatz et al., 2006, pg. 311] Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
34. Introduction Database Applications Regression Testing Conclusion Relational Databases Structured Query Language The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
35. Introduction Database Applications Regression Testing Conclusion Relational Databases Structured Query Language The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) A schema is a collection of table definitions: CREATE TABLE person ( id INT, name VARCHAR(100) NOT NULL, age INT(3), PRIMARY KEY (id) ) Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
36. Introduction Database Applications Regression Testing Conclusion Relational Databases Structured Query Language The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) The data manipulation language supports several operations: SELECT name FROM person WHERE age >= 30 AND age <= 40 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
37. Introduction Database Applications Regression Testing Conclusion Relational Databases Structured Query Language The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) The data manipulation language supports several operations: UPDATE person SET name = Jan WHERE id = 2 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
38. Introduction Database Applications Regression Testing Conclusion Relational Databases Structured Query Language The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) The data manipulation language supports several operations: INSERT INTO person (id, name, age) VALUES (1, John, 38) Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
39. Introduction Database Applications Regression Testing Conclusion Relational Databases Structured Query Language The structured query language (SQL) is an established stan- dard and a query and manipulation language for relational database management systems (RDBMS) The data manipulation language supports several operations: DELETE FROM person WHERE id = 2 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
40. Introduction Database Applications Regression Testing Conclusion Relational Databases Relational Database Tables id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
41. Introduction Database Applications Regression Testing Conclusion Relational Databases Relational Database Tables id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
42. Introduction Database Applications Regression Testing Conclusion Relational Databases Relational Database Tables id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
43. Introduction Database Applications Regression Testing Conclusion Relational Databases Relational Database Tables id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
44. Introduction Database Applications Regression Testing Conclusion Relational Databases Relational Database Tables id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
45. Introduction Database Applications Regression Testing Conclusion Relational Databases Relational Database Tables id name age 1 Chalker Conrad 12 2 Abby Clulow 14 3 David Rogan 18 4 Stacie Reckling 32 5 Megan Hartnup 29 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
46. Introduction Database Applications Regression Testing Conclusion Programs and Databases Database Applications Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
47. Introduction Database Applications Regression Testing Conclusion Programs and Databases Database Applications Program Relational Database Management System Database State Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
48. Introduction Database Applications Regression Testing Conclusion Programs and Databases Database Applications Program Relational Database Management System Relational Database Management System select Database State query Data Manipulation Language (DML) Statements Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
49. Introduction Database Applications Regression Testing Conclusion Programs and Databases Database Applications Program Relational Database Management System Relational Database Management System update Database State modify Data Manipulation Language (DML) Statements Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
50. Introduction Database Applications Regression Testing Conclusion Programs and Databases Database Applications Program Relational Database Management System Data Manipulation Language (DML) Statements Relational Database Management System insert Database State modify Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
51. Introduction Database Applications Regression Testing Conclusion Programs and Databases Database Applications Program Relational Database Management System Data Manipulation Language (DML) Statements Relational Database Management System delete Database State modify Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
52. Introduction Database Applications Regression Testing Conclusion Programs and Databases Database Applications Program Relational Database Management System Relational Database Management System create table Database Structure modify Data Definition Language (DDL) Statements Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
53. Introduction Database Applications Regression Testing Conclusion Programs and Databases Database Applications Program Relational Database Management System Relational Database Management System create table Database Structure modify Data Definition Language (DDL) Statements Can we categorize the dif- ferent ways of implementing database applications? Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
54. Introduction Database Applications Regression Testing Conclusion Programs and Databases Categorizing Database Applications Database Applications Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
55. Introduction Database Applications Regression Testing Conclusion Programs and Databases Categorizing Database Applications Database Applications Interaction Approach Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
56. Introduction Database Applications Regression Testing Conclusion Programs and Databases Categorizing Database Applications Database Applications Interaction Approach Program Location Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
57. Introduction Database Applications Regression Testing Conclusion Programs and Databases Categorizing Database Applications Database Applications Interaction Approach Program Location Interface Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
58. Introduction Database Applications Regression Testing Conclusion Programs and Databases Categorizing Database Applications Database Applications Interaction Approach Program Location Interface Embedded Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
59. Introduction Database Applications Regression Testing Conclusion Programs and Databases Categorizing Database Applications Database Applications Interaction Approach Program Location Interface Embedded Outside RDBMS Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
60. Introduction Database Applications Regression Testing Conclusion Programs and Databases Categorizing Database Applications Database Applications Interaction Approach Program Location Interface Embedded Outside RDBMS Inside RDBMS Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
61. Introduction Database Applications Regression Testing Conclusion Programs and Databases Categorizing Database Applications Database Applications Interaction Approach Program Location Interface Embedded Outside RDBMS Inside RDBMS Interface Outside RDBMS Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
62. Introduction Database Applications Regression Testing Conclusion Programs and Databases Categorizing Database Applications Database Applications Interaction Approach Program Location Interface Embedded Outside RDBMS Inside RDBMS Interface Outside RDBMS Java application that submits SQL strings to HSQLDB using JDBC Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
63. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
64. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
65. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Only the database administrator can add new constraints to the schema! Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
66. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure The programmers encode the constraints in the program’s source code! Constraint Ci Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
67. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure The programmers encode the constraints in the program’s source code! Constraint Ci Constraint Cj Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
68. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure The programmers encode the constraints in the program’s source code! Constraint Ci Constraint Cj Constraint Ck Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
69. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Constraint Ci Constraint Cj Constraint Ck Programmers make other changes to the source code of the program Program Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
70. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Constraint Ci Constraint Cj Constraint Ck External programs can change the state of the relational database Database State Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
71. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Constraint Ci Constraint Cj Constraint Ck Database administrator can change the structure of the database Database Structure Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
72. Introduction Database Applications Regression Testing Conclusion Programs and Databases Evolution of Database Applications Program Relational Database Management System Relational Database Management System DML Command Database State Database Structure Constraint Ci Constraint Cj Constraint Ck How can we test a rapidly changing database application? Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
73. Introduction Database Applications Regression Testing Conclusion Important Techniques Regression Testing to the Rescue Database Applications Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
74. Introduction Database Applications Regression Testing Conclusion Important Techniques Regression Testing to the Rescue Database Applications Pervasive Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
75. Introduction Database Applications Regression Testing Conclusion Important Techniques Regression Testing to the Rescue Database Applications Pervasive Complex Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
76. Introduction Database Applications Regression Testing Conclusion Important Techniques Regression Testing to the Rescue Database Applications Pervasive Complex Evolving Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
77. Introduction Database Applications Regression Testing Conclusion Important Techniques Regression Testing to the Rescue Database Applications Pervasive Complex Evolving Regression Testing supports the efficient construction of database software that is complex and rapidly evolving Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
78. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
79. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Input Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
80. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Input Output Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
81. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
82. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
83. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Output Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
84. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
85. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
86. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
87. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
88. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
89. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Test Verdict The test case passes and the code is correct! Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
90. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
91. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Test Verdict The test case fails and a defect is found! Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
92. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
93. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
94. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 T3 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
95. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 T3 T4 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
96. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 T3 T4 . . . Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
97. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 T3 T4 . . . Tn Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
98. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
99. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
100. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? JUnit Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
101. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? JUnit Apache Ant Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
102. Introduction Database Applications Regression Testing Conclusion Important Techniques What is a Test Suite? T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Tool Support for Software Testing? JUnit Apache Ant DBUnit Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
103. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Organize the Test Cases into a Test Suite Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
104. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Regression Testing Technique Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
105. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Regression Testing Technique What if Some Test Cases are More Effective? T3 Tn Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
106. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Regression Testing Technique What if Some Test Cases are More Effective? T3 Tn Prioritization T3 Tn T1 T4 . . . T2 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
107. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Regression Testing Technique What if Some Test Cases are More Effective? T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
108. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn What if Some Test Cases are Redundant? T1 T2 T3 T4 . . . Tn Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
109. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn What if Some Test Cases are Redundant? T1 T2 T3 T4 . . . Tn Reduction T4 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
110. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn What if Some Test Cases are Redundant? T1 T2 T3 T4 . . . Tn Reduction T4 T1 T2 T3 T4 . . . Tn Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
111. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn T1 T2 T3 T4 . . . Tn Reduction T4 T1 T2 T3 T4 . . . Tn What if Only Certain Tests are Needed? T2 Tn Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
112. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Management T1 T2 T3 T4 . . . Tn Regression Testing Technique T3 Tn Prioritization T3 Tn T1 T4 . . . T2 T1 T2 T3 T4 . . . Tn T1 T2 T3 T4 . . . Tn Reduction T4 T1 T2 T3 T4 . . . Tn What if Only Certain Tests are Needed? T2 Tn Selection T1 T2 T3 T4 . . . Tn Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
113. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
114. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
115. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
116. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
117. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
118. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
119. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
120. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
121. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
122. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Use the Coverage Report During the Next Round of Regression Testing Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
123. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Use the Same Test Suite for the Next Round of Regression Testing Standard Repeat Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
124. Introduction Database Applications Regression Testing Conclusion Important Techniques Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Standard Repeat Make a New Test Suite for the Next Round of Regression Testing Version Specific Repeat Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
125. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
126. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
127. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
128. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
129. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
130. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
131. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
132. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
133. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
134. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
135. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
136. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
137. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
138. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} R12 R12 R12 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
139. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} R12 R12 R12 R12 R12 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
140. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Adequacy T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
141. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
142. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
143. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
144. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 Run Test Case Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
145. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 Passing Test Case: OA = OE Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
146. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
147. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 T2 T3 T4 T5 T6 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
148. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 T2 T3 T4 T5 T6 T7 Run Test Case Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
149. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 T2 T3 T4 T5 T6 T7 T7 Failing Test Case: OA = OE Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
150. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 T2 T3 T4 T5 T6 T7 T7 Failing Test Case: OA = OE Stop Running T Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
151. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 T2 T3 T4 T5 T6 T7 T7 Failing Test Case: OA = OE Stop Running T T8 T9 T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
152. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 T2 T3 T4 T5 T6 T7 T7 Failing Test Case: OA = OE Stop Running T T8 T9 T10 T8 T9 T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
153. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 T2 T3 T4 T5 T6 T7 T7 Failing Test Case: OA = OE Stop Running T T8 T9 T10 T8 T9 T10 Continue Running T Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
154. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Suite Execution T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks T1 T1 T2 T3 T4 T5 T6 T7 T7 Failing Test Case: OA = OE Stop Running T T8 T9 T10 T8 T9 T10 Continue Running T T8 T9 T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
155. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
156. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
157. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks Database-Aware Test Coverage Monitor Proteja Test Suite Manager Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
158. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 JUnit and DBUnit Test Automation Frameworks Database-Aware Test Coverage Monitor Proteja Test Suite Manager T1 Run Test Case Collect Per-Test Case Coverage Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
159. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
160. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
161. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
162. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 Requirements Set R for ... Statement Coverage Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
163. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 Requirements Set R for ... Database Interaction Coverage Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
164. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 Requirements Set R for ... Database Table Coverage Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
165. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 Requirements Set R for ... Database Record Coverage Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
166. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 Requirements Set R for ... Database Attribute Coverage Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
167. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 Requirements Set R for ... Database Attribute-Value Coverage Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
168. Introduction Database Applications Regression Testing Conclusion Important Techniques Test Coverage Monitoring T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 See [Kapfhammer and Soffa, ISEC 2008] for more details Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
169. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
170. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
171. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
172. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
173. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} R12 R12 R12 R12 R12 R12 R12 R12 R12 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
174. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
175. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} R12 ratio(T1) = cost(T1) coverage(T1) Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
176. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} R12 ratio(T1) = 5 3 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
177. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} R12 ratio(T1) = 1.66 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
178. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} 1.66 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
179. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} 1.66 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
180. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} 1.66 R12 ratio(T5) = cost(T5) coverage(T5) Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
181. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} 1.66 R12 ratio(T5) = 8 3 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
182. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} 1.66 R12 ratio(T5) = 2.66 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
183. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} 1.66 2.66 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
184. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} 1.66 2.66 ratio(T1) < ratio(T5) Prefer T1 over T5 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
185. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} 1.66 2.66 Proceed incrementally, picking the test case with the lowest ratio value for the uncovered requirements Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
186. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 Test Suite T = T8, T4, T9, T1, T10, T3, T7, T2, T6, T5 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
187. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
188. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
189. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
190. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
191. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
192. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
193. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
194. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
195. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
196. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 R12 1 2 3 5 2 4 4 6 10 8 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
197. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 T2 T6 T5 Test Suite T = T8, T4, T9, T1, T10, T3, T7 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
198. Introduction Database Applications Regression Testing Conclusion Important Techniques Greedy Algorithms T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 Requirements Set R = {R1, R2, . . . , R11, R12} T8 T4 T9 T1 T10 T3 T7 T2 T6 T5 T2 T6 T5 Test Suite T = T8, T4, T9, T1, T10, T3, T7 R12 R12 R12 R12 R12 R12 R12 Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
199. Introduction Database Applications Regression Testing Conclusion Empirical Results Empirical Results – Test Suite Reduction Program Rel. Attr. Rec. Attr. Val. All RM (13) (7, .46) (7, .46) (10, .30) (9, .31) (8.25, .37) FF (16) (7, .56) (7, .56) (11, .31) (11, .31) (9, .44) PI (15) (6, .60) (6, .60) (8, .70) (7, .53) (6.75, .55) ST (25) (5, .80) (5, .76) (11, .56) (10, .60) (7.75, .690) TM (27) (14, .48) (14, .48) (15, .45) (14, .48) (14.25, .47) GB (51) (33, .35) (33, .35) (33, .35) (32, .37) (32.75, .36) All (24.5) (12, .51) (12.17, .50) (14.67, .40) (13.83, .44) • Reduction values range from .30 to .80 • Reduction level varies depending on interaction granularity • How will the reduction of a test suite impact defect detection? Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
200. Introduction Database Applications Regression Testing Conclusion Empirical Results Empirical Results – Test Suite Reduction Program Rel. Attr. Rec. Attr. Val. All RM (13) (7, .46) (7, .46) (10, .30) (9, .31) (8.25, .37) FF (16) (7, .56) (7, .56) (11, .31) (11, .31) (9, .44) PI (15) (6, .60) (6, .60) (8, .70) (7, .53) (6.75, .55) ST (25) (5, .80) (5, .76) (11, .56) (10, .60) (7.75, .690) TM (27) (14, .48) (14, .48) (15, .45) (14, .48) (14.25, .47) GB (51) (33, .35) (33, .35) (33, .35) (32, .37) (32.75, .36) All (24.5) (12, .51) (12.17, .50) (14.67, .40) (13.83, .44) • Reduction values range from .30 to .80 • Reduction level varies depending on interaction granularity • How will the reduction of a test suite impact defect detection? Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
201. Introduction Database Applications Regression Testing Conclusion Final Remarks Conclusion Conclusion • Databases are widely used in real-world applications • Database applications have complex state and structure • Source code, database state, and relational schema evolve • Prioritization techniques can increase effectiveness • Reduction methods can improve the efficiency of testing Future Work • New empirical studies of database-aware regression testing • Implement and release free and open source testing tools Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
202. Introduction Database Applications Regression Testing Conclusion Final Remarks Conclusion Conclusion • Databases are widely used in real-world applications • Database applications have complex state and structure • Source code, database state, and relational schema evolve • Prioritization techniques can increase effectiveness • Reduction methods can improve the efficiency of testing Future Work • New empirical studies of database-aware regression testing • Implement and release free and open source testing tools Kapfhammer Allegheny College Regression Testing Techniques for Relational Database Applications
203. Regression Testing Techniques for Relational Database Applications Gregory M. Kapfhammer Department of Computer Science Allegheny College http://www.cs.allegheny.edu/∼gkapfham/ Thank you for your attention! I welcome your questions and comments.