Introduction Resource-Constrained Testing Conclusions and Future Work Prioritizing Test Suites by Finding Hamiltonian Paths: Preliminary Studies and Initial Results Suvarshi Bhadra and Gregory M. Kapfhammer Department of Computer Science Allegheny College, Pennsylvania, USA http://www.cs.allegheny.edu/~gkapfham/ TAIC PART 2008, Fast Abstract Track, August 2008 Featuring images from www.pdclipart.org 1 / 10 Prioritizing Test Suites by Finding Hamiltonian Paths: Preliminary Studies and Initial Results
Introduction Resource-Constrained Testing Conclusions and Future Work Regression Testing Techniques Before After Reduction Prunes the Test Suite Before After Prioritization Reorders the Tests It may expensive to run a test suite T = T1, . . . , Tn . Prioritization searches through the n! = n × n − 1 × . . . × 1 orderings for those that avoid costly database restarts, Web service calls, or memory interactions. 3 / 10 Prioritizing Test Suites by Finding Hamiltonian Paths: Preliminary Studies and Initial Results
Introduction Resource-Constrained Testing Conclusions and Future Work Prioritizing When Memory is Constrained ag replacements N0 N1 N2 N3 N4 N5 Frequent Memory Rewrites PSfrag replacements N0 N1 N2 N3 N4 N5 High Testing Costs Frequent reads and writes to memory may increase execution time by as much as 600% when a Java application executes on a virtual machine with a small heap Solution: maximize memory reuse between test cases 4 / 10 Prioritizing Test Suites by Finding Hamiltonian Paths: Preliminary Studies and Initial Results
Introduction Resource-Constrained Testing Conclusions and Future Work The Impact of Test Ordering m1 m2 m3 m4 m5 m6 Test 30 30 30 30 30 30 Size T1 • • • 90 T2 • • • 90 T3 • • • 90 T4 • • • 90 T5 • • 60 T = T1, T2, T3, T4, T5 transfers 750 units to and from memory T = T2, T4, T1, T3, T5 only loads and unloads 180 units 5 / 10 Prioritizing Test Suites by Finding Hamiltonian Paths: Preliminary Studies and Initial Results
Introduction Resource-Constrained Testing Conclusions and Future Work Test Prioritization: Steps One and Two Program Instrumentation Adequacy Criterion Residual Test Requirements Cummulative Adequacy Calculator Instrumented Program Test Suite Execution Covered Requirements ents N0 N1 N2 N3 N4 N5 Collect method invocation and size data using test coverage monitor and profiler 553 256 553 256 256 0 256 553 256 256 O 553 553 256 256 234 553 256 256 256 678 553 256 553 234 234 553 256 553 256 PSfrag replacements N0 N1 N2 N3 N4 N5 Formulate a complete graph using equations that estimate costs for all test pairs 6 / 10 Prioritizing Test Suites by Finding Hamiltonian Paths: Preliminary Studies and Initial Results
Introduction Resource-Constrained Testing Conclusions and Future Work Test Prioritization: Steps Three and Four ements N0 N1 N2 N3 N4 N5 Use TSP solvers to identify a Hamiltonian path with low estimated costs 10 20 30 40 50 60 70 80 90 100 1.0 1.5 2.0 2.5 Size of the Test Suite (n) Execution Time (sec) PSfrag replacements N0 N1 N2 N3 N4 N5 Evaluate the efficiency of the TSP solvers and the effectiveness of the test orderings 7 / 10 Prioritizing Test Suites by Finding Hamiltonian Paths: Preliminary Studies and Initial Results
Introduction Resource-Constrained Testing Conclusions and Future Work Empirical Results 0 200 400 600 800 1000 0.00 0.05 0.10 0.15 0.20 NN RNN NI FI CI AI Time v No of Test Cases (n) (t) ments N0 N1 N2 N3 N4 N5 Efficient Prioritizers 10 20 30 40 50 50 60 70 80 90 100 Score v No of Test Cases − NN (n) (score) PSfrag replacements N0 N1 N2 N3 N4 N5 High Percentile Rankings 8 / 10 Prioritizing Test Suites by Finding Hamiltonian Paths: Preliminary Studies and Initial Results
gkapfham
masakat0
shuntaros
kuri8ive
kyoun
kuanyi_n_n
mhangyo
sumauma
.png){kind=avatar}
researchconf
johannyjm
stktu
signer
misonuma
philnash
paigeccino
jmmastey
shlominoach
zenorocha
revolveconf
62gerente
danielanewman
csswizardry
keathley
samlambert
chriscoyier