mrstudyr: Retrospectively studying the effectiveness of mutant reduction techniques

Interested in learning more about this topic? Visit this web site to read the paper: https://www.gregorykapfhammer.com/research/papers/McCurdy2016/

Transcript

1. mrstudyr Retrospective Mutant Reduction Colton J. McCurdy McCurdyColton ICSME 2016 Gregory M. Kapfhammer and Phil McMinn
2. Motivation Quality software?
3. Motivation Regression Test Suite
4. Motivation Regression Test Suite T= t1 , t2 , . . . , tn
5. Motivation Pass Fail
6. Motivation Pass Fail
7. Motivation Pass Fail
8. Motivation Use testing to assess quality?
9. Motivation Trivial test suite?
10. Background Mutation Testing!
11. Mutation Testing Original Mutant
12. Mutation Testing Original Mutant
13. Mutation Testing a > b
14. Mutation Testing a > b Relational Operator Replacement (ROR)
15. Mutation Testing a > b Relational Operator Replacement (ROR) a ≥ b a ≤ b a == b a != b
16. Mutation Testing Alive Killed
17. Mutation Testing Alive Killed
18. Mutation Testing Alive Killed Killed
19. Mutation Testing MST = Killed Total
20. Mutation Testing MST = Killed Total
21. Mutation Testing MST = Killed Total
22. Mutation Testing MST = Killed Total MST ∈ [0, 1]
23. Mutation Testing MST = Killed Total MST ∈ [0, 1] HIB
24. Mutation Testing Silver Bullet?
25. Mutation Testing Limitations
26. Mutation Testing
27. Mutation Testing T= t1 , t2 , . . . , tn
28. Mutation Testing T= t1 , t2 , . . . , tn Weak
29. Mutation Testing T= t1 , t2 , . . . , tn Weak Strong
30. Mutation Testing T= t1 , t2 , . . . , tn Weak Strong Redundant
31. Mutation Testing T= t1 , t2 , . . . , tn Weak Strong Redundant
32. Mutation Testing Still . . .
33. Mutation Testing Still . . . limitations
34. Mutation Testing
35. Mutation Testing Understand system intricacies
36. Mutation Testing Understand system intricacies
37. Mutation Testing Retrospective Mutant Reduction
38. Mutation Testing Retrospective Mutant Reduction
39. mrstudyr  
40. mrstudyr   mccurdyc/mrstudyr
41. mrstudyr   mccurdyc/mrstudyr Data
42. mrstudyr   mccurdyc/mrstudyr Reduce Data
43. mrstudyr   mccurdyc/mrstudyr Reduce Data Remove
44. Input Database Schema Mutants
45. Demonstration Demonstration!
46. Motivation Random Sampling
47. Output Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90 2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00
48. Output Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90 2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00
49. Output Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90 2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00
50. Output Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90 2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00
51. Conclusion Program Quality
52. Conclusion Program Quality Regression Test Suite
53. Conclusion Program Quality Regression Test Suite Mutation Testing
54. Conclusion Program Quality Regression Test Suite Mutation Testing Reduction Techniques
55. Conclusion Program Quality Regression Test Suite Mutation Testing Reduction Techniques mrstudyr
56. Conclusion Reduced
57. Conclusion Reduced 10%
58. Conclusion Reduced 10%