Empirically evaluating regression testing techniques: Challenges, solutions, and a potential way forward

Interested in learning more about this topic? Visit this web site to read the paper: https://www.gregorykapfhammer.com/research/papers/Kapfhammer2011/

Transcript

1. Introduction Challenges Potential Way Forward Conclusion Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward Gregory M. Kapfhammer Department of Computer Science Allegheny College http://www.cs.allegheny.edu/~gkapfham/ 1st International Workshop on Regression Testing Co-Located with the 4th IEEE International Conference on Software Testing, Verification and Validation Berlin, Germany, March 2011 1 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
2. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
3. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
4. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends Increased Interest 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
5. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends Increased Interest Software Testing Empirical Studies 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
6. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends Increased Interest Software Testing Empirical Studies Techniques 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
7. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends Increased Interest Software Testing Empirical Studies Techniques Challenges 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
8. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends Increased Interest Software Testing Empirical Studies Techniques Challenges Solutions 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
9. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends Increased Interest Software Testing Empirical Studies Techniques Challenges Solutions Statistics 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
10. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends Increased Interest Software Testing Empirical Studies Techniques Challenges Solutions Statistics Medicine 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
11. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends Increased Interest Software Testing Empirical Studies Techniques Challenges Solutions Statistics Medicine Social Sciences 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
12. Introduction Challenges Potential Way Forward Conclusion Presentation Overview Regression Testing Community Current Trends Increased Interest Software Testing Empirical Studies Techniques Challenges Solutions Statistics Medicine Social Sciences Mutually Beneficial Sharing of All Artifacts Used in Experimentation 2 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
13. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
14. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
15. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
16. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
17. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
18. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
19. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
20. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
21. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
22. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Use the Coverage Report During the Next Round of Regression Testing 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
23. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Use the Same Test Suite for the Next Round of Regression Testing Standard Repeat 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
24. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Standard Repeat Make a New Test Suite for the Next Round of Regression Testing Version Specific Repeat 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
25. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Standard Repeat Version Specific Repeat Practitioners are unwilling to use methods for “lack of empirical studies” [12] 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
26. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Standard Repeat Version Specific Repeat Practitioners are unwilling to use methods for “lack of empirical studies” [12] Testing tools do not produce outputs in the best format (e.g., per-test coverage) 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
27. Introduction Challenges Potential Way Forward Conclusion Model of Regression Testing Start Coverage Report Selection Reduction Prioritization Original Test Suite Modified Test Suite Test Suite Execution Test Coverage Monitoring Program Adequacy Criterion Test Results End Standard Repeat Version Specific Repeat Practitioners are unwilling to use methods for “lack of empirical studies” [12] Experimental assessment could stagnate due to inaccessibility of artifacts 3 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
28. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
29. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
30. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
31. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
32. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations Data Sets 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
33. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations Data Sets Visualization Statistical Analysis 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
34. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations Data Sets Visualization Statistical Analysis Graphs Diagrams Summarized Data Sets Models 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
35. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations Data Sets Visualization Statistical Analysis Graphs Diagrams Summarized Data Sets Models Iteratively Perform Visualization and Statistical Analysis Repeat 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
36. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations Data Sets Visualization Statistical Analysis Graphs Diagrams Summarized Data Sets Models Iteratively Perform Visualization and Statistical Analysis Repeat End Conduct Experiments with Additional Programs, Test Suites, and Techniques Repeat 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
37. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations Data Sets Visualization Statistical Analysis Graphs Diagrams Summarized Data Sets Models Repeat End Conduct Experiments with Additional Programs, Test Suites, and Techniques Repeat 56% of Papers Surveyed by Yoo and Harman Only Used the SIR Programs [3] 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
38. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations Data Sets Visualization Statistical Analysis Graphs Diagrams Summarized Data Sets Models Repeat End Conduct Experiments with Additional Programs, Test Suites, and Techniques Repeat Few Papers Report on the Efficiency of the Regression Testing Techniques 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
39. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations Data Sets Visualization Statistical Analysis Graphs Diagrams Summarized Data Sets Models Repeat End Conduct Experiments with Additional Programs, Test Suites, and Techniques Repeat Authors Do Not Release Tools That Conduct Experiments and Analyze Results 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
40. Introduction Challenges Potential Way Forward Conclusion Model of Experimental Evaluation Start Programs Test Suites Conduct Experiments Regression Testing Techniques Technique Configurations Data Sets Visualization Statistical Analysis Graphs Diagrams Summarized Data Sets Models Repeat End Conduct Experiments with Additional Programs, Test Suites, and Techniques Repeat Without Using Data Mining Methods, Researchers May Miss Important Trends 4 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
41. Introduction Challenges Potential Way Forward Conclusion Thought Provoking Assertions Jonathan B. Buckheit and David L. Donoho Department of Statistics Stanford University 5 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
42. Introduction Challenges Potential Way Forward Conclusion Thought Provoking Assertions Jonathan B. Buckheit and David L. Donoho Department of Statistics Stanford University For a field to qualify as a sci- ence, it is important first and foremost that published work be reproducible by others. 5 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
43. Introduction Challenges Potential Way Forward Conclusion Thought Provoking Assertions Jonathan B. Buckheit and David L. Donoho Department of Statistics Stanford University For a field to qualify as a sci- ence, it is important first and foremost that published work be reproducible by others. Does the Regression Testing Community Want to be Scientific? 5 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
44. Introduction Challenges Potential Way Forward Conclusion Thought Provoking Assertions Jonathan B. Buckheit and David L. Donoho Department of Statistics Stanford University For a field to qualify as a sci- ence, it is important first and foremost that published work be reproducible by others. Does the Regression Testing Community Want to be Scientific? What Does it Mean for Re- search to be Reproducible? 5 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
45. Introduction Challenges Potential Way Forward Conclusion Thought Provoking Assertions Gary King Department of Govern- ment Harvard University 5 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
46. Introduction Challenges Potential Way Forward Conclusion Thought Provoking Assertions Gary King Department of Govern- ment Harvard University Reproducible: sufficient infor- mation exists with which to under- stand, evaluate, and build upon a prior work if a third party can replicate the results without any ad- ditional information from the author. 5 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
47. Introduction Challenges Potential Way Forward Conclusion Thought Provoking Assertions Gary King Department of Govern- ment Harvard University Reproducible: sufficient infor- mation exists with which to under- stand, evaluate, and build upon a prior work if a third party can replicate the results without any ad- ditional information from the author. Can the Regression Testing Com- munity Adhere to This Standard? 5 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
48. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
49. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
50. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
51. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
52. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
53. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
54. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
55. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
56. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control Deposit the Artifacts from Experimentation in One or More Repositories 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
57. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control External Researchers External Researchers Access the Repositories 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
58. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control Internal Researchers Internal Researchers Access the Repositories 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
59. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control Internal Researchers Internal Researchers Access the Repositories Costs of Reproducible Research? 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
60. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control Internal Researchers Internal Researchers Access the Repositories Costs of Reproducible Research? Benefits of Reproducible Research? 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
61. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control Internal Researchers Internal Researchers Access the Repositories Costs of Reproducible Research? Benefits of Reproducible Research? Increase the Citation of Your Paper [17] 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
62. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control Internal Researchers Internal Researchers Access the Repositories Costs of Reproducible Research? Benefits of Reproducible Research? Allow Others to Replicate Your Experiments [15] 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
63. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control Internal Researchers Internal Researchers Access the Repositories Costs of Reproducible Research? Benefits of Reproducible Research? Lower the Barriers to Entry for New Researchers [16] 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
64. Introduction Challenges Potential Way Forward Conclusion Reproducible Research Through Sharing Artifacts Used in Experiments Test Suite ... Data Sets Repository Dataverse Web Site Version Control Internal Researchers Internal Researchers Access the Repositories Costs of Reproducible Research? Benefits of Reproducible Research? Find Your Own Deliverables More Easily [13] 6 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
65. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Pick Tools Carefully 7 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
66. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Pick Tools Carefully Use the R Language for Statistical Com- putation When: 7 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
67. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Pick Tools Carefully Use the R Language for Statistical Com- putation When: Conducting Experiments 7 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
68. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Pick Tools Carefully Use the R Language for Statistical Com- putation When: Conducting Experiments Visualizing Data 7 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
69. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Pick Tools Carefully Use the R Language for Statistical Com- putation When: Conducting Experiments Visualizing Data Data Mining 7 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
70. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Pick Tools Carefully Use the R Language for Statistical Com- putation When: Conducting Experiments Visualizing Data Data Mining Simple Shell or Full-Featured IDEs 7 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
71. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Pick Tools Carefully Use the R Language for Statistical Com- putation When: Conducting Experiments Visualizing Data Data Mining Simple Shell or Full-Featured IDEs Advanced Data Analysis Methods 7 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
72. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Tree Models sel_method: TOU3, TOU4, TOU5 RM child_density < 0.875 0.9674 8 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
73. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Tree Models sel_method: TOU3, TOU4, TOU5 RM child_density < 0.875 0.9674 sel_method 8 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
74. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Tree Models sel_method: TOU3, TOU4, TOU5 RM child_density < 0.875 0.9674 sel_method sel_method: TOU3, TOU4, TOU5 GB mutation_rate < 0.17 cross_operator: OX1, VR 8 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
75. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Tree Models sel_method: TOU3, TOU4, TOU5 RM child_density < 0.875 0.9674 sel_method sel_method: TOU3, TOU4, TOU5 GB mutation_rate < 0.17 cross_operator: OX1, VR sel_method 8 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
76. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Tree Models sel_method: TOU3, TOU4, TOU5 RM child_density < 0.875 0.9674 sel_method sel_method: TOU3, TOU4, TOU5 GB mutation_rate < 0.17 cross_operator: OX1, VR sel_method The sel_method variable is always the most important parameter 8 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
77. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Tree Models sel_method: TOU3, TOU4, TOU5 RM child_density < 0.875 0.9674 sel_method sel_method: TOU3, TOU4, TOU5 GB mutation_rate < 0.17 cross_operator: OX1, VR sel_method Importance of sel_method holds for all case study applications 8 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
78. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Tree Models sel_method: TOU3, TOU4, TOU5 RM child_density < 0.875 0.9674 sel_method sel_method: TOU3, TOU4, TOU5 GB mutation_rate < 0.17 cross_operator: OX1, VR sel_method How does the selection method impact efficiency and effectiveness? 8 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
79. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Tree Models sel_method: TOU3, TOU4, TOU5 RM child_density < 0.875 0.9674 sel_method sel_method: TOU3, TOU4, TOU5 GB mutation_rate < 0.17 cross_operator: OX1, VR sel_method For More Details, Please See Conrad et al. [22] 8 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
80. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Open Source 9 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
81. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Open Source Gelations is a Genetic Algorithm-Based Test Suite Prioritizer 9 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
82. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Open Source Visit http://gelations.googlecode.com/ for More Details 9 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
83. Introduction Challenges Potential Way Forward Conclusion Practical Suggestions: Use Open Source Visit http://raise.googlecode.com/ for Greedy Algorithms 9 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
84. Introduction Challenges Potential Way Forward Conclusion Conclusions and Future Work The Potential Way Forward May Seem Daunting 10 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
85. Introduction Challenges Potential Way Forward Conclusion Conclusions and Future Work The Potential Way Forward May Seem Daunting Once you get your courage up and believe that you can do important prob- lems, then you can. Richard Hamming 10 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
86. Introduction Challenges Potential Way Forward Conclusion Conclusions and Future Work The Potential Way Forward May Seem Daunting Once you get your courage up and believe that you can do important prob- lems, then you can. Richard Hamming Share One Item With Your Next Paper 10 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
87. Introduction Challenges Potential Way Forward Conclusion Conclusions and Future Work Once you get your courage up and believe that you can do important prob- lems, then you can. Richard Hamming Share One Item With Your Next Paper Leader of the Regres- sion Testing Community Your Picture Here Your Help is Welcomed and Appreciated! 10 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
88. Introduction Challenges Potential Way Forward Conclusion Conclusions and Future Work Once you get your courage up and believe that you can do important prob- lems, then you can. Richard Hamming Share One Item With Your Next Paper Leader of the Regres- sion Testing Community Your Picture Here Your Help is Welcomed and Appreciated! Read the Paper and Contact Me with Comments 10 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
89. Introduction Challenges Potential Way Forward Conclusion Conclusions and Future Work Share One Item With Your Next Paper Leader of the Regres- sion Testing Community Your Picture Here Your Help is Welcomed and Appreciated! Read the Paper and Contact Me with Comments Future Work: Complete Case Study of Reproducible Research in Regression Testing  Data Sets  Tools  Visualizations 10 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward
90. Introduction Challenges Potential Way Forward Conclusion Conclusions and Future Work Share One Item With Your Next Paper Leader of the Regres- sion Testing Community Your Picture Here Your Help is Welcomed and Appreciated! Read the Paper and Contact Me with Comments Acknowledgments:  Neil Elliott  Jonathan Kauffman  Molly Mattis  Zachary Williams provided feedback on drafts of the paper 10 / 10 Empirically Evaluating Regression Testing Techniques: Challenges, Solutions, and a Potential Way Forward