Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs Gregory M. Kapfhammer† Department of Computer Science Allegheny College http://www.cs.allegheny.edu/∼gkapfham/ University of Sheffield – February 3, 2012 †Joint with Ren´ e Just and Franz Schweiggert (University of Ulm) and Jonathan Miller Kauffman (Allegheny College)
Introduction Mutation Analysis Empirical Evaluation Conclusion Important Points Accessing the Presentation Scan this QR Code with your smartphone! ... or, visit this Web site: http://is.gd/rekiwo ... or, ask me for a USB drive! Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Input Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Input Output Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Output Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Test Verdict The test case passes and the code is correct! Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Method Under Test Test Set Up Input Output Test Clean Up Test Oracle Expected Output Test Verdict Expected Output Output Test Verdict The test case fails and a defect is found! Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 F1 F2 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2 Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2 Requirements R = {R1 , . . . , R6 }, Features F = {F1 , . . . , F4 }, Bug Fixes B = {B1 , B2 } Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2 Requirements R = {R1 , . . . , R6 }, Features F = {F1 , . . . , F4 }, Bug Fixes B = {B1 , B2 } Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2 Requirements R = {R1 , . . . , R6 }, Features F = {F1 , . . . , F4 }, Bug Fixes B = {B1 , B2 } How Good is Test Suite T? Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2 Requirements R = {R1 , . . . , R6 }, Features F = {F1 , . . . , F4 }, Bug Fixes B = {B1 , B2 } How Good is Test Suite T? Coverage Analysis Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Test Suite T = T1, T2, . . . , T9, T10 R1 R2 R3 R4 R5 R6 F1 F2 F3 F4 B1 B2 Requirements R = {R1 , . . . , R6 }, Features F = {F1 , . . . , F4 }, Bug Fixes B = {B1 , B2 } How Good is Test Suite T? Coverage Analysis Mutation Analysis Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Mutation Operator Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Mutation Operator Mutation Operator Mutation Operator Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Mutation Operator Mutation Operator Mutation Operator Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Mutation Operator Mutation Operator Mutation Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Mutation Operator Mutation Operator Mutation Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Mutation Operator Mutation Operator Mutation Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Mutation Operator Mutation Operator Mutation Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Mutation Operator Mutation Operator Mutation Operator Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test The test suite cannot kill the mutant – either a test suite weakness or an equivalent mutant! Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T1 Test Case T2 Test Case T3 Test Case T4 Execute the test suite after enabling a single mutant in the program under test Repeat this process for all of the test cases and mutants – calculate mutation score when finished Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Challenges Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Challenges Solutions Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Challenges Solutions Conditional Mutation Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Compiler Integrated Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Compiler Integrated Comprehensive Empirical Study Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Compiler Integrated Comprehensive Empirical Study Efficient Technique - Fully Integrated into the Java 6 SE Compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } =⇒ =⇒ • y = a - x; • y = a + x; • y = a / x; • if(b < a) • if(b != a) • if(b == a) Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Unbiased and powerful method for assessing oracles and input values Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Unbiased and powerful method for assessing oracles and input values Useful method for fault seeding during the empirical study of testing techniques Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Program Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Program Mutants Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Results Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Results Individually Executing the Mutants is Too Expensive Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Results Individually Executing the Mutants is Too Expensive Prior Solutions? Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Prior Work in Mutation Analysis Improving Mutation Analysis Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Prior Work in Mutation Analysis Improving Mutation Analysis Offutt and Untch Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Prior Work in Mutation Analysis Improving Mutation Analysis Offutt and Untch Do Fewer Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Prior Work in Mutation Analysis Improving Mutation Analysis Offutt and Untch Do Fewer Sampling Selection Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Prior Work in Mutation Analysis Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Prior Work in Mutation Analysis Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Distributed Weak Mutation Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Prior Work in Mutation Analysis Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Do Faster Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Prior Work in Mutation Analysis Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Do Faster Compiler Integrated Bytecode Transformation Mutant Schemata Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Prior Work in Mutation Analysis Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Do Faster Higher Order Mutation Jia and Harman Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Practical Mutation Analysis Practical (adjective): 1 Of or concerned with the actual doing or use of something rather than with theory and ideas 2 (of an idea, plan, or method) Likely to succeed or be effective in real circumstances; feasible 3 Suitable for a particular purpose Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Practical Mutation Analysis Practical (adjective): 1 Of or concerned with the actual doing or use of something rather than with theory and ideas 2 (of an idea, plan, or method) Likely to succeed or be effective in real circumstances; feasible 3 Suitable for a particular purpose What are the practical techniques that MAJOR employs for improving the efficiency and usability of mutation analysis? Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Conditional Mutation Conditional Mutation Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Conditional Mutation Conditional Mutation Encapsulates all mutants within the same block Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Conditional Mutation Conditional Mutation Encapsulates all mutants within the same block Can be inte- grated within the compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Conditional Mutation Conditional Mutation Encapsulates all mutants within the same block Transforms the abstract syntax tree (AST) Can be inte- grated within the compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Conditional Mutation Conditional Mutation Encapsulates all mutants within the same block Transforms the abstract syntax tree (AST) Stmt → Conditional Stmt (if-then-else, switch) Expr → Conditional Expr (conditional operator ?:) Can be inte- grated within the compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Transforming the AST public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Transforming the AST public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } ⇓ ASSIGN IDENT y BINARY ∗ a x Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Transforming the AST public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } ⇓ ASSIGN IDENT y BINARY ∗ a x Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Transforming the AST public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } ⇓ ASSIGN IDENT y BINARY ∗ a x =⇒ =⇒ ASSIGN IDENT y COND-EXPR THEN BINARY + a x COND (M NO ==2) ELSE COND-EXPR THEN BINARY - a x COND (M NO ==1) ELSE BINARY ∗ a x Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Source Code View of Inserting Mutants public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Source Code View of Inserting Mutants public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Source Code View of Inserting Mutants public int eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Source Code View of Inserting Mutants public int eval(int x){ int a=3, b=1, y; y = (M_NO==1)? a - x : a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Source Code View of Inserting Mutants public int eval(int x){ int a=3, b=1, y; y = (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Source Code View of Inserting Mutants public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Source Code View of Inserting Mutants public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } Mutants that are not ex- ecuted cannot be killed 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Collecting and Using Mutation Coverage public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } Mutants that are not ex- ecuted cannot be killed Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Collecting and Using Mutation Coverage public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : (M_NO==0 && COVERED(1,3))? a * x : a * x ; y += b; return y; } Mutants that are not ex- ecuted cannot be killed Determine covered mutants with addi- tional instrumentation Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Collecting and Using Mutation Coverage public int eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : (M_NO==0 && COVERED(1,3))? a * x : a * x ; y += b; return y; } Mutants that are not ex- ecuted cannot be killed Determine covered mutants with addi- tional instrumentation Only execute and investi- gate the covered mutants Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR MAJOR’s Compiler MAJOR’s Compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR MAJOR’s Compiler MAJOR’s Compiler Enhanced Standard Java Compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR MAJOR’s Compiler MAJOR’s Compiler Enhanced Standard Java Compiler Source Files Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR MAJOR’s Compiler MAJOR’s Compiler Enhanced Standard Java Compiler Source Files Common Compiler Options Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR MAJOR’s Compiler MAJOR’s Compiler Enhanced Standard Java Compiler Source Files Common Compiler Options Domain Specific Language Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR MAJOR’s Compiler MAJOR’s Compiler Enhanced Standard Java Compiler Source Files Common Compiler Options Domain Specific Language Bytecode with Embedded Mutants Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Integration into the Java Compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Integration into the Java Compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Integration into the Java Compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Integration into the Java Compiler Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Optimized Mutation Analysis Process 1 Embed and compile all mutants 2 Run test suite on instrumented program 3 Sort tests according to their runtime 4 Perform mutation analysis with reordered test suite Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Optimized Mutation Analysis Process 1 Embed and compile all mutants 2 Run test suite on instrumented program 3 Sort tests according to their runtime 4 Perform mutation analysis with reordered test suite Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Optimized Mutation Analysis Process 1 Embed and compile all mutants 2 Run test suite on instrumented program 3 Sort tests according to their runtime 4 Perform mutation analysis with reordered test suite Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis with MAJOR Optimized Mutation Analysis Process 1 Embed and compile all mutants 2 Run test suite on instrumented program 3 Sort tests according to their runtime 4 Perform mutation analysis with reordered test suite Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Compilation Efficiency Mutant Generation and Compilation 1 2 3 4 5 6 7 8 9 10 11 12 0 20000 40000 60000 80000 100000 120000 140000 Compiler runtime in seconds Number of mutants apache ant jfreechart itext java pathfinder commons math commons lang numerics4j Overhead for generating and compiling mutants is negligible Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Compilation Efficiency Mutant Generation and Compilation 1 2 3 4 5 6 7 8 9 10 11 12 0 20000 40000 60000 80000 100000 120000 140000 Compiler runtime in seconds Number of mutants apache ant jfreechart itext java pathfinder commons math commons lang numerics4j Overhead for generating and compiling mutants is negligible Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis Efficiency Evaluating and Improving Mutation Analysis 0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 Number of killed mutants Runtime in seconds optimized order (using coverage information) random order (using coverage information) original order (using coverage information) optimized order (without coverage information) random order (without coverage information) original order (without coverage information) • Mutation analysis is not feasible without coverage information • Reordering the test suite significantly speeds up the process, especially if runtimes of tests differ by orders of magnitude Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis Efficiency Evaluating and Improving Mutation Analysis 0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 Number of killed mutants Runtime in seconds optimized order (using coverage information) random order (using coverage information) original order (using coverage information) optimized order (without coverage information) random order (without coverage information) original order (without coverage information) • Mutation analysis is not feasible without coverage information • Reordering the test suite significantly speeds up the process, especially if runtimes of tests differ by orders of magnitude Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Mutation Analysis Efficiency Evaluating and Improving Mutation Analysis 0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 Number of killed mutants Runtime in seconds optimized order (using coverage information) random order (using coverage information) original order (using coverage information) optimized order (without coverage information) random order (without coverage information) original order (without coverage information) • Mutation analysis is not feasible without coverage information • Reordering the test suite significantly speeds up the process, especially if runtimes of tests differ by orders of magnitude Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Retrospective Revisiting Practical Mutation Analysis Practical (adjective): 1 Of or concerned with the actual doing or use of something rather than with theory and ideas 2 (of an idea, plan, or method) Likely to succeed or be effective in real circumstances; feasible 3 Suitable for a particular purpose Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Retrospective Revisiting Practical Mutation Analysis Practical (adjective): 1 Of or concerned with the actual doing or use of something rather than with theory and ideas 2 (of an idea, plan, or method) Likely to succeed or be effective in real circumstances; feasible 3 Suitable for a particular purpose The evidence suggests that MAJOR is “likely to succeed or be effective” in real-world software testing circumstances Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Retrospective Reviewing MAJOR’s Contributions Mutation Analysis Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Retrospective Reviewing MAJOR’s Contributions Mutation Analysis Efficiency: MAJOR has ac- ceptable time and space over- heads and scales to large, real-world programs Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Retrospective Reviewing MAJOR’s Contributions Mutation Analysis Efficiency: MAJOR has ac- ceptable time and space over- heads and scales to large, real-world programs Usability: MAJOR’s inte- gration into the Java SE compiler makes it a no- hassle, drop-in tool Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Retrospective Reviewing MAJOR’s Contributions Mutation Analysis Efficiency: MAJOR has ac- ceptable time and space over- heads and scales to large, real-world programs Usability: MAJOR’s inte- gration into the Java SE compiler makes it a no- hassle, drop-in tool We will release MAJOR as free and open source software Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Conclusions and Future Work Conclusion Key Concepts and Features: • Compiler-integrated solution • Conditional mutation with the abstract syntax tree • Furnishes its own domain specific language • Collects and leverages mutation coverage information Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Introduction Mutation Analysis Empirical Evaluation Conclusion Conclusions and Future Work Conclusion Key Concepts and Features: • Compiler-integrated solution • Conditional mutation with the abstract syntax tree • Furnishes its own domain specific language • Collects and leverages mutation coverage information Characteristics of MAJOR: • Fast and scalable technique • Configurable and extensible mutation tool • Enables an optimized workflow for mutation analysis Kapfhammer Allegheny College Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs
Practical Techniques for Improving the Efficiency and Usability of Mutation Analysis for Java Programs Gregory M. Kapfhammer Department of Computer Science Allegheny College http://www.cs.allegheny.edu/∼gkapfham/ Thank you for your attention! I welcome your questions and comments.
gkapfham
tbshiki
nagauta
odasho
miyake
line_developers
hiboma
bun913
jordihofc
matsujirushi
victorrentea
trycycle
soracom
sonatard
shlominoach
lemiorhan
rocio
holman
eileencodes
lara
chriscoyier
bryan
mza
swwweet
rmw