Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using conditional mutation to increase the efficiency of mutation analysis

Using conditional mutation to increase the efficiency of mutation analysis

Interested in learning more about this topic? Visit this web site to read the paper: https://www.gregorykapfhammer.com/research/papers/Just2011a/

Gregory Kapfhammer

May 23, 2011
Tweet

More Decks by Gregory Kapfhammer

Other Decks in Research

Transcript

  1. Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

    Ren´ e Just1 & Gregory M. Kapfhammer2 & Franz Schweiggert1 1Ulm University, Germany 2Allegheny College, USA 6th International Workshop on the Automation of Software Test Waikiki, Honolulu, Hawaii, USA May 23 - 24, 2011
  2. Introduction Conditional Mutation Implementation Conclusion Overview of the Presentation Efficient

    Mutation Analysis Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  3. Introduction Conditional Mutation Implementation Conclusion Overview of the Presentation Efficient

    Mutation Analysis Challenges Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  4. Introduction Conditional Mutation Implementation Conclusion Overview of the Presentation Efficient

    Mutation Analysis Challenges Solutions Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  5. Introduction Conditional Mutation Implementation Conclusion Overview of the Presentation Efficient

    Mutation Analysis Challenges Solutions Conditional Mutation Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  6. Introduction Conditional Mutation Implementation Conclusion Overview of the Presentation Efficient

    Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  7. Introduction Conditional Mutation Implementation Conclusion Overview of the Presentation Efficient

    Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  8. Introduction Conditional Mutation Implementation Conclusion Overview of the Presentation Efficient

    Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Compiler Integrated Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  9. Introduction Conditional Mutation Implementation Conclusion Overview of the Presentation Efficient

    Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Compiler Integrated Comprehensive Empirical Study Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  10. Introduction Conditional Mutation Implementation Conclusion Overview of the Presentation Efficient

    Mutation Analysis Challenges Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Compiler Integrated Comprehensive Empirical Study Efficient Technique - Fully Integrated into the Java 6 SE Compiler Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  11. Introduction Conditional Mutation Implementation Conclusion Overview of Mutation Analysis public

    int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  12. Introduction Conditional Mutation Implementation Conclusion Overview of Mutation Analysis public

    int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Methodically inject small syntactical faults into the program under test Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  13. Introduction Conditional Mutation Implementation Conclusion Overview of Mutation Analysis public

    int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  14. Introduction Conditional Mutation Implementation Conclusion Overview of Mutation Analysis public

    int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } =⇒ =⇒ y = a - x; y = a + x; y = a / x; if(b < a) if(b != a) if(b == a) Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  15. Introduction Conditional Mutation Implementation Conclusion Overview of Mutation Analysis public

    int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Unbiased and powerful method for assessing oracles and input values Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  16. Introduction Conditional Mutation Implementation Conclusion Overview of Mutation Analysis public

    int eval(int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max(int a, int b){ int max = a; if(b>a){ max=b; } return max; } Unbiased and powerful method for assessing oracles and input values Useful method for fault seeding during the empirical study of testing techniques Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  17. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  18. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  19. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Program Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  20. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Program Mutants Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  21. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  22. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  23. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  24. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  25. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Results Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  26. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Results Individually Executing the Mutants is Too Expensive Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  27. Introduction Conditional Mutation Implementation Conclusion Mutation Analysis Challenges Mutant Generation

    Mutation Operators Program Mutants Often Yields a Substantial Num- ber of Mutants High Time Over- head for Generation Mutation Analysis Tests Results Individually Executing the Mutants is Too Expensive Prior Solutions? Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  28. Introduction Conditional Mutation Implementation Conclusion Prior Work in Mutation Analysis

    Improving Mutation Analysis Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  29. Introduction Conditional Mutation Implementation Conclusion Prior Work in Mutation Analysis

    Improving Mutation Analysis Offutt and Untch Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  30. Introduction Conditional Mutation Implementation Conclusion Prior Work in Mutation Analysis

    Improving Mutation Analysis Offutt and Untch Do Fewer Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  31. Introduction Conditional Mutation Implementation Conclusion Prior Work in Mutation Analysis

    Improving Mutation Analysis Offutt and Untch Do Fewer Sampling Selection Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  32. Introduction Conditional Mutation Implementation Conclusion Prior Work in Mutation Analysis

    Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  33. Introduction Conditional Mutation Implementation Conclusion Prior Work in Mutation Analysis

    Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Distributed Weak Mutation Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  34. Introduction Conditional Mutation Implementation Conclusion Prior Work in Mutation Analysis

    Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Do Faster Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  35. Introduction Conditional Mutation Implementation Conclusion Prior Work in Mutation Analysis

    Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Do Faster Compiler Integrated Bytecode Transformation Mutant Schemata Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  36. Introduction Conditional Mutation Implementation Conclusion Prior Work in Mutation Analysis

    Improving Mutation Analysis Offutt and Untch Do Fewer Do Smarter Do Faster Higher Order Mutation Jia and Harman Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  37. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Conditional Mutation Just

    & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  38. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Conditional Mutation Encapsulates

    all mutants within the same block Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  39. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Conditional Mutation Encapsulates

    all mutants within the same block Can be inte- grated within the compiler Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  40. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Conditional Mutation Encapsulates

    all mutants within the same block Transforms the abstract syntax tree (AST) Can be inte- grated within the compiler Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  41. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Conditional Mutation Encapsulates

    all mutants within the same block Transforms the abstract syntax tree (AST) Stmt → Conditional Stmt (if-then-else, switch) Expr → Conditional Expr (conditional operator ?:) Can be inte- grated within the compiler Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  42. Introduction Conditional Mutation Implementation Conclusion Working Example public int eval(int

    x){ int a=3, b=1, y; y = a * x ; y += b; return y; } Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  43. Introduction Conditional Mutation Implementation Conclusion Working Example public int eval(int

    x){ int a=3, b=1, y; y = a * x ; y += b; return y; } ⇓ ASSIGN IDENT y BINARY ∗ a x Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  44. Introduction Conditional Mutation Implementation Conclusion Working Example public int eval(int

    x){ int a=3, b=1, y; y = a * x ; y += b; return y; } ⇓ ASSIGN IDENT y BINARY ∗ a x Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  45. Introduction Conditional Mutation Implementation Conclusion Working Example public int eval(int

    x){ int a=3, b=1, y; y = a * x ; y += b; return y; } ⇓ ASSIGN IDENT y BINARY ∗ a x =⇒ =⇒ ASSIGN IDENT y COND-EXPR THEN BINARY + a x COND (M NO ==2) ELSE COND-EXPR THEN BINARY - a x COND (M NO ==1) ELSE BINARY ∗ a x Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  46. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Algorithm public int

    eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  47. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Algorithm public int

    eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  48. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Algorithm public int

    eval(int x){ int a=3, b=1, y; y = a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  49. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Algorithm public int

    eval(int x){ int a=3, b=1, y; y = (M_NO==1)? a - x : a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  50. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Algorithm public int

    eval(int x){ int a=3, b=1, y; y = (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  51. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Algorithm public int

    eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Versatile approach, can be combined with prior solutions Formal description and implementation details in the paper Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  52. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Algorithm public int

    eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Versatile approach, can be combined with prior solutions Formal description and implementation details in the paper Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  53. Introduction Conditional Mutation Implementation Conclusion Conditional Mutation Algorithm public int

    eval(int x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } 1 Define mutation operators MOP(x ∗ y) = {x − y, x + y, x/y} 2 Determine whether current expression or statement is affected by mutation 3 Apply mutation operators Versatile approach, can be combined with prior solutions Formal description and implementation details in the paper Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  54. Introduction Conditional Mutation Implementation Conclusion Mutation Coverage public int eval(int

    x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  55. Introduction Conditional Mutation Implementation Conclusion Mutation Coverage public int eval(int

    x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : a * x ; y += b; return y; } Mutants not exe- cuted cannot be killed Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  56. Introduction Conditional Mutation Implementation Conclusion Mutation Coverage public int eval(int

    x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : (M_NO==0 && COVERED(1,3))? a * x : a * x ; y += b; return y; } Mutants not exe- cuted cannot be killed Determine covered mutants with addi- tional instrumentation Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  57. Introduction Conditional Mutation Implementation Conclusion Mutation Coverage public int eval(int

    x){ int a=3, b=1, y; y = (M_NO==3)? a / x : (M_NO==2)? a + x : (M_NO==1)? a - x : (M_NO==0 && COVERED(1,3))? a * x : a * x ; y += b; return y; } Mutants not exe- cuted cannot be killed Determine covered mutants with addi- tional instrumentation Only execute and investi- gate the covered mutants Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  58. Introduction Conditional Mutation Implementation Conclusion MAJOR: Mutation Analysis in a

    Java Compiler Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  59. Introduction Conditional Mutation Implementation Conclusion MAJOR: Mutation Analysis in a

    Java Compiler Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  60. Introduction Conditional Mutation Implementation Conclusion MAJOR: Mutation Analysis in a

    Java Compiler Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  61. Introduction Conditional Mutation Implementation Conclusion MAJOR: Mutation Analysis in a

    Java Compiler Configuration: Common compiler options Domain specific language (DSL) http://www.mathematik.uni-ulm.de/sai/major Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  62. Introduction Conditional Mutation Implementation Conclusion Performance Analysis 1 2 3

    4 5 6 7 8 9 10 11 12 0 20000 40000 60000 80000 100000 120000 140000 Compiler runtime in seconds Number of mutants apache ant jfreechart itext java pathfinder commons math commons lang numerics4j Overhead for generating and compiling mutants is negligible Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  63. Introduction Conditional Mutation Implementation Conclusion Performance Analysis 1 2 3

    4 5 6 7 8 9 10 11 12 0 20000 40000 60000 80000 100000 120000 140000 Compiler runtime in seconds Number of mutants apache ant jfreechart itext java pathfinder commons math commons lang numerics4j Overhead for generating and compiling mutants is negligible Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  64. Introduction Conditional Mutation Implementation Conclusion Performance Analysis Application Mutants Runtime

    of test suite Memory consumption original instrumented original instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90 Runtime overhead is application dependent Larger for CPU-bound applications Small for I/O-bound applications Even for large projects, applicable on commodity workstations Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  65. Introduction Conditional Mutation Implementation Conclusion Performance Analysis Application Mutants Runtime

    of test suite Memory consumption original instrumented original instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90 Runtime overhead is application dependent Larger for CPU-bound applications Small for I/O-bound applications Even for large projects, applicable on commodity workstations Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  66. Introduction Conditional Mutation Implementation Conclusion Performance Analysis Application Mutants Runtime

    of test suite Memory consumption original instrumented original instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90 Runtime overhead is application dependent Larger for CPU-bound applications Small for I/O-bound applications Even for large projects, applicable on commodity workstations Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  67. Introduction Conditional Mutation Implementation Conclusion Performance Analysis Application Mutants Runtime

    of test suite Memory consumption original instrumented original instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90 Runtime overhead is application dependent Larger for CPU-bound applications Small for I/O-bound applications Even for large projects, applicable on commodity workstations Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  68. Introduction Conditional Mutation Implementation Conclusion Performance Analysis Application Mutants Runtime

    of test suite Memory consumption original instrumented original instrumented wcs wcs+cov aspectj 406,382 4.3 4.8 5.0 559 813 apache ant 60,258 331.0 335.0 346.0 237 293 jfreechart 68,782 15.0 18.0 23.0 220 303 itext 124,184 5.1 5.6 6.3 217 325 java pathfinder 37,331 17.0 22.0 29.0 182 217 commons math 67,895 67.0 83.0 98.0 153 225 commons lang 25,783 10.3 11.8 14.8 104 149 numerics4j 5,869 1.2 1.3 1.6 73 90 Runtime overhead is application dependent Larger for CPU-bound applications Small for I/O-bound applications Even for large projects, applicable on commodity workstations Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  69. Introduction Conditional Mutation Implementation Conclusion Enabling Efficient Mutation Analysis Just

    & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  70. Introduction Conditional Mutation Implementation Conclusion Enabling Efficient Mutation Analysis Just

    & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  71. Introduction Conditional Mutation Implementation Conclusion Conclusion Conclusion: Largest empirical study

    of mutation analysis to date Mutant generation time reduced to a minimum Mutation coverage provides runtime optimization Versatilely applicable in every Java-based environment Arbitrary conditions enable support for higher order mutation Future Work: Implement new mutation operators Enhance the domain specific language Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  72. Introduction Conditional Mutation Implementation Conclusion Conclusion Conclusion: Largest empirical study

    of mutation analysis to date Mutant generation time reduced to a minimum Mutation coverage provides runtime optimization Versatilely applicable in every Java-based environment Arbitrary conditions enable support for higher order mutation Future Work: Implement new mutation operators Enhance the domain specific language Just & Kapfhammer & Schweiggert Ulm University, Allegheny College Using Conditional Mutation to Increase the Efficiency of Mutation Analysis
  73. Using Conditional Mutation to Increase the Efficiency of Mutation Analysis

    Thank you for your attention! Questions? http://www.mathematik.uni-ulm.de/sai/major