mrstudyr: Retrospectively studying the effectiveness of mutant reduction techniques

mrstudyr: Retrospectively studying the effectiveness of mutant reduction techniques

Interested in learning more about this topic? Visit this web site to read the paper: https://www.gregorykapfhammer.com/research/papers/McCurdy2016/

4ae30d49c8cc07e42d5a871efb9bcfba?s=128

Gregory Kapfhammer

October 02, 2016
Tweet

Transcript

  1. mrstudyr Retrospective Mutant Reduction Colton J. McCurdy McCurdyColton ICSME 2016

    Gregory M. Kapfhammer and Phil McMinn
  2. Motivation Quality software?

  3. Motivation Regression Test Suite

  4. Motivation Regression Test Suite T= t1 , t2 , .

    . . , tn
  5. Motivation Pass Fail

  6. Motivation Pass Fail

  7. Motivation Pass Fail

  8. Motivation Use testing to assess quality?

  9. Motivation Trivial test suite?

  10. Background Mutation Testing!

  11. Mutation Testing Original Mutant

  12. Mutation Testing Original Mutant

  13. Mutation Testing a > b

  14. Mutation Testing a > b Relational Operator Replacement (ROR)

  15. Mutation Testing a > b Relational Operator Replacement (ROR) a

    ≥ b a ≤ b a == b a != b
  16. Mutation Testing Alive Killed

  17. Mutation Testing Alive Killed

  18. Mutation Testing Alive Killed Killed

  19. Mutation Testing MST = Killed Total

  20. Mutation Testing MST = Killed Total

  21. Mutation Testing MST = Killed Total

  22. Mutation Testing MST = Killed Total MST ∈ [0, 1]

  23. Mutation Testing MST = Killed Total MST ∈ [0, 1]

    HIB
  24. Mutation Testing Silver Bullet?

  25. Mutation Testing Limitations

  26. Mutation Testing

  27. Mutation Testing T= t1 , t2 , . . .

    , tn
  28. Mutation Testing T= t1 , t2 , . . .

    , tn Weak
  29. Mutation Testing T= t1 , t2 , . . .

    , tn Weak Strong
  30. Mutation Testing T= t1 , t2 , . . .

    , tn Weak Strong Redundant
  31. Mutation Testing T= t1 , t2 , . . .

    , tn Weak Strong Redundant
  32. Mutation Testing Still . . .

  33. Mutation Testing Still . . . limitations

  34. Mutation Testing

  35. Mutation Testing Understand system intricacies

  36. Mutation Testing Understand system intricacies

  37. Mutation Testing Retrospective Mutant Reduction

  38. Mutation Testing Retrospective Mutant Reduction

  39. mrstudyr  

  40. mrstudyr   mccurdyc/mrstudyr

  41. mrstudyr   mccurdyc/mrstudyr Data

  42. mrstudyr   mccurdyc/mrstudyr Reduce Data

  43. mrstudyr   mccurdyc/mrstudyr Reduce Data Remove

  44. Input Database Schema Mutants

  45. Demonstration Demonstration!

  46. Motivation Random Sampling

  47. Output Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90

    2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00
  48. Output Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90

    2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00
  49. Output Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90

    2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00
  50. Output Percent Correlation MAE RMSE 1 1.00 0.36 9.20 11.90

    2 10.00 0.68 2.96 4.27 3 20.00 0.76 2.02 2.91 4 30.00 0.82 1.35 2.00 5 40.00 0.82 1.19 1.70 6 50.00 0.85 1.07 1.48 7 60.00 0.88 0.75 1.02 8 70.00 0.90 0.59 0.84 9 80.00 0.90 0.47 0.66 10 90.00 0.92 0.34 0.49 11 100.00 1.00 0.00 0.00
  51. Conclusion Program Quality

  52. Conclusion Program Quality Regression Test Suite

  53. Conclusion Program Quality Regression Test Suite Mutation Testing

  54. Conclusion Program Quality Regression Test Suite Mutation Testing Reduction Techniques

  55. Conclusion Program Quality Regression Test Suite Mutation Testing Reduction Techniques

    mrstudyr
  56. Conclusion Reduced

  57. Conclusion Reduced 10%

  58. Conclusion Reduced 10%