Upgrade to Pro — share decks privately, control downloads, hide ads and more …

If You Can't Kill a Supermutant, You Have a Problem

D27cb84e0d30e2778e9b66d6a5f42106?s=47 Rahul Gopinath
April 09, 2018
Research
0
95

If You Can't Kill a Supermutant, You Have a Problem

Mutation Workshop 2018

D27cb84e0d30e2778e9b66d6a5f42106?s=128

Rahul Gopinath

April 09, 2018
Tweet

More Decks by Rahul Gopinath

See All by Rahul Gopinath
D27cb84e0d30e2778e9b66d6a5f42106?s=48 rahulgopinath
0
8
D27cb84e0d30e2778e9b66d6a5f42106?s=48 rahulgopinath
0
27
D27cb84e0d30e2778e9b66d6a5f42106?s=48 rahulgopinath
0
53
D27cb84e0d30e2778e9b66d6a5f42106?s=48 rahulgopinath
0
20
D27cb84e0d30e2778e9b66d6a5f42106?s=48 rahulgopinath
0
55
D27cb84e0d30e2778e9b66d6a5f42106?s=48 rahulgopinath
0
28
D27cb84e0d30e2778e9b66d6a5f42106?s=48 rahulgopinath
0
27
D27cb84e0d30e2778e9b66d6a5f42106?s=48 rahulgopinath
0
85
D27cb84e0d30e2778e9b66d6a5f42106?s=48 rahulgopinath
0
300

Other Decks in Research

See All in Research
B83842a07c4241e2aec3e2dfeedf16fe?s=48 mu_mu_mu
3
780
14da6ebc2e909305afdb348e7970de81?s=48 wingnus
1
1.2k
A8e5603efe2e904e5643163d1e0cb0d6?s=48 butsugiri
0
130
D50d47de32271093b714995cfd267fa9?s=48 questresearch
0
690
14da6ebc2e909305afdb348e7970de81?s=48 wingnus
0
180
0fee20d8bbb7283e1887e7075f638f59?s=48 eumesy
13
4.2k
8ba32afb4b7d6967cb38b2dadf1ddd80?s=48 daigo0927
1
460
7d5c4656c947d0905dfbc5e39f233857?s=48 ayaniwa
0
150
Bf12372d4cd555c92db727869f0c7c03?s=48 emmiartbook
0
360
51535f9347da7e003f77d31c4e0b5cec?s=48 thiagomarzagao
0
100
8a7e83d2e447783ab6d824f553429a09?s=48 stakaya
1
960
0fee20d8bbb7283e1887e7075f638f59?s=48 eumesy
3
1.4k

Featured

See All Featured
39488f9d172ab92fd352f2cd7b73258d?s=48 mza
82
4.4k
5f2da528927a2ec9ba4fec2069cbc958?s=48 kneath
224
15k
79acae287842acf29084005533a7fb3b?s=48 hursman
111
9.5k
A415db3ac9e9668acbc1fb36eead84ac?s=48 mthomps
45
2.5k
6804f1775cb4babfcc3851298566fbce?s=48 tanoku
91
8.9k
F29327647a9cff5c69618bae420792ea?s=48 tenderlove
58
3.8k
5b6a2bd86ef643786a381a212e0ef2f1?s=48 geoffreycrofte
42
2k
B3d6434763caa0ef5dc4b792662c49f7?s=48 smashingmag
236
18k
7edec06b5435e0dac07a353b2cc26253?s=48 geeforr
336
30k
E76911cbe088e5b850d966de3fc7435b?s=48 robhawkes
56
3k
Fde6c3a50ca518a909ef35c22c0ee0e4?s=48 destraynor
148
20k
44b54d2ffcb7857004071cc6795dde11?s=48 cassininazir
350
20k

Transcript

  1. 9. April 2018 Rahul Gopinath, Björn Mathis and Andreas Zeller

    If You Can't Kill a Supermutant You Have a Problem 1

  2. Mutant Kill Matrix 2

  3. Mutant Kill Matrix 2

  4. Mutant Kill Matrix 2

  5. Mutant Kill Matrix 2

  6. Mutant Kill Matrix 2

  7. Mutant Kill Matrix 2

  8. Mutant Kill Matrix 2

  9. Mutant Kill Matrix 2

  10. Mutant Kill Matrix 2 Enormous Amount of Test Executions

  11. Supermutants - Coverage Adequate Test Suite 3

  12. Supermutants - Coverage Adequate Test Suite 0 % 25 %

    50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 3

  13. Supermutants - Coverage Adequate Test Suite 0 Coverage Adequate Tests

    0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 0 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 3

  14. Supermutants - Coverage Adequate Test Suite 1 Coverage Adequate Test

    0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 1 Coverage Adequate Test 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 3

  15. Supermutants - Coverage Adequate Test Suite 2 Coverage Adequate Tests

    0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 2 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 3

  16. Supermutants - Coverage Adequate Test Suite 4

  17. Supermutants - Coverage Adequate Test Suite 4

  18. Supermutants - Coverage Adequate Test Suite 4

  19. Supermutants - Coverage Adequate Test Suite 4

  20. Supermutant Execution 5

  21. Supermutant Execution 5 7 Tests

  22. Supermutant Execution 5 7 Tests

  23. Supermutant Execution 5 7 Tests

  24. Supermutant Execution 5 7 Tests

  25. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  26. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  27. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  28. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  29. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 7 Tests

  30. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 7 Tests

  31. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 7 Tests

  32. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 1 Test 1 Test 7 Tests

  33. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 1 Test 1 Test 7 Tests

  34. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 1 Test 1 Test 7 Tests 34 vs 42 Test Executions

  35. Supermutants - Mutant Kill Matrix 6 7 Tests 3 Tests

    3 Tests 3 Tests 3 Tests 1 Test 1 Test

  36. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  37. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  38. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  39. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  40. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  41. Case Study - Urlparse 7

  42. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite

  43. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 32,000

  44. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 13,009 32,000

  45. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000

  46. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000 Over 70% of Test Executions Are Saved

  47. When Are Supermutants Super? 8

  48. When Are Supermutants Super? 8 Many and Weak Tests

  49. When Are Supermutants Super? 8 Many and Weak Tests Stubborn

    Mutants

  50. When Are Supermutants Super? 8 Many and Weak Tests Stubborn

    Mutants No Fault Masking

  51. When Are Supermutants Super? 8 Many and Weak Tests Stubborn

    Mutants No Fault Masking Automatically Generated Test Suites Can Be Efficiently Analyzed

  52. 9 Summary

  53. 9 Summary

  54. 9 Summary

  55. 9 Summary

  56. 9 Summary