Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

If You Can't Kill a Supermutant, You Have a Pr...

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Rahul Gopinath Rahul Gopinath
April 09, 2018
Research
360
0

If You Can't Kill a Supermutant, You Have a Problem

Mutation Workshop 2018

Avatar for Rahul Gopinath

Rahul Gopinath

April 09, 2018

More Decks by Rahul Gopinath

See All by Rahul Gopinath
Specification for Free: Behavior-Driven Fuzzing with Inferred Specifications
Avatar for Rahul Gopinath rahulgopinath
0
30
Evaluating Impact of Coverage Feedback on Estimators for Maximum Reachability in Fuzzing (Registered Report)
Avatar for Rahul Gopinath rahulgopinath
0
54
Automatic Data Repair without Format Specifications
Avatar for Rahul Gopinath rahulgopinath
0
67
From Theory to Practice: Blackbox Testing in an Industrial Telecom Environment
Avatar for Rahul Gopinath rahulgopinath
0
79
Assessing Reliability of Statistical Maximum Coverage Estimators in Fuzzing
Avatar for Rahul Gopinath rahulgopinath
0
79
Fuzzing Without Specifications: Learning Structure from Behaviour Part I
Avatar for Rahul Gopinath rahulgopinath
0
93
Fuzzing Without Specifications: Learning Structure from Behaviour Part II
Avatar for Rahul Gopinath rahulgopinath
0
82
How to Compare Fuzzers
Avatar for Rahul Gopinath rahulgopinath
0
170
Mutation Analysis: Answering the Fuzzing Challenge
Avatar for Rahul Gopinath rahulgopinath
0
120

Other Decks in Research

See All in Research
クラウド・AI 時代の研究開発 DX / R&D Digital Transformation
Avatar for Yoshitaka Haribara hariby
0
190
OWASP AISVS - C7
Avatar for shieLL shiell
2
820
Karkada さんの論文 × 2 の紹介: (1) Closed-Form Training Dynamics Reveal Learned Features and Linear Structure in Word2Vec-like Models, (2) Symmetry in language statistics shapes the geometry of model representations
Avatar for Sho Yokoi eumesy
PRO
1
830
CVPR2026論文紹介_VLMにとって​良いvision encoderとは何か？​Rethinking Model Selection in VLM Through the Lens of Gromov-Wasserstein Distance​
Avatar for 小林優斗 kobayashi31
1
260
【ローカルAIに向き合う展示会vol.2】液体時間定数型モジュールを用いた オリジナルの双方向エンコーダーモデルNexteraBERT 推論速度向上検討並びにダウンストリーム評価
Avatar for Rikka Botan rikkabotan7
0
230
研究室単位での自律的 IPv6接続性確立に向けたAS共同運用モデルの提案と実証
Avatar for Hiroki (REO) Kashiwazaki reokashiwa
PRO
0
230
最先端NLP 2026 論文紹介: Wait, Wait, Wait... Why Do Reasoning Models Loop? / SNLP Paper Review: Wait, Wait, Wait... Why Do Reasoning Models Loop?
Avatar for tkng tkng
0
250
第64回CV・PRML勉強会　論文紹介：Linguistic Priors for Visual Decoupling: Towards Symmetric Vision-Brain Alignment
Avatar for Soki Katayama sokikatayama
0
240
論文紹介：Doc-to-LoRA: Learning to Instantly Internalize Contexts
Avatar for Yukako Nakano yukako_nakano
0
190
Bilan travail en élevage : première restitution des enquêtes 2025-2026
Avatar for Idele institutdelelevage
PRO
0
160
進化？迷走？CasualConc ファミリーアプリの現在地 @ 英語コーパス学会 2026
Avatar for Yasu Imao casualconc
0
140
【Zozo Research 技術共有会】三次元領域の現在と展望
Avatar for 小島瑞貴 mickey_0226
3
670

Featured

See All Featured
AI Search: Implications for SEO and How to Move Forward - #ShenzhenSEOConference
Avatar for Aleyda Solis aleyda
1
1.4k
brightonSEO & MeasureFest 2025 - Winning Strategies for Black Friday CRO & PPC - Christian Goodrich
Avatar for Christian Goodrich cargoodrich
3
860
Agile that works and the tools we love
Avatar for Rasmus Luckow-Nielsen rasmusluckow
331
22k
Impact Scores and Hybrid Strategies: The future of link building
Avatar for Tamara Novitovic tamaranovitovic
0
450
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
Avatar for Liv Day livdayseo
0
210
A better future with KSS
Avatar for Kyle Neath kneath
240
18k
Building the Perfect Custom Keyboard
Avatar for Naoto Takai takai
2
900
WCS-LA-2024
Avatar for Leonardo Collado-Torres lcolladotor
0
860
SEO Brein meetup: CTRL+C is not how to scale international SEO
Avatar for Linda Hogenes lindahogenes
2
2.9k
KATA
Avatar for Megan Lloyd mclloyd
PRO
35
16k
Chasing Engaging Ingredients in Design
Avatar for Sebastian Deterding codingconduct
0
350
The Mindset for Success: Future Career Progression
Avatar for Greg Gifford greggifford
PRO
0
530

Transcript

  1. 9. April 2018 Rahul Gopinath, Björn Mathis and Andreas Zeller

    If You Can't Kill a Supermutant You Have a Problem 1

  2. Mutant Kill Matrix 2

  3. Mutant Kill Matrix 2

  4. Mutant Kill Matrix 2

  5. Mutant Kill Matrix 2

  6. Mutant Kill Matrix 2

  7. Mutant Kill Matrix 2

  8. Mutant Kill Matrix 2

  9. Mutant Kill Matrix 2

  10. Mutant Kill Matrix 2 Enormous Amount of Test Executions

  11. Supermutants - Coverage Adequate Test Suite 3

  12. Supermutants - Coverage Adequate Test Suite 0 % 25 %

    50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 3

  13. Supermutants - Coverage Adequate Test Suite 0 Coverage Adequate Tests

    0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 0 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 3

  14. Supermutants - Coverage Adequate Test Suite 1 Coverage Adequate Test

    0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 1 Coverage Adequate Test 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 3

  15. Supermutants - Coverage Adequate Test Suite 2 Coverage Adequate Tests

    0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 2 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 3

  16. Supermutants - Coverage Adequate Test Suite 4

  17. Supermutants - Coverage Adequate Test Suite 4

  18. Supermutants - Coverage Adequate Test Suite 4

  19. Supermutants - Coverage Adequate Test Suite 4

  20. Supermutant Execution 5

  21. Supermutant Execution 5 7 Tests

  22. Supermutant Execution 5 7 Tests

  23. Supermutant Execution 5 7 Tests

  24. Supermutant Execution 5 7 Tests

  25. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  26. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  27. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  28. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  29. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 7 Tests

  30. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 7 Tests

  31. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 7 Tests

  32. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 1 Test 1 Test 7 Tests

  33. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 1 Test 1 Test 7 Tests

  34. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 1 Test 1 Test 7 Tests 34 vs 42 Test Executions

  35. Supermutants - Mutant Kill Matrix 6 7 Tests 3 Tests

    3 Tests 3 Tests 3 Tests 1 Test 1 Test

  36. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  37. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  38. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  39. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  40. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  41. Case Study - Urlparse 7

  42. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite

  43. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 32,000

  44. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 13,009 32,000

  45. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000

  46. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000 Over 70% of Test Executions Are Saved

  47. When Are Supermutants Super? 8

  48. When Are Supermutants Super? 8 Many and Weak Tests

  49. When Are Supermutants Super? 8 Many and Weak Tests Stubborn

    Mutants

  50. When Are Supermutants Super? 8 Many and Weak Tests Stubborn

    Mutants No Fault Masking

  51. When Are Supermutants Super? 8 Many and Weak Tests Stubborn

    Mutants No Fault Masking Automatically Generated Test Suites Can Be Efficiently Analyzed

  52. 9 Summary

  53. 9 Summary

  54. 9 Summary

  55. 9 Summary

  56. 9 Summary