Upgrade to Pro — share decks privately, control downloads, hide ads and more …

If You Can't Kill a Supermutant, You Have a Pr...

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Rahul Gopinath Rahul Gopinath
April 09, 2018
Research
360
0

If You Can't Kill a Supermutant, You Have a Problem

Mutation Workshop 2018

Avatar for Rahul Gopinath

Rahul Gopinath

April 09, 2018

More Decks by Rahul Gopinath

See All by Rahul Gopinath
Specification for Free: Behavior-Driven Fuzzing with Inferred Specifications
Avatar for Rahul Gopinath rahulgopinath
0
25
Evaluating Impact of Coverage Feedback on Estimators for Maximum Reachability in Fuzzing (Registered Report)
Avatar for Rahul Gopinath rahulgopinath
0
42
Automatic Data Repair without Format Specifications
Avatar for Rahul Gopinath rahulgopinath
0
58
From Theory to Practice: Blackbox Testing in an Industrial Telecom Environment
Avatar for Rahul Gopinath rahulgopinath
0
64
Assessing Reliability of Statistical Maximum Coverage Estimators in Fuzzing
Avatar for Rahul Gopinath rahulgopinath
0
61
Fuzzing Without Specifications: Learning Structure from Behaviour Part I
Avatar for Rahul Gopinath rahulgopinath
0
87
Fuzzing Without Specifications: Learning Structure from Behaviour Part II
Avatar for Rahul Gopinath rahulgopinath
0
74
How to Compare Fuzzers
Avatar for Rahul Gopinath rahulgopinath
0
160
Mutation Analysis: Answering the Fuzzing Challenge
Avatar for Rahul Gopinath rahulgopinath
0
110

Other Decks in Research

See All in Research
医療LLMの現在地〜最新研究から社会実装までを考える〜
Avatar for kento sugimoto kento1109
1
1.7k
MM-OVSeg: Multimodal Optical–SAR Fusion for Open-Vocabulary Segmentation in Remote Sensing
Avatar for SatAI.challenge satai
3
120
[BlackHatAsia2026] Hidden Telemetry: Uncovering TraceLogging ETW Providers You're Not Using (Yet)
Avatar for Asuka Nakajima asuna_jp
1
690
某助成金プロジェクト採択に向けて企業研究所のアウトリーチ専任者がやったこと
Avatar for afroscript afroscript
0
180
[最先端NLP勉強会2026] Agentic Rubrics as Contextual Verifiers for SWE Agents
Avatar for Ryo Fujii rfujii
1
310
Spatial Active Noise Control Based on Sound Field Interpolation Incorporating Physical Constraints
Avatar for NII S. Koyama's Lab skoyamalab
0
170
高性能計算機クラスタを用いた大規模点群処理による森林の単木抽出と構造解析
Avatar for Kenta Itakura kentaitakura
1
110
2026年度 生成AI を活用した論文執筆ガイド／ワークショップ / 2026 Academic Year Guide to Writing Papers Using Generative AI - Workshop
Avatar for Kenji Saito ks91
PRO
0
220
Research Engineerという仕事 / Research Engineering: Bridging Research and Business
Avatar for chck chck
1
290
「AIとWhyを深堀る」をAIと深堀る
Avatar for なつ iflection
0
600
[SNLP2026] Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
Avatar for 池田航 wataruuuuu
0
270
RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent
Avatar for SatAI.challenge satai
3
510

Featured

See All Featured
The Anti-SEO Checklist Checklist. Pubcon Cyber Week
Avatar for Ryan Jones ryanjones
0
220
Discover your Explorer Soul
Avatar for Emna emna__ayadi
2
1.3k
Unsuck your backbone
Avatar for Amy Palamountain ammeep
672
58k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
Avatar for Addy Osmani addyosmani
10
1.3k
Bridging the Design Gap: How Collaborative Modelling removes blockers to flow between stakeholders and teams @FastFlow conf
Avatar for Kenny Baas-Schwegler baasie
0
680
The SEO Collaboration Effect
Avatar for Kristina Bergwall kristinabergwall1
1
540
End of SEO as We Know It (SMX Advanced Version)
Avatar for Michael King ipullrank
3
4.4k
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
430
Color Theory Basics | Prateek | Gurzu
Avatar for Gurzu gurzu
0
450
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
1.5k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
Avatar for soudai sone soudai
PRO
201
75k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
239
140k

Transcript

  1. 9. April 2018 Rahul Gopinath, Björn Mathis and Andreas Zeller

    If You Can't Kill a Supermutant You Have a Problem 1

  2. Mutant Kill Matrix 2

  3. Mutant Kill Matrix 2

  4. Mutant Kill Matrix 2

  5. Mutant Kill Matrix 2

  6. Mutant Kill Matrix 2

  7. Mutant Kill Matrix 2

  8. Mutant Kill Matrix 2

  9. Mutant Kill Matrix 2

  10. Mutant Kill Matrix 2 Enormous Amount of Test Executions

  11. Supermutants - Coverage Adequate Test Suite 3

  12. Supermutants - Coverage Adequate Test Suite 0 % 25 %

    50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 3

  13. Supermutants - Coverage Adequate Test Suite 0 Coverage Adequate Tests

    0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 0 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 3

  14. Supermutants - Coverage Adequate Test Suite 1 Coverage Adequate Test

    0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 1 Coverage Adequate Test 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 3

  15. Supermutants - Coverage Adequate Test Suite 2 Coverage Adequate Tests

    0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 2 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 3

  16. Supermutants - Coverage Adequate Test Suite 4

  17. Supermutants - Coverage Adequate Test Suite 4

  18. Supermutants - Coverage Adequate Test Suite 4

  19. Supermutants - Coverage Adequate Test Suite 4

  20. Supermutant Execution 5

  21. Supermutant Execution 5 7 Tests

  22. Supermutant Execution 5 7 Tests

  23. Supermutant Execution 5 7 Tests

  24. Supermutant Execution 5 7 Tests

  25. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  26. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  27. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  28. Supermutant Execution 5 3 Tests 3 Tests 7 Tests

  29. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 7 Tests

  30. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 7 Tests

  31. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 7 Tests

  32. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 1 Test 1 Test 7 Tests

  33. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 1 Test 1 Test 7 Tests

  34. Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3

    Tests 1 Test 1 Test 7 Tests 34 vs 42 Test Executions

  35. Supermutants - Mutant Kill Matrix 6 7 Tests 3 Tests

    3 Tests 3 Tests 3 Tests 1 Test 1 Test

  36. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  37. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  38. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  39. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  40. Supermutants - Mutant Kill Matrix 6 Test 1 Test 2

    Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test

  41. Case Study - Urlparse 7

  42. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite

  43. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 32,000

  44. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 13,009 32,000

  45. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000

  46. Case Study - Urlparse 7 0 exec. 10000 exec. 20000

    exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000 Over 70% of Test Executions Are Saved

  47. When Are Supermutants Super? 8

  48. When Are Supermutants Super? 8 Many and Weak Tests

  49. When Are Supermutants Super? 8 Many and Weak Tests Stubborn

    Mutants

  50. When Are Supermutants Super? 8 Many and Weak Tests Stubborn

    Mutants No Fault Masking

  51. When Are Supermutants Super? 8 Many and Weak Tests Stubborn

    Mutants No Fault Masking Automatically Generated Test Suites Can Be Efficiently Analyzed

  52. 9 Summary

  53. 9 Summary

  54. 9 Summary

  55. 9 Summary

  56. 9 Summary