Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
If You Can't Kill a Supermutant, You Have a Pr...
Search
Rahul Gopinath
April 09, 2018
Research
0
320
If You Can't Kill a Supermutant, You Have a Problem
Mutation Workshop 2018
Rahul Gopinath
April 09, 2018
Tweet
Share
More Decks by Rahul Gopinath
See All by Rahul Gopinath
Assessing Reliability of Statistical Maximum Coverage Estimators in Fuzzing
rahulgopinath
0
4
Fuzzing Without Specifications: Learning Structure from Behaviour Part I
rahulgopinath
0
23
Fuzzing Without Specifications: Learning Structure from Behaviour Part II
rahulgopinath
0
20
How to Compare Fuzzers
rahulgopinath
0
110
Mutation Analysis: Answering the Fuzzing Challenge
rahulgopinath
0
60
An Empirical Evaluation of Frequency Based Statistical Models for Estimating Killable Mutants
rahulgopinath
0
78
Look Ma No Hands! Testing Software Without Specifications
rahulgopinath
0
120
How to Talk to Strange Programs and Find Bugs
rahulgopinath
0
150
Dancing to an Unknown Music: Grammar Inferrence with Prefix Queries
rahulgopinath
0
140
Other Decks in Research
See All in Research
EarthSynth: Generating Informative Earth Observation with Diffusion Models
satai
3
380
CoRL2025速報
rpc
1
2.1k
問いを起点に、社会と共鳴する知を育む場へ
matsumoto_r
PRO
0
660
HoliTracer:Holistic Vectorization of Geographic Objects from Large-Size Remote Sensing Imagery
satai
3
120
IMC の細かすぎる話 2025
smly
2
690
20250605_新交通システム推進議連_熊本都市圏「車1割削減、渋滞半減、公共交通2倍」から考える地方都市交通政策
trafficbrain
0
870
2025/7/5 応用音響研究会招待講演@北海道大学
takuma_okamoto
1
220
ロボット学習における大規模検索技術の展開と応用
denkiwakame
1
140
日本語・韓国語オノマトペにおける意味的・音韻的類似ペア抽出のための言語埋め込みを用いた分析
shunnosukemotomura
0
100
Google Agent Development Kit (ADK) 入門 🚀
mickey_kubo
2
2.1k
Time to Cash: The Full Stack Breakdown of Modern ATM Attacks
ratatata
0
160
AWSで実現した大規模日本語VLM学習用データセット "MOMIJI" 構築パイプライン/buiding-momiji
studio_graph
2
740
Featured
See All Featured
How STYLIGHT went responsive
nonsquared
100
5.8k
How to Ace a Technical Interview
jacobian
280
24k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
620
The Cult of Friendly URLs
andyhume
79
6.6k
Done Done
chrislema
185
16k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
230
22k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
140
34k
Fireside Chat
paigeccino
40
3.7k
The Invisible Side of Design
smashingmag
302
51k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
46
2.5k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
30
2.9k
Transcript
9. April 2018 Rahul Gopinath, Björn Mathis and Andreas Zeller
If You Can't Kill a Supermutant You Have a Problem 1
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2 Enormous Amount of Test Executions
Supermutants - Coverage Adequate Test Suite 3
Supermutants - Coverage Adequate Test Suite 0 % 25 %
50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 3
Supermutants - Coverage Adequate Test Suite 0 Coverage Adequate Tests
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 0 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 3
Supermutants - Coverage Adequate Test Suite 1 Coverage Adequate Test
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 1 Coverage Adequate Test 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 3
Supermutants - Coverage Adequate Test Suite 2 Coverage Adequate Tests
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 2 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 3
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutant Execution 5
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests 34 vs 42 Test Executions
Supermutants - Mutant Kill Matrix 6 7 Tests 3 Tests
3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Case Study - Urlparse 7
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 13,009 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000 Over 70% of Test Executions Are Saved
When Are Supermutants Super? 8
When Are Supermutants Super? 8 Many and Weak Tests
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants No Fault Masking
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants No Fault Masking Automatically Generated Test Suites Can Be Efficiently Analyzed
9 Summary
9 Summary
9 Summary
9 Summary
9 Summary