Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
If You Can't Kill a Supermutant, You Have a Problem
Search
Rahul Gopinath
April 09, 2018
Research
0
220
If You Can't Kill a Supermutant, You Have a Problem
Mutation Workshop 2018
Rahul Gopinath
April 09, 2018
Tweet
Share
More Decks by Rahul Gopinath
See All by Rahul Gopinath
How to Talk to Strange Programs and Find Bugs
rahulgopinath
0
20
Dancing to an Unknown Music: Grammar Inferrence with Prefix Queries
rahulgopinath
0
22
Systematic Assessment of Fuzzers using Mutation Analysis
rahulgopinath
0
89
Building Blocks for Fuzzing
rahulgopinath
0
130
"Synthesizing Input Grammars" A Replication Study
rahulgopinath
0
180
CLIFuzzer: Mining Grammars for Command-Line Invocations
rahulgopinath
0
65
Learning And Refining Input Grammars For Effective Fuzzing
rahulgopinath
0
110
Input Languages for Effective and Focused Fuzzing
rahulgopinath
0
150
Fuzzing -- From Alchemy to a Science
rahulgopinath
0
210
Other Decks in Research
See All in Research
Rの機械学習フレームワークの紹介〜tidymodelsを中心に〜 / machine_learning_with_r2024
s_uryu
0
220
[ICLR'24] Towards Assessing and Benchmarking Risk-Return Tradeoff of OPE
harukakiyohara_
0
200
第14回対話システムシンポジウム EMNLP 2023 参加報告
atsumoto
0
150
時系列解析と疫学
kingqwert
2
920
データで診て考える合志市の渋滞と公共交通 ~めざせ 車1割削減、渋滞半減、公共交通2倍~
trafficbrain
0
460
クロスモーダル表現学習の研究動向: 音声関連を中心として
ryomasumura
3
590
F0に基づいて伸縮された画像文字からの音声合成 [ASJ2024春]
nehi0615
0
120
Equivalence of Geodesics and Importance Weighting from the Perspective of Information Geometry
mkimura
0
140
Discovering Universal Geometry in Embeddings with ICA
momoseoyama
1
340
FMP L3 Year 1 Project Proposal
haiinya
0
150
ゼロからわかるリザバーコンピューティング
kurotaky
1
290
待機電力を削減したネットワーク更新型電子ペーパーサイネージの開発と評価 / IOT64
yumulab
0
100
Featured
See All Featured
[RailsConf 2023] Rails as a piece of cake
palkan
23
3.9k
10 Git Anti Patterns You Should be Aware of
lemiorhan
648
58k
Creatively Recalculating Your Daily Design Routine
revolveconf
210
11k
What’s in a name? Adding method to the madness
productmarketing
PRO
16
2.6k
A Tale of Four Properties
chriscoyier
151
22k
What the flash - Photography Introduction
edds
64
11k
Producing Creativity
orderedlist
PRO
337
39k
Writing Fast Ruby
sferik
621
60k
The MySQL Ecosystem @ GitHub 2015
samlambert
243
12k
Debugging Ruby Performance
tmm1
70
11k
How STYLIGHT went responsive
nonsquared
92
4.8k
Teambox: Starting and Learning
jrom
128
8.4k
Transcript
9. April 2018 Rahul Gopinath, Björn Mathis and Andreas Zeller
If You Can't Kill a Supermutant You Have a Problem 1
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2 Enormous Amount of Test Executions
Supermutants - Coverage Adequate Test Suite 3
Supermutants - Coverage Adequate Test Suite 0 % 25 %
50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 3
Supermutants - Coverage Adequate Test Suite 0 Coverage Adequate Tests
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 0 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 3
Supermutants - Coverage Adequate Test Suite 1 Coverage Adequate Test
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 1 Coverage Adequate Test 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 3
Supermutants - Coverage Adequate Test Suite 2 Coverage Adequate Tests
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 2 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 3
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutant Execution 5
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests 34 vs 42 Test Executions
Supermutants - Mutant Kill Matrix 6 7 Tests 3 Tests
3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Case Study - Urlparse 7
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 13,009 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000 Over 70% of Test Executions Are Saved
When Are Supermutants Super? 8
When Are Supermutants Super? 8 Many and Weak Tests
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants No Fault Masking
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants No Fault Masking Automatically Generated Test Suites Can Be Efficiently Analyzed
9 Summary
9 Summary
9 Summary
9 Summary
9 Summary