Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
If You Can't Kill a Supermutant, You Have a Problem
Rahul Gopinath
April 09, 2018
Research
0
130
If You Can't Kill a Supermutant, You Have a Problem
Mutation Workshop 2018
Rahul Gopinath
April 09, 2018
Tweet
Share
More Decks by Rahul Gopinath
See All by Rahul Gopinath
Learning And Refining Input Grammars For Effective Fuzzing
rahulgopinath
0
4
Input Languages for Effective and Focused Fuzzing
rahulgopinath
0
42
Fuzzing -- From Alchemy to a Science
rahulgopinath
0
71
Input Algebras
rahulgopinath
0
84
Inducing Subtle Mutations with Program Repair
rahulgopinath
0
64
Mining Input Grammars from Dynamic Control Flow
rahulgopinath
0
90
Revisiting the Relationship Between Fault Detection, Test Adequacy Criteria, and Test Set Size
rahulgopinath
0
62
Abstracting Failure Inducing Inputs
rahulgopinath
0
65
Learning Grammars without Samples
rahulgopinath
0
120
Other Decks in Research
See All in Research
第10回チャンピオンズミーティング・アクエリアス杯ラウンド2集計 / Umamusume Aquarius 2022 Round2
kitachan_black
0
1.1k
CompilerGym CGO 2022 Tutorial Part 2
chriscummins
0
460
機械学習ベースの動画像処理における近似計算手法の検討 (CPSY 2022/03)
hashi0203
0
150
ICCV 2021 ただただVision and Languageの論文を読んでみた
yushiku
3
1.7k
CVIM_2022_01
keiichirokagawa
0
440
時間情報表現抽出とルールベース解析器のこれから / Temporal Expression Analysis in Japanese and Future of Rule-based Approach
yag_ays
PRO
0
740
深層学習を用いた三次元点群処理入門
nnchiba
4
2.3k
AIOps研究録―SREのための システム障害の自動原因診断 / SRE NEXT 2022
yuukit
5
1.1k
ライティング支援のための文法誤り訂正
chemical_tree
1
1k
ICSE22: An Exploratory Study of Deep Learning Supply Chain
sunflowerpku
0
200
民間企業におけるデータ整備の課題と工夫 / 20220305
yuzutas0
4
3.1k
Iterative source steering を用いたオンライン補助関数型独立ベクトル分析に基づくブラインド音源分離 / Blind source separation using online auxiliary-function-based independent vector analysis with iterative source steering
taishi
0
140
Featured
See All Featured
A better future with KSS
kneath
225
15k
What's in a price? How to price your products and services
michaelherold
229
9.3k
Clear Off the Table
cherdarchuk
79
280k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
7
1k
Embracing the Ebb and Flow
colly
73
3.3k
Large-scale JavaScript Application Architecture
addyosmani
499
110k
How to train your dragon (web standard)
notwaldorf
57
3.8k
Stop Working from a Prison Cell
hatefulcrawdad
261
17k
Side Projects
sachag
449
37k
Visualization
eitanlees
124
11k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
224
49k
Building a Scalable Design System with Sketch
lauravandoore
447
30k
Transcript
9. April 2018 Rahul Gopinath, Björn Mathis and Andreas Zeller
If You Can't Kill a Supermutant You Have a Problem 1
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2 Enormous Amount of Test Executions
Supermutants - Coverage Adequate Test Suite 3
Supermutants - Coverage Adequate Test Suite 0 % 25 %
50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 3
Supermutants - Coverage Adequate Test Suite 0 Coverage Adequate Tests
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 0 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 3
Supermutants - Coverage Adequate Test Suite 1 Coverage Adequate Test
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 1 Coverage Adequate Test 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 3
Supermutants - Coverage Adequate Test Suite 2 Coverage Adequate Tests
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 2 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 3
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutant Execution 5
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests 34 vs 42 Test Executions
Supermutants - Mutant Kill Matrix 6 7 Tests 3 Tests
3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Case Study - Urlparse 7
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 13,009 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000 Over 70% of Test Executions Are Saved
When Are Supermutants Super? 8
When Are Supermutants Super? 8 Many and Weak Tests
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants No Fault Masking
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants No Fault Masking Automatically Generated Test Suites Can Be Efficiently Analyzed
9 Summary
9 Summary
9 Summary
9 Summary
9 Summary