Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
If You Can't Kill a Supermutant, You Have a Pr...
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
Rahul Gopinath
April 09, 2018
Research
350
0
Share
If You Can't Kill a Supermutant, You Have a Problem
Mutation Workshop 2018
Rahul Gopinath
April 09, 2018
More Decks by Rahul Gopinath
See All by Rahul Gopinath
Specification for Free: Behavior-Driven Fuzzing with Inferred Specifications
rahulgopinath
0
7
Evaluating Impact of Coverage Feedback on Estimators for Maximum Reachability in Fuzzing (Registered Report)
rahulgopinath
0
27
Automatic Data Repair without Format Specifications
rahulgopinath
0
37
From Theory to Practice: Blackbox Testing in an Industrial Telecom Environment
rahulgopinath
0
47
Assessing Reliability of Statistical Maximum Coverage Estimators in Fuzzing
rahulgopinath
0
37
Fuzzing Without Specifications: Learning Structure from Behaviour Part I
rahulgopinath
0
69
Fuzzing Without Specifications: Learning Structure from Behaviour Part II
rahulgopinath
0
58
How to Compare Fuzzers
rahulgopinath
0
140
Mutation Analysis: Answering the Fuzzing Challenge
rahulgopinath
0
96
Other Decks in Research
See All in Research
R&Dチームを起ち上げる
shibuiwilliam
1
260
Apache Gravitinoで実現する Icebergカタログ統合とアクセスの一元化
matsumooon
0
250
論文紹介 "ReSim: Reliable World Simulation for Autonomous Driving"
kogo
0
610
明日から使える!研究効率化ツール入門
matsui_528
13
7.2k
Fukui Shibiten 39 - AI Art
butchi
0
110
AIエージェント時代のLLM-jpモデルのあるべき姿
k141303
0
430
通時的な類似度行列に基づく単語の意味変化の分析
rudorudo11
0
310
進学校の生徒にはア行の苗字が多いのか
ozekinote
0
420
コーディングエージェントとABNを再考
hf149
2
700
Ghost in the 7‑Zip: The Shadow of Residential Proxies Creeping into Your Life
nttcom
0
830
Sequences of Logits Reveal the Low Rank Structure of Language Models
sansantech
PRO
1
260
セマンティック通信勉強会 6Gに向けたデバイス間効率的な通信の技術紹介・課題・今後展望
satai
2
140
Featured
See All Featured
Build The Right Thing And Hit Your Dates
maggiecrowley
39
3.2k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
25
1.9k
Designing for Performance
lara
611
70k
Why You Should Never Use an ORM
jnunemaker
PRO
61
9.9k
How People are Using Generative and Agentic AI to Supercharge Their Products, Projects, Services and Value Streams Today
helenjbeal
1
200
Abbi's Birthday
coloredviolet
2
7.9k
What does AI have to do with Human Rights?
axbom
PRO
1
2.2k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
38
2.9k
Stewardship and Sustainability of Urban and Community Forests
pwiseman
0
220
Max Prin - Stacking Signals: How International SEO Comes Together (And Falls Apart)
techseoconnect
PRO
0
170
Reflections from 52 weeks, 52 projects
jeffersonlam
356
21k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4.3k
Transcript
9. April 2018 Rahul Gopinath, Björn Mathis and Andreas Zeller
If You Can't Kill a Supermutant You Have a Problem 1
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2
Mutant Kill Matrix 2 Enormous Amount of Test Executions
Supermutants - Coverage Adequate Test Suite 3
Supermutants - Coverage Adequate Test Suite 0 % 25 %
50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 3
Supermutants - Coverage Adequate Test Suite 0 Coverage Adequate Tests
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 0 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 0 % 93 % 3
Supermutants - Coverage Adequate Test Suite 1 Coverage Adequate Test
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 1 Coverage Adequate Test 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 70 % 93 % 3
Supermutants - Coverage Adequate Test Suite 2 Coverage Adequate Tests
0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 2 Coverage Adequate Tests 0 % 25 % 50 % 75 % 100 % Full Suite (7 Tests) Coverage Adequate 93 % 93 % 3
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutants - Coverage Adequate Test Suite 4
Supermutant Execution 5
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests
Supermutant Execution 5 3 Tests 3 Tests 3 Tests 3
Tests 1 Test 1 Test 7 Tests 34 vs 42 Test Executions
Supermutants - Mutant Kill Matrix 6 7 Tests 3 Tests
3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Supermutants - Mutant Kill Matrix 6 Test 1 Test 2
Test 3 Test 4 Test 5 Test 6 Test 7 ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✕ ✕ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✕ ✓ ✓ ✓ ✓ ✓ ✕ ✕ ✕ ✕ ✕ ✕ ✕ 7 Tests 3 Tests 3 Tests 3 Tests 3 Tests 1 Test 1 Test
Case Study - Urlparse 7
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 13,009 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000
Case Study - Urlparse 7 0 exec. 10000 exec. 20000
exec. 30000 exec. 40000 exec. Full Matrix Strong Testsuite Weak Testsuite 9,037 13,009 32,000 Over 70% of Test Executions Are Saved
When Are Supermutants Super? 8
When Are Supermutants Super? 8 Many and Weak Tests
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants No Fault Masking
When Are Supermutants Super? 8 Many and Weak Tests Stubborn
Mutants No Fault Masking Automatically Generated Test Suites Can Be Efficiently Analyzed
9 Summary
9 Summary
9 Summary
9 Summary
9 Summary