Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Eeek, my tests are mutating
Search
Lander Vanderstraeten
November 07, 2017
Programming
1
94
Eeek, my tests are mutating
Presentation given for
https://www.meetup.com/phpgent/events/242279704
Lander Vanderstraeten
November 07, 2017
Tweet
Share
Other Decks in Programming
See All in Programming
Introduction to Git & GitHub
latte72
0
120
AIエージェント開発、DevOps and LLMOps
ymd65536
1
350
レガシープロジェクトで最大限AIの恩恵を受けられるようClaude Codeを利用する
tk1351
3
1.3k
Flutter로 Gemini와 MCP를 활용한 Agentic App 만들기 - 박제창 2025 I/O Extended Seoul
itsmedreamwalker
0
150
なぜ今、Terraformの本を書いたのか? - 著者陣に聞く!『Terraformではじめる実践IaC』登壇資料
fufuhu
4
660
The state patternの実践 個人開発で培ったpractice集
miyanokomiya
0
150
コーディングは技術者(エンジニア)の嗜みでして / Learning the System Development Mindset from Rock Lady
mackey0225
2
580
TanStack DB ~状態管理の新しい考え方~
bmthd
2
340
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
190
CSC305 Summer Lecture 06
javiergs
PRO
0
100
Oracle Database Technology Night 92 Database Connection control FAN-AC
oracle4engineer
PRO
1
220
コーディングエージェント時代のNeovim
key60228
1
110
Featured
See All Featured
Balancing Empowerment & Direction
lara
2
590
Why Our Code Smells
bkeepers
PRO
338
57k
Visualization
eitanlees
147
16k
How GitHub (no longer) Works
holman
315
140k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.5k
The Art of Programming - Codeland 2020
erikaheidi
55
13k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.9k
[RailsConf 2023] Rails as a piece of cake
palkan
56
5.8k
4 Signs Your Business is Dying
shpigford
184
22k
The Straight Up "How To Draw Better" Workshop
denniskardys
236
140k
Code Reviewing Like a Champion
maltzj
525
40k
GitHub's CSS Performance
jonrohan
1031
460k
Transcript
HI, I'M LANDER VANDERSTRAETEN 1
3 QUIZ QUESTIONS 2
WHO THINKS UNIT TESTS ARE WASTE OF TIME? 3
4
WHO ALWAYS WRITES UNIT TESTS? 5
6
WHO ALREADY USES MUTATION TESTING? 7
8
CODE HAS BUGS TESTS ARE CODE TESTS HAVE BUGS! 9
10
TESTS ARE A GOOD TOOL TO VERIFY YOUR CODE BUT
HOW DO YOU VERIFY YOUR TESTS? 11
CODE COVERAGE IS A START, BUT IT CAN GIVE A
“GOOD” SCORE WITH USELESS TESTS 12
ALL TESTS PASSED! 13
HOW WE DETECT? 14
MUTATION TESTING 15
Mutation testing is a technique to evaluate the quality of
your tests by programmatically mutating and making a series of small modifications to your code so that the tests no longer pass. 16
17
IF BUGS ARE CRIMES AND YOUR TESTS ARE THE POLICE
MUTATIONS ARE FAKE CRIMES THAT LET YOU SEE THE POLICE IS DOING THEIR JOB 18
TO ASSESS THE QUALITY OF A GIVEN TEST, MUTANTS ARE
EXECUTED AGAINST THE INPUT TEST TO SEE IF THE SEEDED FAULTS CAN BE DETECTED. 19
MUTATION, MUTANTS, MUTAFUCK WHAT? 20
Each mutated version: Mutant Mutated program + failing tests: killed
mutant Mutated program + passed tests: escaped mutant 21
MEASURE ALL THE THINGS! 22
Test suites are measured by the percentage of mutants that
they kill. New tests can be designed to kill additional mutants. 23
HOW DOES IT MODIFY? 24
MUTATORS 25
EXAMPLE public function foobar(int $number): int { return $number +
1; // original } public function foobar(int $number): int { return $number - 1; // mutated version } 26
ANOTHER EXAMPLE public function foobar(?Bar $foo): string { if (null
=== $foo) { // original return 'foo'; } // ... } public function foobar(?Bar $foo): string { if (null !== $foo) { // mutated version return 'foo'; } // ... } 27
DEMO TIME WITH INFECTION 28
WHAT DID WE LEARN TODAY? 29
1. Always write unit tests 2. Code coverage gives a
false positive feeling 3. Mutation testing also gives a false positive feeling 30
QUESTIONS? 31