Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Eeek, my tests are mutating
Search
Lander Vanderstraeten
November 07, 2017
Programming
1
94
Eeek, my tests are mutating
Presentation given for
https://www.meetup.com/phpgent/events/242279704
Lander Vanderstraeten
November 07, 2017
Tweet
Share
Other Decks in Programming
See All in Programming
20250704_教育事業におけるアジャイルなデータ基盤構築
hanon52_
5
780
Deep Dive into ~/.claude/projects
hiragram
14
2.5k
AI コーディングエージェントの時代へ:JetBrains が描く開発の未来
masaruhr
1
140
技術同人誌をMCP Serverにしてみた
74th
1
640
VS Code Update for GitHub Copilot
74th
2
640
「Cursor/Devin全社導入の理想と現実」のその後
saitoryc
0
820
What Spring Developers Should Know About Jakarta EE
ivargrimstad
0
470
Discover Metal 4
rei315
2
130
AIプログラマーDevinは PHPerの夢を見るか?
shinyasaita
1
220
Is Xcode slowly dying out in 2025?
uetyo
1
270
ISUCON研修おかわり会 講義スライド
arfes0e2b3c
1
440
Rubyでやりたい駆動開発 / Ruby driven development
chobishiba
1
700
Featured
See All Featured
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Site-Speed That Sticks
csswizardry
10
690
Docker and Python
trallard
44
3.5k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
820
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.4k
Producing Creativity
orderedlist
PRO
346
40k
Balancing Empowerment & Direction
lara
1
430
How to Think Like a Performance Engineer
csswizardry
25
1.7k
Practical Orchestrator
shlominoach
189
11k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
510
Navigating Team Friction
lara
187
15k
Transcript
HI, I'M LANDER VANDERSTRAETEN 1
3 QUIZ QUESTIONS 2
WHO THINKS UNIT TESTS ARE WASTE OF TIME? 3
4
WHO ALWAYS WRITES UNIT TESTS? 5
6
WHO ALREADY USES MUTATION TESTING? 7
8
CODE HAS BUGS TESTS ARE CODE TESTS HAVE BUGS! 9
10
TESTS ARE A GOOD TOOL TO VERIFY YOUR CODE BUT
HOW DO YOU VERIFY YOUR TESTS? 11
CODE COVERAGE IS A START, BUT IT CAN GIVE A
“GOOD” SCORE WITH USELESS TESTS 12
ALL TESTS PASSED! 13
HOW WE DETECT? 14
MUTATION TESTING 15
Mutation testing is a technique to evaluate the quality of
your tests by programmatically mutating and making a series of small modifications to your code so that the tests no longer pass. 16
17
IF BUGS ARE CRIMES AND YOUR TESTS ARE THE POLICE
MUTATIONS ARE FAKE CRIMES THAT LET YOU SEE THE POLICE IS DOING THEIR JOB 18
TO ASSESS THE QUALITY OF A GIVEN TEST, MUTANTS ARE
EXECUTED AGAINST THE INPUT TEST TO SEE IF THE SEEDED FAULTS CAN BE DETECTED. 19
MUTATION, MUTANTS, MUTAFUCK WHAT? 20
Each mutated version: Mutant Mutated program + failing tests: killed
mutant Mutated program + passed tests: escaped mutant 21
MEASURE ALL THE THINGS! 22
Test suites are measured by the percentage of mutants that
they kill. New tests can be designed to kill additional mutants. 23
HOW DOES IT MODIFY? 24
MUTATORS 25
EXAMPLE public function foobar(int $number): int { return $number +
1; // original } public function foobar(int $number): int { return $number - 1; // mutated version } 26
ANOTHER EXAMPLE public function foobar(?Bar $foo): string { if (null
=== $foo) { // original return 'foo'; } // ... } public function foobar(?Bar $foo): string { if (null !== $foo) { // mutated version return 'foo'; } // ... } 27
DEMO TIME WITH INFECTION 28
WHAT DID WE LEARN TODAY? 29
1. Always write unit tests 2. Code coverage gives a
false positive feeling 3. Mutation testing also gives a false positive feeling 30
QUESTIONS? 31