Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Eeek, my tests are mutating
Search
Lander Vanderstraeten
November 07, 2017
Programming
1
94
Eeek, my tests are mutating
Presentation given for
https://www.meetup.com/phpgent/events/242279704
Lander Vanderstraeten
November 07, 2017
Tweet
Share
Other Decks in Programming
See All in Programming
テスターからテストエンジニアへ ~新米テストエンジニアが歩んだ9ヶ月振り返り~
non0113
2
220
PHPでWebSocketサーバーを実装しよう2025
kubotak
0
320
Vibe Codingの幻想を超えて-生成AIを現場で使えるようにするまでの泥臭い話.ai
fumiyakume
10
4.3k
코딩 에이전트 체크리스트: Claude Code ver.
nacyot
0
930
ご注文の差分はこちらですか? 〜 AWS CDK のいろいろな差分検出と安全なデプロイ
konokenj
3
580
ニーリーにおけるプロダクトエンジニア
nealle
0
950
AI時代のソフトウェア開発を考える(2025/07版) / Agentic Software Engineering Findy 2025-07 Edition
twada
PRO
99
37k
ソフトウェア設計とAI技術の活用
masuda220
PRO
17
3.8k
「テストは愚直&&網羅的に書くほどよい」という誤解 / Test Smarter, Not Harder
munetoshi
0
200
チームのテスト力を総合的に鍛えて品質、スピード、レジリエンスを共立させる/Testing approach that improves quality, speed, and resilience
goyoki
5
1.1k
Modern Angular with Signals and Signal Store:New Rules for Your Architecture @enterJS Advanced Angular Day 2025
manfredsteyer
PRO
0
270
はじめてのWeb API体験 ー 飲食店検索アプリを作ろうー
akinko_0915
0
140
Featured
See All Featured
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.5k
Done Done
chrislema
184
16k
The Straight Up "How To Draw Better" Workshop
denniskardys
235
140k
How to train your dragon (web standard)
notwaldorf
96
6.1k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Java REST API Framework Comparison - PWX 2021
mraible
31
8.7k
RailsConf 2023
tenderlove
30
1.1k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
60k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
8
700
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
GraphQLとの向き合い方2022年版
quramy
49
14k
Embracing the Ebb and Flow
colly
86
4.8k
Transcript
HI, I'M LANDER VANDERSTRAETEN 1
3 QUIZ QUESTIONS 2
WHO THINKS UNIT TESTS ARE WASTE OF TIME? 3
4
WHO ALWAYS WRITES UNIT TESTS? 5
6
WHO ALREADY USES MUTATION TESTING? 7
8
CODE HAS BUGS TESTS ARE CODE TESTS HAVE BUGS! 9
10
TESTS ARE A GOOD TOOL TO VERIFY YOUR CODE BUT
HOW DO YOU VERIFY YOUR TESTS? 11
CODE COVERAGE IS A START, BUT IT CAN GIVE A
“GOOD” SCORE WITH USELESS TESTS 12
ALL TESTS PASSED! 13
HOW WE DETECT? 14
MUTATION TESTING 15
Mutation testing is a technique to evaluate the quality of
your tests by programmatically mutating and making a series of small modifications to your code so that the tests no longer pass. 16
17
IF BUGS ARE CRIMES AND YOUR TESTS ARE THE POLICE
MUTATIONS ARE FAKE CRIMES THAT LET YOU SEE THE POLICE IS DOING THEIR JOB 18
TO ASSESS THE QUALITY OF A GIVEN TEST, MUTANTS ARE
EXECUTED AGAINST THE INPUT TEST TO SEE IF THE SEEDED FAULTS CAN BE DETECTED. 19
MUTATION, MUTANTS, MUTAFUCK WHAT? 20
Each mutated version: Mutant Mutated program + failing tests: killed
mutant Mutated program + passed tests: escaped mutant 21
MEASURE ALL THE THINGS! 22
Test suites are measured by the percentage of mutants that
they kill. New tests can be designed to kill additional mutants. 23
HOW DOES IT MODIFY? 24
MUTATORS 25
EXAMPLE public function foobar(int $number): int { return $number +
1; // original } public function foobar(int $number): int { return $number - 1; // mutated version } 26
ANOTHER EXAMPLE public function foobar(?Bar $foo): string { if (null
=== $foo) { // original return 'foo'; } // ... } public function foobar(?Bar $foo): string { if (null !== $foo) { // mutated version return 'foo'; } // ... } 27
DEMO TIME WITH INFECTION 28
WHAT DID WE LEARN TODAY? 29
1. Always write unit tests 2. Code coverage gives a
false positive feeling 3. Mutation testing also gives a false positive feeling 30
QUESTIONS? 31