Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Eeek, my tests are mutating
Search
Lander Vanderstraeten
November 07, 2017
Programming
1
94
Eeek, my tests are mutating
Presentation given for
https://www.meetup.com/phpgent/events/242279704
Lander Vanderstraeten
November 07, 2017
Tweet
Share
Other Decks in Programming
See All in Programming
副作用と戦う PHP リファクタリング ─ ドメインイベントでビジネスロジックを解きほぐす
kajitack
3
520
AIに安心して任せるためにTypeScriptで一意な型を作ろう
arfes0e2b3c
0
330
オホーツクでコミュニティを立ち上げた理由―地方出身プログラマの挑戦 / TechRAMEN 2025 Conference
lemonade_37
1
430
DataformでPythonする / dataform-de-python
snhryt
0
150
知って得する@cloudflare_vite-pluginのあれこれ
chimame
1
140
decksh - a little language for decks
ajstarks
4
21k
AIコーディングエージェント全社導入とセキュリティ対策
hikaruegashira
15
9.3k
MCPで実現できる、Webサービス利用体験について
syumai
7
2.4k
Jakarta EE Meets AI
ivargrimstad
0
570
Strands Agents で実現する名刺解析アーキテクチャ
omiya0555
1
110
Terraform やるなら公式スタイルガイドを読もう 〜重要項目 10選〜
hiyanger
11
2.8k
画像コンペでのベースラインモデルの育て方
tattaka
3
1k
Featured
See All Featured
The Language of Interfaces
destraynor
158
25k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3k
Bash Introduction
62gerente
614
210k
Thoughts on Productivity
jonyablonski
69
4.8k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
4 Signs Your Business is Dying
shpigford
184
22k
GitHub's CSS Performance
jonrohan
1031
460k
Adopting Sorbet at Scale
ufuk
77
9.5k
Side Projects
sachag
455
43k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.6k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
182
54k
Transcript
HI, I'M LANDER VANDERSTRAETEN 1
3 QUIZ QUESTIONS 2
WHO THINKS UNIT TESTS ARE WASTE OF TIME? 3
4
WHO ALWAYS WRITES UNIT TESTS? 5
6
WHO ALREADY USES MUTATION TESTING? 7
8
CODE HAS BUGS TESTS ARE CODE TESTS HAVE BUGS! 9
10
TESTS ARE A GOOD TOOL TO VERIFY YOUR CODE BUT
HOW DO YOU VERIFY YOUR TESTS? 11
CODE COVERAGE IS A START, BUT IT CAN GIVE A
“GOOD” SCORE WITH USELESS TESTS 12
ALL TESTS PASSED! 13
HOW WE DETECT? 14
MUTATION TESTING 15
Mutation testing is a technique to evaluate the quality of
your tests by programmatically mutating and making a series of small modifications to your code so that the tests no longer pass. 16
17
IF BUGS ARE CRIMES AND YOUR TESTS ARE THE POLICE
MUTATIONS ARE FAKE CRIMES THAT LET YOU SEE THE POLICE IS DOING THEIR JOB 18
TO ASSESS THE QUALITY OF A GIVEN TEST, MUTANTS ARE
EXECUTED AGAINST THE INPUT TEST TO SEE IF THE SEEDED FAULTS CAN BE DETECTED. 19
MUTATION, MUTANTS, MUTAFUCK WHAT? 20
Each mutated version: Mutant Mutated program + failing tests: killed
mutant Mutated program + passed tests: escaped mutant 21
MEASURE ALL THE THINGS! 22
Test suites are measured by the percentage of mutants that
they kill. New tests can be designed to kill additional mutants. 23
HOW DOES IT MODIFY? 24
MUTATORS 25
EXAMPLE public function foobar(int $number): int { return $number +
1; // original } public function foobar(int $number): int { return $number - 1; // mutated version } 26
ANOTHER EXAMPLE public function foobar(?Bar $foo): string { if (null
=== $foo) { // original return 'foo'; } // ... } public function foobar(?Bar $foo): string { if (null !== $foo) { // mutated version return 'foo'; } // ... } 27
DEMO TIME WITH INFECTION 28
WHAT DID WE LEARN TODAY? 29
1. Always write unit tests 2. Code coverage gives a
false positive feeling 3. Mutation testing also gives a false positive feeling 30
QUESTIONS? 31