Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Eeek, my tests are mutating
Search
Lander Vanderstraeten
November 07, 2017
Programming
1
91
Eeek, my tests are mutating
Presentation given for
https://www.meetup.com/phpgent/events/242279704
Lander Vanderstraeten
November 07, 2017
Tweet
Share
Other Decks in Programming
See All in Programming
PHP8.3の機能を振り返る / Review of PHP 8.3 features
seike460
PRO
1
120
冗長なエラーログを削減し、スタックトレースを手に入れる / Reducing Verbose Error Logs and Obtaining Stack Traces
upamune
0
1k
#phpcon_odawara オープン・クローズドなテストフィクスチャを求めて / open closed test fixtures
77web
3
240
Polars入門
daikikatsuragawa
1
170
Go製Webアプリケーションのエラーとの向き合い方大全、あるいはやっぱりスタックトレース欲しいやん / Kyoto.go #50
utgwkk
6
1.8k
Deep Dive into React Stream/Serialize
mugi_uno
3
670
Try creating your own orderedmap
kazamori
1
190
Next.js App Router
quramy
12
1.8k
Sheets API使ってみた
toshi0383
2
160
Elm 0.19.0 Changes
bkuhlmann
0
510
サイコロで理解する統計的仮説検定の考え方
tatamiya
4
1k
Apache Hive 4 on Treasure Data
ryukobayashi
1
420
Featured
See All Featured
The Mythical Team-Month
searls
216
42k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
221
21k
Statistics for Hackers
jakevdp
790
220k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
117
18k
Building Applications with DynamoDB
mza
88
5.6k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
660
120k
Build your cross-platform service in a week with App Engine
jlugia
226
17k
Building Flexible Design Systems
yeseniaperezcruz
320
37k
Building a Scalable Design System with Sketch
lauravandoore
457
32k
Large-scale JavaScript Application Architecture
addyosmani
504
110k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
275
13k
Documentation Writing (for coders)
carmenintech
60
4k
Transcript
HI, I'M LANDER VANDERSTRAETEN 1
3 QUIZ QUESTIONS 2
WHO THINKS UNIT TESTS ARE WASTE OF TIME? 3
4
WHO ALWAYS WRITES UNIT TESTS? 5
6
WHO ALREADY USES MUTATION TESTING? 7
8
CODE HAS BUGS TESTS ARE CODE TESTS HAVE BUGS! 9
10
TESTS ARE A GOOD TOOL TO VERIFY YOUR CODE BUT
HOW DO YOU VERIFY YOUR TESTS? 11
CODE COVERAGE IS A START, BUT IT CAN GIVE A
“GOOD” SCORE WITH USELESS TESTS 12
ALL TESTS PASSED! 13
HOW WE DETECT? 14
MUTATION TESTING 15
Mutation testing is a technique to evaluate the quality of
your tests by programmatically mutating and making a series of small modifications to your code so that the tests no longer pass. 16
17
IF BUGS ARE CRIMES AND YOUR TESTS ARE THE POLICE
MUTATIONS ARE FAKE CRIMES THAT LET YOU SEE THE POLICE IS DOING THEIR JOB 18
TO ASSESS THE QUALITY OF A GIVEN TEST, MUTANTS ARE
EXECUTED AGAINST THE INPUT TEST TO SEE IF THE SEEDED FAULTS CAN BE DETECTED. 19
MUTATION, MUTANTS, MUTAFUCK WHAT? 20
Each mutated version: Mutant Mutated program + failing tests: killed
mutant Mutated program + passed tests: escaped mutant 21
MEASURE ALL THE THINGS! 22
Test suites are measured by the percentage of mutants that
they kill. New tests can be designed to kill additional mutants. 23
HOW DOES IT MODIFY? 24
MUTATORS 25
EXAMPLE public function foobar(int $number): int { return $number +
1; // original } public function foobar(int $number): int { return $number - 1; // mutated version } 26
ANOTHER EXAMPLE public function foobar(?Bar $foo): string { if (null
=== $foo) { // original return 'foo'; } // ... } public function foobar(?Bar $foo): string { if (null !== $foo) { // mutated version return 'foo'; } // ... } 27
DEMO TIME WITH INFECTION 28
WHAT DID WE LEARN TODAY? 29
1. Always write unit tests 2. Code coverage gives a
false positive feeling 3. Mutation testing also gives a false positive feeling 30
QUESTIONS? 31