Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Who watches the watchmen: testing our tests
Search
Maria Clara Santana
July 05, 2018
Programming
1
210
Who watches the watchmen: testing our tests
The talk was given at JS Experience 2018.
https://eventos.imasters.com.br/jsexperience
Maria Clara Santana
July 05, 2018
Tweet
Share
More Decks by Maria Clara Santana
See All by Maria Clara Santana
You might also like...
olarclara
0
110
Accessibility beyond the aria-label
olarclara
1
160
React nas Trincheiras
olarclara
3
280
ImmutableJS 101
olarclara
0
110
Other Decks in Programming
See All in Programming
Google's Recipe for Scaling (Web) Security – LocoMocoSec 2024
lweichselbaum
0
170
Clean Architecture by TypeScript & NestJS
ryounasso
0
150
20240706_CDKConf
takuyay0ne
0
1.2k
実用的かつリーズナブルな 「Azure × Gemini × LINE」~キャラクターBot 実装ライブデモ~
tomodo_ysys
1
170
Javaの現状2024夏 / Java current status 2024 summer
kishida
4
1.4k
Prompt FlowによるLLMアプリケーション開発
yuto2000
1
1k
CSC307 Lecture 09
javiergs
PRO
1
500
Advanced App Shrinking Techniques
cbeyls
2
150
生成AIをkintoneに連携してみた
hideg
0
230
Jetpack for KMP
fornewid
1
290
スクラムマスターって孤独じゃないですか?
yoshitaroyoyo
1
140
君たちはどうコードをレビューする (される) か / 大吉祥寺.pm
utgwkk
15
8.5k
Featured
See All Featured
Designing on Purpose - Digital PM Summit 2013
jponch
113
6.6k
5 minutes of I Can Smell Your CMS
philhawksworth
200
19k
Happy Clients
brianwarren
94
6.6k
Why Our Code Smells
bkeepers
PRO
332
56k
4 Signs Your Business is Dying
shpigford
178
21k
Rails Girls Zürich Keynote
gr2m
93
13k
The MySQL Ecosystem @ GitHub 2015
samlambert
248
12k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
26
1.8k
Raft: Consensus for Rubyists
vanstee
134
6.5k
Making Projects Easy
brettharned
111
5.7k
Ruby is Unlike a Banana
tanoku
96
10k
The Illustrated Children's Guide to Kubernetes
chrisshort
39
47k
Transcript
“WHO WATCHES THE WATCHMEN” Testing our tests Maria C. Santana
JS Experience ‘18
Maria Clara • Front-end Developer @ Picter • Live Coding
Instructor @ Udacity • Likes dogs and sparkling water
Types of software errors • Construction errors; • Specification errors;
• Design errors; • Requirements errors; Toward a theory of test data selection, Softech Inc.
BLACK-BOX TESTING
Input Output
None
None
•Pro: Allows to check the functionality of a program without
worrying with the structure behind it; •Con: Does not offer insights about the tested program’s structure;
High code coverage !== Effective Test Suite
Source: https://gph.is/1jE2Wzg
None
None
Take it with a grain of salt.
“Quis custodiet ipsos custodes?” Satires, 2a.c.
WHITE-BOX TESTING
Input Output
None
None
•Pro: More thorough tests; •Con: Can be complex and require
a lot of work due to implementation changes;
MUTATION TESTING
CONCEPT
#1 STEP Test data Original code
•If it fails, then the original program is wrong; •If
it passes, then we should proceed with creating mutants;
#2 STEP Test data N modified versions of the original
code
•#1: given the test data, the modified versions have different
outputs from the original one; •#2: some (or all of them) versions have the same output of the original one;
MUTATORS TYPES
Not this kind, though…
AAR (array for array replacement)
ABS (absolute value insertion)
AOR (arithmetic operator replacement)
UOI (unary operation insertion)
Many others… • Array constant replacement; • Logical connector replacement;
• Relational operator replacement; • Constant replacement; • Return statement replacement; • Statement deletion; • …
EXAMPLE
Original program
Test suite
Mutant #1 - Conditional operator replacement
Mutant #1 - Test suite results Mutant was killed!
Mutant #2 - Conditional operator replacement
Mutant #2 - Test suite results Mutant has survived… ☹
EQUIVALENT MUTANTS
None
Mutant #3 - Return statement replacement
Mutant #3 - Test suite results Mutant was killed!
mutation score = mutants killed / total of mutants; ->
100%
TOOLS
https://stryker-mutator.io/
Trade-offs •Open-source; •Active maintenance and improvements; •Supports most of the
mainstream runners (karma, jasmine, jest, …); •Does not support React;
Other languages… • Python: https://github.com/mutpy/mutpy; • Ruby: https://github.com/mbj/mutant; • Java:
https://github.com/hcoles/pitest;
PROS
•Application safety; •Evaluation of edge cases; •Better test design;
CONS
•High computational costs; •Filtering equivalent mutants is still a non-optimal
task;
Q&A
THANKS! olarclara.github.io