Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Who watches the watchmen: testing our tests
Search
Maria Clara Santana
July 05, 2018
Programming
1
210
Who watches the watchmen: testing our tests
The talk was given at JS Experience 2018.
https://eventos.imasters.com.br/jsexperience
Maria Clara Santana
July 05, 2018
Tweet
Share
More Decks by Maria Clara Santana
See All by Maria Clara Santana
You might also like...
olarclara
0
160
Accessibility beyond the aria-label
olarclara
1
200
React nas Trincheiras
olarclara
3
310
ImmutableJS 101
olarclara
0
120
Other Decks in Programming
See All in Programming
What's new in Spring Modulith?
olivergierke
1
160
Cursorハンズオン実践!
eltociear
2
1.2k
デミカツ切り抜きで面倒くさいことはPythonにやらせよう
aokswork3
0
260
Leading Effective Engineering Teams in the AI Era
addyosmani
7
570
CSC509 Lecture 05
javiergs
PRO
0
310
Goで実践するドメイン駆動開発 AIと歩み始めた新規プロダクト開発の現在地
imkaoru
4
880
20251016_Rails News ~Rails 8.1の足音を聴く~
morimorihoge
2
670
pnpm に provenance のダウングレード を検出する PR を出してみた
ryo_manba
1
150
SODA - FACT BOOK(JP)
sodainc
1
8.6k
はじめてのDSPy - 言語モデルを『プロンプト』ではなく『プログラミング』するための仕組み
masahiro_nishimi
3
12k
チームの境界をブチ抜いていけ
tokai235
0
210
XP, Testing and ninja testing ZOZ5
m_seki
3
810
Featured
See All Featured
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.2k
Code Review Best Practice
trishagee
72
19k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3.1k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.6k
Typedesign – Prime Four
hannesfritz
42
2.8k
Leading Effective Engineering Teams in the AI Era
addyosmani
7
570
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.7k
Building a Scalable Design System with Sketch
lauravandoore
463
33k
Thoughts on Productivity
jonyablonski
70
4.9k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
34
2.3k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.5k
How to train your dragon (web standard)
notwaldorf
97
6.3k
Transcript
“WHO WATCHES THE WATCHMEN” Testing our tests Maria C. Santana
JS Experience ‘18
Maria Clara • Front-end Developer @ Picter • Live Coding
Instructor @ Udacity • Likes dogs and sparkling water
Types of software errors • Construction errors; • Specification errors;
• Design errors; • Requirements errors; Toward a theory of test data selection, Softech Inc.
BLACK-BOX TESTING
Input Output
None
None
•Pro: Allows to check the functionality of a program without
worrying with the structure behind it; •Con: Does not offer insights about the tested program’s structure;
High code coverage !== Effective Test Suite
Source: https://gph.is/1jE2Wzg
None
None
Take it with a grain of salt.
“Quis custodiet ipsos custodes?” Satires, 2a.c.
WHITE-BOX TESTING
Input Output
None
None
•Pro: More thorough tests; •Con: Can be complex and require
a lot of work due to implementation changes;
MUTATION TESTING
CONCEPT
#1 STEP Test data Original code
•If it fails, then the original program is wrong; •If
it passes, then we should proceed with creating mutants;
#2 STEP Test data N modified versions of the original
code
•#1: given the test data, the modified versions have different
outputs from the original one; •#2: some (or all of them) versions have the same output of the original one;
MUTATORS TYPES
Not this kind, though…
AAR (array for array replacement)
ABS (absolute value insertion)
AOR (arithmetic operator replacement)
UOI (unary operation insertion)
Many others… • Array constant replacement; • Logical connector replacement;
• Relational operator replacement; • Constant replacement; • Return statement replacement; • Statement deletion; • …
EXAMPLE
Original program
Test suite
Mutant #1 - Conditional operator replacement
Mutant #1 - Test suite results Mutant was killed!
Mutant #2 - Conditional operator replacement
Mutant #2 - Test suite results Mutant has survived… ☹
EQUIVALENT MUTANTS
None
Mutant #3 - Return statement replacement
Mutant #3 - Test suite results Mutant was killed!
mutation score = mutants killed / total of mutants; ->
100%
TOOLS
https://stryker-mutator.io/
Trade-offs •Open-source; •Active maintenance and improvements; •Supports most of the
mainstream runners (karma, jasmine, jest, …); •Does not support React;
Other languages… • Python: https://github.com/mutpy/mutpy; • Ruby: https://github.com/mbj/mutant; • Java:
https://github.com/hcoles/pitest;
PROS
•Application safety; •Evaluation of edge cases; •Better test design;
CONS
•High computational costs; •Filtering equivalent mutants is still a non-optimal
task;
Q&A
THANKS! olarclara.github.io