Mutation Testing - Backend Meetup @ STRV

Slide 1

Slide 1 text

MUTATION TESTING Ruan Martinelli, Backend Developer at STRV

Slide 2

Slide 2 text

2 https://en.wikipedia.org/wiki/Category:Software_testing

Slide 3

Slide 3 text

1. REPLACE "AND" WITH "OR" 3

Slide 4

Slide 4 text

2. REPLACE "+" WITH "-" 4

Slide 5

Slide 5 text

3. CODE COVERAGE 5

Slide 6

Slide 6 text

3. CODE COVERAGE 6

Slide 7

Slide 7 text

3. CODE COVERAGE 7

Slide 8

Slide 8 text

“What if I told you there is a codebase that is not yours and it passes all your tests? What if told you there are 200?” 8

Slide 9

Slide 9 text

Mutations are small "faults" or "bugs" inserted into your code. When a mutation is applied to the code, a mutant is created. Mutation testing: 1. Change your code in small ways 2. Run the test suite once for each changed version 3. Analyze results - If one or more test breaks, it’s said that the mutant was killed - If all tests passes, it’s said that the mutant survived MUTATIONS, MUTANTS & MUTATION TESTING 9

Slide 10

Slide 10 text

4. LIST OF COMMON MUTATIONS 10

Slide 11

Slide 11 text

List of Mutation Testing tools: ● Java - PIT ● Ruby - Mutant ● PHP - Infection ● C# - VisualMutator & Stryker.NET ● Python - MutPy ● JavaScript - Stryker TOOLING 11

Slide 12

Slide 12 text

DEMO 12

Slide 13

Slide 13 text

KOA.JS 13 ● Koa.js has 345 tests which take ~3s to run ● 4 ﬁles were mutated, 793 mutants were created ● Total execution time was 12m28s (8 cores)

Slide 14

Slide 14 text

KOA.JS 14

Slide 15

Slide 15 text

KOA.JS 15

Slide 16

Slide 16 text

KOA.JS 16

Slide 17

Slide 17 text

KOA.JS 17

Slide 18

Slide 18 text

KOA.JS 18

Slide 19

Slide 19 text

KOA.JS 19

Slide 20

Slide 20 text

KOA.JS 20

Slide 21

Slide 21 text

Mutation testing is resource and time-consuming ● Run tests in the cloud periodically ● Parallelism and distributed computing ● Diff-based approach ● AI & Machine learning ● Run only on critical parts of your system GOING FURTHER 21