About me
Matthew Vern
Twitter
Github
Mercari, Inc
Software Engineer (Android)
@panini_ja
panpanini
2/45
Slide 3
Slide 3 text
Client Engineer
Solving problems for our customers
Shipping features
Improve existing functionality
My job
3/45
Slide 4
Slide 4 text
A non-shipped feature doesn't provide benefit
Ship features as quick as possible
My job
4/45
Slide 5
Slide 5 text
A shipped, broken feature doesn't provide benefit
Ship quality features as quick as possible
My job
5/45
Slide 6
Slide 6 text
Maintaining quality
6/45
Slide 7
Slide 7 text
QA
Maintaining quality
6/45
Slide 8
Slide 8 text
QA
Code Review
Maintaining quality
6/45
Slide 9
Slide 9 text
QA
Code Review
Tests
Maintaining quality
6/45
Slide 10
Slide 10 text
How do we know our tests are providing quality
Maintaining quality
7/45
Slide 11
Slide 11 text
How do we know our tests are providing quality
Use coverage to make sure that our tests are
calling production code
Maintaining quality
7/45
Slide 12
Slide 12 text
How do we know our tests are providing quality
Use coverage to make sure that our tests are
calling production code
changes introduced will not break existing code
Maintaining quality
7/45
Slide 13
Slide 13 text
How do we know our tests are providing quality
Use coverage to make sure that our tests are
calling production code
changes introduced will not break existing code
new code does what it says on the tin
Maintaining quality
7/45
Slide 14
Slide 14 text
Who watches the
watchmen?
8/45
Slide 15
Slide 15 text
How do we know that our tests
are quality?
Maintaining quality
9/45
Slide 16
Slide 16 text
What are tests
10/45
Slide 17
Slide 17 text
asserting that our assumptions about a piece of
code are correct
binary assertions of code correctness
What are tests
11/45
Slide 18
Slide 18 text
Lets fail
some tests
12/45
Slide 19
Slide 19 text
Unit tests assert code behaviour
change code behaviour
tests fail
????
profit
Lets fail some tests
13/45
Slide 20
Slide 20 text
Mutation
testing
14/45
Slide 21
Slide 21 text
proposed by Richard Lipton in 1971
computationally expensive, not a viable testing
solution until recently
Mutation testing
15/45
Slide 22
Slide 22 text
1. Create a mutant
2. Run test suite
3. Confirm if mutant was detected or not
4. Repeat
Mutation testing steps
16/45
Slide 23
Slide 23 text
A mutant is a biological entity which has
undergone a change in its genetic structure.
What is a mutant?
17/45
Slide 24
Slide 24 text
A mutant is a code block which has undergone a
change in its structure.
What is a mutant?
18/45
Slide 25
Slide 25 text
class SessionController(
private val sessions: MutableList
) : EpoxyController() {
fun setSessions(sessions: List) {}
override fun buildModels() {}
fun generateModels(sessions: List): List {}
}
Creating mutations
19/45
Replaces relational operators with boundary
counterpart
Original
Original Mutated
Mutated
< <=
<= <
> >=
>= >
Conditionals boundary
24/45
Slide 33
Slide 33 text
Conditionals boundary
original
if (currentTime < startTime) {
do something
}
mutated
if (currentTime startTime) {
do something
}
25/45
Slide 34
Slide 34 text
Negates conditional checks
Original
Original Mutated
Mutated
== !=
!= ==
<= >
> <=
Negate Conditionals
26/45
Slide 35
Slide 35 text
Negate Conditionals
original
fun buildModels() {
SessionModel_()
.title(session.title)
.imageUrl(
if (session.speaker.profileImage "") {
session.speaker.profileImage
} else {
null
}
)
}
mutated
fun buildModels() {
SessionModel_()
.title(session.title)
.imageUrl(
if (session.speaker.profileImage "") {
session.speaker.profileImage
} else {
null
}
)
}
27/45
Slide 36
Slide 36 text
removes void method calls
Remove void calls
28/45
Slide 37
Slide 37 text
Remove void calls
original
fun setSessions(sessions: List) {
this.sessions.clear()
this.sessions.addAll(sessions)
requestModelBuild()
}
mutated
fun setSessions(sessions: List) {
this.sessions.clear()
this.sessions.addAll(sessions)
}
29/45
Slide 38
Slide 38 text
So what?
30/45
Slide 39
Slide 39 text
Code coverage, but better
Why mutation testing
31/45
Slide 40
Slide 40 text
1. Introduce a fault into production code
2. Use code coverage to determine which tests to
run
3. Run tests
4. Confirm if fault was detected or not
5. Repeat
Mutation testing
The better way
32/45
Slide 41
Slide 41 text
That's a lot of work you
expect us to do there
bud
33/45
Slide 42
Slide 42 text
Pitest
34/45
Slide 43
Slide 43 text
mutation testing system
mutants stored in memory
outputs pretty reports
Gradle plugin
Pitest
pitest.org
35/45
written by Karol Wrótniak, forked from
szpak/gradle-pitest-plugin
works with Android projects
has some Android specific helpers (eg: generating
mockable Android jar)
Android Gradle plugin
koral--/gradle-pitest-plugin
38/45