Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Fixing Broken Robots - Android Mutation Testing
Panini
July 18, 2019
Programming
0
58
Fixing Broken Robots - Android Mutation Testing
Panini
July 18, 2019
Tweet
Share
More Decks by Panini
See All by Panini
panini
0
200
panini
0
130
panini
2
200
panini
2
1.7k
panini
1
530
panini
0
700
panini
2
650
panini
1
360
panini
1
940
Other Decks in Programming
See All in Programming
hr01
0
1.7k
nbkouhou
1
1.4k
azdaroth
0
180
ajstarks
2
550
supikiti
4
1.4k
line_developers_tw
0
580
standfm
1
310
thatjeffsmith
0
490
manfredsteyer
PRO
0
110
rishitdagli
0
180
wasabeef
1
580
line_developers_tw
0
1.6k
Featured
See All Featured
smashingmag
229
18k
roundedbygravity
241
21k
carmenhchung
26
1.4k
caitiem20
308
17k
colly
187
14k
cherdarchuk
71
260k
gr2m
83
11k
philhawksworth
192
8.8k
jnunemaker
PRO
40
4.6k
tammielis
237
23k
brianwarren
83
4.7k
schacon
145
6.6k
Transcript
Fixing broken robots - Android Mutation Testing DroidConSG 2019 1/45
About me Matthew Vern Twitter Github Mercari, Inc Software Engineer
(Android) @panini_ja panpanini 2/45
Client Engineer Solving problems for our customers Shipping features Improve
existing functionality My job 3/45
A non-shipped feature doesn't provide benefit Ship features as quick
as possible My job 4/45
A shipped, broken feature doesn't provide benefit Ship quality features
as quick as possible My job 5/45
Maintaining quality 6/45
QA Maintaining quality 6/45
QA Code Review Maintaining quality 6/45
QA Code Review Tests Maintaining quality 6/45
How do we know our tests are providing quality Maintaining
quality 7/45
How do we know our tests are providing quality Use
coverage to make sure that our tests are calling production code Maintaining quality 7/45
How do we know our tests are providing quality Use
coverage to make sure that our tests are calling production code changes introduced will not break existing code Maintaining quality 7/45
How do we know our tests are providing quality Use
coverage to make sure that our tests are calling production code changes introduced will not break existing code new code does what it says on the tin Maintaining quality 7/45
Who watches the watchmen? 8/45
How do we know that our tests are quality? Maintaining
quality 9/45
What are tests 10/45
asserting that our assumptions about a piece of code are
correct binary assertions of code correctness What are tests 11/45
Lets fail some tests 12/45
Unit tests assert code behaviour change code behaviour tests fail
???? profit Lets fail some tests 13/45
Mutation testing 14/45
proposed by Richard Lipton in 1971 computationally expensive, not a
viable testing solution until recently Mutation testing 15/45
1. Create a mutant 2. Run test suite 3. Confirm
if mutant was detected or not 4. Repeat Mutation testing steps 16/45
A mutant is a biological entity which has undergone a
change in its genetic structure. What is a mutant? 17/45
A mutant is a code block which has undergone a
change in its structure. What is a mutant? 18/45
class SessionController( private val sessions: MutableList<Session> ) : EpoxyController() {
fun setSessions(sessions: List<Session>) {} override fun buildModels() {} fun generateModels(sessions: List<Session>): List<SessionModel> {} } Creating mutations 19/45
Creating mutations fun setSessions(sessions: List<Session>) { this.sessions.clear() this.sessions.addAll(sessions) requestModelBuild() }
20/45
Creating mutations override fun buildModels() { generateModels(sessions) .forEach { it.addTo(this)
} } 21/45
Creating mutations fun generateModels(sessions: List<Session>): List<SessionModel> { return sessions .map
{ session SessionModel_() .title(session.title) .imageUrl( if (session.speaker.profileImage "") { session.speaker.profileImage } else { null } ) } } 22/45
Creating mutations 23/45
Competent Programmer Hypothesis Creating mutations 23/45
Competent Programmer Hypothesis Coupling Effect Creating mutations 23/45
Replaces relational operators with boundary counterpart Original Original Mutated Mutated
< <= <= < > >= >= > Conditionals boundary 24/45
Conditionals boundary original if (currentTime < startTime) { do something
} mutated if (currentTime startTime) { do something } 25/45
Negates conditional checks Original Original Mutated Mutated == != !=
== <= > > <= Negate Conditionals 26/45
Negate Conditionals original fun buildModels() { SessionModel_() .title(session.title) .imageUrl( if
(session.speaker.profileImage "") { session.speaker.profileImage } else { null } ) } mutated fun buildModels() { SessionModel_() .title(session.title) .imageUrl( if (session.speaker.profileImage "") { session.speaker.profileImage } else { null } ) } 27/45
removes void method calls Remove void calls 28/45
Remove void calls original fun setSessions(sessions: List<Session>) { this.sessions.clear() this.sessions.addAll(sessions)
requestModelBuild() } mutated fun setSessions(sessions: List<Session>) { this.sessions.clear() this.sessions.addAll(sessions) } 29/45
So what? 30/45
Code coverage, but better Why mutation testing 31/45
1. Introduce a fault into production code 2. Use code
coverage to determine which tests to run 3. Run tests 4. Confirm if fault was detected or not 5. Repeat Mutation testing The better way 32/45
That's a lot of work you expect us to do
there bud 33/45
Pitest 34/45
mutation testing system mutants stored in memory outputs pretty reports
Gradle plugin Pitest pitest.org 35/45
apply plugin: pitest generates pitest<Variant> tasks Gradle plugin szpak/gradle-pitest-plugin 36/45
37/45
written by Karol Wrótniak, forked from szpak/gradle-pitest-plugin works with Android
projects has some Android specific helpers (eg: generating mockable Android jar) Android Gradle plugin koral--/gradle-pitest-plugin 38/45
plugins { id("pl.droidsonroids.pitest") } pitest { excludeMockableAndroidJar = false targetClasses
= setOf("jp.co.panpanini.mypackage.*") outputFormats = setOf("XML", "HTML") } Android Gradle plugin 39/45
id("pl.droidsonroids.pitest") plugins { } pitest { excludeMockableAndroidJar = false targetClasses
= setOf("jp.co.panpanini.mypackage.*") outputFormats = setOf("XML", "HTML") } Android Gradle plugin 39/45
pitest { excludeMockableAndroidJar = false targetClasses = setOf("jp.co.panpanini.mypackage.*") outputFormats =
setOf("XML", "HTML") } plugins { id("pl.droidsonroids.pitest") } Android Gradle plugin 39/45
excludeMockableAndroidJar = false plugins { id("pl.droidsonroids.pitest") } pitest { targetClasses
= setOf("jp.co.panpanini.mypackage.*") outputFormats = setOf("XML", "HTML") } Android Gradle plugin 39/45
targetClasses = setOf("jp.co.panpanini.mypackage.*") plugins { id("pl.droidsonroids.pitest") } pitest { excludeMockableAndroidJar
= false outputFormats = setOf("XML", "HTML") } Android Gradle plugin 39/45
outputFormats = setOf("XML", "HTML") plugins { id("pl.droidsonroids.pitest") } pitest {
excludeMockableAndroidJar = false targetClasses = setOf("jp.co.panpanini.mypackage.*") } Android Gradle plugin 39/45
Demo 40/45
Pitest tips & tricks 41/45
MutationInterceptor Removes mutants for Kotlin generated code Pitest kotlin pitest/pitest-kotlin
42/45
Run PITest on Unit tests only 43/45
Run PITest on CI 44/45
Mutation Testing panpanini/mutation_testing Github: panpanini Twitter: panini_ja 45/45