What's the Fuzz with Unit Tests Droidcon London 2018

About me… ● Xavier F. Gouchet ● Writing Android apps since Android 1.5 (Cupcake) ● Lead Android Engineer at WorkWell ● @xgouchet on Github / StackOverflow / LinkedIn

No content

When the tests go wrong

The test method… @Test fun testItemClicked() { val user = User(id = 42 /* … */) testedPresenter.onUserSelected(user) verify(mockView).navigateToChat(user.id) }

The tested method… fun onUserSelected(user: User) { view?.navigateToChat(company.id) }

… and the test setup method @Before fun setUp() { val company = Company(id = 42 /* … */) testedPresenter = Presenter(company) testedPresenter.view = mockView }

All my tests involved users named Bob. Aged 42. Working at FooBar Inc.

When all your fake data are “foo” and 42, how can you be sure that your tests are valid ?

The problem when using static test data ● Where does the 42 come from ? ● Lots of hardcoded values ! ● Only one value is tested. Ever.

Is there a solution ?

What we can do about it ● Keep track of test values ? ● Parameterized Tests ? ● Random test data !

Using fuzzy data in your tests ● Random data ● Help define boundaries of valid / invalid inputs ● Help find edge cases

Without fuzz @Test fun invalidateDataAfterTimeout() { testedObject.setLastCallTimestamp(0) testedObject.setDataTTL(42) val ts = 666 val result = testedObject.isDataValid(ts) assertFalse(result) }

With fuzz @Test fun invalidateDataAfterTimeout() { val fakeTS = rand.nextLong() val fakeTTL = rand.nextLong() val fakeDelay = rand.nextLong() testedObject.setLastCallTimestamp(fakeTS) testedObject.setDataTTL(fakeTTL) val ts = fakeTS + fakeTTL + fakeDelay val result = testedObject.isDataValid(ts) assertFalse(result) }

● Long overflow ● negative TTL ● Invalid values ● … … but not too random

https://github.com/xgouchet/Elmyr

Elmyr setup class FooTest { @JvmField @Rule val forger = new JUnitForger() // … }

Elmyr forging numbers forger.anInt(min = 0, max =100) forger.aSmallInt() forger.aPositiveLong() forger.aGaussianFloat(mean = 42f, standardDeviation = 100f) forger.aDoubleArray(DoubleConstraint.NEGATIVE_STRICT)

Elmyr forging Strings forger.anHexadecimalString() forger.aWord() forger.aSentence() forger.anEmail() forger.aUrl() forger.aStringArray(StringConstraint.WORD) forger.aStringMatching("(0|+44)\\d{10}")

Elmyr forging (from) Collections forger.aList { aUrl() } forger.aSubListOf(myList) forger.anElementFrom(myCollection) forger.anElementFrom("foo", "bar", "baz" /* … */) forger.aValueFrom(MyEnum::class) forger.aNullableFrom { aUrl() }

Elmyr forging (from) Collections forger.aList { aUrl() } forger.aSubListOf(myList) forger.anElementFrom(myCollection) forger.anElementFrom("foo", "bar", "baz" /* … */) forger.aValueFrom(MyEnum::class) forger.aNullableFrom { aUrl() }

Without fuzz @Test fun invalidateDataAfterTimeout() { val fakeTS = forger.aTimestamp() val fakeTTL = forger.aLong(1, 86400000) val fakeDelay = forger.aLong(1, 86400000) testedObject.setLastCallTimestamp(fakeTS) testedObject.setDataTTL(fakeTTL) val ts = fakeTS + fakeTTL + fakeDelay val result = testedObject.isDataValid(ts) assertFalse(result) }

Making tests reproducible

failed with fake seed 0x4815162342 Add the following line in your setUp method to reproduce : forger.reset(0x4815162342L) Elmyr libary

Adapting to your use cases

Custom POJO / Data Class data class User( val id: String, val firstName: String, val lastName: String, val jobTitle: String?, val avatarUrl: String?, val email: String?, val tags: List? )

Custom extension function fun Forger.aUser(): User { return User( id = aNumericalString(), firstName = aWord(Case.CAPITALIZE), lastName = aWord(Case.CAPITALIZE), jobTitle = aNullable { aSentence() }, avatarUrl = aNullable { aUrl() }, email = aNullable { anEmail() }, tags = aList { aWord() } ) }

Our original test… @Before fun setUp() { val company = forger.aCompany() testedPresenter = Presenter(company) testedPresenter.view = mockView } @Test fun testItemClicked() { val user = forger.aUser() testedPresenter.onUserSelected(user) verify(mockView).navigateToChat(user.id) }

“You need to be as confident in the tests we code as we are in the code we test.” Xavier F. Gouchet

Thanks for your attention Any question ? @xgouchet http://github.com/xgouchet/Elmyr