What's the Fuzz about Unit Testing

What's the Fuzz about Unit Testing

All my tests used to involve users named Alice and Bob. Aged 42. Working at FooBar Inc. The problem when using hard-coded values in your tests, means that you only test one single path of your code. Ever. During this talk, you will discover a fuzzy testing library to make your test use a wider range of data.

Cf95f93e78f6d6dd0630049396f723c6?s=128

Xavier Gouchet

October 25, 2018
Tweet

Transcript

  1. 2.

    About me… • Xavier F. Gouchet • Writing Android apps

    since Android 1.5 (Cupcake) • Lead Android Engineer at WorkWell • @xgouchet on Github / StackOverflow / LinkedIn
  2. 3.
  3. 5.

    The test method… @Test fun testItemClicked() { val user =

    User(id = 42 /* … */) testedPresenter.onUserSelected(user) verify(mockView).navigateToChat(user.id) }
  4. 7.

    … and the test setup method @Before fun setUp() {

    val company = Company(id = 42 /* … */) testedPresenter = Presenter(company) testedPresenter.view = mockView }
  5. 9.

    When all your fake data are “foo” and 42, how

    can you be sure that your tests are valid ?
  6. 10.

    The problem when using static test data • Where does

    the 42 come from ? • Lots of hardcoded values ! • Only one value is tested. Ever.
  7. 12.

    What we can do about it • Keep track of

    test values ? • Parameterized Tests ? • Random test data !
  8. 13.

    Using fuzzy data in your tests • Random data •

    Help define boundaries of valid / invalid inputs • Help find edge cases
  9. 15.

    With fuzz @Test fun invalidateDataAfterTimeout() { val fakeTS = rand.nextLong()

    val fakeTTL = rand.nextLong() val fakeDelay = rand.nextLong() testedObject.setLastCallTimestamp(fakeTS) testedObject.setDataTTL(fakeTTL) val ts = fakeTS + fakeTTL + fakeDelay val result = testedObject.isDataValid(ts) assertFalse(result) }
  10. 19.

    Elmyr forging numbers forger.anInt(min = 0, max =100) forger.aSmallInt() forger.aPositiveLong()

    forger.aGaussianFloat(mean = 42f, standardDeviation = 100f) forger.aDoubleArray(DoubleConstraint.NEGATIVE_STRICT)
  11. 21.

    Elmyr forging (from) Collections forger.aList { aUrl() } forger.aSubListOf(myList) forger.anElementFrom(myCollection)

    forger.anElementFrom("foo", "bar", "baz" /* … */) forger.aValueFrom(MyEnum::class) forger.aNullableFrom { aUrl() }
  12. 22.

    Elmyr forging (from) Collections forger.aList { aUrl() } forger.aSubListOf(myList) forger.anElementFrom(myCollection)

    forger.anElementFrom("foo", "bar", "baz" /* … */) forger.aValueFrom(MyEnum::class) forger.aNullableFrom { aUrl() }
  13. 23.

    Without fuzz @Test fun invalidateDataAfterTimeout() { val fakeTS = forger.aTimestamp()

    val fakeTTL = forger.aLong(1, 86400000) val fakeDelay = forger.aLong(1, 86400000) testedObject.setLastCallTimestamp(fakeTS) testedObject.setDataTTL(fakeTTL) val ts = fakeTS + fakeTTL + fakeDelay val result = testedObject.isDataValid(ts) assertFalse(result) }
  14. 25.

    <FooTest.testSomething()> failed with fake seed 0x4815162342 Add the following line

    in your setUp method to reproduce : forger.reset(0x4815162342L) Elmyr libary
  15. 27.

    Custom POJO / Data Class data class User( val id:

    String, val firstName: String, val lastName: String, val jobTitle: String?, val avatarUrl: String?, val email: String?, val tags: List<String>? )
  16. 28.

    Custom extension function fun Forger.aUser(): User { return User( id

    = aNumericalString(), firstName = aWord(Case.CAPITALIZE), lastName = aWord(Case.CAPITALIZE), jobTitle = aNullable { aSentence() }, avatarUrl = aNullable { aUrl() }, email = aNullable { anEmail() }, tags = aList { aWord() } ) }
  17. 29.

    Our original test… @Before fun setUp() { val company =

    forger.aCompany() testedPresenter = Presenter(company) testedPresenter.view = mockView } @Test fun testItemClicked() { val user = forger.aUser() testedPresenter.onUserSelected(user) verify(mockView).navigateToChat(user.id) }
  18. 30.

    “You need to be as confident in the tests we

    code as we are in the code we test.” Xavier F. Gouchet