Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing bulletproof code with property-based testing

Writing bulletproof code with property-based testing

Slides of my "Writing bulletproof code with property-based testing" tech talk at Droidcon Lisbon & Droidcon Berlin
-------------------------------------------------------------------------------
Unit tests are an important part of the development process. Among other things, they help us verify the correctness of our code.

Thus, when writing tests, we usually pick one or two relevant examples, let the tests run green and call it a day. This is called example-based testing.

But, is that enough? Have you missed any significant case? Have you considered all edge-cases? What if any of those non-tested examples would make the app crash?

Fear not, we can fill that gap with property-based tests. Let me show to you how, step by step.

In this tech-talk you will learn:
- some easy tips to improve your unit tests
- why example-based unit tests are not always enough
- how property-based testing can make your tests more robust
- property-based testing in very detail:
* what it is
* its pros and cons
* which tools/frameworks are available for implementing them
* how to make the most out of them

And all that by using as examples some features you would find in most Android apps, like "create account" or "text editing".
----------------------------------------------------------------
Repo with code examples:
https://github.com/sergio-sastre/Multiplying_the_quality_of_unit_tests

Sergio Sastre Flórez

April 28, 2022
Tweet

More Decks by Sergio Sastre Flórez

Other Decks in Programming

Transcript

  1. PROPERTY - BASED BY SERGIO SASTRE WRITING BULLETPROOF CODE WITH

    TESTING
  2. Example-based testing of your software writing examples of it tests

    specify the behaviour where by @Gio_Sastre
  3. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString =

    “1221”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
  4. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString =

    “1221”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
  5. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString =

    “12a21”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
  6. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString =

    “”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
  7. Text editor @Gio_Sastre Strong password validation

  8. @Gio_Sastre Text editor Strong password validation

  9. @Gio_Sastre Strong password validation Text editor

  10. @Gio_Sastre fun clickSignUpButton() { mutablePasswordError.value = passwordValidator.validate(passwordInput)
 } ViewModel

  11. @Gio_Sastre fun clickSignUpButton() { mutablePasswordError.value = passwordValidator.validate(passwordInput)
 } ViewModel

  12. @Gio_Sastre The password contains at least 1 digit Show “must

    contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @Test
 fun passwordValidatorDigitsTest() { assertThat ( passwordValidator.validate(“ABCDEF”) ).contains(“must contain digits”)
 }
  13. @Gio_Sastre The password contains at least 1 digit Show “must

    contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @Test
 fun passwordValidatorDigitsTest() { assertThat ( passwordValidator.validate(“ABCDEF”) ).contains(“must contain digits”)
 assertThat ( passwordValidator.validate(“123456”) ).contains(“must contain upper case letters”) } @Test
 fun passwordValidatorUpperCaseTest() { }
  14. @Gio_Sastre The password contains at least 1 digit Show “must

    contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @ParameterizedTest(
 name = “when password \”{0}\”, error contains \”{1}\”” ) @CsvSource ( “ABCDEF, must contain digits”, “123456, must contain upper case letters” , “12345, must contain at least 6 chars”,
 “12 3 456, must not contain blanks” , “HELLO, must contain lower case letters”, )
 fun passwordValidatorTest ( password: String?, expectedError: String ? ) { assertThat ( passwordValidator.validate(password) ).contains(expectedError) }
  15. @Gio_Sastre The password contains at least 1 digit Show “must

    contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @ParameterizedTest(
 name = “when password \”{0}\”, error contains \”{1}\”” ) @CsvSource ( “ABCDEF, must contain digits”, “123456, must contain upper case letters” , “12345, must contain at least 6 chars”,
 “12 3 456, must not contain blanks” , “HELLO, must contain lower case letters”, )
 fun passwordValidatorTest ( password: String?, expectedError: String ? ) { assertThat ( passwordValidator.validate(password) ).contains(expectedError) }
  16. @Gio_Sastre The password contains at least 1 digit Show “must

    contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @ParameterizedTest(
 name = “when password \”{0}\”, error contains \”{1}\”” ) @CsvSource ( “ABCDEF, must contain digits”, “123456, must contain upper case letters” , “12345, must contain at least 6 chars”,
 “12 3 456, must not contain blanks” , “HELLO, must contain lower case letters”, )
 fun passwordValidatorTest ( password: String?, expectedError: String ? ) { assertThat ( passwordValidator.validate(password) ).contains(expectedError) }
  17. @Gio_Sastre The password contains at least 1 digit Show “must

    contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @ParameterizedTest(
 name = “when password \”{0}\”, error contains \”{1}\”” ) @CsvSource ( “ABCDEF, must contain digits”, “123456, must contain upper case letters” , “12345, must contain at least 6 chars”,
 “12 3 456, must not contain blanks” , “HELLO, must contain lower case letters”, )
 fun passwordValidatorTest ( password: String?, expectedError: String ? ) { assertThat ( passwordValidator.validate(password) ).contains(expectedError) }
  18. Are enough? Example-based tests @Gio_Sastre

  19. class PasswordValidator(…) { override fun isValid(password: String?): Boolean { …

    // validate contains upper case letter s if (!password?.contains(“[A-Z]”.toRegex())) { … } … } } @Gio_Sastre
  20. class PasswordValidator(…) { override fun isValid(password: String?): Boolean { …

    // validate contains upper case letter s if (!password?.contains(“[a-z]”.toRegex())) { … } … } } @Gio_Sastre
  21. @Gio_Sastre

  22. 🧐 Upper case: “[a-z]”.toRegex() @Gio_Sastre

  23. 🧐 @Gio_Sastre Upper case: “[a-z]”.toRegex() show “must contain upper case

    letters ” if “123456” has no
  24. 🧐 @Gio_Sastre Lower case: “[a-z]”.toRegex() show “must contain lower case

    letters ” 🧐 if “123456” has no
  25. 🧐 @Gio_Sastre Upper case: “[a-z]”.toRegex() if “123456” has no show

    “must contain upper case letters ” 🤦
  26. @Gio_Sastre Upper case: “[a-z]”.toRegex() if “123456” has no show “must

    contain upper case letters ” 🤦 🧐
  27. @Gio_Sastre 🙃 Upper case: “[a-z]”.toRegex() if “abcdef” has no show

    “must contain upper case letters ”
  28. 1 example-based tests Problems with 2 Finding good examples @Gio_Sastre

    Covering all relevant cases
  29. Can those issues? we mitigate @Gio_Sastre

  30. Example-based testing of your software writing examples of it tests

    specify the behaviour where by @Gio_Sastre
  31. Property-based testing generating thousands of tests specify the behaviour where

    by @Gio_Sastre semi-random examples of it of your software
  32. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString =

    “1221”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
  33. fun is_string_palindrome() { val inputString = SemiRandomString(seed)
 val revertedInput =

    inputString.reversed( ) assertTrue(inputString == revertedInput) 
 } @Gio_Sastre Property-based testing @Test(tries = 1_000)
  34. fun is_string_palindrome() { val inputString = SemiRandomString(seed)
 val revertedInput =

    inputString.reversed( ) assertTrue(inputString == revertedInput) 
 } @Gio_Sastre Property-based testing @Test(tries = 1_000)
  35. @Gio_Sastre Kotlin support Nullables for non-top-level types requires @WithNull Missing

    features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Smooth integration Full featured & Configurable via Gradle Stateful testing, statistics, etc. Kotlin support Kotlin Native too Kotlin multiplatform support Jqwik otest
  36. @Gio_Sastre Smooth integration Full featured & Configurable via Gradle Kotlin

    support Kotlin support Stateful testing, statistics, etc. Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
  37. @Gio_Sastre Smooth integration Full featured & Configurable via Gradle Kotlin

    support Stateful testing, statistics, etc. Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Kotlin support Kotlin Native too Kotlin multiplatform support Jqwik otest
  38. @Gio_Sastre Smooth integration Full featured & Configurable via Gradle Kotlin

    support Stateful testing, statistics, etc. Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Kotlin support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
  39. @Gio_Sastre Kotlin support Nullables for non-top-level types requires @WithNull Missing

    features & configurability Previous failing seed not saved, etc. String? Smooth integration Full featured & Configurable via Gradle Stateful testing, statistics, etc. Kotlin support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
  40. @Gio_Sastre Kotlin support Nullables for non-top-level types requires @WithNull Missing

    features & configurability Previous failing seed not saved, etc. List<@WithNull String> Smooth integration Full featured & Configurable via Gradle Stateful testing, statistics, etc. Kotlin support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
  41. @Gio_Sastre Kotlin support Nullables for non-top-level types requires @WithNull Missing

    features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Smooth integration Full featured & Configurable via Gradle Stateful testing, statistics, etc. Kotlin support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
  42. How @Gio_Sastre to write property-based tests?

  43. @Gio_Sastre Strong password validation upper case

  44. @Gio_Sastre 1 Define the property If password without upper case

    Show “must contain upper case letters” 3 Test the property Default is 1.000 times 2 Create a generator Password without upper case chars String without upper case chars
  45. @Gio_Sastre 1 Define the property If password without upper case

    Show “must contain upper case letters” fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) } 3 Test the property Default is 1.000 times 2 Create a generator Password without upper case chars String without upper case chars
  46. @Gio_Sastre 1 Define the property If password without upper case

    Show “must contain upper case letters” 2 Create a generator Password without upper case chars String without upper case chars fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) } 3 Test the property Default is 1.000 times fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) }
  47. @Gio_Sastre 1 Define the property If password without upper case

    Show “must contain upper case letters” 3 Test the property Default is 1.000 times fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } 2 Create a generator Password without upper case chars String without upper case chars fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) }
  48. @Gio_Sastre 1 Define the property If password without upper case

    Show “must contain upper case letters” 3 Test the property Default is 1.000 times fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) 2 Create a generator Password without upper case chars String without upper case chars fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) }
  49. @Gio_Sastre 1 Define the property If password without upper case

    Show “must contain upper case letters” 3 Test the property Default is 1.000 times 2 Create a generator Password without upper case chars String without upper case chars fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) }
  50. @Gio_Sastre

  51. @Gio_Sastre

  52. @Gio_Sastre

  53. @Gio_Sastre “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“
  54. @Gio_Sastre Shrinking Trying to find the simplest example that fails

    for the same reason “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“
  55. @Gio_Sastre “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“
  56. @Gio_Sastre “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “abc_1234567890“

  57. @Gio_Sastre

  58. @Gio_Sastre “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“ null
  59. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“ null
  60. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“ fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) null
  61. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“ fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) null
  62. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“ fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100, seed = -3508921172508693187) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) null
  63. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“ No flaky tests Use the seed to reliably reproduce the results of a failing test null
  64. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“ null
  65. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“ Total edge cases fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) } null (jqwik only)
  66. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%”

    𡨸 o字9=“ fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) } noUpperCase().edgeCases() null (Java.lang.Character.MIN_VALUE) “/u0000” “” (Empty char) “ ” (Space char) Total edge cases (jqwik only)
  67. 1 Property-based testing main features 2 Shrinking (minimum failing sample)

    Reproduce failing tests (seeds) 3 Tests for edge cases @Gio_Sastre
  68. What else @Gio_Sastre can Jqwik do?

  69. @Gio_Sastre Stateful testing Text editor

  70. @Gio_Sastre Model val currentText: String? val undoActions: List<String?> val redoActions:

    List<String?> data class TextEditor ( )
  71. @Gio_Sastre Actions on model 1 fun changeTextTo(newText: String?) { …

    } 2 fun undo() { … } 3 fun redo() { … }
  72. (if something to redo) (if something to undo) @Gio_Sastre 1

    On “text change” one more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements
  73. (if something to redo) (if something to undo) @Gio_Sastre 1

    On “text change” one more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements
  74. (if something to redo) @Gio_Sastre 1 On “text change” one

    more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements (if something to undo)
  75. (if something to undo) @Gio_Sastre 1 On “text change” one

    more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements (if something to redo)
  76. @Gio_Sastre 1 On “text change” one more “undo” action possible,

    max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements (if something to undo) (if something to redo)
  77. @Gio_Sastre 1 On “text change” one more “undo” action possible,

    max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements (if something to undo) (if something to redo)
  78. @Gio_Sastre 1 On “text change” one more “undo” action possible,

    max 20 no “redo” actions possible fun onTextChange_max_undo_actions_is_20() { @Test val editor = TextEditor( ) repeat(21){ editor.textChange(“abc”) } assertThat(editor.undoActions).size(20) } Requirements
  79. @Gio_Sastre 1 On “text change” one more “undo” action possible,

    max 20 no “redo” actions possible fun onTextChange_max_undo_actions_is_20() { @Test val editor = TextEditor( ) repeat(21){ editor.textChange(“abc”) } assertThat(editor.undoActions).size(20) } Requirements
  80. @Gio_Sastre 1 On “text change” one more “undo” action possible,

    max 20 no “redo” actions possible
  81. @Gio_Sastre 1 On “text change” one more “undo” action possible,

    max 20 no “redo” actions possible class TextChange(val text: String?): Action<TextEditor> { override fun run(editor: TextEditor) { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } }
  82. @Gio_Sastre 1 On “text change” one more “undo” action possible,

    max 20 no “redo” actions possible class TextChange(val text: String?): Action<TextEditor> { override fun run(editor: TextEditor) { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } }
  83. class TextChange(val text: String?): Action<TextEditor> { override fun run(editor: TextEditor)

    { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } } @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible
  84. @Gio_Sastre class TextChange(val text: String?): Action<TextEditor> { override fun run(editor:

    TextEditor) { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } } 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible
  85. @Gio_Sastre class TextChange(val text: String?): Action<TextEditor> { override fun run(editor:

    TextEditor) { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } } 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible
  86. @Gio_Sastre 2 On “undo” one more “redo” action possible, max

    20 next “redo” brings the previous text back (if something to undo)
  87. @Gio_Sastre 2 On “undo” one more “redo” action possible, max

    20 next “redo” brings the previous text back (if something to undo) class Undo(): Action<TextEditor> { override fun run(editor: TextEditor) { val preEditor = editor.copy( ) editor.undo( ) assertThat(editor ) .redoActionsEquals(min(preEditor.redoActions() + 1, 20) ) .nextRedoTextEquals(preEditor.text ) } } override fun precondition(editor: TextEditor) = editor.undoActionsCount() > 0
  88. @Gio_Sastre 2 On “undo” one more “redo” action possible, max

    20 next “redo” brings the previous text back (if something to undo) class Undo(): Action<TextEditor> { override fun run(editor: TextEditor) { val preEditor = editor.copy( ) editor.undo( ) assertThat(editor ) .redoActionsEquals(min(preEditor.redoActions() + 1, 20) ) .nextRedoTextEquals(preEditor.text ) } } override fun precondition(editor: TextEditor) = editor.undoActionsCount() > 0
  89. @Gio_Sastre 2 On “undo” one more “redo” action possible, max

    20 next “redo” brings the previous text back (if something to undo) class Undo(): Action<TextEditor> { override fun run(editor: TextEditor) { val preEditor = editor.copy( ) editor.undo( ) assertThat(editor ) .redoActionsEquals(min(preEditor.redoActions() + 1, 20) ) .nextRedoTextEquals(preEditor.text ) } } override fun precondition(editor: TextEditor) = editor.undoActionsCount() > 0
  90. fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() =

    Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo()) @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo())
  91. @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun

    randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo())
  92. @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun

    randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo())
  93. @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun

    randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo())
  94. @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun

    randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo())
  95. @Gio_Sastre fun randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun textEditorActionGenerator() =

    @Provide Arbitraries.sequences(randomActions()) Property test fun statefulTextEditorTest ( @ForAll(“textEditorActionGenerator”) actions: ActionSequence<TextEditor > ) = actions.run(TextEditor()) @Property fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo()) class TextChange(val text: String?): Action<TextEditor> { … } class Undo(): Action<TextEditor> { … } class Redo(): Action<TextEditor> { … }
  96. @Gio_Sastre class TextChange(val text: String?): Action<TextEditor> { … } class

    Undo(): Action<TextEditor> { … } class Redo(): Action<TextEditor> { … } fun statefulTextEditorTest ( @ForAll(“textEditorActionGenerator”) actions: ActionSequence<TextEditor > ) = actions.run(TextEditor()) @Property fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) Property test
  97. 1 Property-based stateful testing 2 Identify actions Preconditions in actions

    3 Assertions in actions @Gio_Sastre with Jqwik
  98. @Gio_Sastre pros & cons Property-based testing of

  99. Cons @Gio_Sastre Pros @Gio_Sastre

  100. Cons @Gio_Sastre Pros @Gio_Sastre More robust tests Cover edge cases

    we might forget Random but deterministic tests
  101. Cons Hard to find properties Properties are not always sufficient

    @Gio_Sastre More robust tests Cover edge cases we might forget Random but deterministic tests Pros @Gio_Sastre Run more tests, so run more slowly
  102. Cons Hard to find properties @Gio_Sastre @Gio_Sastre fun String?.reversed() :

    String? { … } Create method that reverses a given string Properties are not always sufficient Run more tests, so run more slowly
  103. Cons Hard to find properties @Gio_Sastre @Gio_Sastre Patterns to find

    properties Business rule as property If password contains blanks show error message… fun String?.reversed() : String? { … } Create method that reverses a given string Properties are not always sufficient Run more tests, so run more slowly
  104. Cons @Gio_Sastre @Gio_Sastre Patterns to find properties Test Oracle list.bubbleSort()

    == list.moreEfficientSort() fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  105. Cons @Gio_Sastre @Gio_Sastre Invariant functions Patterns to find properties string.reverse()?.length

    == string?.length fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  106. Cons @Gio_Sastre @Gio_Sastre fun String?.reversed() : String? { … }

    @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  107. Cons @Gio_Sastre @Gio_Sastre Inverse functions Patterns to find properties string.reversed().reversed()

    == string @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  108. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string:

    String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  109. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string:

    String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  110. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string:

    String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? = this Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  111. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string:

    String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? = this this this Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  112. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string:

    String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? = this this this Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  113. Cons @Gio_Sastre @Gio_Sastre Create method that reverses a given string

    @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? = this this this Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  114. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string:

    String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  115. Cons @Gio_Sastre @Gio_Sastre @Property(times = 100 ) fun reversedStringKeepsLength (

    @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Property(times = 100 ) fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
  116. 1 Property-based testing (PBT) opinionated advice 2 Example + property

    tests PBT for widely used code 3 @Gio_Sastre PBT for obvious properties
  117. attention for your THANKS @Gio_Sastre

  118. @Gio_Sastre code @sergio-sastre Sergio Sastre Flórez Lead & Senior Android

    developer appdev.de blogs @SergioSastre sergiosastre.hashnode.dev