Writing bulletproof code with property-based testing

Slides of my "Writing bulletproof code with property-based testing" tech talk at Droidcon Lisbon & Droidcon Berlin
-------------------------------------------------------------------------------
Unit tests are an important part of the development process. Among other things, they help us verify the correctness of our code.

Thus, when writing tests, we usually pick one or two relevant examples, let the tests run green and call it a day. This is called example-based testing.

But, is that enough? Have you missed any significant case? Have you considered all edge-cases? What if any of those non-tested examples would make the app crash?

Fear not, we can fill that gap with property-based tests. Let me show to you how, step by step.

In this tech-talk you will learn:
- some easy tips to improve your unit tests
- why example-based unit tests are not always enough
- how property-based testing can make your tests more robust
- property-based testing in very detail:
* what it is
* its pros and cons
* which tools/frameworks are available for implementing them
* how to make the most out of them

And all that by using as examples some features you would find in most Android apps, like "create account" or "text editing".
----------------------------------------------------------------
Repo with code examples:
https://github.com/sergio-sastre/Multiplying_the_quality_of_unit_tests

Transcript

1. PROPERTY - BASED BY SERGIO SASTRE WRITING BULLETPROOF CODE WITH TESTING
2. Example-based testing of your software writing examples of it tests specify the behaviour where by @Gio_Sastre
3. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString = “1221”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
4. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString = “1221”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
5. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString = “12a21”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
6. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString = “”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
7. Text editor @Gio_Sastre Strong password validation
8. @Gio_Sastre Text editor Strong password validation
9. @Gio_Sastre Strong password validation Text editor
10. @Gio_Sastre fun clickSignUpButton() { mutablePasswordError.value = passwordValidator.validate(passwordInput)
 } ViewModel
11. @Gio_Sastre fun clickSignUpButton() { mutablePasswordError.value = passwordValidator.validate(passwordInput)
 } ViewModel
12. @Gio_Sastre The password contains at least 1 digit Show “must contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @Test
 fun passwordValidatorDigitsTest() { assertThat ( passwordValidator.validate(“ABCDEF”) ).contains(“must contain digits”)
 }
13. @Gio_Sastre The password contains at least 1 digit Show “must contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @Test
 fun passwordValidatorDigitsTest() { assertThat ( passwordValidator.validate(“ABCDEF”) ).contains(“must contain digits”)
 assertThat ( passwordValidator.validate(“123456”) ).contains(“must contain upper case letters”) } @Test
 fun passwordValidatorUpperCaseTest() { }
14. @Gio_Sastre The password contains at least 1 digit Show “must contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @ParameterizedTest(
 name = “when password \”{0}\”, error contains \”{1}\”” ) @CsvSource ( “ABCDEF, must contain digits”, “123456, must contain upper case letters” , “12345, must contain at least 6 chars”,
 “12 3 456, must not contain blanks” , “HELLO, must contain lower case letters”, )
 fun passwordValidatorTest ( password: String?, expectedError: String ? ) { assertThat ( passwordValidator.validate(password) ).contains(expectedError) }
15. @Gio_Sastre The password contains at least 1 digit Show “must contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @ParameterizedTest(
 name = “when password \”{0}\”, error contains \”{1}\”” ) @CsvSource ( “ABCDEF, must contain digits”, “123456, must contain upper case letters” , “12345, must contain at least 6 chars”,
 “12 3 456, must not contain blanks” , “HELLO, must contain lower case letters”, )
 fun passwordValidatorTest ( password: String?, expectedError: String ? ) { assertThat ( passwordValidator.validate(password) ).contains(expectedError) }
16. @Gio_Sastre The password contains at least 1 digit Show “must contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @ParameterizedTest(
 name = “when password \”{0}\”, error contains \”{1}\”” ) @CsvSource ( “ABCDEF, must contain digits”, “123456, must contain upper case letters” , “12345, must contain at least 6 chars”,
 “12 3 456, must not contain blanks” , “HELLO, must contain lower case letters”, )
 fun passwordValidatorTest ( password: String?, expectedError: String ? ) { assertThat ( passwordValidator.validate(password) ).contains(expectedError) }
17. @Gio_Sastre The password contains at least 1 digit Show “must contain digits" otherwise 1 at least 1 latin char in upper case. Show “must contain upper case letters” otherwise 2 at least 6 chars. Show “must contain at least 6 chars” otherwise 3 no blanks. Show “must not contain blanks” otherwise 4 at least 1 latin char in lower case. Show “must contain lower case letters” otherwise 5 @ParameterizedTest(
 name = “when password \”{0}\”, error contains \”{1}\”” ) @CsvSource ( “ABCDEF, must contain digits”, “123456, must contain upper case letters” , “12345, must contain at least 6 chars”,
 “12 3 456, must not contain blanks” , “HELLO, must contain lower case letters”, )
 fun passwordValidatorTest ( password: String?, expectedError: String ? ) { assertThat ( passwordValidator.validate(password) ).contains(expectedError) }
18. Are enough? Example-based tests @Gio_Sastre
19. class PasswordValidator(…) { override fun isValid(password: String?): Boolean { … // validate contains upper case letter s if (!password?.contains(“[A-Z]”.toRegex())) { … } … } } @Gio_Sastre
20. class PasswordValidator(…) { override fun isValid(password: String?): Boolean { … // validate contains upper case letter s if (!password?.contains(“[a-z]”.toRegex())) { … } … } } @Gio_Sastre
21. @Gio_Sastre
22. 🧐 Upper case: “[a-z]”.toRegex() @Gio_Sastre
23. 🧐 @Gio_Sastre Upper case: “[a-z]”.toRegex() show “must contain upper case letters ” if “123456” has no
24. 🧐 @Gio_Sastre Lower case: “[a-z]”.toRegex() show “must contain lower case letters ” 🧐 if “123456” has no
25. 🧐 @Gio_Sastre Upper case: “[a-z]”.toRegex() if “123456” has no show “must contain upper case letters ” 🤦
26. @Gio_Sastre Upper case: “[a-z]”.toRegex() if “123456” has no show “must contain upper case letters ” 🤦 🧐
27. @Gio_Sastre 🙃 Upper case: “[a-z]”.toRegex() if “abcdef” has no show “must contain upper case letters ”
28. 1 example-based tests Problems with 2 Finding good examples @Gio_Sastre Covering all relevant cases
29. Can those issues? we mitigate @Gio_Sastre
30. Example-based testing of your software writing examples of it tests specify the behaviour where by @Gio_Sastre
31. Property-based testing generating thousands of tests specify the behaviour where by @Gio_Sastre semi-random examples of it of your software
32. Example-based testing @Gio_Sastre @Test
 fun is_string_palindrome() { val inputString = “1221”
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 }
33. fun is_string_palindrome() { val inputString = SemiRandomString(seed)
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 } @Gio_Sastre Property-based testing @Test(tries = 1_000)
34. fun is_string_palindrome() { val inputString = SemiRandomString(seed)
 val revertedInput = inputString.reversed( ) assertTrue(inputString == revertedInput) 
 } @Gio_Sastre Property-based testing @Test(tries = 1_000)
35. @Gio_Sastre Kotlin support Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Smooth integration Full featured & Configurable via Gradle Stateful testing, statistics, etc. Kotlin support Kotlin Native too Kotlin multiplatform support Jqwik otest
36. @Gio_Sastre Smooth integration Full featured & Configurable via Gradle Kotlin support Kotlin support Stateful testing, statistics, etc. Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
37. @Gio_Sastre Smooth integration Full featured & Configurable via Gradle Kotlin support Stateful testing, statistics, etc. Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Kotlin support Kotlin Native too Kotlin multiplatform support Jqwik otest
38. @Gio_Sastre Smooth integration Full featured & Configurable via Gradle Kotlin support Stateful testing, statistics, etc. Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Kotlin support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
39. @Gio_Sastre Kotlin support Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. String? Smooth integration Full featured & Configurable via Gradle Stateful testing, statistics, etc. Kotlin support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
40. @Gio_Sastre Kotlin support Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. List<@WithNull String> Smooth integration Full featured & Configurable via Gradle Stateful testing, statistics, etc. Kotlin support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
41. @Gio_Sastre Kotlin support Nullables for non-top-level types requires @WithNull Missing features & configurability Previous failing seed not saved, etc. Kotlin multiplatform support Smooth integration Full featured & Configurable via Gradle Stateful testing, statistics, etc. Kotlin support Kotlin Native too Kotlin multiplatform support Requires IntelliJ plugin to run from IDE Jqwik otest
42. How @Gio_Sastre to write property-based tests?
43. @Gio_Sastre Strong password validation upper case
44. @Gio_Sastre 1 Define the property If password without upper case Show “must contain upper case letters” 3 Test the property Default is 1.000 times 2 Create a generator Password without upper case chars String without upper case chars
45. @Gio_Sastre 1 Define the property If password without upper case Show “must contain upper case letters” fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) } 3 Test the property Default is 1.000 times 2 Create a generator Password without upper case chars String without upper case chars
46. @Gio_Sastre 1 Define the property If password without upper case Show “must contain upper case letters” 2 Create a generator Password without upper case chars String without upper case chars fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) } 3 Test the property Default is 1.000 times fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) }
47. @Gio_Sastre 1 Define the property If password without upper case Show “must contain upper case letters” 3 Test the property Default is 1.000 times fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } 2 Create a generator Password without upper case chars String without upper case chars fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) }
48. @Gio_Sastre 1 Define the property If password without upper case Show “must contain upper case letters” 3 Test the property Default is 1.000 times fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) 2 Create a generator Password without upper case chars String without upper case chars fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) }
49. @Gio_Sastre 1 Define the property If password without upper case Show “must contain upper case letters” 3 Test the property Default is 1.000 times 2 Create a generator Password without upper case chars String without upper case chars fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) }
50. @Gio_Sastre
51. @Gio_Sastre
52. @Gio_Sastre
53. @Gio_Sastre “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“
54. @Gio_Sastre Shrinking Trying to find the simplest example that fails for the same reason “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“
55. @Gio_Sastre “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“
56. @Gio_Sastre “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “abc_1234567890“
57. @Gio_Sastre
58. @Gio_Sastre “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null
59. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null
60. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) null
61. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) null
62. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ fun testPasswordValidator ( @ForAll(“noUpperCase”) password: String ? ) { @Property(tries = 100, seed = -3508921172508693187) val error = passwordValidator.validate(password ) assertThat(error ) .contains(“must contain upper case letters”) } @Label(“if password without upper case, error ” + “shows ‘must contain upper case letters’”) null
63. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ No flaky tests Use the seed to reliably reproduce the results of a failing test null
64. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ null
65. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ Total edge cases fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) } null (jqwik only)
66. @Gio_Sastre null “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ “漢!字) 𡨸 $&指>事<字9§漢0=漢*字```+#会意@字<指?%” 𡨸 o字9=“ fun noUpperCase() = @Provide Arbitraries.strings().ascii( ) .filter { it.matches(“[^A-Z]”.toRegex()) } noUpperCase().edgeCases() null (Java.lang.Character.MIN_VALUE) “/u0000” “” (Empty char) “ ” (Space char) Total edge cases (jqwik only)
67. 1 Property-based testing main features 2 Shrinking (minimum failing sample) Reproduce failing tests (seeds) 3 Tests for edge cases @Gio_Sastre
68. What else @Gio_Sastre can Jqwik do?
69. @Gio_Sastre Stateful testing Text editor
70. @Gio_Sastre Model val currentText: String? val undoActions: List<String?> val redoActions: List<String?> data class TextEditor ( )
71. @Gio_Sastre Actions on model 1 fun changeTextTo(newText: String?) { … } 2 fun undo() { … } 3 fun redo() { … }
72. (if something to redo) (if something to undo) @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements
73. (if something to redo) (if something to undo) @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements
74. (if something to redo) @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements (if something to undo)
75. (if something to undo) @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements (if something to redo)
76. @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements (if something to undo) (if something to redo)
77. @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back 3 On “redo” one more “undo” action possible, max 20 next “undo” brings the previous text back Requirements (if something to undo) (if something to redo)
78. @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible fun onTextChange_max_undo_actions_is_20() { @Test val editor = TextEditor( ) repeat(21){ editor.textChange(“abc”) } assertThat(editor.undoActions).size(20) } Requirements
79. @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible fun onTextChange_max_undo_actions_is_20() { @Test val editor = TextEditor( ) repeat(21){ editor.textChange(“abc”) } assertThat(editor.undoActions).size(20) } Requirements
80. @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible
81. @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible class TextChange(val text: String?): Action<TextEditor> { override fun run(editor: TextEditor) { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } }
82. @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible class TextChange(val text: String?): Action<TextEditor> { override fun run(editor: TextEditor) { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } }
83. class TextChange(val text: String?): Action<TextEditor> { override fun run(editor: TextEditor) { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } } @Gio_Sastre 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible
84. @Gio_Sastre class TextChange(val text: String?): Action<TextEditor> { override fun run(editor: TextEditor) { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } } 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible
85. @Gio_Sastre class TextChange(val text: String?): Action<TextEditor> { override fun run(editor: TextEditor) { val previousUndoCount = editor.undoActionsCount( ) editor.changeTextTo(text ) assertThat(editor ) .undoActionsEquals(min(previousUndoCount + 1, 20) ) .redoActionsEquals(0 ) } } 1 On “text change” one more “undo” action possible, max 20 no “redo” actions possible
86. @Gio_Sastre 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back (if something to undo)
87. @Gio_Sastre 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back (if something to undo) class Undo(): Action<TextEditor> { override fun run(editor: TextEditor) { val preEditor = editor.copy( ) editor.undo( ) assertThat(editor ) .redoActionsEquals(min(preEditor.redoActions() + 1, 20) ) .nextRedoTextEquals(preEditor.text ) } } override fun precondition(editor: TextEditor) = editor.undoActionsCount() > 0
88. @Gio_Sastre 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back (if something to undo) class Undo(): Action<TextEditor> { override fun run(editor: TextEditor) { val preEditor = editor.copy( ) editor.undo( ) assertThat(editor ) .redoActionsEquals(min(preEditor.redoActions() + 1, 20) ) .nextRedoTextEquals(preEditor.text ) } } override fun precondition(editor: TextEditor) = editor.undoActionsCount() > 0
89. @Gio_Sastre 2 On “undo” one more “redo” action possible, max 20 next “redo” brings the previous text back (if something to undo) class Undo(): Action<TextEditor> { override fun run(editor: TextEditor) { val preEditor = editor.copy( ) editor.undo( ) assertThat(editor ) .redoActionsEquals(min(preEditor.redoActions() + 1, 20) ) .nextRedoTextEquals(preEditor.text ) } } override fun precondition(editor: TextEditor) = editor.undoActionsCount() > 0
90. fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo()) @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo())
91. @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo())
92. @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo())
93. @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo())
94. @Gio_Sastre Semi-random action generator fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) fun randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo())
95. @Gio_Sastre fun randomActions() = Arbitraries.oneOf(randomTextChange(), undo(), redo()) fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) Property test fun statefulTextEditorTest ( @ForAll(“textEditorActionGenerator”) actions: ActionSequence<TextEditor > ) = actions.run(TextEditor()) @Property fun randomTextChange() = Arbitraries.strings().map { TextChange(it) } fun undo() = Arbitraries.just(Undo()) fun redo() = Arbitraries.just(Redo()) class TextChange(val text: String?): Action<TextEditor> { … } class Undo(): Action<TextEditor> { … } class Redo(): Action<TextEditor> { … }
96. @Gio_Sastre class TextChange(val text: String?): Action<TextEditor> { … } class Undo(): Action<TextEditor> { … } class Redo(): Action<TextEditor> { … } fun statefulTextEditorTest ( @ForAll(“textEditorActionGenerator”) actions: ActionSequence<TextEditor > ) = actions.run(TextEditor()) @Property fun textEditorActionGenerator() = @Provide Arbitraries.sequences(randomActions()) Property test
97. 1 Property-based stateful testing 2 Identify actions Preconditions in actions 3 Assertions in actions @Gio_Sastre with Jqwik
98. @Gio_Sastre pros & cons Property-based testing of
99. Cons @Gio_Sastre Pros @Gio_Sastre
100. Cons @Gio_Sastre Pros @Gio_Sastre More robust tests Cover edge cases we might forget Random but deterministic tests
101. Cons Hard to find properties Properties are not always sufficient @Gio_Sastre More robust tests Cover edge cases we might forget Random but deterministic tests Pros @Gio_Sastre Run more tests, so run more slowly
102. Cons Hard to find properties @Gio_Sastre @Gio_Sastre fun String?.reversed() : String? { … } Create method that reverses a given string Properties are not always sufficient Run more tests, so run more slowly
103. Cons Hard to find properties @Gio_Sastre @Gio_Sastre Patterns to find properties Business rule as property If password contains blanks show error message… fun String?.reversed() : String? { … } Create method that reverses a given string Properties are not always sufficient Run more tests, so run more slowly
104. Cons @Gio_Sastre @Gio_Sastre Patterns to find properties Test Oracle list.bubbleSort() == list.moreEfficientSort() fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
105. Cons @Gio_Sastre @Gio_Sastre Invariant functions Patterns to find properties string.reverse()?.length == string?.length fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
106. Cons @Gio_Sastre @Gio_Sastre fun String?.reversed() : String? { … } @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
107. Cons @Gio_Sastre @Gio_Sastre Inverse functions Patterns to find properties string.reversed().reversed() == string @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
108. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
109. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
110. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? = this Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
111. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? = this this this Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
112. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? = this this this Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
113. Cons @Gio_Sastre @Gio_Sastre Create method that reverses a given string @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? = this this this Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
114. Cons @Gio_Sastre @Gio_Sastre @Propert y fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Propert y fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
115. Cons @Gio_Sastre @Gio_Sastre @Property(times = 100 ) fun reversedStringKeepsLength ( @ForAll string: String ? ) { assertTrue ( string.reversed()?.length == string?.lengt h ) } @Property(times = 100 ) fun reversedStringTwiceReturnsOriginal ( @ForAll string: String ? ) { assertTrue ( string.reversed().reversed() == string ) } fun String?.reversed() : String? { … } Create method that reverses a given string Hard to find properties Properties are not always sufficient Run more tests, so run more slowly
116. 1 Property-based testing (PBT) opinionated advice 2 Example + property tests PBT for widely used code 3 @Gio_Sastre PBT for obvious properties
117. attention for your THANKS @Gio_Sastre
118. @Gio_Sastre code @sergio-sastre Sergio Sastre Flórez Lead & Senior Android developer appdev.de blogs @SergioSastre sergiosastre.hashnode.dev