Property-Based Testing with SwiftCheck

These are the slides for my talk at try! swift NYC on 2016-09-02

Transcript

1. Property-Based Testing with SwiftCheck 1
2. Tests are difficult to write 2
3. Tests are difficult to write 3
4. Test cases are difficult to write 4
5. How are test cases difficult to write? 5
6. How are test cases difficult to write? • We need to find all of the meaninful test cases 5
7. How are test cases difficult to write? • We need to find all of the meaninful test cases • We need to organize them in a meaningful way 5
8. How are test cases difficult to write? • We need to find all of the meaninful test cases • We need to organize them in a meaningful way • How many cases are enough? 5
9. How can we find test cases? 6
10. How can we find test cases? • Randomness 6
11. How can we find test cases? • Randomness • Reproducible 6
12. How can we find test cases? • Randomness • Reproducible • Simplify failing case (when possible) 6
14. Let's Get Testing public struct Rational { public let numerator : Int public let denominator : Int } 8
15. Arbitrary public protocol Arbitrary { /// The generator for this particular type. public static var arbitrary: SwiftCheck.Gen<Self> { get } } 9
16. Gen Gen represents a generator for random arbitrary values of a type. public struct Gen<A> 10
17. Conforming to Arbitrary public static var arbitrary: SwiftCheck.Gen<Rational> { return Gen.compose { comp in return Rational(comp.generate(), comp.generate()) } } 11
18. Conforming to Arbitrary return Rational(comp.generate(), comp.generate()) 12
19. Filtering Generated values Int.arbitrary.suchThat { $0 != 0 } // Denominator 13
20. Filtering Generated values extension Rational : Arbitrary { public static var arbitrary: SwiftCheck.Gen<Rational> { return Gen<Rational>.compose { comp in let denGen = Int.arbitrary.suchThat { $0 != 0 } return Rational( comp.generate(), comp.generate(using: denGen) } } 14
21. OK, What to test? 15
22. Properties! 16
23. Round Trip Properties property("round trip to string") <- forAll { (i: Int) in return Int(i.description)! == i } 17
24. Round Trip Properties // for all foo: [Int] foo.reverse().reverse() == foo 18
25. Commuting Diagram Properties • ----- | | | | | ! -----" 19
26. Commuting Diagram Properties property("base64 encoding commutes with Apple's ") <- forAll { (data: Data) in return data.base64Encoding == data.myBase64Encoding } 20
27. Properties func testAddition() { property("Addition is commutative") <- forAll { (i: Rational, j: Rational) in return (i + j) == (j + i) } } 21
28. Properties property("Addition is commutative 2") <- forAll { (i: Rational, j: Rational, k: Rational) in let caseOne = (i + j + k) let caseTwo = ((i + j) + k) let caseThree = (i + (j + k)) return (caseOne == caseTwo) && (caseTwo == caseThree) } 22
29. Replay 23
30. Replay let (seedl, seedr) = (1640744780, 1403884642) let replayArgs = CheckerArguments(replay: .some(StdGen(seedl, seedr), 3)) property("x / y", arguments: replayArgs) <- forAll { … 24
31. Mixed Testing func testEquality() { let oneOverTwo = Rational(1, 2)! let twoOverFour = Rational(2, 4)! let ichiUeNi = Rational(1, 2)! XCTAssertEqual(oneOverTwo == twoOverFour, true) XCTAssertEqual(twoOverFour == ichiUeNi, true) XCTAssertEqual(ichiUeNi == oneOverTwo, true) XCTAssertEqual(oneOverTwo ≡ twoOverFour, false) XCTAssertEqual(twoOverFour ≡ ichiUeNi, false) XCTAssertEqual(ichiUeNi ≡ oneOverTwo, true) let halfIntMax = (Int.max / 2) - 1 property("value holds") <- forAll { (value: Rational) in return ((value.numerator < halfIntMax) && (value.denominator < halfIntMax)) ==> { let doubledSame = Rational(value.numerator * 2, value.denominator * 2)! return (value == doubledSame) && ((value ≡ doubledSame) == false) } } } 25
32. Shrinking 26
33. So? 27
34. So? • Less time writing tests? 27
35. So? • Less time writing tests? • Better tests 27
36. So? • Less time writing tests? • Better tests • Easier diagnosis 27
37. So? • Less time writing tests? • Better tests • Easier diagnosis • You will and should still write test cases! 27
38. For Our Friends in Obj-C https://github.com/jeffh/Fox 28
39. TJ Usiyan @griotspeak 29