Check Your Work: An Introduction to Property-Based Testing

Transcript

1. Check Your Work An introduction to property-based testing Connor Mendenhall @ecmendenhall 8th Light
3. Can we do better?
5. Have you performed the right tests? —“Out of the Tar Pit”
9. Values “add(1,2) equals 3” “sort([1,5,3,4]) returns [1,3,4,5]” “MyCoolModel.save() is the same as MyCoolModel.save().save()”
10. Properties “add(a,b) equals a+b for all values of a and b” “sort(list) returns all values in list in ascending order.” “Model.save() is idempotent.”
11. Prime factors
12. Prime factors 4 = 2·2 17 = 17 30 = 2·3·5 31878 = 2·3·3·7·11·23
13. Prime factors 4 = 2·2 17 = 17 30 = 2·3·5 31878 = 2·3·3·7·11·23
14. Prime factors 4 = 2·2 17 = 17 30 = 2·3·5 31878 = 2·3·3·7·11·23
15. Prime factors 4 = 2·2 17 = 17 30 = 2·3·5 31878 = 2·3·3·7·11·23
16. Prime factors user=> (prime-factors 4) [2 2] user=> (prime-factors 17) [17] user=> (prime-factors 30) [2 3 5] user=> (prime-factors 31878) [2 3 3 7 11 23]
17. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
18. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
19. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
20. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
21. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
22. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
23. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
24. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
25. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
26. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
27. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
28. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
29. (defn trial-division [n divisor factors] (if (< n 2) factors (if (= 0 (mod n divisor)) (recur (/ n divisor) divisor (conj factors divisor)) (recur n (inc divisor) factors)))) (defn prime-factors [n] (trial-division n 2 []))
30. (deftest test-factors-of-one (testing "returns [] for 1" (is (= [] (prime-factors 1)))))
31. (deftest test-factors-of-two (testing "returns [2] for 2" (is (= [2] (prime-factors 2)))))
32. (def primes (load-primes "primes.txt")) (defspec primes-have-one-prime-factor 1000 (testing "A prime's only prime factor is itself" (prop/for-all [prime (gen/elements primes)] (= [prime] (prime-factors prime)))))
33. (def primes (load-primes "primes.txt")) (defspec primes-have-one-prime-factor 1000 (testing "A prime's only prime factor is itself" (prop/for-all [prime (gen/elements primes)] (= [prime] (prime-factors prime)))))
34. (def primes (load-primes "primes.txt")) (defspec primes-have-one-prime-factor 1000 (testing "A prime's only prime factor is itself" (prop/for-all [prime (gen/elements primes)] (= [prime] (prime-factors prime)))))
35. (def primes (load-primes "primes.txt")) (defspec primes-have-one-prime-factor 1000 (testing "A prime's only prime factor is itself" (prop/for-all [prime (gen/elements primes)] (= [prime] (prime-factors prime)))))
36. (def primes (load-primes "primes.txt")) (defspec primes-have-one-prime-factor 1000 (testing "A prime's only prime factor is itself" (prop/for-all [prime (gen/elements primes)] (= [prime] (prime-factors prime)))))
37. (def primes (load-primes "primes.txt")) (defspec primes-have-one-prime-factor 1000 (testing "A prime's only prime factor is itself" (prop/for-all [prime (gen/elements primes)] (= [prime] (prime-factors prime)))))
38. (def primes (load-primes "primes.txt")) (defspec primes-have-one-prime-factor 1000 (testing "A prime's only prime factor is itself" (prop/for-all [prime (gen/elements primes)] (= [prime] (prime-factors prime)))))
39. (deftest test-factors-of-two (testing "returns [2 2] for 4" (is (= [2 2] (prime-factors 4))))) (defspec two-times-prime-returns-two-and-prime 1000 (testing "Factors of 2 * prime = 2 and the prime" (prop/for-all [prime (gen/elements primes)] (= [2 prime] (prime-factors (* 2 prime))))))
40. (deftest test-factors-of-two (testing "returns [2 2] for 4" (is (= [2 2] (prime-factors 4))))) (defspec two-times-prime-returns-two-and-prime 1000 (testing "Factors of 2 * prime = 2 and the prime" (prop/for-all [prime (gen/elements primes)] (= [2 prime] (prime-factors (* 2 prime))))))
41. (deftest test-factors-of-two (testing "returns [2 2] for 4" (is (= [2 2] (prime-factors 4))))) (defspec two-times-prime-returns-two-and-prime 1000 (testing "Factors of 2 * prime = 2 and the prime" (prop/for-all [prime (gen/elements primes)] (= [2 prime] (prime-factors (* 2 prime))))))
42. (deftest test-factors-of-nine (testing "returns [3 3] for 9" (is (= [3 3] (prime-factors 9))))) (defspec check-prime-times-prime-has-two-prime-factors 1000 (testing "Factoring prime * prime returns both primes" (prop/for-all [prime-one (gen/elements primes) prime-two (gen/elements primes)] (= (set [prime-one prime-two]) (set (prime-factors (* prime-one prime-two)))))))
43. (deftest test-factors-of-nine (testing "returns [3 3] for 9" (is (= [3 3] (prime-factors 9))))) (defspec check-prime-times-prime-has-two-prime-factors 1000 (testing "Factoring prime * prime returns both primes" (prop/for-all [prime-one (gen/elements primes) prime-two (gen/elements primes)] (= (set [prime-one prime-two]) (set (prime-factors (* prime-one prime-two)))))))
44. (deftest test-factors-of-nine (testing "returns [3 3] for 9" (is (= [3 3] (prime-factors 9))))) (defspec check-prime-times-prime-has-two-prime-factors 1000 (testing "Factoring prime * prime returns both primes" (prop/for-all [prime-one (gen/elements primes) prime-two (gen/elements primes)] (= (set [prime-one prime-two]) (set (prime-factors (* prime-one prime-two)))))))
45. (deftest test-factors-of-nine (testing "returns [3 3] for 9" (is (= [3 3] (prime-factors 9))))) (defspec check-prime-times-prime-has-two-prime-factors 1000 (testing "Factoring prime * prime returns both primes" (prop/for-all [prime-one (gen/elements primes) prime-two (gen/elements primes)] (= (set [prime-one prime-two]) (set (prime-factors (* prime-one prime-two)))))))
46. (defspec check-product-of-primes-returns-all-primes 1000 (testing "Factoring the product of a series of primes returns all the primes" (prop/for-all [prime-seq (gen/vector (gen/elements primes))] (= (sort prime-seq) (prime-factors (apply *' prime-seq))))))
47. (defspec check-product-of-primes-returns-all-primes 1000 (testing "Factoring the product of a series of primes returns all the primes" (prop/for-all [prime-seq (gen/vector (gen/elements primes))] (= (sort prime-seq) (prime-factors (apply *' prime-seq))))))
48. (deftest test-factors-of-some-big-number (testing "returns [2 2 2 3 3 5 13 23 131] for 14100840" (is (= [2 2 2 3 3 5 13 23 131] (prime-factors (* 2 2 3 3 5 13 23 131)))))
49. $ lein test lein test :only prime-factors.core-test/test-factors-of-some-big-number FAIL in (test-factors-of-some-big-number) (core_test.clj:32) returns [2 2 2 3 3 5 13 23 131] for 14100840 expected: (= [2 2 2 3 3 5 13 23 131] (prime-factors (* 2 2 3 3 5 13 23 131))) actual: (not (= [2 2 2 3 3 5 13 23 131] [2 2 3 3 5 13 23 131])) Ran 13 tests containing 13 assertions. 1 failures, 0 errors. Tests failed.
50. $ lein test lein test :only prime-factors.core-test/test-factors-of-some-big-number FAIL in (test-factors-of-some-big-number) (core_test.clj:32) returns [2 2 2 3 3 5 13 23 131] for 14100840 expected: (= [2 2 2 3 3 5 13 23 131] (prime-factors (* 2 2 3 3 5 13 23 131))) actual: (not (= [2 2 2 3 3 5 13 23 131] [2 2 3 3 5 13 23 131])) Ran 13 tests containing 13 assertions. 1 failures, 0 errors. Tests failed.
51. (defspec check-product-of-primes-returns-all-primes 1000 (testing "Factoring the product of a series of primes returns all the primes" (prop/for-all [prime-seq (gen/vector (gen/elements primes))] (= prime-seq (prime-factors (apply *' prime-seq))))))
52. $ lein test lein test prime-factors.core-check {:result false, :seed 1469999806348, :failing-size 5, :num-tests 6, :fail [[4483 2251 7309 971 6199]], :shrunk {:total-nodes-visited 87, :depth 17, :result false, :smallest [[3 2]]}, :test-var "check-product-of-primes-returns-all-primes"} FAIL in (check-product-of-a-series-of-primes-returns-all-primes) (clojure_test.cljc:21) expected: result actual: false {:result true, :num-tests 1000, :seed 1469999806440, :test-var "check-two-times-prime-has-two-prime-factors"} {:result true, :num-tests 1000, :seed 1469999807020, :test-var "check-primes-have-one-prime-factor"} {:result true, :num-tests 1000, :seed 1469999807577, :test-var "check-prime-times-prime-has-two-prime-factors"} Tests failed.
53. $ lein test lein test prime-factors.core-check {:result false, :seed 1469999806348, :failing-size 5, :num-tests 6, :fail [[4483 2251 7309 971 6199]], :shrunk {:total-nodes-visited 87, :depth 17, :result false, :smallest [[3 2]]}, :test-var "check-product-of-primes-returns-all-primes"} FAIL in (check-product-of-a-series-of-primes-returns-all-primes) (clojure_test.cljc:21) expected: result actual: false {:result true, :num-tests 1000, :seed 1469999806440, :test-var "check-two-times-prime-has-two-prime-factors"} {:result true, :num-tests 1000, :seed 1469999807020, :test-var "check-primes-have-one-prime-factor"} {:result true, :num-tests 1000, :seed 1469999807577, :test-var "check-prime-times-prime-has-two-prime-factors"} Tests failed.
54. $ lein test lein test prime-factors.core-check {:result false, :seed 1469999806348, :failing-size 5, :num-tests 6, :fail [[4483 2251 7309 971 6199]], :shrunk {:total-nodes-visited 87, :depth 17, :result false, :smallest [[3 2]]}, :test-var "check-product-of-primes-returns-all-primes"} FAIL in (check-product-of-a-series-of-primes-returns-all-primes) (clojure_test.cljc:21) expected: result actual: false {:result true, :num-tests 1000, :seed 1469999806440, :test-var "check-two-times-prime-has-two-prime-factors"} {:result true, :num-tests 1000, :seed 1469999807020, :test-var "check-primes-have-one-prime-factor"} {:result true, :num-tests 1000, :seed 1469999807577, :test-var "check-prime-times-prime-has-two-prime-factors"} Tests failed.
55. $ lein test lein test prime-factors.core-check {:result false, :seed 1469999806348, :failing-size 5, :num-tests 6, :fail [[4483 2251 7309 971 6199]], :shrunk {:total-nodes-visited 87, :depth 17, :result false, :smallest [[3 2]]}, :test-var "check-product-of-primes-returns-all-primes"} FAIL in (check-product-of-a-series-of-primes-returns-all-primes) (clojure_test.cljc:21) expected: result actual: false {:result true, :num-tests 1000, :seed 1469999806440, :test-var "check-two-times-prime-has-two-prime-factors"} {:result true, :num-tests 1000, :seed 1469999807020, :test-var "check-primes-have-one-prime-factor"} {:result true, :num-tests 1000, :seed 1469999807577, :test-var "check-prime-times-prime-has-two-prime-factors"} Tests failed.
56. $ lein test lein test prime-factors.core-check {:result false, :seed 1469999806348, :failing-size 5, :num-tests 6, :fail [[4483 2251 7309 971 6199]], :shrunk {:total-nodes-visited 87, :depth 17, :result false, :smallest [[3 2]]}, :test-var "check-product-of-primes-returns-all-primes"} FAIL in (check-product-of-a-series-of-primes-returns-all-primes) (clojure_test.cljc:21) expected: result actual: false {:result true, :num-tests 1000, :seed 1469999806440, :test-var "check-two-times-prime-has-two-prime-factors"} {:result true, :num-tests 1000, :seed 1469999807020, :test-var "check-primes-have-one-prime-factor"} {:result true, :num-tests 1000, :seed 1469999807577, :test-var "check-prime-times-prime-has-two-prime-factors"} Tests failed.
57. Can we do better?
58. Show Your Work
59. Check Your Work Show Your Work