Bucharest FP
August 27, 2014
340

QuickCheck: First Dive

August 27, 2014

Transcript

testing?

to do

6. QuickCheck is 80/20 80% of the benefit of formal verification

for 20% of the effort

n features
8. n features 3-4 manually written units tests per feature O(n)

Testing involves a linear amount of work But we all know you won't find all the bugs that way
9. Some bugs only manifest themselves when you use two distinct

features in combination
10. Testing pairs of features How many tests do we need

to write to test all pairs of features? Quadratically many How about those subtle bugs which only manifest themselves when 3 distinct features interact?

12. Testing triplets of features Are we really going to allow

testing to take up O(n3) amount of effort?

conditions

16. None
17. None
18. None
19. None
20. None
21. None

23. The problem In traditional / unit testing, we write test

cases by hand we specify the input exactly we can predict what the expected output should be

25. When we generate test cases we don't know ahead of

time what input will be sent to the API under test therefore we can't rely on expected results for the assert part
26. Question #1: How do we know whether an arbitrarily generated

test has passed or failed?

y + x
29. Property: "For each integer n, if n is even, then

n+1 should be odd." \n -> even(n) ==> odd(n+1)

< size input
31. Property: "Sorting a list twice in a row should give

the same result as sorting only once." (sort . sort) list == sort list
32. Property: "The leftmost element in a sorted list is less

than or equal to each of the first ten elements in the original list before the sort."

in and [ headOfSortedList <= i | i <- take 10 myList ]
34. None

equivalent."
36. clampA :: Int -> Int -> Int -> Int clampA

lowerBound upperBound i | i > upperBound = upperBound | i < lowerBound = lowerBound | otherwise = i clampB :: Int -> Int -> Int -> Int clampB lowerBound upperBound i = (!! 1) . sort \$ [lowerBound, upperBound, i]
37. Property: "The MD5 algorithm should always generate a string of

length 32." \$ echo -n "abc" | md5sum 900150983cd24fb0d6963f7d28e17f72 \$
38. Property: "The MD5 algorithm should always generate a sufficient amount

of entropy." \$ echo -n "abc" | md5sum 900150983cd24fb0d6963f7d28e17f72 \$
39. Question #3: How do I capture the essence of entropy

so that I can write a general property for it? "Entropy" is a highly abstract concept.
40. \$ echo -n "abc" | md5sum 900150983cd24fb0d6963f7d28e17f72 \$ 900150983 cd

24 fb 0 d 6963 f 7 d 28 e 17 f 72 Key info here: 23 digits 9 letters
41. 23 digits and 9 letters. That's not at all balanced,

therefore there is not enough entropy. Test failed.

43. "The software-powered semaphores at a simple intersection should never greenlight

both roads, not even after being hit by 10 lightning strikes in a row."

46. QuickCheck pictures credit: John Hughes "Testing the Really Hard Stuff

and Staying Sane"