Upgrade to Pro — share decks privately, control downloads, hide ads and more …

QuickCheck: First Dive

QuickCheck: First Dive

0286822f506fc4621bd3ea0bcbfef238?s=128

Bucharest FP

August 27, 2014
Tweet

More Decks by Bucharest FP

Other Decks in Programming

Transcript

  1. QuickCheck First Dive 27.08.2014 • Dan Șerban • github.com/dserban

  2. Let's start with WHY

  3. Let's start with WHY Q: WHY QuickCheck? Q: WHY property-based

    testing?
  4. A: Because testing is a very hard thing for developers

    to do
  5. QuickCheck is 80/20

  6. QuickCheck is 80/20 80% of the benefit of formal verification

    for 20% of the effort
  7. Why is testing so hard? Think of a system with

    n features
  8. n features 3-4 manually written units tests per feature O(n)

    Testing involves a linear amount of work But we all know you won't find all the bugs that way
  9. Some bugs only manifest themselves when you use two distinct

    features in combination
  10. Testing pairs of features How many tests do we need

    to write to test all pairs of features? Quadratically many How about those subtle bugs which only manifest themselves when 3 distinct features interact?
  11. This is starting to sound like a lot of work

  12. Testing triplets of features Are we really going to allow

    testing to take up O(n3) amount of effort?
  13. And that's just for sequential code

  14. We haven't even started talking about testing for intermittent race

    conditions
  15. So how exactly does QuickCheck work?

  16. None
  17. None
  18. None
  19. None
  20. None
  21. None
  22. There is a problem, however

  23. The problem In traditional / unit testing, we write test

    cases by hand we specify the input exactly we can predict what the expected output should be
  24. But we can't use expected results in randomized testing

  25. When we generate test cases we don't know ahead of

    time what input will be sent to the API under test therefore we can't rely on expected results for the assert part
  26. Question #1: How do we know whether an arbitrarily generated

    test has passed or failed?
  27. Question #2: What is a general property?

  28. Property: "Plus is a commutative operator." x + y ==

    y + x
  29. Property: "For each integer n, if n is even, then

    n+1 should be odd." \n -> even(n) ==> odd(n+1)
  30. Property: "Compression should make things smaller" (size . compress) input

    < size input
  31. Property: "Sorting a list twice in a row should give

    the same result as sorting only once." (sort . sort) list == sort list
  32. Property: "The leftmost element in a sorted list is less

    than or equal to each of the first ten elements in the original list before the sort."
  33. propSort myList = let headOfSortedList = (head . sort) myList

    in and [ headOfSortedList <= i | i <- take 10 myList ]
  34. None
  35. Property: "The following two implementations for the clamp function are

    equivalent."
  36. clampA :: Int -> Int -> Int -> Int clampA

    lowerBound upperBound i | i > upperBound = upperBound | i < lowerBound = lowerBound | otherwise = i clampB :: Int -> Int -> Int -> Int clampB lowerBound upperBound i = (!! 1) . sort $ [lowerBound, upperBound, i]
  37. Property: "The MD5 algorithm should always generate a string of

    length 32." $ echo -n "abc" | md5sum 900150983cd24fb0d6963f7d28e17f72 $
  38. Property: "The MD5 algorithm should always generate a sufficient amount

    of entropy." $ echo -n "abc" | md5sum 900150983cd24fb0d6963f7d28e17f72 $
  39. Question #3: How do I capture the essence of entropy

    so that I can write a general property for it? "Entropy" is a highly abstract concept.
  40. $ echo -n "abc" | md5sum 900150983cd24fb0d6963f7d28e17f72 $ 900150983 cd

    24 fb 0 d 6963 f 7 d 28 e 17 f 72 Key info here: 23 digits 9 letters
  41. 23 digits and 9 letters. That's not at all balanced,

    therefore there is not enough entropy. Test failed.
  42. Properties can be expressed for mission-critical systems too

  43. "The software-powered semaphores at a simple intersection should never greenlight

    both roads, not even after being hit by 10 lightning strikes in a row."
  44. Demo Time!

  45. Follow me on GitHub github.com/dserban

  46. QuickCheck pictures credit: John Hughes "Testing the Really Hard Stuff

    and Staying Sane"