A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs Wednesday, 29 August 12
A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs “All reductions on lists of Boolean values to a single Boolean value can be expressed as a foldr.” Wednesday, 29 August 12
A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs “All reductions on lists of Boolean values to a single Boolean value can be expressed as a foldr.” Functional values Wednesday, 29 August 12
A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs “All reductions on lists of Boolean values to a single Boolean value can be expressed as a foldr.” Existential quantifier Functional values Wednesday, 29 August 12
A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs “All reductions on lists of Boolean values to a single Boolean value can be expressed as a foldr.” Existential quantifier Functional values Nested quantification Wednesday, 29 August 12
LSC? • Lazy SmallCheck (Runciman et al., 2008). • Check – Property-based testing library. • Small – Exhaustive search for minimal counterexamples in bounded test-data space. • Lazy – Space includes partial values and evaluation order guides search. Wednesday, 29 August 12
>>> depthCheck 7 prop_insertSet Depth 7: Completed 109600 test(s) without failure. But 108576 did not meet ==> condition. Benefit of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) In SC Wednesday, 29 August 12
>>> depthCheck 7 prop_insertSet Depth 7: Completed 109600 test(s) without failure. But 108576 did not meet ==> condition. Benefit of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) In SC Wednesday, 29 August 12
Benefit of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) >>> depthCheck 7 prop_insertSet OK, required 1716 tests at depth 7 In LSC 2008 1.6% of tests performed by SC Wednesday, 29 August 12
Benefit of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) Lazy antecedent >>> depthCheck 7 prop_insertSet OK, required 1716 tests at depth 7 In LSC 2008 1.6% of tests performed by SC Wednesday, 29 August 12
• xs = (1:0:⊥) falsifies the antecedent. • Therefore, the LSC doesn’t need to test; xs = [1,0] xs = [1,0,2,3] xs = [1,0,5,4] e.t.c. • Or even any value of x for this class of xs. Benefit of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) Lazy antecedent Wednesday, 29 August 12
Functional values II • LSC now generates partial functions including wildcard patterns. • Tries in disguise! • Wildcards explicit but partiality of functions is a result of partial values. • Users need to implement ‘Argument’ instance for functional value argument types. • ‘deriveArgument’ does this automatically using Template Haskell. Wednesday, 29 August 12
Quantification II • Lazy pruning is beneficial for existentials too. • Nested quantification necessary for existentials to be useful. • Adds forAll and exists to Property DSL. • Required a complete rethink of underlying structure and refutation algorithm. Wednesday, 29 August 12
Performance Name Ratio Catch Circuits1 Circuits2 Circuits3 Countdown1 Countdown2 Huffman1 0.28 1.00 1.04 0.56 0.55 1.01 0.67 Name Ratio Huffman2 ListSet1 Mate RedBlack SumPuz Turner Geo. Mean 0.59 0.80 0.60 0.66 0.97 0.62 0.68 Ratio = LSC2012 execution time LSC2008 execution time Ratio < 1 is improvement. Wednesday, 29 August 12
Related work • Koen Claessen, Shrinking and Showing Functions (Functional Pearl), Haskell 2012. • Extends QuickCheck’s functional value capabilities. • Uses tries (different formulation) to provide additional features. • Must wrap functional values in a ‘modifier’. Wednesday, 29 August 12
Further work • Looking at Claessen’s trie formulation. • Could use SYB instead of TH for automatic instances. • Difficult to judge the depth of a functional value. Wednesday, 29 August 12
Further work • Looking at Claessen’s trie formulation. • Could use SYB instead of TH for automatic instances. • Difficult to judge the depth of a functional value. • Parallel LSC (with JMCT) • Naive so far. Testing on 8 cores. • Scales well for most examples. Wednesday, 29 August 12
Conclusions I • SCs handling of functional values wasn’t entirely satisfying. • New formulation for LSC leverages the ‘lazy’ for maximum effect. • Displaying partial counterexample gives more information than a totally defined one. Wednesday, 29 August 12
Conclusions II • Existentials also benefit from laziness. • Making things more complicated can strangely make them faster? • Broader range of functionality, looking for interesting applications. Wednesday, 29 August 12
chumajin
leveragestech
tkikuchi1002
cybozuinsideout
nwiizo
onlinesalon
coconala_engineer
k1low
optimisuke
bigwheel
yukimatsuyoshi
nishiuma
scottboms
addyosmani
eileencodes
palkan
panda_program
deanohume
chrislema
marcelosomers
mthomps
danielanewman
vanstee
zakiyama