A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs Wednesday, 29 August 12

A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs “All reductions on lists of Boolean values to a single Boolean value can be expressed as a foldr.” Wednesday, 29 August 12

A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs “All reductions on lists of Boolean values to a single Boolean value can be expressed as a foldr.” Functional values Wednesday, 29 August 12

A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs “All reductions on lists of Boolean values to a single Boolean value can be expressed as a foldr.” Existential quantiﬁer Functional values Wednesday, 29 August 12

A ‘conjectured’ property prop_ReduceFold :: ([Bool] -> Bool) -> Property prop_ReduceFold r = exists $ \f z -> forAll $ \xs -> foldr f z xs == r xs “All reductions on lists of Boolean values to a single Boolean value can be expressed as a foldr.” Existential quantiﬁer Functional values Nested quantiﬁcation Wednesday, 29 August 12

LSC? • Lazy SmallCheck (Runciman et al., 2008). • Check – Property-based testing library. • Small – Exhaustive search for minimal counterexamples in bounded test-data space. • Lazy – Space includes partial values and evaluation order guides search. Wednesday, 29 August 12

>>> depthCheck 7 prop_insertSet Depth 7: Completed 109600 test(s) without failure. But 108576 did not meet ==> condition. Beneﬁt of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) In SC Wednesday, 29 August 12

>>> depthCheck 7 prop_insertSet Depth 7: Completed 109600 test(s) without failure. But 108576 did not meet ==> condition. Beneﬁt of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) In SC Wednesday, 29 August 12

Beneﬁt of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) >>> depthCheck 7 prop_insertSet OK, required 1716 tests at depth 7 In LSC 2008 1.6% of tests performed by SC Wednesday, 29 August 12

Beneﬁt of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) Lazy antecedent >>> depthCheck 7 prop_insertSet OK, required 1716 tests at depth 7 In LSC 2008 1.6% of tests performed by SC Wednesday, 29 August 12

• xs = (1:0:⊥) falsiﬁes the antecedent. • Therefore, the LSC doesn’t need to test; xs = [1,0] xs = [1,0,2,3] xs = [1,0,5,4] e.t.c. • Or even any value of x for this class of xs. Beneﬁt of being lazy prop_insertSet :: Char -> [Char] -> Property prop_insertSet x xs = isOrdered xs ==> isOrdered (insert x xs) Lazy antecedent Wednesday, 29 August 12

Functional values II • LSC now generates partial functions including wildcard patterns. • Tries in disguise! • Wildcards explicit but partiality of functions is a result of partial values. • Users need to implement ‘Argument’ instance for functional value argument types. • ‘deriveArgument’ does this automatically using Template Haskell. Wednesday, 29 August 12

Quantiﬁcation II • Lazy pruning is beneﬁcial for existentials too. • Nested quantiﬁcation necessary for existentials to be useful. • Adds forAll and exists to Property DSL. • Required a complete rethink of underlying structure and refutation algorithm. Wednesday, 29 August 12

Performance Name Ratio Catch Circuits1 Circuits2 Circuits3 Countdown1 Countdown2 Huffman1 0.28 1.00 1.04 0.56 0.55 1.01 0.67 Name Ratio Huffman2 ListSet1 Mate RedBlack SumPuz Turner Geo. Mean 0.59 0.80 0.60 0.66 0.97 0.62 0.68 Ratio = LSC2012 execution time LSC2008 execution time Ratio < 1 is improvement. Wednesday, 29 August 12

Related work • Koen Claessen, Shrinking and Showing Functions (Functional Pearl), Haskell 2012. • Extends QuickCheck’s functional value capabilities. • Uses tries (different formulation) to provide additional features. • Must wrap functional values in a ‘modiﬁer’. Wednesday, 29 August 12

Further work • Looking at Claessen’s trie formulation. • Could use SYB instead of TH for automatic instances. • Difﬁcult to judge the depth of a functional value. Wednesday, 29 August 12

Further work • Looking at Claessen’s trie formulation. • Could use SYB instead of TH for automatic instances. • Difﬁcult to judge the depth of a functional value. • Parallel LSC (with JMCT) • Naive so far. Testing on 8 cores. • Scales well for most examples. Wednesday, 29 August 12

Conclusions I • SCs handling of functional values wasn’t entirely satisfying. • New formulation for LSC leverages the ‘lazy’ for maximum effect. • Displaying partial counterexample gives more information than a totally deﬁned one. Wednesday, 29 August 12

Conclusions II • Existentials also beneﬁt from laziness. • Making things more complicated can strangely make them faster? • Broader range of functionality, looking for interesting applications. Wednesday, 29 August 12