Share
Embed
Intro to QuickCheck + Quviq EQC
Slides from the QuickCheck unsession at StrangeLoop 2013
Sean Cribbs
September 18, 2013
Transcript
23 slides
-
Introduction Quviq Erlang QuickCheck Intro to QuickCheck + Quviq EQC
Sean Cribbs 2013-09-18 Wed -
Introduction Quviq Erlang QuickCheck Problems with xUnit/BDD/etc Hand-coded test cases
are tedious and brittle. When have you written enough of them? How do you know? -
Introduction Quviq Erlang QuickCheck What is QuickCheck? A testing tool,
but not like xUnit Write abstract properties, not specific examples Generates data to test over, with increasing entropy Shrinks failing cases to the smallest similar example -
Introduction Quviq Erlang QuickCheck Sample properties Reversing a list twice
should equal the original list. Reversing and sorting a list should preserve its length. Popping an element from a queue should reduce its size by one. -
Introduction Quviq Erlang QuickCheck Sample property prop_reverse() -> ?FORALL(X, %%
Binding list(int()), %% Generator %% Property lists:reverse(lists:reverse(X)) == X ). -
Introduction Quviq Erlang QuickCheck Thinking in properties A “roundtrip”, e.g
encode/decode? An existing implementation with similar behavior A relationship between inputs/outputs? -
Introduction Quviq Erlang QuickCheck General advice Separate pure and impure
code Write small functions Write functions that manipulate data, side-effect free -
Introduction Quviq Erlang QuickCheck Quviq Erlang Quickcheck Caveat Quviq QuickCheck
(“eqc”) is closed-source, licensed. On the other hand, it is probably the most advanced QuickCheck. -
Introduction Quviq Erlang QuickCheck Other Erlang QuickChecks Triq Trifork, Apache
2.0 license PropEr Uppsala University, GPLv3 license -
Introduction Quviq Erlang QuickCheck Building blocks ?FORALL macro wrapping the
property eqc:quickcheck(Property) Generators for simple types -
Introduction Quviq Erlang QuickCheck Setup -include_lib("eqc/include/eqc.hrl"). Imports the macros and
functions from the eqc module and generators (eqc_gen) -
Introduction Quviq Erlang QuickCheck Executing a property 3> eqc:quickcheck(eqc1:prop_reverse()). ............................................................
............................. OK, passed 100 tests true -
Introduction Quviq Erlang QuickCheck Run more examples 5> eqc:quickcheck(eqc:numtests(250, eqc1:prop_reverse())).
............................................................ ............................................................ ............................................................ ..................................... OK, passed 250 tests true -
Introduction Quviq Erlang QuickCheck Failing test prop_delete() -> ?FORALL(Ls, non_empty(list(int())),
?FORALL(X, elements(Ls), not lists:member(X, lists:delete(X, Ls)))). 10> eqc:quickcheck(eqc1:prop_delete()). ....................................................Failed! [-15,-12,1,1,9,17] 1 Shrinking...(3 times) [1,1] 1 false -
Introduction Quviq Erlang QuickCheck Generating inputs Simple: bool int, largeint,
nat choose char real binary, bitstring function0, function1, function2, function3, function4 Compositions: list, orderedlist, shuffle default elements, oneof, frequency non_empty -
Introduction Quviq Erlang QuickCheck Custom generators Static terms generate themselves
Compound terms can contain generators Modifying a value before usage: ?LET(X, list(int()), lists:usort(X)) Discarding/filtering generated values: ?SUCHTHAT(X, int(), X > 10) Lazy generators: oneof([ int(), ?LAZY(gen_foo()) ]). Binding the generation size into the generator: gen_queue() -> ?SIZED(N, gen_queue(N)). gen_queue(0) -> queue:new(); gen_queue(N) -> queue:in(N, gen_queue(N-1)). -
Introduction Quviq Erlang QuickCheck Sampling generators 13> eqc_gen:sample(eqc_gen:int()). 3 11
-12 -5 3 14 1 6 -13 -19 20 -
Introduction Quviq Erlang QuickCheck Sampling generators (2) 14> eqc_gen:sample(eqc1:gen_queue()). {[10,9,8,7,6,5,4,3,2],[1]}
{[11,10,9,8,7,6,5,4,3,2],[1]} {[12,11,10,9,8,7,6,5,4,3,2],[1]} {[13,12,11,10,9,8,7,6,5,4,3,2],[1]} {[14,13,12,11,10,9,8,7,6,5,4,3,2],[1]} {[15,14,13,12,11,10,9,8,7,6,5,4,3,2],[1]} {[16,15,14,13,12,11,10,9,8,7,6,5,4,3,2],[1]} {[17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2],[1]} {[18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2],[1]} {[19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2],[1]} {[20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2],[1]} -
Introduction Quviq Erlang QuickCheck Sample shrinking 15> eqc_gen:sampleshrink(eqc_gen:list(eqc_gen:int())). [-5,-8] -->
[[],[-5],[-8],[0,-8],[2,-8],[4,-8],[5,-8],[-1,-8],[-3,-8],[- -
Introduction Quviq Erlang QuickCheck Other cool features Stateful tests: eqc_statem
& eqc_fsm Generate a sequence of commands Compute an abstract model Run the commands and compare results Driving other systems eqc_c: automagic binding of C code eqc_socket: send commands over sockets Testing parsers: eqc_grammar Create token generators Test roundtrips: AST -> pretty print -> lex/parse -> AST -
Introduction Quviq Erlang QuickCheck PULSE Automates random interleavings of: message
sending/delivery process birth/death Finds race conditions automatically! Reorderings of messages Receiver died while sender waits Timeouts, DOWN, EXIT arrive sooner/later