Tweet
Tweet

Slide 1

Slide 1 text

Steve Miner @miner Generating Generators Clojure Conj 2014

Slide 2

Slide 2 text

Background • The Way to EDN at Conj 2013 • Testing the Hard Stuff and Staying Sane
 John Hughes at Clojure/West 2014 • Powerful Testing with test.check
 Reid Draper at Clojure/West 2014

Slide 3

Slide 3 text

Overview • Herbert schemas • Property-based testing • Generating generators

Slide 4

Slide 4 text

Conj 2013 @miner

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

edn-format.org edn is a system for the conveyance of values. It is not a type system, and has no schemas.

Slide 7

Slide 7 text

Herbert a schema language for EDN

Slide 8

Slide 8 text

Schema the shape of data

Slide 9

Slide 9 text

Herbert Schemas • Documentation • Validation • Pattern matching • test.check generators

Slide 10

Slide 10 text

Patterns • Literals - :kw, “foo”, 42, nil, true, false • Symbols - int, kw, str, sym, list, vec • Maps - {:a int :b sym} • Vectors - [int kw] • Operators - (or nil (and int (not 42))

Slide 11

Slide 11 text

Quantifiers • (? kw) • (* int) • (+ sym) • kw? int* sym+

Slide 12

Slide 12 text

{:a int :b? sym :c [str*]} matches: {:a 42 :b foo :c [“abc” “def”]} {:a 42 :c [“x” “y”]} {:a 42 :b foo :c []}

Slide 13

Slide 13 text

Herbert API (conforms? pattern value) (conform pattern) (def my-test? (conform ‘[(:= K kw) sym+])) (my-test? ‘[:a foo bar baz])
 ;=> {K :a}

Slide 14

Slide 14 text

Property-based testing test.check

Slide 15

Slide 15 text

Property-based testing • Test against randomly generated inputs • Universally quantified
 (for-all [n gen/int] (> (* n n) n)))) • Shrinking failures
 {:result false, :shrunk {:smallest [0]}} • Predicate holds across generated data • Round trip consistency • Oracle, known-good implementation

Slide 16

Slide 16 text

Properties • for-all
 (for-all [a gen/int b gen/int] (>= (+ a b) a))
 (for-all* [gen/int gen/int] (fn [a b] (>= (+ a b) a))) • quick-check
 (quick-check 100 (for-all [a gen/s-pos-int] (> (* a a) a)))
 {:result false, :shrunk {:smallest [1]}}

Slide 17

Slide 17 text

Monad

Slide 18

Slide 18 text

Generators & Combinators • choose, return, int, string, keyword, etc • hash-map, map, vector, list, tuple • one-of, element, frequency, such-that • fmap • bind • new: recursive-gen, shuffle

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

Vector and an Element (gen/bind (gen/not-empty (gen/vector gen/int))
 #(gen/tuple (gen/return %) (gen/elements %)))) [[2 2] 2], [[4 4 -1 2] 4], [[-3 -1 -5 3 2] -5] (hg/generator ‘[(:= V [int+]) (in V)])

Slide 21

Slide 21 text

Optional Keys {:a? int :b? sym} {:b L} {:a 0} {:b ZB, :a -9} {:b 7B.i2x/P, :a 0} {:a -9} {} {:b XV2/o?---} {:b !h+?0, :a 1} {:a 12} {} 
 {:b Q?-JM9*9/M, :a 10}

Slide 22

Slide 22 text

Regexs (str #"fo*bar+") "fbarrr" "foobarrrrrrrrrrrrrrrrr" "fooooooooooooobarrrr" “fbarr" (kw #":[a-z]") :m :l :u :v :j :l :k :n :p :b :x :u :x :j :h :d :m :d :l :q

Slide 23

Slide 23 text

Bindings [[(:= Low int 9) (:= Hi int 10 99)] (int+ Low Hi)]
 [[0 99] 86] [[9 71] 9 66 9 9 9 71 39 7] [[0 99] 99 0 0 63 42 0 0 17 26 0 0 99 0 0 99]

Slide 24

Slide 24 text

Property from Schemas (defn property 
 ([pred schema] (prop/for-all* [(generator schema)] pred))
 
 ([pred schema1 schema2] 
 (prop/for-all* [(generator schema1) (generator schema2)] pred))
 
 ([pred schema1 schema2 & more]
 (prop/for-all* (list* (generator schema1) (generator schema2) 
 (map generator more)) 
 pred)))

Slide 25

Slide 25 text

Herbert check (defn check
 ([pred schema] (check 100 pred schema))
 ([trials pred schema] 
 (tc/quick-check trials (property pred schema)))) (check #(pos? (count %)) ‘[int*]) {:result false, :failing-size 0, :fail [[]], :shrunk {:smallest [[]]}}

Slide 26

Slide 26 text

Self-testing (defn gen-test
 ([schema] (gen-test schema 100))
 ([schema num]
 (let [confn (conform schema)
 result (hg/check num confn schema)]
 (is (:result result) 
 (str "Schema: " schema " failed "
 (first (get-in result [:shrunk :smallest])))))))

Slide 27

Slide 27 text

Herbert Summary • schema = EDN in EDN • test.check property = predicate + generator • (conform schema) — predicate • (generator schema) — generator

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

Generator Limitations • Negation • (not 42) — works with literals and simple types • (not (and int pos)) — throws • (or (not int) (not pos))) — works • Conjunction • (and int (not 42)) — works • (and (str #“foo.*”) (str #“.*bar”)) — throws • (or (and even odd) 42) — throws

Slide 30

Slide 30 text

To Do • Performance • Nested binding generators • Extensible generators • Cognitect Transit types • Debugging and visualization

Slide 31

Slide 31 text

Thanks • Rich Hickey - edn-format.org • Eric Normand - squarepeg parser • Reid Draper - test.check • Gary Fredericks - re-rand and test.chuck • Zach Tellman - collection-check • http://tos.trekcore.com for Star Trek photos

Slide 32

Slide 32 text

github.com/miner/herbert

Slide 33

Slide 33 text

Star Trek II: 
 The Wrath of Khan 1982 Genesis Device Clojure Conj 2014

Slide 34

Slide 34 text

THE END