Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Way to EDN

miner
November 15, 2013

The Way to EDN

This presentation was given as a lightning talk at Clojure/conj 2013. It introduced the Herbert project, which implements a schema language for Clojure data. The whiteboard-compatible notation (represented as EDN values) is useful for documentation and testing.

video: http://www.youtube.com/watch?v=bmHTFo2Rf2w&t=19m55s

miner

November 15, 2013
Tweet

More Decks by miner

Other Decks in Programming

Transcript

  1. Steve Miner

    @miner

    View Slide

  2. edn-format.org
    edn is a system for the conveyance of values.

    It is not a type system, and has no schemas.

    View Slide

  3. Schema
    the shape of data

    View Slide

  4. Herbert
    a schema language for EDN

    View Slide

  5. Whiteboard
    Compatible
    simple

    terse

    readable

    View Slide

  6. EDN in EDN

    View Slide

  7. Patterns
    • Literals - :kw, “foo”, 42, nil, true, false

    • Symbols - int, kw, str, sym, list, vec

    • Maps - {:a int}

    • Vectors - [int kw]

    • Operators - (or nil (and int (not 42))

    View Slide

  8. Quantifiers
    • (? kw)

    • (* int)

    • (+ sym)

    • kw? int* sym+

    View Slide

  9. {:a int :b? sym :c [str*]}

    matches:

    {:a 42 :b foo :c [“abc” “def”]}

    {:a 42 :c [“x” “y”]}

    {:a 42 :b foo :c []}

    View Slide

  10. Grammar
    (grammar [person+]

    name (and str (not (pred clojure.string/blank?)))

    handle (str “@.+”)


    person {:first name :last name :twitter? handle})

    !
    [{:first “Steve” :last “Miner” :twitter “@miner”}

    {:first “James” :last “Kirk”}]

    !
    (grammar (list 'grammar pattern (* term pattern))

    term sym

    pattern any)

    View Slide

  11. Bindings
    [(:= n int) n n] matches [3 3 3]

    {:a (:= s sym) :b (not s)} matches {:a foo :b bar}

    [(:= low int) (:= hi int) (int* low hi)]

    matches [3 7 4 6 5]

    !
    {:len (:= n int) :vals (and (cnt n) [sym*])}

    matches {:len 5 :vals [a b c d e]}

    View Slide

  12. Tags
    • (tag inst) matches any java.util.Date,
    java.util.Calendar and java.sql.Timestamp

    • (tag my.ns/Rec) matches any record my.ns.Rec

    • (tag my.ns/Rec {:a int}) matches #my.ns.Rec{:a 42}

    • protocol miner.tagged.EdnTag in tagged

    View Slide

  13. Herbert API
    (conforms? pattern value)

    (conform pattern)

    (def my-test? (conform ‘[(:= k kw) sym+]))

    (my-test? ‘[:a foo bar baz])

    ;=> {k :a}

    View Slide

  14. Herbert
    • Open source on
    github and clojars

    • https://github.com/
    miner/herbert

    • https://clojars.org/
    com.velisco/herbert

    View Slide

  15. Thanks
    • Eric Normand - squarepeg parser

    • tagged library - tagged records

    • edn-format.org

    • Prismatic schema

    • runa-dev clj-schema
    • http://tos.trekcore.com for Star Trek photos

    View Slide

  16. Star Trek
    {:title

    “The Way to Eden”

    :stardate

    #tos/stardate 5832.3

    :airdate

    #inst “1969-02-21”

    :season 3

    :episode 20}

    View Slide

  17. Steve Miner

    @miner

    View Slide

  18. Examples
    • [kw str int] - [:a “foo” 4]

    • {:a int :b sym :c kw} - {:a 42 :b bar :c :baz}

    • (keys kw sym) - {:a foo :b bar :c baz}

    • [int+ kw?] - [1 2 3 :end]

    View Slide

  19. Martin Fowler
    • “schemaless” means implicit schema

    • prefer explicit schemas

    • http://martinfowler.com/articles/schemaless/

    • https://www.youtube.com/watch?v=8kotnF6hfd8

    View Slide

  20. Uses for Schemata
    • Documentation

    • Validation

    • Pattern matching

    • Data transformation

    View Slide