Clojure — One Level Above Meta

http://bucharestfp.ro/2014/10/29/clojure-one-level-above-meta/

Transcript

1. CLOJURE one level above meta Mihnea Dobrescu-Balaur mihnea@linux.com @mihneadb
2. ABOUT ME • student at • ongoing master's in AI • shipping code at • interested in functional programming
3. ABOUT THIS TALK • Clojure • core.async • transducers • demos
4. CLOJURE • LISP - functional, dynamic • hosted language (JVM as main target) • excellent data structures - persistent, lazy etc.
5. HELLO, WORLD! (println "Hello, world!") ! (+ 1 2) ! (def x 3) ! ((fn [x] (+ x 2)) 5) ! (defn say-hello [name] (println "Hello" name))
6. DATA STRUCTURES • list: (1 2 3) • vector: [1 2 3] • set: #{1 3 2} • map: {:name "Tom", :age 24}
7. PERSISTENT DATA STRUCTURES • immutable • any change creates a new instance • memory is reused • O(1) -> O(log_32(n)) [Wikipedia]
8. SEQUENCES • abstraction over all collections • most of stdlib is defined in terms of seqs • "It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures." - Alan J. Perlis • map, filter, reduce, some, concat, first, rest, partition, flatten, distinct, take, drop, interleave, next, sort, reverse, split-at, mapcat, nth, group-by, ...
9. DEMO
10. CORE.ASYNC • library (not part of the language!) • introduces "go blocks" and channels • think sockets + select • works in ClojureScript too (no threads)
11. CORE.ASYNC - WHY? • decoupling • independent threads of activity • communication via channels • pipeline
12. HELLO, CORE.ASYNC! (def c (chan)) ! (go (while true (let [msg (<! c)] (println "Got message" msg)))) ! (go (>! c "Hello, core.async!"))
13. USING WHAT WE'VE LEARNED https://github.com/mihneadb/clj-reddit-pipeline
14. NOW FOR THE META STUFF
15. (fn [x] (+ 1 x)) Simple Functions
16. (map (fn [x] (+ 1 x) [1 2 3]) (filter odd? (range 5)) (reduce * 1 (range 1 10) Higher Order Functions
17. (defn map [f coll] (reduce (fn [acc x] (conj acc (f x))) [] coll)) ! Higher Order Functions implemented in terms of reduce (defn filter [pred coll] (reduce (fn [acc x] (if (pred x) (conj acc x) acc)) [] coll)) !
18. HIGHER ORDER FUNCTIONS ARE COOL (def stream (->> "all" r/subreddit-new r/items (filter pred) (map uppercase-title) (map make-json) (map send-to-listener))) (def my-pipeline (comp (partial map send-to-listener) (partial map make-json) (partial map uppercase-title) (partial filter pred)))
19. BUT THEY.. • only work on collections • use intermediate results • describe the big picture • not only the transformation
20. WE WANT • to work with anything • data comes from a list today, from a channel tomorrow • to focus on the transformation • I carry stuff from A to B. • doesn't matter if I use bags or my bare hands • efficiency - no intermediate data
21. (defn map [f coll] (reduce (fn [acc x] (conj acc (f x))) [] coll)) ! Higher Order Functions implemented in terms of reduce (defn filter [pred coll] (reduce (fn [acc x] (if (pred x) (conj acc x) acc)) [] coll)) !
22. (defn map [f coll] (reduce (fn [acc x] (conj acc (f x))) [] coll)) ! Higher Order Functions implemented in terms of reduce (defn filter [pred coll] (reduce (fn [acc x] (if (pred x) (conj acc x) acc)) [] coll)) !
23. (defn mapping [f] (fn [step] (fn [acc x] (step acc (f x))))) ! Parametrizing the specifics (defn filtering [f] (fn [step] (fn [acc x] (if (f x) (step acc x) acc)))) !
24. (defn mapping [f] (fn [step] (fn [acc x] (step acc (f x))))) ! Parametrizing the specifics (defn filtering [f] (fn [step] (fn [acc x] (if (f x) (step acc x) acc)))) ! Transducer
25. (defn map [f coll] (reduce ((mapping f) conj) [] coll)) ! Using transducers to (re)define map and filter (defn filter [f coll] (reduce ((filtering f) conj) [] coll)) !
26. COMPOSING TRANSDUCERS (reduce ((comp (mapping inc) (mapping str)) conj) [] [1 2 3])
27. COMPOSING TRANSDUCERS ((comp (mapping inc) (mapping str)) conj) ((comp f g) x) === (f (g x))
28. COMPOSING TRANSDUCERS ((comp (fn [step] (fn [acc x] (step acc (inc x)))) (fn [step] (fn [acc x] (step acc (str x))))) conj) !
29. COMPOSING TRANSDUCERS ((fn [step] (fn [acc x] (step acc (inc x)))) ((fn [step] (fn [acc x] (step acc (str x)))) conj)) !
30. COMPOSING TRANSDUCERS ((fn [step] (fn [acc x] (step acc (inc x)))) ((fn [step] (fn [acc x] (step acc (str x)))) conj)) !
31. COMPOSING TRANSDUCERS ((fn [step] (fn [acc x] (step acc (inc x)))) (fn [acc x] (conj acc (str x)))) !
32. COMPOSING TRANSDUCERS ((fn [step] (fn [acc x] (step acc (inc x)))) (fn [acc x] (conj acc (str x)))) !
33. COMPOSING TRANSDUCERS (fn [acc x] ((fn [acc x] (conj acc (str x))) acc (inc x))) !
34. COMPOSING TRANSDUCERS (fn [acc x] ((fn [acc x] (conj acc (str x))) acc (inc x))) !
35. COMPOSING TRANSDUCERS (fn [acc x] (conj acc (str (inc x)))) !
36. COMPOSING TRANSDUCERS (fn [acc x] (conj acc (str (inc x)))) ! (defn mapping [f] (fn [step] (fn [acc x] (step acc (f x))))) !
37. NO INTERMEDIARY COLLECTIONS. #perfmatters
38. TRANSDUCERS IN CLOJURE • available as of 1.7 • arity-1 map, filter, mapcat etc. return transducers • apply them with sequence, into etc. • they even work on channels!
39. DEMO
40. BONUS: PROOF • map - http://ow.ly/Duda1 • sequence - http://ow.ly/Dudla • LazyTransformer.create - http://ow.ly/DudzD • Stepper constructor - http://ow.ly/DudJG (sequence (map inc) [1 2 3])
41. CLOJURE • almost no syntax • smart data structures • rich set of operations • useful abstractions