Transducers

Clojure Transducers

Transcript

1. Transducers
2. • recast sequence processing functions (map, filter, etc.) as process transformations
3. processes • succession of steps • each step ingests an input
4. Reducing functions • whatever, input -> whatever
5. • sources/sinks are irrelevant/unspecified
6. • sum, number -> sum • (+ 1 2)
 => 3
7. • person, attribute -> person (into {:name "Alex"} {:company “VTS”}) => {:name "Alex" :company "VTS"}
8. • vector, input -> vector • (conj [1] 2)
 => [1, 2]
9. • transducers are: 1. reusable 2. avoid intermediary collections 3. composable
10. reusable • every new collection defines own version of map, filter, etc. • MyCollection->MyCollection (append list 1) • Channel->Channel (>! chan 1) • Sequence->Sequence (conj [] 1)
11. intermediate collections (1..10).map { |x| x * 2 } .select { |x| x > 20 } .reduce { |sum, x| sum + x }
12. composable (1..10).transduce(&method(:push))
13. composable (1..10).transduce(&method(:merge))
14. • Clojure 101
16. (def name "Alex")
17. (println "Hello " name) => Hello Alex
18. (fn [x] (* x x))
19. (defn square [x] (* x x))
20. ((comp inc inc) 1) (inc (inc 1))
21. (def plus-two (comp inc inc))
22. • Transducers
24. (map inc [1 2 3 4 5 6]) => (2 3 4 5 7) (filter even? [1 2 3 4 5 6]) => (2 4 6)
25. • write map, filter in terms of reduce?
26. map (reduce (fn [result input] (conj result (inc input))) [] [1 2 3 4])
27. filter (reduce (fn [result input] (if (even? input) (conj result input) result)) [] [1 2 3 4])
28. (reduce (fn [result input] (conj result (inc input))) [] [1 2 3 4]) (reduce (fn [result input] (if (even? input) (conj result input) result)) [] [1 2 3 4])
29. (reduce (fn [result input] (conj result (inc input))) [] [1 2 3 4]) (reduce (fn [result input] (if (even? input) (conj result input) result)) [] [1 2 3 4])
30. map (reduce (fn [result input] (conj result (inc input))) [] [1 2 3 4])
31. map (reduce (fn [result input] (conj result (inc input))) [] [1 2 3 4]) (reduce ((fn [f] (fn [result input] (conj result (f input)))) inc) [] [1 2 3 4])
32. map (reduce (fn [result input] (conj result (inc input))) [] [1 2 3 4]) (reduce ((fn [f] (fn [result input] (conj result (f input)))) dec) [] [1 2 3 4])
33. filter (reduce (fn [result input] (if (even? input) (conj result input) result)) [] [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (conj result input) result))) even?) [] [1 2 3 4])
34. (reduce (fn [result input] (if (even? input) (conj result input) result)) [] [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (conj result input) result))) odd?) [] [1 2 3 4])
35. (reduce (fn [result input] (if (even? input) (conj result input) result)) [] [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (conj result input) result)))(fn [x] (< x 10))) [] [1 2 3 4])
37. what’s the problem? (reduce ((fn [f] (fn [result input] (conj result (f input))) inc) [] [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (conj result input) even?) result))) [] [1 2 3 4])
38. (reduce ((fn [f] (fn [result input] (conj result (f input))) inc) [] [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (conj result input) even?) result))) [] [1 2 3 4])
39. (reduce ((fn [f] (fn [result input] (conj result (f input))) inc) [] [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (conj result input) even?) result))) [] [1 2 3 4])
40. (reduce ((fn [f] (fn [result input] (conj result (f input))) inc) 0 [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (conj result input) even?) result))) 0 [1 2 3 4])
41. (reduce ((fn [f] (fn [result input] (conj result (f input))) inc) stream [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (conj result input) even?) result))) stream [1 2 3 4])
42. (reduce ((fn [f] (fn [result input] (conj result (f input))) inc) chan [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (conj result input) even?) result))) chan [1 2 3 4])
43. (reduce ((fn [f] (fn [result input] (step result (f input))) inc) file [1 2 3 4]) (reduce ((fn [predicate] (fn [result input] (if (predicate input) (step result input) even?) result))) file [1 2 3 4])
44. (defn mapping [f] (fn [step] (fn [result input] (step result (f input))))) (defn filtering [predicate] (fn [step] (fn [result input] (if (predicate input) (step result input) result))))
45. (defn mapping [f] ;; fn to apply to each input (fn [step] (fn [result input] (step result (f input)))))
46. (defn mapping [f] (fn [step] ;; reducing fn that moves reduction forward (fn [result input] (step result (f input)))))
47. (defn mapping [f] (fn [step] (fn [result input] ;; reducing function (step result (f input)))))
48. • mapping/filtering return transducers • Transducers are functions that accept a reducing (step) function and return a reducing function
49. • Note to future self: Alex. Do not go to the next slide yet. Open vim and show the repl examples.
50. • map, cat • mapcat, filter • remove, take • take-while, take-nth • drop, drop-while • replace, partition-by • partition-all, keep • keep-indexed, map-indexed • distinct, interpose • dedupe random-sample