Dynamic Programming + Clojure

Some techniques showing how to leverage the dynamic programming technique in an idiomatic way in Clojure, without resorting to reassignment.

Transcript

1. Dynamic Programming + Clojure Chris Mowforth | @m0wfo
2. Dynamic Programming "is a method for solving complex problems by breaking them down into simpler subproblems" (Wikipedia entry1)
3. Optimal Substructure “A solution to problem (P) can be constituted from optimal sub-solutions P1, P2, PN“
4. Before we begin (loop! ...! (recur ...))! Let’s pretend it doesn’t exist
5. Before we begin core.match is your friend2 (match [x]! [something] ...! [something-else] ...! [_] ...)!
6. Fibonacci
7. Fibonacci (defn pure [n]! "A simple recursive definition of a Fibonacci function."! (match [n]! [0] 0! [1] 1! [_] (+ (pure (- n 1)) (pure (- n 2)))))! user=> (time (fib/pure 40))! "Elapsed time: 4442.101648 msecs"! 102334155!
8. Fibonacci State-space explosion! (Structure and Interpretation of Computer Programs, §1.2.2)3
9. Fibonacci overlapping sub-problems (Structure and Interpretation of Computer Programs, §1.2.2)3
10. Fibonacci (defn top-down [n]! "A top-down implementation using memoization.”! ; we need to rebind the recursive call inside 'pure'! (with-redefs [pure (memoize pure)]! (pure n)))! memoization? capture the sub-problems, restrict the state-space user=> (time (fib/top-down 40))! "Elapsed time: 0.413518 msecs"! 102334155!
11. Fibonacci fib(n) 0 0 1 1 2 1 3 2 4 3 5 5 6 8 memoization? (this lookup table could get pretty big…)
12. Fibonacci dynamic solution? (assignment is tolerable in Java, awkward in Clojure) public long fib(int n) {! if (n <= 1) {! return n;! } else {! long i = 0, j = 1, sum = 0;! for (long k = 2; k < n; k++) {! sum = i + j;! i = j;! j = sum;! }! return sum;! }! }!
13. Streams "Streams are delayed lists" (Structure and Interpretation of Computer Programs, §3.5.1)4
14. Streams == laziness
15. Streams == iterate
16. Laziness (defn iterate! "Returns a lazy sequence of x, (f x), (f (f x)) etc.! f must be free of side-effects"! {:added "1.0"! :static true}! [f x] (cons x (lazy-seq (iterate f (f x)))))!
17. Laziness ;; iterate Ad Infinitum starting at 5 using the inc (increment) function! user=> (iterate inc 5)! (5 6 7 8 9 10 11 12 13 14 15 ... n (blocks forever!)! ! ! ;; limit results! user=> (take 5 (iterate inc 1))! (1 2 3 4 5) (clojuredoc)5
18. Back to Fibonacci (defn bottom-up [n]! "A bottom-up dynamic implementation"! (nth (map first! (iterate! (fn [[x y]]! [y (+ x y)]) [0 1])) n))! user=> (time (fib/bottom-up 40))! "Elapsed time: 0.1028 msecs"! 102334155!
19. Levenshtein
20. Levenshtein (defn pure [a b]! "A simple recursive definition of the Levenshtein function"! (match [(count a) (count b)]! [0 _] (count b)! [_ 0] (count a)! [_ _] (let [cost (if (= (last a) (last b)) 0 1)]! (apply min [(+ (pure (butlast a) b) 1)! (+ (pure a (butlast b)) 1)! (+ (pure (butlast a) (butlast b)) cost)]))))! (three recursive calls- even worse than Fibonacci!) user=> (time (lev/pure "s0mething" "something"))! "Elapsed time: 850.185935 msecs"! 1!
21. Levenshtein (defn top-down [a b]! "A top-down implementation using memoization"! (with-redefs [pure (memoize pure)]! (pure a b)))! (I could get used to with-redefs :) user=> (time (lev/top-down "s0mething" "something"))! "Elapsed time: 2.329673 msecs"! 1!
22. Levenshtein ! public static long levenshtein(String s, String t) {! ! ! // degenerate cases! ! ! if (s == t) return 0;! ! ! if (s.length() == 0) return t.length();! ! ! if (t.length() == 0) return s.length();! ! ! ! // create two work vectors of integer distances! ! ! int[] v0 = new int[t.length() + 1];! ! ! int[] v1 = new int[t.length() + 1];! ! ! ! // initialize v0 (the previous row of distances)! ! ! // this row is A[0][i]: edit distance for an empty s! ! ! // the distance is just the number of characters to delete from t! ! ! for (int i = 0; i < v0.length; i++) {! ! ! ! v0[i] = i;! ! ! }! ! ! ! for (int i = 0; i < s.length(); i++) {! ! ! ! // calculate v1 (current row distances) from the previous row v0! ! ! ! ! // first element of v1 is A[i+1][0]! ! ! ! // edit distance is delete (i+1) chars from s to match empty t! ! ! ! v1[0] = i + 1;! ! ! ! ! // use formula to fill in the rest of the row! ! ! ! for (int j = 0; j < t.length(); j++)! ! ! ! {! ! ! ! ! int cost = (s.charAt(i) == t.charAt(j)) ? 0 : 1;! ! ! ! ! v1[j + 1] = Math.min(v1[j] + 1, Math.min(v0[j + 1] + 1, v0[j] + cost));! ! ! ! }! ! ! ! ! // copy v1 (current row) to v0 (previous row) for next iteration! ! ! ! for (int j = 0; j < v0.length; j++) {! ! ! ! ! v0[j] = v1[j];! ! ! ! }! ! ! }! ! ! ! return v1[t.length()];! ! }!
23. Levenshtein dynamic solution? (defn bottom-up [a b]! "A bottom-up dynamic implementation using two vectors"! (match [(count a) (count b)]! ; couple of degenerate cases! [0 _] (count b)! [_ 0] (count a)! [_ _] (let [count-a (count a)! count-b (count b)! init-v0 (vec (range (+ 1 count-b)))! init-v1 (vec (take (+ 1 count-b) (iterate (constantly 0) 0)))]! (last (take (count a)! (iterate (fn [[i v0 v1]]! (let [updated-v1 (assoc v1 0 i)! inner (last (take (count b)! (iterate (fn [[j v1-inner]]! (let [cost (if (= (get a i) (get b j)) 0 1)! min-val (min! (+ 1 (get v1-inner j))! (+ 1 (get v0 (+ 1 j)))! (+ cost (get v0 j)))]! [(+ 1 j)! (assoc v1-inner (+ 1 j) min-val)]))! [0 updated-v1])))]! [(+ 1 i) (last inner) v0])) [0 init-v0 init-v1]))))))! (oh god…)
24. Links 1. Wikipedia definition http://en.wikipedia.org/wiki/Dynamic_programming 2. core.match https://github.com/clojure/core.match/wiki/Overview 3. SICP- §1.2.2 http://mitpress.mit.edu/sicp/full-text/book
 /book-Z-H-11.html#%_sec_1.2.2 4. SICP- §3.5.1 http://mitpress.mit.edu/sicp/full-text/book/
 book-Z-H-24.html#%_sec_3.5.1 5. Iterate clojuredoc http://clojuredocs.org/clojure_core/clojure.core/iterate