You Wouldn't Fold a Tree...?

Functional programming uses a lot of recursion and pattern matching to transform inductive data, but sometimes our functions are mostly plumbing data around which makes it hard to understand what the important part of the transformation is. We’ll look at some techniques that help with these patterns, and how they can be applied to lists and different kinds of trees.

Links:
- Functional Programming with Bananas, Lenses, Envelopes, and Barbed Wire: https://maartenfokkinga.github.io/utwente/mmf91m.pdf

- Scrap Your Boilerplate: A Practical Design Pattern for Generic Programming: https://www.microsoft.com/en-us/research/wp-content/uploads/2003/01/hmap.pdf

- The implementation of these combinators in Erlang: https://github.com/robotlolita/generique

Transcript

1. You Wouldn’t Fold a Tree…? (recursion schemes for Erlang)
2. I’m Quil. I love languages. (aka @robotlolita) actual Quils may vary due to reality enhancements
3. RECURSION SCHEMES I’ll assume that you’re familiar with recursion, pattern matching, and map/filter/fold TODAY’S TOPIC:
4. The (FP) Agenda WHY? (motivation) FOLDS (you might know this already!) GENERALISED FOLDS I. II. III.
5. PROGRAMMING IS TRANSFORMING DATA
6. FP IS REALLY GOOD AT THIS DEFINING DATA type list(A) :: empty | {cons, A, list(A)}. TRANSFORMING DATA rest(empty) -> empty; rest({cons, _, Rest}) -> Rest.
7. SO MUCH WE CAN USE THE MAXIM Function follows Form a.k.a. data
8. ANOTHER INTERESTING FACT Most data can be represented as some sort of tree.
9. LISTS ARE A TREE type list(A) :: empty | {cons, A, list(A)} 1 CONS 2 CONS 3 CONS EMPTY
10. 1 fib(0) fib(3) fib(2) fib(1) 1 fib(1) 1 RECURSION IS A TREE fib(0) -> 0; fib(1) -> 1; fib(N) -> fib(N-1) + fib(N-2).
11. IT’S NOT ENTIRELY ABOUT TREES. But we want uniformity.
12. TREES ARE VERY FLEXIBLE, THOUGH • • • • •
13. LET’S GO BACK TO LISTS type list(A) :: empty | {cons, A, list(A)}.
14. There are three important points I want you to focus on.
15. TRANSFORMING ELEMENTS OF A LIST map({cons, X, Rest}, Fun) -> {cons, Fun(X), map(Rest, Fun)}; map(empty, Fun) -> empty.
16. REMOVING ELEMENTS FROM A LIST filter({cons, X, Rest}, Pred) -> case Pred(X) of true -> {cons, X, filter(Rest, Pred)}; false -> filter(Rest, Pred) end; filter(empty, Pred) -> empty.
17. SUMMING UP ELEMENTS OF A LIST sum({cons, X, Rest}) -> X + sum(Rest); sum(empty) -> 0.
18. CONCATENATING TWO LISTS concat({cons, X, Xs}, Ys) -> {cons, X, concat(Xs, Ys)}; concat(empty, Ys) -> Ys.
19. THREE THINGS TO NOTE • • • empty
20. CAPTURING THE COMMON PATTERN fold(Cons, Empty, {cons, X, Rest}) -> Cons(X, fold(Cons, Empty, Rest)); fold(Cons, Empty, empty) -> Empty().
21. TRANSFORMING ELEMENTS OF A LIST map(List, Fun) -> fold( fun(X, Xs) -> {cons, Fun(X), Xs} end, fun() -> empty end, List ).
22. REMOVING ELEMENTS FROM A LIST filter(List, Pred) -> fold( fun(X, Xs) -> case Pred(X) of true -> {cons, X, Xs}; false -> Xs end end, fun() -> empty end, List ).
23. SUMMING UP ELEMENTS OF A LIST sum(List) -> fold( fun(A, B) -> A + B end, fun() -> 0 end, List ).
24. CONCATENATING TWO LISTS concat(Xs, Ys) -> fold( fun(A, As) -> {cons, A, As} end, fun() -> Ys end, Xs ).
25. INTERLUDE: LIGHTWEIGHT NOTATIONS map f xs = foldr (\x xs -> f x : xs) [] filter p xs = foldr (\x xs -> if p x then x : xs else xs) [] sum xs = foldr (+) 0 concat xs ys = foldr (:) ys xs
26. FUNCTIONAL PROGRAMMING WITH BANANAS, LENSES, ENVELOPES, AND BARBED WIRE IT’S DANGEROUS TO GO ALONE! TAKE THESE WITH YOU.
27. OPTIMISING FP Dr. Gabriele Keller and others have been using this to optimise functional programs with e.g.: stream fusion.
28. Let’s go back to folds.
29. RIGHT NOW We need to write one fold (and its combinators) for each data structure! this doesn’t scale!
30. What we really want is a fold that works for EVERYTHING
31. There is a recursion scheme for that! Lucky us!
32. SCRAP YOUR BOILERPLATE Ralf Lämmel and Simon Peyton Jones provided a type-safe way for generalising folds.
33. So, let’s try coming up with a universal fold.
34. I. Visiting every value in the universe ufold([]) -> []; ufold([X|Xs]) -> [ufold(X)|ufold(Xs)]; ufold({}) -> {}; ufold({A}) -> {ufold(A)}; ufold({A, B}) -> {ufold(A), ufold(B)};
35. II. Restricting the universe ufold(T) when is_tuple(T) -> map_tuple(fun ufold/1, T); ufold([X|Xs]) -> [ufold(X)|ufold(Xs)]; ufold(M) when is_map(M) -> map_values( fun ufold/1, M); ufold(X) -> X.
36. III. Transforming the values ufold(F,T) when is_tuple(T) -> map_tuple( fun(X) -> ufold(F, X) end, T); ufold(F, [X|Xs]) -> [ufold(F, X)|ufold(F, Xs)]; (...) ufold(F, X) -> F(X).
37. IV. Transforming ALL of the values ufold(F,T) when is_tuple(T) -> F(map_tuple( fun(X) -> ufold(F,X) end, T)); ufold(F, [X|Xs]) -> F([ ufold(F, X) | ufold(F, Xs)]); (...) ufold(F, X) -> F(X).
38. EXAMPLE 1: SALARY INCREASE Company = {company, [ {dep, “Research”, {e, {n, “Ralf”, “Amsterdam”}, {s, 800}}, [ {e, {n, “Joost”, “Amsterdam”}, {s, 100}} , {e, {n, “Marlow”, “Cambridge”}, {s, 200}} ] }, {dep, “Strategy”, {e, {n, “Blair”, “London”}, {s, 100000}}, []} ]}.
39. EXAMPLE 1: SALARY INCREASE inc({company, Xs}) -> lists:map(fun inc_dep/1, Xs). inc_dep({dep, N, M, Ps}) -> {dep, N, inc_emp(M), lists:map(fun inc_emp/1, Ps)}. inc_emp({e, N, {s, X}}) -> {e, N, {s, X * Rate}}.
40. EXAMPLE 1: SALARY INCREASE Rate = 2, ufold( fun({s, X}) -> {s, X * Rate}; (X) -> X end, company() ).
41. EXAMPLE 1: SALARY INCREASE Company = {company, [ {dep, “Research”, {e, {n, “Ralf”, “Amsterdam”}, {s, 1600}}, [ {e, {n, “Joost”, “Amsterdam”}, {s, 200}} , {e, {n, “Marlow”, “Cambridge”}, {s, 400}} ] }, {dep, “Strategy”, {e, {n, “Blair”, “London”}, {s, 200000}}, []} ]}.
42. EXAMPLE 2: ALGEBRAIC SIMPLIFICATION Program = {prog, [ ... {‘fun’, f, [x], [ {‘let’, y, {‘+’, x, {‘*’, {int, 3}, {int, 1000}}}, {‘*’, x, {int, 2}}}]}, ... ]}.
43. EXAMPLE 2: ALGEBRAIC SIMPLIFICATION ufold( fun({‘*’, {int, A}, {int, B}}) -> {int, A * B}; ({‘*’, X, {int, 2}}) when is_atom(X) -> {‘+’, X, X}; (X) -> X end, Program ).
44. EXAMPLE 2: ALGEBRAIC SIMPLIFICATION Program = {prog, [ ... {fun, f, [x], [ {let, y, {‘+’, x, {int, 3000}}} {‘+’, x, x}}]}, ... ]}.
45. SMALL VARIANT: QUERIES map_query(F, X) when is_tuple(X) -> map_tuple_to_list(F, X); map_query(F, [X | Xs]) -> [F(X), F(Xs)]; map_query(F, X) -> []. collect(Combine, Q, X) -> lists:foldl(Combine, Q(X), map_query(fun(X1) -> collect(Combine, Q, X1) end, X)).
46. EXAMPLE 3: ALL INTEGER NODES Program = {prog, [ ... {‘fun’, f, [x], [ {‘let’, y, {‘+’, x, {‘*’, {int, 3}, {int, 1000}}}, {‘*’, x, x}}]}, ... ]}.
47. EXAMPLE 3: ALL INTEGER NODES >> collect( fun(A, B) -> A ++ B end, fun({int, X}) -> [{int, X}]; (_) -> [] end, Program ). [{int, 3}, {int, 1000}]
48. Let’s do a short review!
49. Most data can be thought of as a tree. 1 CONS 2 CONS 3 CONS EMPTY
50. Fold abstracts the recursion in a tree-like structure. fold( fun(A, B) -> A + B end, fun() -> 0 end, {cons, 1, {cons, 2, empty}} ).
51. A universal fold is simply a fold over all possible types. ufold(F,T) when is_tuple(T) -> F(map_tuple( fun(X) -> ufold(F,X) end, T)); ufold(F, [X|Xs]) -> F([ ufold(F, X) | ufold(F, Xs)]); (...) ufold(F, X) -> F(X).
52. This forms a basis for other recursive operators, like querying. collect(K, Q, X) -> lists:foldl( K, Q(X), map_query( fun(X1) -> collect(K, Q, X1) end, X)).
53. It’s context-free; practical usage needs tagged data. ufold( fun (X) when is_integer(X) -> X + 1; (X) -> X end, [1, sets:from_list([1, 2])] ). => [2, {set, …}]
54. AND THEN THERE WERE LINKS... Functional Programming with Bananas, Lenses, ... Scrap Your Boilerplate: A Practical Design Pattern ... Generique: Generic Programming for Erlang
55. THAT’S ALL, FOLKS talks.robotlolita.me