Point-free or Die - Tacit Programming in Haskell and Beyond

I really liked Amar Shah's talk on point-free style as a tool for producing tacit code which communicates better because it is quiet and so easier to understand: https://www.youtube.com/watch?v=seVSlKazsNk

This slide deck is my effort to capture the content of the talk in a way that can aid its consumption, digestion and retention.

Download for better quality.

Transcript

1. So what is this thing, point-free? Point-free is a style of writing function definitions. A point-free expression, is a kind of function definition. But point free is kind of bigger than that. It’s a way of talking about transformations that emphasizes the space instead of the individual points that make up the space. So what’s this other thing: tacit? So tacit is just a synomym for quiet. And tacit code is quieter than noisy code, and here is an example of a point- free definition, down here at the bottom, and its pointful counterpart: So both of these definitions describe the same function. It’s the sum function and you could say the first one reads like this: to take the sum of a list of things, do a right fold over that list, starting at zero and using addition to combine items. Or you could just say sum is a right fold, using addition, starting at zero. https://www.youtube.com/watch?v=seVSlKazsNk @amar47shah Amar Shah point-free definition pointful definition
2. But why would you want to use the point-free definition? What’s the point of removing all these arguments from your functions? So that means that when you want to make your code communicate better, you can use tacitness, and you can use point-free style to make it quiet. And why would you want to do that? To One reason is that tacit code can keep you talking at the right level of abstraction, so that you are not constantly switching between low level and high level constructions. Here is another example. lengths can take a list of lists, and map the length function over that list of lists, or you could just say lengths is a map of length. So point-free definitions are a way that you can be more expressive with your code @amar47shah Amar Shah So here is one more example. Here is a function totalNumber, of a list of lists. To get he total number of the items in my sublists, I want to take the lengths of all my sublists and then I want to add all those together. Or I could just say the total number is the composition of sum and length. So I can use the composition operator, right here.
3. so, that little dot, it is not a point, it is composition and you are better off thinking of it as a pipe. And then you might say something like: If I have two functions, outside and inside, their composition works like this: Composition is a function that takes an argument, applies inside and then applies outside. I can do some Haskell magic here, I can use the $ sign operator here, instead of parentheses, it means the same thing essentially, and I can move the dollar sign to the end here, and replace it with composition. Of course, when I have a lambda abstraction that takes an argument and does nothing but apply that function to the argument, then I don’t even need the lambda abstraction. So you can read this as outside composed with inside. That’s probably the best way to read it if you want to remember that it is a composition. @amar47shah Amar Shah So when should you use a point-free definition, because you can use definitions that are point-free and you can use ones that aren’t? And here are my two rules, which are really just one rule: Use it when it is good and don’t use it when it is bad.
4. So this was the definition we came up with. totalNumber is the composition of sum and a map of length (I decided to ditch lengths). And this is the point free definition, but if you ask me, this is another useless function, because for one, it doesn’t communicate a lot more. totalNumber does not communicate a lot more than the composition of sum and map length, really if you think about it, I am mapping lengths and adding them up. That kind of gets across the idea. And another problem is that if we take the total number of a bunch of sublists, that’s like one way that we can summarize all the sublists, but it’s not the only way. We might decide that we want the total max, like we want the maximum value in each sublist and we want to add those together. Or we might want the number of items in each sublist that meets some criteria, and we get the total match by adding them all together at the end. So one thing I really like about FP is that it encourages you to think in a general way about problems, whereas my experience with OO code, and dynamically typed OO code is that it really encourages you to do the opposite. It encourages you to say YAGNI, so don’t bother thinking about the case that you haven’t started to use yet. FP on the other hand, because you are using pure functions, you are restricting side effects, you can use wholemeal programming, where you think in the broad, think about when the general constraint applies and then you can narrow it down to the specifics. @amar47shah Amar Shah So to see an example of that, let’s extract a little helper here. totalNumber is this sum composed with the map of length, but it is also just a way of aggregating these sublists, and we are aggregating here using length. We could aggregate here using another function. aggregate is here a local helper function and the definition here is the composition of the sum and the map of that metric, which I am calling f. You look at this and you think this is not a point-free definition, because I have an argument over here, but it is not really a pointful definition either, because what it returns is a function.
5. So we can see all of the arguments by using a proces called eta-abstraction. eta-abstraction looks like this So you saw eta-abstraction but the trick here is that that definition of aggregate that we saw, this one up here at the top, it is also not point-free. So we can make it point-free, we can remove arguments, using eta-reduction, which is the opposite of eta-abstraction. This looks complicated, but I am going to walk you through this. … So on line 7 we have half of the composition of sum, composed with map. That’s great. That makes a lot of sense, right? That’s easy to read, it’s easy to understand (yeah, right, <laughter>) @amar47shah Amar Shah So we have these two things, we have this not-point-free and not pointful definition, and we have this one, which is completely point free, and you are already probably thinking this is terrible, because I don’t even know how to say it. So let’s look at it a bit more closely. It’s a function that takes two functions, f and g and it composes them in a weird way, it combines them, not quite composes them, it composes this part with that, and that thing is half of a composition.
6. So this ‘thing’, we can eta-abstract this ‘thing’, so that it looks a little bit better, and you don’t have to go through all these steps, just take my word for it. You can see the x filtering in here and then I realise I can add a y too. And I end up with this down here at the bottom. And what I have is a pipe. I have something, I have a function here that takes two arguments, one argument after the other, and then applies another function to the result. @amar47shah Amar Shah So this thing is a combinator. There is nothing that it does that isn’t here. The only stuff it does that isn’t here is part of the system, the application of a function. Here are some examples of combinators. We have the one we are currently investigating (the second one), this one is composition (the first one), which you have already seen, and these things, which you have also seen, are not combinators, because they make reference to items that are not here, in the argument list.
7. So, we have our combinator, and it turns out that this combinator has a name. It is the “blackbird”. Where does that name come from? This is Raymond Smullyan. He was a professor of philosophy and logic, and he wrote a logic puzzle book: “To Mock a Mocking Bird – and other logic puzzles - Including an Amazing Adventure in Combinatory logic” @amar47shah Amar Shah The book takes you on a tour of the forest, you get to meet a lot of birds, and the birds say different things when you tell them the names of other birds. This bird is a blackbird. Raymond Smullyan
8. So here is our blackbird – I have decided to go ahead and give it a name. I am going to give it the name dot dot dot: ‘…’, and it looks like this: f dot dot dot g, f blackbird of g, it means that. And I can eta-reduce again, and this is just the same distance we already travelled, and we can get back from this expression to this expression It’s a pipe with two inputs. You know what, it turns out that since this isn’t a point-free expression, I can go ahead and write a point-free definition of blackbird, if you want to see it. So this is crazy: the blackbird is the composition of composition and composition @amar47shah Amar Shah
9. So we have a new way to write our function aggregate: aggregate is the blackbird of sum and map. I am actually serious here. I don’t think this is a joke. I think this is great (hear me out), because just like composition is not a thing that we think about, it is not like when I try to understand composition I go back to the Latin words and I say, oh it is the com of the posing, instead I am just like, composition is a pattern that I am familiar with, I see it and I become familiar with it. Blackbird is a pattern that you can also become familiar with. You can remember, to start off, that it is a pipe that takes two inputs, applies a function, and then applies another function. But after a while, you just remember that it is the blackbird. f … g = \x y -> f(g x y ) sum … map = \f xs -> sum(map f xs) ‡ So what if we wanted to work with n-dimensional distance?…say vectors in space, and how close these vectors might be to each other? Well then we might use coordinates in a list, instead of just two of them. @amar47shah Amar Shah euclidean xs = (sqrt.(aggregate sqr)) xs = (sqrt.((sum … map)sqr)) xs = sqrt(sum(map sqr xs)) manhattan xs = aggregate abs xs = (sum … map) abs xs = sum(map abs xs) ‡ ‡ (informal pseudo-code added to aid comprehension - Philip Schwarz) I am going to give you an example.
10. But there is something here, between these two definitions, that is trying to get out, there is something they have in common, that maybe we can extract and maybe we can understand distance a little bit better in general. So let’s try that, what is happening here? Here we aggregate absolute value and, well, it doesn’t fit the pattern, there is something missing here, we don’t do anything here. Here we aggregate sqr and we apply sqrt to the the result. But it turns out that not doing anything is also a function! So now we have these in the same form, which means we can write a definition that we can use to derive these. @amar47shah Amar Shah And here it is: distance outer inner is the outer composed with the aggregate of inner. So these point-free definitions here are special cases of the generalised distance.
11. But you notice that this definition is not point-free. Well, we can look at a couple of point-free definitions, or we can look at one in particular. You’ll eventually get here with eta-reduction (laughter). But of course if you know about the blackbird, then you’ll get here, which is not as ugly, but still, it is not really good, honestly, it is not good, because what does half a blackbird mean? Half a blackbird of aggregate? I don’t know. This definition isn’t bad, but it only mentions the o and not the i and you kind of think of these functions, when you think about distances, like they are on the same plane of importance. So I would stick with this second one here. And I wouldn’t go to the pointful definition, because I don’t want to have to talk about the coordinates if I am talking about the distance. @amar47shah Amar Shah I have been showing you examples in Haskell, but point-free isn’t really about Haskell, it is just a concept. It is easy to do in Haskell and F#, and Ocaml, you can jump through some hoops and do it in Clojure and Elixir, I don’t know any of those languages, to be honest, so I am going to show you one that I do know, Ruby ‡ So you should learn to do eta-abstraction and eta-reduction by hand. You can find tools to do it for you, because it is completely mechanical, so you can let the computer do it, but the result that you get might not be useful. ‡ (I gloss over this part of the presentation, in the next slide - Philip Schwarz)
12. compared with Haskell? @amar47shah Amar Shah In Haskell, functions are automatically curried
13. @amar47shah Amar Shah I discovered point-free notation just a few months after I kind of got into Haskell, and I got really excited by it, and I kind of went overboard with it, and I was doing some exercises involving the game of Mastermind, so if you want a little refresher, Mastermind looks like this. So here is my definition for exactMatches. Don’t kind of parse this in your head, just notice right away that this is a point-free definition. We just need a pairwise comparison – is the first pair the same? True. The next one? False. Next one? False, Next one? False. Then we add up the True values, and we get 1. And notice also, that I have got this little pattern here, which you might remember as the motivation behind naming the blackbird. Let’s go ahead and fix that, so now this is somewhat less noisy to look at, sort of easier to see, and we have a blackbird of this pipe and zip with equals.
14. So here is the comparison of those two functions. It is my opinion that the one at the top communicates better, and maybe that’s something that I need to sell you on a little bit better, but, a point-free definition here, lets you stay at the level of abstraction where you are talking about transformations on lists. I have one transformation that brings two lists together, I have another transformation that filters out the ones that have the values that I like, I have a third transformation that counts up the values. So basically I have a pipe that is in three parts, it takes two lists of pegs, zips them together with the equals comparison, filters the ones that pass the comparison, where the value is true, and then takes the length of the result. f … g = \x y -> f(g x y ) (filter id) … (zipWith ==) = \ps qs -> (filter id)((zipWith ==) ps qs) exactMatches ps qs = (length . ((filter id) … (zipWith ==))) ps qs = length((filter id)(zipWith == ps qs) ‡ ‡ (informal pseudo-code added to aid comprehension - Philip Schwarz) @amar47shah Amar Shah So let’s go look at how I deal with colour matches. Same scenario as before. But now we need to ignore position. So we can run both of these collections through a function that changes they way they are structured. so I got two matches plus one match
15. So here is some code that does that. So this was the code that I attempted to pass off as code that is easy to understand, that communicates what it intends to do. So that didn’t work, so let’s go back to a fully pointful definition. To get the colour matches of two lists of pegs, what I want to do is count the colours in each of them, I want to zip them together with the minimum of the counts, and I want to add up those counts together. But if you ask me, this really irks me, that I have to say countColours twice here. I have this transformation that kind of looks like this: I have these two lists, that come in together like this, and I have to do something here to put them together, and I have to do something after I have done that, but then before I put them together, I have to run this same transformation on both of them. There is a way around this that is much more elegant. In fact, I have this kind of problem involving combining functions, I wonder, if there might be a combinator? And there is. It’s called the psi combinator, and it actually lives in a Haskell, in a module called Data.Function and it is called on’ And here is the type of on. This looks a little scary, if you are not used to it. So on takes a binary operator that takes two bs and makes them into a c, it also takes a function that takes an a and transforms it into a b, and it will give you back an operator that will take two as and give you a c as a result. So I can use it here, I can take the ‘zipWith min’ on countColours. And here I am using an infix notation instead of a prefix notation, the backticks mean I can put ’on’ in the middle, so this actually reads well. So I countColours with both, zip them together with min, and I get a function that looks like this: two lists of pegs, and that gives me back a list of numbers. @amar47shah Amar Shah It is objectively awful. And if you went to any presentation today where someone told you that you can prove things with code, somebody please prove that this is awful.
16. so let’s go ahead and try it out so now using on I can do something like this and of course, I don’t need to use the $ sign, I can use parentheses instead but if I am looking at this, there is a pattern here that we have seen a few times so I have a blackbird! So here is the colourMatch definition that I think communicates very well. It wants to ‘zipWith min’, on countColours, and we want to take that result, and sum it. If I want to get rid of these I wonder how we’d do it, and it turns out, it is not too hard. What I have here is a pipe of a function taking two inputs and then using an outer function. And here is what the two functions look like Now here is the pointful function, the function that irked me, I guess, and here is the version that I came up with, just by using eta-reduction, with what I knew, and here is what we could do today. So when someone says to you, don’t bother with point-free, you are just going to end with obfuscated code, it’s not worth it, remember that there’s more than one way to do point-free, more than way to arrive at a point-free definition. @amar47shah Amar Shah
17. But what did we do differently? What was different? Why did we come up with something that was better, cleaner, more elegant? The difference is that we knew about ‘on’ and we knew about this way (the blackbird) of composing things in a pipe that take two inputs. So some advice: if you want to do point-free style, learn eta-reduction, because you want to be able to find the intermediate step that works best for your communication. And learn combinators, so that you can use them. So when you see a pattern that comes up again and a gain, you’ll be familiar with it, because this is what we do, when we learn to program, we learn to see patterns that come up again and again and we learn to apply them. So this is the guide to point-free, point-free without dying So point-free and die?, Point-free or die? The choice is yours. @amar47shah Amar Shah
18. Point-free style has a really long pedigree, has a long history, and it comes from conversations in computer science about how we can make programming, how we can make computing more composable, how we can actually build abstractions that are concise and still don’t leak, and that’s something that we are still chasing today, and I know a lot of people will look at syntax, and point free is all about questions of syntax, a lot of people will look at syntax and say syntax is not important, but I think nothing could be further from the truth. Syntax is where we communicate. And how we choose to communicate will make programming what it is, for days and years to come.. @amar47shah Amar Shah
19. http://www.angelfire.com/tx4/cus/combinator/birds.html Combinator Birds