pure fun in python yeah Christopher Armstrong https://github.com/radix/effect @radix - Introduction - Twisted, if you know it - Effect (talk about later) - Rackspace, Auto Scale, we do FP! - Go! 

i am not going to say it monad I will not say the word on this slide at any point during this talk. 

hundreds of years ago http://yellow5.com/pokey/archive/index264.html — @stevehavelka heard from someone that every great talk starts with a lengthy section on history. - Pascal, the first programming language, had two features: functions and procedures. - Functions return, procedures had side-effects. - we made a distinction then, most modern languages don’t. 

“pure” Function, n. (plural functions) A relation between a set of inputs and a set of permissible outputs with the property that each input is related to exactly one output. -Wikipedia - title of this talk is “PURELY functional programming” - shouldn’t have to say that. - a distinction was lost: “functions” changed meaning. - - I’ve also heard that every great talk has definition slides. - - No room for side-effects. Can’t delete money from your bank account. - Most don’t know/care when they write funcs or procedures. So, what is pure FP? just programming with functions. :-) 

not for fun >:( - not here to tell you this for fun - not just encouraging you to expand your mind - people just go back to familiarity - YOLO state manipulation and OO design are a big source of problems - pure FP seriously improves our code 

j/k it’s super fun :D - ok, ok, it’s actually really fun to do functional programming! - check it out! - learn some stuff! 

definition #2 YOLO, v, t. Modifying state or performing IO in a function. For the rest of this talk, I’ll refer to side-effects as “YOLO”. 

why 1. reason 2. express 3. test 4. perform Let me tell you about why FP is good for you. - Reason about code in a way impossible with YOLO. - express code concisely/meaningfully. abstract mechanisms for composing funcs. - WAY easier to test. Call func with args, check result. - Opportunity for speed for PFP, parallelism 

1. reason "See, I told you they'd listen to Reason.” -Fisheye (Snow Crash reference) - substantial effect on understanding of code - greater confidence - “equational reasoning”, function call = return value. - for snow crash reference - an example… 

reasoning f(a(‘foo’), b(c(‘bar’))) - code - If these functions YOLO, we need to carefully search for side-effects. - If we know they’re pure, it’s much easier. Let me demonstrate. 

reasoning >>> a(‘foo’)
 1
 >>> c(‘bar’)
 2
 >>> b(2)
 3
 >>> f(1, 3)
 “Everything is awesome!” - This is what it’s like to experiment and trace execution. - It’s just like when you learned Python. Try stuff out. - May seem trite. Surely, anyone can do this kind of stuff with their Python code? 

sad REPL syndrome foo.bar(3)? Foo(dbconn, auth) INSERT INTO txn_log (event, owner, value mock.return_value = MockAuthenticationResponse() def foo(val): … print stuff return stuff Q: ever worked on a big project where it’s hard use the REPL? “understand foo.bar(3)”. Easy? but class needs DB connobj, authobj -> HTTP req, and method YOLOs txn logs into DB. Have mocks in tests, -> 5 minutes to find & put together End up adding prints to func, try reproduce behavior in testing env. Happens to me! All the time! It’s a tragedy! slows dev cycle. 

happy REPL pure functions better factored dependencies :-) - If most code PF, you CAN use the REPL. - Functional code naturally minimizes deps between comps - end up w/ wonderfully factored little funcs, easy to evaluate in REPL. 

2. express result = map(comp(f, g), inputs) not result = []
 for input in inputs:
 x = f(g(input))
 result.append(x) - FP lets us specify things in a way that exposes *meaning* more clearly - lots of FP advocates say that FP style is “say what you want, not how to get it” - it’s rhetoric, but has a grain of truth - more on this later when I talk about “style” 

3. test call a function. pass it some arguments. check its return value. - Call func w/ args, check return. That’s it. - Not how most Python code is tested these days. - Mocking has become absolute *nightmare*. mocks -> mocks -> mocks - need track complex YOLOs - Tests too coupled to implementation. very hard to change your code. - care about this a lot. Really into unit testing. I like 100% test coverage. - I still mock (or parameterize deps), but the mocks are much more tasteful in FP 

4. perform PyPy. STM. (go fast) - PyPy. Obviously. - Watch PyPy’s STM - STM optimistically runs your code concurrently - PFP = trivially parallel. - STM needs more work, but it’s the future. So much that it hurts. And so is FP! - Basically, STM and PFP will kill the GIL. 

purity FP in the large - Let’s talk about HOW we do pure functional programming. - Most people focus on stylistic parts of FP - FP in the large 

don’t change anything • list.append(“foo”) • self.foo = bar • global • x = 1; x = 2 purity - don’t change anything - seems crazy? like many paradigms, not so hard once you get used to it 

lies def foo(coolstuff):
 coolstuff = list(coolstuff.items())
 coolstuff.append(('extrathing', 'value'))
 return coolstuff purity - but those are lies - only thing that matters is you don’t *expose* side-effects. 

python def assoc(d, k, v):
 new = d.copy()
 new[k] = v
 return new purity - local YOLO often necessary when doing pure FP in Python. - This function replaces a value in a dictionary by copying and mutating. - Pure FP, but local mutation. Easiest way to do it in this case. 

functional data purity/data toolz warehouse.python.org/project/toolz - Python has poor built-in data structures for FP. - Get stuff done by explicit copy/modify, but it’s inefficient and error-prone. - (e.g. nested mutable objects) - toolz is a nice lib that supplies useful functions. 

toolz purity/data >>> from toolz.dicttoolz import assoc
 >>> assoc({'a': 1}, 'b', 2)
 {'a': 1, 'b': 2} - here’s the assoc function from earlier. pretty much does what it looks like. - this is nice, for small data…. 

big data (structures) • Zone.actors : [Actor] • Actor.health : int • (Actor.mana, Actor.inventory) • Actor.attack(target) -> reduce target’s health purity/data - e.g.: game simulation with big world. - thousands of actors, with health/mana/inventory nested inside - inventory may contain hundreds of items too - Copying the whole zone is insanity! How do we do this efficiently without mutation? 

persistent data purity/data pyrsistent warehouse.python.org/project/pyrsistent/ - wonderful implementation of “persistent” data structures - (not on-disk; it means immutable) - incredibly, surprisingly efficient - shared structure! 

linked list a = (1, (2, ())) b = (0, a) purity/data - before I explain pyrsistent, an example of the idea of shared structure - pushing/popping a linked list is very efficient. Structure is shared. 

slow • push/pop: O(1) • anything else: O(n) purity/data - linked lists are limited. - push/pop is fast, but insert/index is slow. - Much more clever data structures exist now, like… 

(H)AMT purity/data (Hash) Array Mapped Tries - (HASH) Array Mapped Tries - becoming very common - used in the Clojure core data types - in the pyrsistent library for Python. 

(H)AMT purity/data - array of integers 0-8 - array is chunked up into the leaves of the trie. - Indexing this structure is very essentially O(1), unlike linked lists, which require traversal 

(H)AMT purity/data - let’s replace ‘5’ with “beef” - mutation is also essentially O(1) (O(log32 N))! - only the nodes on the way to that chunk need to be allocated - all shared structure just gets referenced in the new vector. - brown is old, blue is new. 

zone -> zone purity/data from pyrsistent import m, v
 zone = m(actors=v(m(health=100, mana=100,
 inventory=v())))
 updated_zone = zone.set_in(
 (‘actors’, 0, ‘health’), 90) - Example of pyrsistent. Let’s see how to change someone’s health. - What do you do with a new world, though? - Well, honestly, to be practical you could just overwrite the old world. - Or you can treat event-handlers as functions of world -> world and continually run them in a purely functional way. - Magic word here would be FRP, but I’m not going to talk too much about that. 

IO purity/effects - You’ve been biting your tongue. - “This is just a bunch of faffing about. Can’t do _real_ programming with pure FP!” - Useful programs have to communicate with the outside world. - You’re right. But maybe we can still do something useful 

functional core,
 imperative shell purity/effects — Gary Bernhardt - Start simple. - the majority of your logic should be written as pure functions. - wrap the imperative logic that talks to DB or makes HTTP requests 

typical IO def foo():
 name = input("Name?")
 print(“Oh, %s is a nice name" % (name,)) purity/effects - Here’s a typical IO-using function. - It looks tiny, right? - Let’s reduce it to its essence. Which parts can be factored out to be pure? 

refactoring def compliment(name):
 return "Oh, %s is a nice name" % (name,) def foo():
 print(compliment(input(“Name?”))) purity/effects - extract the part that takes the result of first IO and does the pure calculation based on it. - This is a trivial example of functional core/imperative shell, but you can extrapolate this idea to a lot of code. - In practice, a lot of your functions are already pure, there are a lot that aren’t, and could be. - Thinking in these terms will help you factor your apps much better. - I encourage you to go back to your code bases, find the side effects, and try to factor them out. 

down the rabbit hole def foo():
 print(compliment(input(“Name?”))) - But can we go deeper!???? - Here are our side-effecting functions. - Could we represent the ideas in this code in a way that maintains purity? 

down the rabbit hole def foo():
 eff = Input("Name?") purity/effects @do
 def foo():
 name = yield Input(“Name?”)
 return Print(compliment(n)) or return eff.on(lambda n: Print(compliment(n))) - Here’s an idea. - represent our side-effects as objects describing *intent* to perform an effect. - input became an Input object, print became a Print object. - `on` attaches a callback to deal with the eventual result of that IO. - This is a liiiiittle but of a tedious syntax, so… - Maybe we can take advantage of generator syntax to make it look nicer. - This is just like asyncio’s coroutines. - twisted also has inlineCallbacks 

results purity/effects @do
 def foo():
 name = yield Input(“Name?”)
 return Print(compliment(n)) def main():
 perform(foo()) - So what’s the point? - When we call it, we get back an introspectable object. - Test it. - In the end, actually perform the side-effects with a “perform” function. 

wink, nudge • Effect, warehouse.python.org/project/effect purity/effects - With a few minor syntactic differences, this is exactly how Effect works. - It’s rather extreme compared to the typical Python program - but similar to asyncio, Twisted, etc. - Not just for IO-based side-effects. also useful for RNG, mutable data writes, etc. 

style In the beginning was the function.
 All things issue from it; all things return to it. 
 —Tao Te FP - That’s all I’m going to say about purity and side-effects. - Could say a lot more, but TIME. - Now I want to talk about the *style* of functional programming. - Lisp, Erlang, OCaml, are all about stylistic FP. They don’t isolate side-effects, but they do love everything-as-functions. 

style • higher-order functions • use functions for control flow • don’t reassign • be an abstraction astronaut style - “higher-order” DEFINE - Use functions for control flow. Ex, recursion == looping. - HO functions are used for control flow and abstraction - Don’t reassign variables - already talked about this - combine above = abstraction astronaut. 

higher-order result = map(comp(f, g), inputs) not result = []
 for input in inputs:
 x = f(g(input))
 result.append(x) style - I showed this earlier when talking about expressiveness. - Seems like a win! 2 HO funcs: map&compose. - map: apply a func to all elements in a list. - compose: double-whammy. accepts and returns functions. 

compose h = comp(f, g) same as def h(*args, **kwargs):
 return f(g(*args, **kwargs)) style - FPers love these tiny abstractions, use them to great effect. - not just shorter lexically: avoids mention “arg” to funcs,*conceptually* simpler (“number of referenced names”). 

compose def comp(f, g):
 return lambda *a, **kw: f(g(*a, **kw)) or from toolz import comp style We can define compose, or use toolz. 

filter/reduce >>> filter(lambda x: x % 2 == 0,
 ... [0,1,2])
 [0, 2] >>> reduce(operator.add, [1,2,3])
 6 style more HO funcs 

list comprehensions >>> [x + 1 for x in [0,1,2,3,4]
 ... if x % 2 == 0]
 [1, 3, 5] >>> map(lambda x: x + 1,
 ... filter(lambda x: x % 2 == 0,
 … [0,1,2,3,4]))
 [1, 3, 5] style - syntax sugar for map/filter (not reduce) - often nicer > map/filter+lambdas, 

map/filter >>> map(add1, filter(is_even, [0,1,2]))
 [1, 3] style map/filter shine w/ named funcs 

recursion def factorial(n):
 if n == 0:
 return 1
 else:
 return n * factorial(n-1) style - Not a lot of recursion in Python. Uses up stack space, no TCE. 

abstract it away def factorial(n):
 if n == 0:
 return 1
 else:
 return n * factorial(n-1) vs def factorial(n):
 reduce(operator.mul, range(1, n)) style But that implementation is actually pretty gross conceptually as well as performance-wise. Reduce is powerful. Reduce usually recursive in FPL, Py = iterative. large inputs = no p. 

astronaut add1all (first:rest) = first + 1 : add1all rest
 add1all [] = [] or, def add1all(inputs):
 if not inputs: return []
 return [inputs[0] + 1] + add1all(inputs[1:]) style - Functional programmers love abstractions. - Many patterns of recursion have been identified. - In ancient history, people often wrote things like this, and realized a pattern 

astronaut map f (head:rest) = f head : map f rest
 map f [] = [] or def map(f, inputs):
 if not inputs: return []
 return [f(inputs[0])] + map(f, inputs[1:]) style - Parameterize the per-element calculation and create `map`. - map, reduce, filter are all simple recursion recipes, extracted into HO funcs. - though recursion is an important part of FP, people prefer not to do it explicitly - Using the recipes makes it easier and quicker to understand what the code is doing - Even though Python hates recursion, these abstractions are implemented iteratively so we can use the same style. 

compose functional purity + functional style - Coming up on the end! - functional purity: idea of not exposing side-effects - functional style: use functions for everything! - doing pure functional programming leads to - easier reading - easier adding features/fixing bugs - easier refactoring - easier testing - Go forth and invoke a function! 

questions/refs • Pyrsistent, warehouse.python.org/project/pyrsistent • Toolz, warehouse.python.org/project/toolz • Effect, warehouse.python.org/project/effect • HAMT, http://hypirion.com/musings/understanding- persistent-vector-pt-1 • Pokey the Penguin, http://yellow5.com/pokey/