Monads, In My Python? (PyConAU 2015)

The associated talk can be found here:
https://www.youtube.com/watch?v=WNwV3wR4JjA

Transcript

1. MONADS, IN MY PYTHON? PYCON AU 2015 @chewxy on Twitter
2. What Are Monads Monads are just monoids in the category of endofunctors. @chewxy on Twitter
3. Example Monad with  open('blah.txt',  r)  as  f:    lines  =  doSomething(f)   @chewxy on Twitter
4. The End Thank you. See you next year. @chewxy on Twitter
5. Real Life Code def  anomalyDetection(data,  obsPerPeriod):        if  not  obsPerPeriod:      raise  Exception("errorMessageHere")        #  other  related  checks...      decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)        lessSeasonal  =  data  –  decomposed.seasonal        lessMedian  =  lessSeasonal  –  np.median(data)        #  and  so  on  and  so  forth...   @chewxy on Twitter
6. Real Life Code def  anomalyDetection(data,  obsPerPeriod):        if  not  obsPerPeriod:      raise  Exception("errorMessageHere")        #  other  related  checks...      decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)        lessSeasonal  =  data  –  decomposed.seasonal        lessMedian  =  lessSeasonal  –  np.median(data)        #  and  so  on  and  so  forth...   @chewxy on Twitter
7. Composition x  =  foo()   y  =  bar(x)   z  =  baz(y)       @chewxy on Twitter
8. Composition x  =  foo()   y  =  bar(x)   z  =  baz(y)   Equivalent to z  =  baz(bar(foo()))   @chewxy on Twitter
9. Composition a  =  1  +  2  +  3   b  =  1  /  (3  –  sqrt(9))       @chewxy on Twitter
10. Composition a  =  1  +  2  +  3   b  =  1  /  (3  –  sqrt(9))   Equivalent to a  =  add(add(1,  2),  3)   b  =  div(1,  sub(3,  sqrt(9)))   @chewxy on Twitter
11. Then the universe implodes @chewxy on Twitter
12. Functions We know what functions are… @chewxy on Twitter
13. Functions We know what functions are… def  foo(x):    #do  something   @chewxy on Twitter
14. Procedures Back in the old days before C won, Pascal differentiated procedures and functions. SQL does the same. @chewxy on Twitter
15. Procedures Back in the old days before C won, Pascal differentiated procedures and functions. SQL does the same. Functions returned values. Procedures didn't. @chewxy on Twitter
16. Pure Functions In mathematics, a function[1] is a relation between a set of inputs and a set of permissible outputs with the property that each input is related to exactly one output. [1] – "Function" Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc. 2015- ish @chewxy on Twitter
17. Pure Functions 1 2 3 4 2 3 4 5 fn = (+1) def fn(x: int) -> int: return x + 1 @chewxy on Twitter
18. Programmers' Definition of Functions   May return a value (or not).   Return values are not mapped 1:1 with input values   Side effects!   Do this, then do that (aka procedures). @chewxy on Twitter
19. Functions and Pure Functions   Both take parameters*   Both return values*   Both are composable @chewxy on Twitter
20. Categories @chewxy on Twitter
21. Categories A collection of:   Things (values)   Have an identity function where I(x)  =  x     Arrows (functions)   Have domain (input) and range (output)   Can be composed   Obey some form of associativeness laws TL;DR – Pretty pictures to help us think about functions @chewxy on Twitter
22. Categories @chewxy on Twitter X Y Z f g gŸf
23. Categories* 1 2 3 4 2 3 4 5 f = (+1) 3 4 5 6 g = (+1) gŸf @chewxy on Twitter *The objects are expanded to show the values for visualization purposes
24. Big Deal…   Categories are cool… so what? @chewxy on Twitter
25. Big Deal…   Categories are cool… so what?   Category theory gives us the intuitions needed to think about issues that plague developers @chewxy on Twitter
26. Awkward Squad*   IO   Concurrency   Exception   FFI * With apologies to Simon Peyton Jones @chewxy on Twitter
27. Awkward Squad   IO   Permanent side effects   Concurrency   Non-determinism happens   Exception   Errors – What do we do with malformed/unexpected inputs?   FFI   Dependence on outside information Even the simple print can fail! @chewxy on Twitter
28. Print Fail! while  True:    print("Hello  World")   @chewxy on Twitter
29. Print Fail! while  True:    print("Hello  World")     >  python3  test.py  |  head  -­‐n  1   Hello  World   Traceback  (most  recent  call  last):      File  "test.py",  line  2,  in  <module>          print("Hello  World")   IOError:  [Errno  32]  Broken  pipe   @chewxy on Twitter
30. Dealing with Failure @chewxy on Twitter
31. Example def  div(num,  denom):    return  num/denom     def  sqrt(x):    return  math.sqrt(x)   @chewxy on Twitter
32. Domains (input)   The domain for div to function correctly:   For denom: Any numeric type (int, float, double … )   For num: Any numeric type   The domain for sqrt to function correctly:   For x: Any numeric type @chewxy on Twitter
33. Ranges (output)   Range of div:   Float   Range of sqrt:   Float @chewxy on Twitter
34. Example in Categories 2 4 5 0 50 25 20 ???? f = div(100, denom) 7.07 5.00 4.47 ????? g = sqrt(x) gŸf denom x @chewxy on Twitter
35. Extend the Range def  cleanDiv(num,  denom):    try:      return  num/denom    except  ZeroDivisionError:      return  None     def  cleanSqrt(x):    if  x  <  0:      return  None    return  math.sqrt(x)   @chewxy on Twitter
36. New Ranges (output)   Range of cleanDiv:   Float NoneType     Range of cleanSqrt:   Float NoneType   @chewxy on Twitter
37. With Extended Ranges 2 4 5 0 50 25 20 7.07 5.00 4.47 TypeError gŸf denom x None @chewxy on Twitter
38. Extend the Domain and Ranges def  cleanDiv(num,  denom):    try:      return  num/denom    except  ZeroDivisionError:      return  None     def  cleanSqrt(x):    #  special  check  for  x  is  None,  because  sqrt(0)  ==  0    if  x  <  0  or  x  is  None:      return  None    return  math.sqrt(x)   @chewxy on Twitter
39. Extended Domains + Ranges 2 4 5 0 50 25 20 f = cleanDiv(100, denom) 7.07 5.00 4.47 g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None
40. Tedious   To compose functions, you basically need to create new functions that have different ranges and domains   Or do it like what we do normally @chewxy on Twitter
41. How To Deal With It v  =  foo()  #  may  return  None   if  v:    v  =  bar(v)   else:    v  =  None         try:    v  =  bar(v)   except  Exception  as  pokémon:    v  =  None   @chewxy on Twitter
42. Tedious   To compose functions, you basically need to create new functions that have different ranges and domains   Or do it like what we do normally   Is there a way to generalize? @chewxy on Twitter
43. Map from This Category Numeric Float f = div(100, denom) Float g = sqrt(x) gŸf denom x @chewxy on Twitter
44. To This Category Numeric Float f = cleanDiv(100, denom) Float g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None
45. Recall Monads are just monoids in the category of endofunctors @chewxy on Twitter
46. Functor In mathematics, a functor[0] is a type of mapping between categories … [0] – More wikipedia citation?? This essay will fail if it's a uni essay @chewxy on Twitter
47. Implementing Monads Reality called. It wants its Python back @chewxy on Twitter
48. Interfaces Recap >>>  v  =  Value(1,2,3)   >>>  for  i  in  v:    print(i)   >>>  v2  =  ValueNoIter(1,2,3)   >>>  for  i  in  v2:    print(i)   Traceback  (most  recent  call  last):      File  "<stdin>",  line  1,  in  <module>   TypeError:  'ValueNoIter'  object  is  not  iterable     @chewxy on Twitter
49. Interfaces Recap class  Value(object):    def  __init__(self,  *args):      self.value  =  args    def  __iter__(self):      for  v  in  self.value:        yield  v   @chewxy on Twitter
50. Interfaces*   Monads are anything with bind() and unit().   bind(): applies a function on a monadic value.   unit(): makes something a monadic value   There are different types of monads, each with different functionalities to their bind() and unit().   Monad Laws apply *Haskell typeclasses are like compile time type safe interfaces @chewxy on Twitter
51. Monadic Values Numeric Float f = cleanDiv(100, denom) Float g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None
52. Monadic Values Numeric Float f = cleanDiv(100, denom) Float g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None
53. Maybe Monad   Deals with failure conditions   bind(): apply a function if the input is not None, else return None     unit(): return the value* * For clarity's sake, we're going to wrap the value in a Value class, which is strictly not necessary @chewxy on Twitter
54. Example Time def  bind(f):      def  inner(*args,  **kwargs):      a  =  [unbox(i)  for  i  in  args]      kw  =  {k:unbox(v)  for  k,  v  in  kwargs.items()}      if  None  in  a  or  None  in  kw:        return  None      return  f(*a,  **kw)    return  inner     def  unit(f):    def  inner(*args,  **kwargs):      return  Value(f(*args,  **kwargs))    return  inner     def  unbox(v):    try:      return  v.value    except  AttributeError:      return  v     @chewxy on Twitter
55. Example Time class  Value(object):    __slots__  =  ['value']    def  __init__(self,  v=None):      if  type(v)  is  Value:        self.value  =  v.value      else:        self.value  =  v    def  __repr__(self):      if  self.value  is  not  None:        return  "Just({})".format(self.value)      else:        return  "Nothing"   @chewxy on Twitter
56. Example Time @unit   @bind   def  div(num,  denom):    if  denom  ==  0:      return  None    return  num/denom     @unit   @bind   def  sqrt(x):    if  x  <  0:      return  None    return  math.sqrt(x)   @chewxy on Twitter
57. Sanity Check >>>  sqrt(div(100,  4))   Just(5.0)   >>>  sqrt(div(100,  0))   Nothing   >>>  div(sqrt(100),  2)   Just(5.0)   >>>  div(sqrt(100),  0)   Nothing   >>>  div(div(div(100,  0),2),10)   Nothing   @chewxy on Twitter
58. List Monad   Deals with collections   bind(): applies a function on the elements of the list   unit(): puts an element into a list @chewxy on Twitter
59. Example Time def  bind(f,  x):    return  [j  for  i  in  x  for  j  in  f(i)]     def  unit(x):    return  [x]   @chewxy on Twitter
60. Example Time def  sqrt(x):    if  x  <  0:      return  []    else:      s  =  math.sqrt(x)      return  [s,  -­‐s]     @chewxy on Twitter
61. Example Time >>>  bind(sqrt,  unit(100))   [10.0,  -­‐10.0]     >>>  bind(sqrt,  bind(sqrt,  [100]))   [3.1622776601683795,  -­‐3.1622776601683795]     >>>  bind(sqrt,  [4,  9,  16,  25,  36])   [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,   -­‐5.0,  6.0,  -­‐6.0]   @chewxy on Twitter
62. Seems Familiar… >>>  bind(sqrt,  [4,  9,  16,  25,  36])   [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,   -­‐5.0,  6.0,  -­‐6.0]     >>>  [sqrt(x)  for  x  in  [4,  9,  16,  25,  36]]   >>>  list(map(sqrt,  [4,  9,  16,  25,  36]))   @chewxy on Twitter
63. The General Idea Monads help us compose functions that have different return types. Solution: 1.  Wrap the value with their contexts in something called a monadic value (which can be anything – you define unit()) 2.  Define a function bind() that defines out how to compose the functions that applies within that context @chewxy on Twitter
64. Opening Files def  unit(n):    return  open(n,  'r').__enter__()      #  type:    'file'     def  bind(f):    def  inner(x):      retVal  =  None      try:        retVal  =  f(x)      except  Exception  as  pokemon:              pass  #  do  something  with  pokemon        x.__exit__()      return  retVal    return  inner       @bind   def  doStuff(f):    return  f.readlines()       lines  =  doStuff(unit(fileName))     @chewxy on Twitter
65. Sounds Familiar? with  open('blah.txt',  'r')  as  f:    lines  =  doStuff(f)   @chewxy on Twitter
66. Other (Possible) Monads Twisted's Deferred (JS promises 10 years before Promises!) djangoORM's .query() and .filter()  *   Scikits-Learn's Pipeline  *   * Yet to check whether it's actually a monad @chewxy on Twitter
67. Why Does This Matter? @chewxy on Twitter
68. Why Does This Matter?   We compose functions all the time.   Programs should be easy for humans to reason about.   We haphazardly recreate solutions to solve certain classes of problems.   Understanding the underlying pattern to most of the issues programmers face makes us better (I hope?)   BYO paradigm @chewxy on Twitter
69. The End (for Realz) See you next year! @chewxy on Twitter
70. Appendix: Monad Laws #  left  bind   assert  bind(unit(v),  f)  ==  f(v)     #  right  bind   assert  bind(unit(v),  unit)  ==  unit(v)     #  associativeness   assert  bind(bind(unit(v),  f),  g)  ==  g(f(v))   @chewxy on Twitter
71. Appendix: Monad Transformers Build your own monad transformer @chewxy on Twitter
72. Appendix: Arrow Associativity @chewxy on Twitter hŸgŸf == (hŸg)Ÿf == hŸ(gŸf)