Monads, In My Python? (PyConAU 2015)

The associated talk can be found here:
https://www.youtube.com/watch?v=WNwV3wR4JjA

Transcript

1. MONADS, IN MY
   PYTHON?
   PYCON AU 2015
   @chewxy on Twitter
   

   View Slide

2. What Are Monads
   Monads are just monoids in the category of endofunctors.
   @chewxy on Twitter
   

   View Slide

3. Example Monad
   with  open('blah.txt',  r)  as  f:  
    lines  =  doSomething(f)  
   @chewxy on Twitter
   

   View Slide

4. The End
   Thank you. See you next year.
   @chewxy on Twitter
   

   View Slide

5. Real Life Code
   def  anomalyDetection(data,  obsPerPeriod):  
      
    if  not  obsPerPeriod:  
      raise  Exception("errorMessageHere")  
      
    #  other  related  checks...  
    
    decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)  
      
    lessSeasonal  =  data  –  decomposed.seasonal  
      
    lessMedian  =  lessSeasonal  –  np.median(data)  
      
    #  and  so  on  and  so  forth...  
   @chewxy on Twitter
   

   View Slide

6. Real Life Code
   def  anomalyDetection(data,  obsPerPeriod):  
      
    if  not  obsPerPeriod:  
      raise  Exception("errorMessageHere")  
      
    #  other  related  checks...  
    
    decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)  
      
    lessSeasonal  =  data  –  decomposed.seasonal  
      
    lessMedian  =  lessSeasonal  –  np.median(data)  
      
    #  and  so  on  and  so  forth...  
   @chewxy on Twitter
   

   View Slide

7. Composition
   x  =  foo()  
   y  =  bar(x)  
   z  =  baz(y)  
    
    
   @chewxy on Twitter
   

   View Slide

8. Composition
   x  =  foo()  
   y  =  bar(x)  
   z  =  baz(y)  
   Equivalent to
   z  =  baz(bar(foo()))  
   @chewxy on Twitter
   

   View Slide

9. Composition
   a  =  1  +  2  +  3  
   b  =  1  /  (3  –  sqrt(9))  
    
    
   @chewxy on Twitter
   

   View Slide

10. Composition
    a  =  1  +  2  +  3  
    b  =  1  /  (3  –  sqrt(9))  
    Equivalent to
    a  =  add(add(1,  2),  3)  
    b  =  div(1,  sub(3,  sqrt(9)))  
    @chewxy on Twitter
    

    View Slide

11. Then the universe implodes
    @chewxy on Twitter
    

    View Slide

12. Functions
    We know what functions are…
    @chewxy on Twitter
    

    View Slide

13. Functions
    We know what functions are…
    def  foo(x):  
     #do  something  
    @chewxy on Twitter
    

    View Slide

14. Procedures
    Back in the old days before C won, Pascal differentiated
    procedures and functions.
    SQL does the same.
    @chewxy on Twitter
    

    View Slide

15. Procedures
    Back in the old days before C won, Pascal differentiated
    procedures and functions.
    SQL does the same.
    Functions returned values. Procedures didn't.
    @chewxy on Twitter
    

    View Slide

16. Pure Functions
    In mathematics, a function[1] is a relation between a set of
    inputs and a set of permissible outputs with the property
    that each input is related to exactly one output.
    [1] – "Function" Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc. 2015-
    ish
    @chewxy on Twitter
    

    View Slide

17. Pure Functions
    1
    2
    3
    4
    2
    3
    4
    5
    fn = (+1)
    def fn(x: int) -> int: return x + 1
    @chewxy on Twitter
    

    View Slide

18. Programmers' Definition of
    Functions
      May return a value (or not).
      Return values are not mapped 1:1 with input values
      Side effects!
      Do this, then do that (aka procedures).
    @chewxy on Twitter
    

    View Slide

19. Functions and Pure Functions
      Both take parameters*
      Both return values*
      Both are composable
    @chewxy on Twitter
    

    View Slide

20. Categories
    @chewxy on Twitter
    

    View Slide

21. Categories
    A collection of:
      Things (values)
      Have an identity function where I(x)  =  x  
      Arrows (functions)
      Have domain (input) and range (output)
      Can be composed
      Obey some form of associativeness laws
    TL;DR – Pretty pictures to help us think about functions
    @chewxy on Twitter
    

    View Slide

22. Categories
    @chewxy on Twitter
    X Y
    Z
    f
    g
    gŸf
    

    View Slide

23. Categories*
    1
    2
    3
    4
    2
    3
    4
    5
    f = (+1)
    3
    4
    5
    6
    g = (+1)
    gŸf
    @chewxy on Twitter
    *The objects are expanded to show the values for visualization purposes
    

    View Slide

24. Big Deal…
      Categories are cool… so what?
    @chewxy on Twitter
    

    View Slide

25. Big Deal…
      Categories are cool… so what?
      Category theory gives us the intuitions needed to think
    about issues that plague developers
    @chewxy on Twitter
    

    View Slide

26. Awkward Squad*
      IO
      Concurrency
      Exception
      FFI
    * With apologies to Simon Peyton Jones
    @chewxy on Twitter
    

    View Slide

27. Awkward Squad
      IO
      Permanent side effects
      Concurrency
      Non-determinism happens
      Exception
      Errors – What do we do with malformed/unexpected
    inputs?
      FFI
      Dependence on outside information
    Even the simple print can fail!
    @chewxy on Twitter
    

    View Slide

28. Print Fail!
    while  True:  
     print("Hello  World")  
    @chewxy on Twitter
    

    View Slide

29. Print Fail!
    while  True:  
     print("Hello  World")  
     
    >  python3  test.py  |  head  -­‐n  1  
    Hello  World  
    Traceback  (most  recent  call  last):  
       File  "test.py",  line  2,  in    
           print("Hello  World")  
    IOError:  [Errno  32]  Broken  pipe  
    @chewxy on Twitter
    

    View Slide

30. Dealing with Failure
    @chewxy on Twitter
    

    View Slide

31. Example
    def  div(num,  denom):  
     return  num/denom  
     
    def  sqrt(x):  
     return  math.sqrt(x)  
    @chewxy on Twitter
    

    View Slide

32. Domains (input)
      The domain for div to function correctly:
      For denom: Any numeric type (int, float, double … )
      For num: Any numeric type
      The domain for sqrt to function correctly:
      For x: Any numeric type
    @chewxy on Twitter
    

    View Slide

33. Ranges (output)
      Range of div:
      Float
      Range of sqrt:
      Float
    @chewxy on Twitter
    

    View Slide

34. Example in Categories
    2
    4
    5
    0
    50
    25
    20
    ????
    f = div(100, denom)
    7.07
    5.00
    4.47
    ?????
    g = sqrt(x)
    gŸf
    denom x
    @chewxy on Twitter
    

    View Slide

35. Extend the Range
    def  cleanDiv(num,  denom):  
     try:  
       return  num/denom  
     except  ZeroDivisionError:  
       return  None  
     
    def  cleanSqrt(x):  
     if  x  <  0:  
       return  None  
     return  math.sqrt(x)  
    @chewxy on Twitter
    

    View Slide

36. New Ranges (output)
      Range of cleanDiv:
      Float
    NoneType  
      Range of cleanSqrt:
      Float
    NoneType  
    @chewxy on Twitter
    

    View Slide

37. With Extended Ranges
    2
    4
    5
    0
    50
    25
    20 7.07
    5.00
    4.47
    TypeError
    gŸf
    denom x
    None
    @chewxy on Twitter
    

    View Slide

38. Extend the Domain and Ranges
    def  cleanDiv(num,  denom):  
     try:  
       return  num/denom  
     except  ZeroDivisionError:  
       return  None  
     
    def  cleanSqrt(x):  
     #  special  check  for  x  is  None,  because  sqrt(0)  ==  0  
     if  x  <  0  or  x  is  None:  
       return  None  
     return  math.sqrt(x)  
    @chewxy on Twitter
    

    View Slide

39. Extended Domains + Ranges
    2
    4
    5
    0
    50
    25
    20
    f = cleanDiv(100, denom)
    7.07
    5.00
    4.47
    g = cleanSqrt(x)
    gŸf
    denom x
    None None
    @chewxy on Twitter
    None
    

    View Slide

40. Tedious
      To compose functions, you basically need to create new
    functions that have different ranges and domains
      Or do it like what we do normally
    @chewxy on Twitter
    

    View Slide

41. How To Deal With It
    v  =  foo()  #  may  return  None  
    if  v:  
     v  =  bar(v)  
    else:  
     v  =  None  
     
     
     
    try:  
     v  =  bar(v)  
    except  Exception  as  pokémon:  
     v  =  None  
    @chewxy on Twitter
    

    View Slide

42. Tedious
      To compose functions, you basically need to create new
    functions that have different ranges and domains
      Or do it like what we do normally
      Is there a way to generalize?
    @chewxy on Twitter
    

    View Slide

43. Map from This Category
    Numeric Float
    f = div(100, denom)
    Float
    g = sqrt(x)
    gŸf
    denom x
    @chewxy on Twitter
    

    View Slide

44. To This Category
    Numeric Float
    f = cleanDiv(100, denom)
    Float
    g = cleanSqrt(x)
    gŸf
    denom x
    None None
    @chewxy on Twitter
    None
    

    View Slide

45. Recall
    Monads are just monoids in the category of endofunctors
    @chewxy on Twitter
    

    View Slide

46. Functor
    In mathematics, a functor[0] is a type of mapping between
    categories …
    [0] – More wikipedia citation?? This essay will fail if it's a uni essay
    @chewxy on Twitter
    

    View Slide

47. Implementing Monads
    Reality called. It wants its Python back
    @chewxy on Twitter
    

    View Slide

48. Interfaces Recap
    >>>  v  =  Value(1,2,3)  
    >>>  for  i  in  v:  
     print(i)  
    >>>  v2  =  ValueNoIter(1,2,3)  
    >>>  for  i  in  v2:  
     print(i)  
    Traceback  (most  recent  call  last):  
       File  "",  line  1,  in    
    TypeError:  'ValueNoIter'  object  is  not  iterable  
     
    @chewxy on Twitter
    

    View Slide

49. Interfaces Recap
    class  Value(object):  
     def  __init__(self,  *args):  
       self.value  =  args  
     def  __iter__(self):  
       for  v  in  self.value:  
         yield  v  
    @chewxy on Twitter
    

    View Slide

50. Interfaces*
      Monads are anything with bind() and unit().
      bind(): applies a function on a monadic value.
      unit(): makes something a monadic value
      There are different types of monads, each with different
    functionalities to their bind() and unit().
      Monad Laws apply
    *Haskell typeclasses are like compile time type safe interfaces
    @chewxy on Twitter
    

    View Slide

51. Monadic Values
    Numeric Float
    f = cleanDiv(100, denom)
    Float
    g = cleanSqrt(x)
    gŸf
    denom x
    None None
    @chewxy on Twitter
    None
    

    View Slide

52. Monadic Values
    Numeric Float
    f = cleanDiv(100, denom)
    Float
    g = cleanSqrt(x)
    gŸf
    denom x
    None None
    @chewxy on Twitter
    None
    

    View Slide

53. Maybe Monad
      Deals with failure conditions
      bind(): apply a function if the input is not None, else return
    None  
      unit(): return the value*
    * For clarity's sake, we're going to wrap the value in a Value class, which is strictly not necessary
    @chewxy on Twitter
    

    View Slide

54. Example Time
    def  bind(f):    
     def  inner(*args,  **kwargs):  
       a  =  [unbox(i)  for  i  in  args]  
       kw  =  {k:unbox(v)  for  k,  v  in  kwargs.items()}  
       if  None  in  a  or  None  in  kw:  
         return  None  
       return  f(*a,  **kw)  
     return  inner  
     
    def  unit(f):  
     def  inner(*args,  **kwargs):  
       return  Value(f(*args,  **kwargs))  
     return  inner  
     
    def  unbox(v):  
     try:  
       return  v.value  
     except  AttributeError:  
       return  v  
     
    @chewxy on Twitter
    

    View Slide

55. Example Time
    class  Value(object):  
     __slots__  =  ['value']  
     def  __init__(self,  v=None):  
       if  type(v)  is  Value:  
         self.value  =  v.value  
       else:  
         self.value  =  v  
     def  __repr__(self):  
       if  self.value  is  not  None:  
         return  "Just({})".format(self.value)  
       else:  
         return  "Nothing"  
    @chewxy on Twitter
    

    View Slide

56. Example Time
    @unit  
    @bind  
    def  div(num,  denom):  
     if  denom  ==  0:  
       return  None  
     return  num/denom  
     
    @unit  
    @bind  
    def  sqrt(x):  
     if  x  <  0:  
       return  None  
     return  math.sqrt(x)  
    @chewxy on Twitter
    

    View Slide

57. Sanity Check
    >>>  sqrt(div(100,  4))  
    Just(5.0)  
    >>>  sqrt(div(100,  0))  
    Nothing  
    >>>  div(sqrt(100),  2)  
    Just(5.0)  
    >>>  div(sqrt(100),  0)  
    Nothing  
    >>>  div(div(div(100,  0),2),10)  
    Nothing  
    @chewxy on Twitter
    

    View Slide

58. List Monad
      Deals with collections
      bind(): applies a function on the elements of the list
      unit(): puts an element into a list
    @chewxy on Twitter
    

    View Slide

59. Example Time
    def  bind(f,  x):  
     return  [j  for  i  in  x  for  j  in  f(i)]  
     
    def  unit(x):  
     return  [x]  
    @chewxy on Twitter
    

    View Slide

60. Example Time
    def  sqrt(x):  
     if  x  <  0:  
       return  []  
     else:  
       s  =  math.sqrt(x)  
       return  [s,  -­‐s]  
     
    @chewxy on Twitter
    

    View Slide

61. Example Time
    >>>  bind(sqrt,  unit(100))  
    [10.0,  -­‐10.0]  
     
    >>>  bind(sqrt,  bind(sqrt,  [100]))  
    [3.1622776601683795,  -­‐3.1622776601683795]  
     
    >>>  bind(sqrt,  [4,  9,  16,  25,  36])  
    [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,  
    -­‐5.0,  6.0,  -­‐6.0]  
    @chewxy on Twitter
    

    View Slide

62. Seems Familiar…
    >>>  bind(sqrt,  [4,  9,  16,  25,  36])  
    [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,  
    -­‐5.0,  6.0,  -­‐6.0]  
     
    >>>  [sqrt(x)  for  x  in  [4,  9,  16,  25,  36]]  
    >>>  list(map(sqrt,  [4,  9,  16,  25,  36]))  
    @chewxy on Twitter
    

    View Slide

63. The General Idea
    Monads help us compose functions that have different return
    types.
    Solution:
    1.  Wrap the value with their contexts in something called a
    monadic value (which can be anything – you define
    unit())
    2.  Define a function bind() that defines out how to compose
    the functions that applies within that context
    @chewxy on Twitter
    

    View Slide

64. Opening Files
    def  unit(n):  
     return  open(n,  'r').__enter__()      #  type:    'file'  
     
    def  bind(f):  
     def  inner(x):  
       retVal  =  None  
       try:  
         retVal  =  f(x)  
       except  Exception  as  pokemon:        
         pass  #  do  something  with  pokemon    
       x.__exit__()  
       return  retVal  
     return  inner  
     
     
    @bind  
    def  doStuff(f):  
     return  f.readlines()  
     
     
    lines  =  doStuff(unit(fileName))  
     
    @chewxy on Twitter
    

    View Slide

65. Sounds Familiar?
    with  open('blah.txt',  'r')  as  f:  
     lines  =  doStuff(f)  
    @chewxy on Twitter
    

    View Slide

66. Other (Possible) Monads
    Twisted's Deferred (JS promises 10 years before
    Promises!)
    djangoORM's .query() and .filter()  *  
    Scikits-Learn's Pipeline  *  
    * Yet to check whether it's actually a monad
    @chewxy on Twitter
    

    View Slide

67. Why Does This Matter?
    @chewxy on Twitter
    

    View Slide

68. Why Does This Matter?
      We compose functions all the time.
      Programs should be easy for humans to reason about.
      We haphazardly recreate solutions to solve certain
    classes of problems.
      Understanding the underlying pattern to most of the
    issues programmers face makes us better (I hope?)
      BYO paradigm
    @chewxy on Twitter
    

    View Slide

69. The End (for Realz)
    See you next year!
    @chewxy on Twitter
    

    View Slide

70. Appendix: Monad Laws
    #  left  bind  
    assert  bind(unit(v),  f)  ==  f(v)  
     
    #  right  bind  
    assert  bind(unit(v),  unit)  ==  unit(v)  
     
    #  associativeness  
    assert  bind(bind(unit(v),  f),  g)  ==  g(f(v))  
    @chewxy on Twitter
    

    View Slide

71. Appendix: Monad Transformers
    Build your own monad transformer
    @chewxy on Twitter
    

    View Slide

72. Appendix: Arrow Associativity
    @chewxy on Twitter
    hŸgŸf == (hŸg)Ÿf == hŸ(gŸf)
    

    View Slide