Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Monads, In My Python? (PyConAU 2015)

Xuanyi
August 01, 2015

Monads, In My Python? (PyConAU 2015)

The associated talk can be found here:
https://www.youtube.com/watch?v=WNwV3wR4JjA

Xuanyi

August 01, 2015
Tweet

More Decks by Xuanyi

Other Decks in Programming

Transcript

  1. MONADS, IN MY
    PYTHON?
    PYCON AU 2015
    @chewxy on Twitter

    View Slide

  2. What Are Monads
    Monads are just monoids in the category of endofunctors.
    @chewxy on Twitter

    View Slide

  3. Example Monad
    with  open('blah.txt',  r)  as  f:  
     lines  =  doSomething(f)  
    @chewxy on Twitter

    View Slide

  4. The End
    Thank you. See you next year.
    @chewxy on Twitter

    View Slide

  5. Real Life Code
    def  anomalyDetection(data,  obsPerPeriod):  
       
     if  not  obsPerPeriod:  
       raise  Exception("errorMessageHere")  
       
     #  other  related  checks...  
     
     decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)  
       
     lessSeasonal  =  data  –  decomposed.seasonal  
       
     lessMedian  =  lessSeasonal  –  np.median(data)  
       
     #  and  so  on  and  so  forth...  
    @chewxy on Twitter

    View Slide

  6. Real Life Code
    def  anomalyDetection(data,  obsPerPeriod):  
       
     if  not  obsPerPeriod:  
       raise  Exception("errorMessageHere")  
       
     #  other  related  checks...  
     
     decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)  
       
     lessSeasonal  =  data  –  decomposed.seasonal  
       
     lessMedian  =  lessSeasonal  –  np.median(data)  
       
     #  and  so  on  and  so  forth...  
    @chewxy on Twitter

    View Slide

  7. Composition
    x  =  foo()  
    y  =  bar(x)  
    z  =  baz(y)  
     
     
    @chewxy on Twitter

    View Slide

  8. Composition
    x  =  foo()  
    y  =  bar(x)  
    z  =  baz(y)  
    Equivalent to
    z  =  baz(bar(foo()))  
    @chewxy on Twitter

    View Slide

  9. Composition
    a  =  1  +  2  +  3  
    b  =  1  /  (3  –  sqrt(9))  
     
     
    @chewxy on Twitter

    View Slide

  10. Composition
    a  =  1  +  2  +  3  
    b  =  1  /  (3  –  sqrt(9))  
    Equivalent to
    a  =  add(add(1,  2),  3)  
    b  =  div(1,  sub(3,  sqrt(9)))  
    @chewxy on Twitter

    View Slide

  11. Then the universe implodes
    @chewxy on Twitter

    View Slide

  12. Functions
    We know what functions are…
    @chewxy on Twitter

    View Slide

  13. Functions
    We know what functions are…
    def  foo(x):  
     #do  something  
    @chewxy on Twitter

    View Slide

  14. Procedures
    Back in the old days before C won, Pascal differentiated
    procedures and functions.
    SQL does the same.
    @chewxy on Twitter

    View Slide

  15. Procedures
    Back in the old days before C won, Pascal differentiated
    procedures and functions.
    SQL does the same.
    Functions returned values. Procedures didn't.
    @chewxy on Twitter

    View Slide

  16. Pure Functions
    In mathematics, a function[1] is a relation between a set of
    inputs and a set of permissible outputs with the property
    that each input is related to exactly one output.
    [1] – "Function" Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc. 2015-
    ish
    @chewxy on Twitter

    View Slide

  17. Pure Functions
    1
    2
    3
    4
    2
    3
    4
    5
    fn = (+1)
    def fn(x: int) -> int: return x + 1
    @chewxy on Twitter

    View Slide

  18. Programmers' Definition of
    Functions
      May return a value (or not).
      Return values are not mapped 1:1 with input values
      Side effects!
      Do this, then do that (aka procedures).
    @chewxy on Twitter

    View Slide

  19. Functions and Pure Functions
      Both take parameters*
      Both return values*
      Both are composable
    @chewxy on Twitter

    View Slide

  20. Categories
    @chewxy on Twitter

    View Slide

  21. Categories
    A collection of:
      Things (values)
      Have an identity function where I(x)  =  x  
      Arrows (functions)
      Have domain (input) and range (output)
      Can be composed
      Obey some form of associativeness laws
    TL;DR – Pretty pictures to help us think about functions
    @chewxy on Twitter

    View Slide

  22. Categories
    @chewxy on Twitter
    X Y
    Z
    f
    g
    gŸf

    View Slide

  23. Categories*
    1
    2
    3
    4
    2
    3
    4
    5
    f = (+1)
    3
    4
    5
    6
    g = (+1)
    gŸf
    @chewxy on Twitter
    *The objects are expanded to show the values for visualization purposes

    View Slide

  24. Big Deal…
      Categories are cool… so what?
    @chewxy on Twitter

    View Slide

  25. Big Deal…
      Categories are cool… so what?
      Category theory gives us the intuitions needed to think
    about issues that plague developers
    @chewxy on Twitter

    View Slide

  26. Awkward Squad*
      IO
      Concurrency
      Exception
      FFI
    * With apologies to Simon Peyton Jones
    @chewxy on Twitter

    View Slide

  27. Awkward Squad
      IO
      Permanent side effects
      Concurrency
      Non-determinism happens
      Exception
      Errors – What do we do with malformed/unexpected
    inputs?
      FFI
      Dependence on outside information
    Even the simple print can fail!
    @chewxy on Twitter

    View Slide

  28. Print Fail!
    while  True:  
     print("Hello  World")  
    @chewxy on Twitter

    View Slide

  29. Print Fail!
    while  True:  
     print("Hello  World")  
     
    >  python3  test.py  |  head  -­‐n  1  
    Hello  World  
    Traceback  (most  recent  call  last):  
       File  "test.py",  line  2,  in    
           print("Hello  World")  
    IOError:  [Errno  32]  Broken  pipe  
    @chewxy on Twitter

    View Slide

  30. Dealing with Failure
    @chewxy on Twitter

    View Slide

  31. Example
    def  div(num,  denom):  
     return  num/denom  
     
    def  sqrt(x):  
     return  math.sqrt(x)  
    @chewxy on Twitter

    View Slide

  32. Domains (input)
      The domain for div to function correctly:
      For denom: Any numeric type (int, float, double … )
      For num: Any numeric type
      The domain for sqrt to function correctly:
      For x: Any numeric type
    @chewxy on Twitter

    View Slide

  33. Ranges (output)
      Range of div:
      Float
      Range of sqrt:
      Float
    @chewxy on Twitter

    View Slide

  34. Example in Categories
    2
    4
    5
    0
    50
    25
    20
    ????
    f = div(100, denom)
    7.07
    5.00
    4.47
    ?????
    g = sqrt(x)
    gŸf
    denom x
    @chewxy on Twitter

    View Slide

  35. Extend the Range
    def  cleanDiv(num,  denom):  
     try:  
       return  num/denom  
     except  ZeroDivisionError:  
       return  None  
     
    def  cleanSqrt(x):  
     if  x  <  0:  
       return  None  
     return  math.sqrt(x)  
    @chewxy on Twitter

    View Slide

  36. New Ranges (output)
      Range of cleanDiv:
      Float
    NoneType  
      Range of cleanSqrt:
      Float
    NoneType  
    @chewxy on Twitter

    View Slide

  37. With Extended Ranges
    2
    4
    5
    0
    50
    25
    20 7.07
    5.00
    4.47
    TypeError
    gŸf
    denom x
    None
    @chewxy on Twitter

    View Slide

  38. Extend the Domain and Ranges
    def  cleanDiv(num,  denom):  
     try:  
       return  num/denom  
     except  ZeroDivisionError:  
       return  None  
     
    def  cleanSqrt(x):  
     #  special  check  for  x  is  None,  because  sqrt(0)  ==  0  
     if  x  <  0  or  x  is  None:  
       return  None  
     return  math.sqrt(x)  
    @chewxy on Twitter

    View Slide

  39. Extended Domains + Ranges
    2
    4
    5
    0
    50
    25
    20
    f = cleanDiv(100, denom)
    7.07
    5.00
    4.47
    g = cleanSqrt(x)
    gŸf
    denom x
    None None
    @chewxy on Twitter
    None

    View Slide

  40. Tedious
      To compose functions, you basically need to create new
    functions that have different ranges and domains
      Or do it like what we do normally
    @chewxy on Twitter

    View Slide

  41. How To Deal With It
    v  =  foo()  #  may  return  None  
    if  v:  
     v  =  bar(v)  
    else:  
     v  =  None  
     
     
     
    try:  
     v  =  bar(v)  
    except  Exception  as  pokémon:  
     v  =  None  
    @chewxy on Twitter

    View Slide

  42. Tedious
      To compose functions, you basically need to create new
    functions that have different ranges and domains
      Or do it like what we do normally
      Is there a way to generalize?
    @chewxy on Twitter

    View Slide

  43. Map from This Category
    Numeric Float
    f = div(100, denom)
    Float
    g = sqrt(x)
    gŸf
    denom x
    @chewxy on Twitter

    View Slide

  44. To This Category
    Numeric Float
    f = cleanDiv(100, denom)
    Float
    g = cleanSqrt(x)
    gŸf
    denom x
    None None
    @chewxy on Twitter
    None

    View Slide

  45. Recall
    Monads are just monoids in the category of endofunctors
    @chewxy on Twitter

    View Slide

  46. Functor
    In mathematics, a functor[0] is a type of mapping between
    categories …
    [0] – More wikipedia citation?? This essay will fail if it's a uni essay
    @chewxy on Twitter

    View Slide

  47. Implementing Monads
    Reality called. It wants its Python back
    @chewxy on Twitter

    View Slide

  48. Interfaces Recap
    >>>  v  =  Value(1,2,3)  
    >>>  for  i  in  v:  
     print(i)  
    >>>  v2  =  ValueNoIter(1,2,3)  
    >>>  for  i  in  v2:  
     print(i)  
    Traceback  (most  recent  call  last):  
       File  "",  line  1,  in    
    TypeError:  'ValueNoIter'  object  is  not  iterable  
     
    @chewxy on Twitter

    View Slide

  49. Interfaces Recap
    class  Value(object):  
     def  __init__(self,  *args):  
       self.value  =  args  
     def  __iter__(self):  
       for  v  in  self.value:  
         yield  v  
    @chewxy on Twitter

    View Slide

  50. Interfaces*
      Monads are anything with bind() and unit().
      bind(): applies a function on a monadic value.
      unit(): makes something a monadic value
      There are different types of monads, each with different
    functionalities to their bind() and unit().
      Monad Laws apply
    *Haskell typeclasses are like compile time type safe interfaces
    @chewxy on Twitter

    View Slide

  51. Monadic Values
    Numeric Float
    f = cleanDiv(100, denom)
    Float
    g = cleanSqrt(x)
    gŸf
    denom x
    None None
    @chewxy on Twitter
    None

    View Slide

  52. Monadic Values
    Numeric Float
    f = cleanDiv(100, denom)
    Float
    g = cleanSqrt(x)
    gŸf
    denom x
    None None
    @chewxy on Twitter
    None

    View Slide

  53. Maybe Monad
      Deals with failure conditions
      bind(): apply a function if the input is not None, else return
    None  
      unit(): return the value*
    * For clarity's sake, we're going to wrap the value in a Value class, which is strictly not necessary
    @chewxy on Twitter

    View Slide

  54. Example Time
    def  bind(f):    
     def  inner(*args,  **kwargs):  
       a  =  [unbox(i)  for  i  in  args]  
       kw  =  {k:unbox(v)  for  k,  v  in  kwargs.items()}  
       if  None  in  a  or  None  in  kw:  
         return  None  
       return  f(*a,  **kw)  
     return  inner  
     
    def  unit(f):  
     def  inner(*args,  **kwargs):  
       return  Value(f(*args,  **kwargs))  
     return  inner  
     
    def  unbox(v):  
     try:  
       return  v.value  
     except  AttributeError:  
       return  v  
     
    @chewxy on Twitter

    View Slide

  55. Example Time
    class  Value(object):  
     __slots__  =  ['value']  
     def  __init__(self,  v=None):  
       if  type(v)  is  Value:  
         self.value  =  v.value  
       else:  
         self.value  =  v  
     def  __repr__(self):  
       if  self.value  is  not  None:  
         return  "Just({})".format(self.value)  
       else:  
         return  "Nothing"  
    @chewxy on Twitter

    View Slide

  56. Example Time
    @unit  
    @bind  
    def  div(num,  denom):  
     if  denom  ==  0:  
       return  None  
     return  num/denom  
     
    @unit  
    @bind  
    def  sqrt(x):  
     if  x  <  0:  
       return  None  
     return  math.sqrt(x)  
    @chewxy on Twitter

    View Slide

  57. Sanity Check
    >>>  sqrt(div(100,  4))  
    Just(5.0)  
    >>>  sqrt(div(100,  0))  
    Nothing  
    >>>  div(sqrt(100),  2)  
    Just(5.0)  
    >>>  div(sqrt(100),  0)  
    Nothing  
    >>>  div(div(div(100,  0),2),10)  
    Nothing  
    @chewxy on Twitter

    View Slide

  58. List Monad
      Deals with collections
      bind(): applies a function on the elements of the list
      unit(): puts an element into a list
    @chewxy on Twitter

    View Slide

  59. Example Time
    def  bind(f,  x):  
     return  [j  for  i  in  x  for  j  in  f(i)]  
     
    def  unit(x):  
     return  [x]  
    @chewxy on Twitter

    View Slide

  60. Example Time
    def  sqrt(x):  
     if  x  <  0:  
       return  []  
     else:  
       s  =  math.sqrt(x)  
       return  [s,  -­‐s]  
     
    @chewxy on Twitter

    View Slide

  61. Example Time
    >>>  bind(sqrt,  unit(100))  
    [10.0,  -­‐10.0]  
     
    >>>  bind(sqrt,  bind(sqrt,  [100]))  
    [3.1622776601683795,  -­‐3.1622776601683795]  
     
    >>>  bind(sqrt,  [4,  9,  16,  25,  36])  
    [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,  
    -­‐5.0,  6.0,  -­‐6.0]  
    @chewxy on Twitter

    View Slide

  62. Seems Familiar…
    >>>  bind(sqrt,  [4,  9,  16,  25,  36])  
    [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,  
    -­‐5.0,  6.0,  -­‐6.0]  
     
    >>>  [sqrt(x)  for  x  in  [4,  9,  16,  25,  36]]  
    >>>  list(map(sqrt,  [4,  9,  16,  25,  36]))  
    @chewxy on Twitter

    View Slide

  63. The General Idea
    Monads help us compose functions that have different return
    types.
    Solution:
    1.  Wrap the value with their contexts in something called a
    monadic value (which can be anything – you define
    unit())
    2.  Define a function bind() that defines out how to compose
    the functions that applies within that context
    @chewxy on Twitter

    View Slide

  64. Opening Files
    def  unit(n):  
     return  open(n,  'r').__enter__()      #  type:    'file'  
     
    def  bind(f):  
     def  inner(x):  
       retVal  =  None  
       try:  
         retVal  =  f(x)  
       except  Exception  as  pokemon:        
         pass  #  do  something  with  pokemon    
       x.__exit__()  
       return  retVal  
     return  inner  
     
     
    @bind  
    def  doStuff(f):  
     return  f.readlines()  
     
     
    lines  =  doStuff(unit(fileName))  
     
    @chewxy on Twitter

    View Slide

  65. Sounds Familiar?
    with  open('blah.txt',  'r')  as  f:  
     lines  =  doStuff(f)  
    @chewxy on Twitter

    View Slide

  66. Other (Possible) Monads
    Twisted's Deferred (JS promises 10 years before
    Promises!)
    djangoORM's .query() and .filter()  *  
    Scikits-Learn's Pipeline  *  
    * Yet to check whether it's actually a monad
    @chewxy on Twitter

    View Slide

  67. Why Does This Matter?
    @chewxy on Twitter

    View Slide

  68. Why Does This Matter?
      We compose functions all the time.
      Programs should be easy for humans to reason about.
      We haphazardly recreate solutions to solve certain
    classes of problems.
      Understanding the underlying pattern to most of the
    issues programmers face makes us better (I hope?)
      BYO paradigm
    @chewxy on Twitter

    View Slide

  69. The End (for Realz)
    See you next year!
    @chewxy on Twitter

    View Slide

  70. Appendix: Monad Laws
    #  left  bind  
    assert  bind(unit(v),  f)  ==  f(v)  
     
    #  right  bind  
    assert  bind(unit(v),  unit)  ==  unit(v)  
     
    #  associativeness  
    assert  bind(bind(unit(v),  f),  g)  ==  g(f(v))  
    @chewxy on Twitter

    View Slide

  71. Appendix: Monad Transformers
    Build your own monad transformer
    @chewxy on Twitter

    View Slide

  72. Appendix: Arrow Associativity
    @chewxy on Twitter
    hŸgŸf == (hŸg)Ÿf == hŸ(gŸf)

    View Slide