Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Monads, In My Python? (PyConAU 2015)

3e45c02f2ae5f812a55c4975124da6b2?s=47 Xuanyi
August 01, 2015

Monads, In My Python? (PyConAU 2015)

The associated talk can be found here:
https://www.youtube.com/watch?v=WNwV3wR4JjA

3e45c02f2ae5f812a55c4975124da6b2?s=128

Xuanyi

August 01, 2015
Tweet

Transcript

  1. MONADS, IN MY PYTHON? PYCON AU 2015 @chewxy on Twitter

  2. What Are Monads Monads are just monoids in the category

    of endofunctors. @chewxy on Twitter
  3. Example Monad with  open('blah.txt',  r)  as  f:    lines  =

     doSomething(f)   @chewxy on Twitter
  4. The End Thank you. See you next year. @chewxy on

    Twitter
  5. Real Life Code def  anomalyDetection(data,  obsPerPeriod):        if

     not  obsPerPeriod:      raise  Exception("errorMessageHere")        #  other  related  checks...      decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)        lessSeasonal  =  data  –  decomposed.seasonal        lessMedian  =  lessSeasonal  –  np.median(data)        #  and  so  on  and  so  forth...   @chewxy on Twitter
  6. Real Life Code def  anomalyDetection(data,  obsPerPeriod):        if

     not  obsPerPeriod:      raise  Exception("errorMessageHere")        #  other  related  checks...      decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)        lessSeasonal  =  data  –  decomposed.seasonal        lessMedian  =  lessSeasonal  –  np.median(data)        #  and  so  on  and  so  forth...   @chewxy on Twitter
  7. Composition x  =  foo()   y  =  bar(x)   z

     =  baz(y)       @chewxy on Twitter
  8. Composition x  =  foo()   y  =  bar(x)   z

     =  baz(y)   Equivalent to z  =  baz(bar(foo()))   @chewxy on Twitter
  9. Composition a  =  1  +  2  +  3   b

     =  1  /  (3  –  sqrt(9))       @chewxy on Twitter
  10. Composition a  =  1  +  2  +  3   b

     =  1  /  (3  –  sqrt(9))   Equivalent to a  =  add(add(1,  2),  3)   b  =  div(1,  sub(3,  sqrt(9)))   @chewxy on Twitter
  11. Then the universe implodes @chewxy on Twitter

  12. Functions We know what functions are… @chewxy on Twitter

  13. Functions We know what functions are… def  foo(x):    #do

     something   @chewxy on Twitter
  14. Procedures Back in the old days before C won, Pascal

    differentiated procedures and functions. SQL does the same. @chewxy on Twitter
  15. Procedures Back in the old days before C won, Pascal

    differentiated procedures and functions. SQL does the same. Functions returned values. Procedures didn't. @chewxy on Twitter
  16. Pure Functions In mathematics, a function[1] is a relation between

    a set of inputs and a set of permissible outputs with the property that each input is related to exactly one output. [1] – "Function" Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc. 2015- ish @chewxy on Twitter
  17. Pure Functions 1 2 3 4 2 3 4 5

    fn = (+1) def fn(x: int) -> int: return x + 1 @chewxy on Twitter
  18. Programmers' Definition of Functions   May return a value (or

    not).   Return values are not mapped 1:1 with input values   Side effects!   Do this, then do that (aka procedures). @chewxy on Twitter
  19. Functions and Pure Functions   Both take parameters*   Both

    return values*   Both are composable @chewxy on Twitter
  20. Categories @chewxy on Twitter

  21. Categories A collection of:   Things (values)   Have an

    identity function where I(x)  =  x     Arrows (functions)   Have domain (input) and range (output)   Can be composed   Obey some form of associativeness laws TL;DR – Pretty pictures to help us think about functions @chewxy on Twitter
  22. Categories @chewxy on Twitter X Y Z f g gŸf

  23. Categories* 1 2 3 4 2 3 4 5 f

    = (+1) 3 4 5 6 g = (+1) gŸf @chewxy on Twitter *The objects are expanded to show the values for visualization purposes
  24. Big Deal…   Categories are cool… so what? @chewxy on

    Twitter
  25. Big Deal…   Categories are cool… so what?   Category

    theory gives us the intuitions needed to think about issues that plague developers @chewxy on Twitter
  26. Awkward Squad*   IO   Concurrency   Exception   FFI

    * With apologies to Simon Peyton Jones @chewxy on Twitter
  27. Awkward Squad   IO   Permanent side effects   Concurrency

      Non-determinism happens   Exception   Errors – What do we do with malformed/unexpected inputs?   FFI   Dependence on outside information Even the simple print can fail! @chewxy on Twitter
  28. Print Fail! while  True:    print("Hello  World")   @chewxy on

    Twitter
  29. Print Fail! while  True:    print("Hello  World")     >

     python3  test.py  |  head  -­‐n  1   Hello  World   Traceback  (most  recent  call  last):      File  "test.py",  line  2,  in  <module>          print("Hello  World")   IOError:  [Errno  32]  Broken  pipe   @chewxy on Twitter
  30. Dealing with Failure @chewxy on Twitter

  31. Example def  div(num,  denom):    return  num/denom     def

     sqrt(x):    return  math.sqrt(x)   @chewxy on Twitter
  32. Domains (input)   The domain for div to function correctly:

      For denom: Any numeric type (int, float, double … )   For num: Any numeric type   The domain for sqrt to function correctly:   For x: Any numeric type @chewxy on Twitter
  33. Ranges (output)   Range of div:   Float   Range

    of sqrt:   Float @chewxy on Twitter
  34. Example in Categories 2 4 5 0 50 25 20

    ???? f = div(100, denom) 7.07 5.00 4.47 ????? g = sqrt(x) gŸf denom x @chewxy on Twitter
  35. Extend the Range def  cleanDiv(num,  denom):    try:    

     return  num/denom    except  ZeroDivisionError:      return  None     def  cleanSqrt(x):    if  x  <  0:      return  None    return  math.sqrt(x)   @chewxy on Twitter
  36. New Ranges (output)   Range of cleanDiv:   Float NoneType

        Range of cleanSqrt:   Float NoneType   @chewxy on Twitter
  37. With Extended Ranges 2 4 5 0 50 25 20

    7.07 5.00 4.47 TypeError gŸf denom x None @chewxy on Twitter
  38. Extend the Domain and Ranges def  cleanDiv(num,  denom):    try:

         return  num/denom    except  ZeroDivisionError:      return  None     def  cleanSqrt(x):    #  special  check  for  x  is  None,  because  sqrt(0)  ==  0    if  x  <  0  or  x  is  None:      return  None    return  math.sqrt(x)   @chewxy on Twitter
  39. Extended Domains + Ranges 2 4 5 0 50 25

    20 f = cleanDiv(100, denom) 7.07 5.00 4.47 g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None
  40. Tedious   To compose functions, you basically need to create

    new functions that have different ranges and domains   Or do it like what we do normally @chewxy on Twitter
  41. How To Deal With It v  =  foo()  #  may

     return  None   if  v:    v  =  bar(v)   else:    v  =  None         try:    v  =  bar(v)   except  Exception  as  pokémon:    v  =  None   @chewxy on Twitter
  42. Tedious   To compose functions, you basically need to create

    new functions that have different ranges and domains   Or do it like what we do normally   Is there a way to generalize? @chewxy on Twitter
  43. Map from This Category Numeric Float f = div(100, denom)

    Float g = sqrt(x) gŸf denom x @chewxy on Twitter
  44. To This Category Numeric Float f = cleanDiv(100, denom) Float

    g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None
  45. Recall Monads are just monoids in the category of endofunctors

    @chewxy on Twitter
  46. Functor In mathematics, a functor[0] is a type of mapping

    between categories … [0] – More wikipedia citation?? This essay will fail if it's a uni essay @chewxy on Twitter
  47. Implementing Monads Reality called. It wants its Python back @chewxy

    on Twitter
  48. Interfaces Recap >>>  v  =  Value(1,2,3)   >>>  for  i

     in  v:    print(i)   >>>  v2  =  ValueNoIter(1,2,3)   >>>  for  i  in  v2:    print(i)   Traceback  (most  recent  call  last):      File  "<stdin>",  line  1,  in  <module>   TypeError:  'ValueNoIter'  object  is  not  iterable     @chewxy on Twitter
  49. Interfaces Recap class  Value(object):    def  __init__(self,  *args):    

     self.value  =  args    def  __iter__(self):      for  v  in  self.value:        yield  v   @chewxy on Twitter
  50. Interfaces*   Monads are anything with bind() and unit().  

    bind(): applies a function on a monadic value.   unit(): makes something a monadic value   There are different types of monads, each with different functionalities to their bind() and unit().   Monad Laws apply *Haskell typeclasses are like compile time type safe interfaces @chewxy on Twitter
  51. Monadic Values Numeric Float f = cleanDiv(100, denom) Float g

    = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None
  52. Monadic Values Numeric Float f = cleanDiv(100, denom) Float g

    = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None
  53. Maybe Monad   Deals with failure conditions   bind(): apply

    a function if the input is not None, else return None     unit(): return the value* * For clarity's sake, we're going to wrap the value in a Value class, which is strictly not necessary @chewxy on Twitter
  54. Example Time def  bind(f):      def  inner(*args,  **kwargs):  

       a  =  [unbox(i)  for  i  in  args]      kw  =  {k:unbox(v)  for  k,  v  in  kwargs.items()}      if  None  in  a  or  None  in  kw:        return  None      return  f(*a,  **kw)    return  inner     def  unit(f):    def  inner(*args,  **kwargs):      return  Value(f(*args,  **kwargs))    return  inner     def  unbox(v):    try:      return  v.value    except  AttributeError:      return  v     @chewxy on Twitter
  55. Example Time class  Value(object):    __slots__  =  ['value']    def

     __init__(self,  v=None):      if  type(v)  is  Value:        self.value  =  v.value      else:        self.value  =  v    def  __repr__(self):      if  self.value  is  not  None:        return  "Just({})".format(self.value)      else:        return  "Nothing"   @chewxy on Twitter
  56. Example Time @unit   @bind   def  div(num,  denom):  

     if  denom  ==  0:      return  None    return  num/denom     @unit   @bind   def  sqrt(x):    if  x  <  0:      return  None    return  math.sqrt(x)   @chewxy on Twitter
  57. Sanity Check >>>  sqrt(div(100,  4))   Just(5.0)   >>>  sqrt(div(100,

     0))   Nothing   >>>  div(sqrt(100),  2)   Just(5.0)   >>>  div(sqrt(100),  0)   Nothing   >>>  div(div(div(100,  0),2),10)   Nothing   @chewxy on Twitter
  58. List Monad   Deals with collections   bind(): applies a

    function on the elements of the list   unit(): puts an element into a list @chewxy on Twitter
  59. Example Time def  bind(f,  x):    return  [j  for  i

     in  x  for  j  in  f(i)]     def  unit(x):    return  [x]   @chewxy on Twitter
  60. Example Time def  sqrt(x):    if  x  <  0:  

       return  []    else:      s  =  math.sqrt(x)      return  [s,  -­‐s]     @chewxy on Twitter
  61. Example Time >>>  bind(sqrt,  unit(100))   [10.0,  -­‐10.0]    

    >>>  bind(sqrt,  bind(sqrt,  [100]))   [3.1622776601683795,  -­‐3.1622776601683795]     >>>  bind(sqrt,  [4,  9,  16,  25,  36])   [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,   -­‐5.0,  6.0,  -­‐6.0]   @chewxy on Twitter
  62. Seems Familiar… >>>  bind(sqrt,  [4,  9,  16,  25,  36])  

    [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,   -­‐5.0,  6.0,  -­‐6.0]     >>>  [sqrt(x)  for  x  in  [4,  9,  16,  25,  36]]   >>>  list(map(sqrt,  [4,  9,  16,  25,  36]))   @chewxy on Twitter
  63. The General Idea Monads help us compose functions that have

    different return types. Solution: 1.  Wrap the value with their contexts in something called a monadic value (which can be anything – you define unit()) 2.  Define a function bind() that defines out how to compose the functions that applies within that context @chewxy on Twitter
  64. Opening Files def  unit(n):    return  open(n,  'r').__enter__()    

     #  type:    'file'     def  bind(f):    def  inner(x):      retVal  =  None      try:        retVal  =  f(x)      except  Exception  as  pokemon:              pass  #  do  something  with  pokemon        x.__exit__()      return  retVal    return  inner       @bind   def  doStuff(f):    return  f.readlines()       lines  =  doStuff(unit(fileName))     @chewxy on Twitter
  65. Sounds Familiar? with  open('blah.txt',  'r')  as  f:    lines  =

     doStuff(f)   @chewxy on Twitter
  66. Other (Possible) Monads Twisted's Deferred (JS promises 10 years before

    Promises!) djangoORM's .query() and .filter()  *   Scikits-Learn's Pipeline  *   * Yet to check whether it's actually a monad @chewxy on Twitter
  67. Why Does This Matter? @chewxy on Twitter

  68. Why Does This Matter?   We compose functions all the

    time.   Programs should be easy for humans to reason about.   We haphazardly recreate solutions to solve certain classes of problems.   Understanding the underlying pattern to most of the issues programmers face makes us better (I hope?)   BYO paradigm @chewxy on Twitter
  69. The End (for Realz) See you next year! @chewxy on

    Twitter
  70. Appendix: Monad Laws #  left  bind   assert  bind(unit(v),  f)

     ==  f(v)     #  right  bind   assert  bind(unit(v),  unit)  ==  unit(v)     #  associativeness   assert  bind(bind(unit(v),  f),  g)  ==  g(f(v))   @chewxy on Twitter
  71. Appendix: Monad Transformers Build your own monad transformer @chewxy on

    Twitter
  72. Appendix: Arrow Associativity @chewxy on Twitter hŸgŸf == (hŸg)Ÿf ==

    hŸ(gŸf)