Slide 1

Slide 1 text

MONADS, IN MY PYTHON? PYCON AU 2015 @chewxy on Twitter

Slide 2

Slide 2 text

What Are Monads Monads are just monoids in the category of endofunctors. @chewxy on Twitter

Slide 3

Slide 3 text

Example Monad with  open('blah.txt',  r)  as  f:    lines  =  doSomething(f)   @chewxy on Twitter

Slide 4

Slide 4 text

The End Thank you. See you next year. @chewxy on Twitter

Slide 5

Slide 5 text

Real Life Code def  anomalyDetection(data,  obsPerPeriod):        if  not  obsPerPeriod:      raise  Exception("errorMessageHere")        #  other  related  checks...      decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)        lessSeasonal  =  data  –  decomposed.seasonal        lessMedian  =  lessSeasonal  –  np.median(data)        #  and  so  on  and  so  forth...   @chewxy on Twitter

Slide 6

Slide 6 text

Real Life Code def  anomalyDetection(data,  obsPerPeriod):        if  not  obsPerPeriod:      raise  Exception("errorMessageHere")        #  other  related  checks...      decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)        lessSeasonal  =  data  –  decomposed.seasonal        lessMedian  =  lessSeasonal  –  np.median(data)        #  and  so  on  and  so  forth...   @chewxy on Twitter

Slide 7

Slide 7 text

Composition x  =  foo()   y  =  bar(x)   z  =  baz(y)       @chewxy on Twitter

Slide 8

Slide 8 text

Composition x  =  foo()   y  =  bar(x)   z  =  baz(y)   Equivalent to z  =  baz(bar(foo()))   @chewxy on Twitter

Slide 9

Slide 9 text

Composition a  =  1  +  2  +  3   b  =  1  /  (3  –  sqrt(9))       @chewxy on Twitter

Slide 10

Slide 10 text

Composition a  =  1  +  2  +  3   b  =  1  /  (3  –  sqrt(9))   Equivalent to a  =  add(add(1,  2),  3)   b  =  div(1,  sub(3,  sqrt(9)))   @chewxy on Twitter

Slide 11

Slide 11 text

Then the universe implodes @chewxy on Twitter

Slide 12

Slide 12 text

Functions We know what functions are… @chewxy on Twitter

Slide 13

Slide 13 text

Functions We know what functions are… def  foo(x):    #do  something   @chewxy on Twitter

Slide 14

Slide 14 text

Procedures Back in the old days before C won, Pascal differentiated procedures and functions. SQL does the same. @chewxy on Twitter

Slide 15

Slide 15 text

Procedures Back in the old days before C won, Pascal differentiated procedures and functions. SQL does the same. Functions returned values. Procedures didn't. @chewxy on Twitter

Slide 16

Slide 16 text

Pure Functions In mathematics, a function[1] is a relation between a set of inputs and a set of permissible outputs with the property that each input is related to exactly one output. [1] – "Function" Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc. 2015- ish @chewxy on Twitter

Slide 17

Slide 17 text

Pure Functions 1 2 3 4 2 3 4 5 fn = (+1) def fn(x: int) -> int: return x + 1 @chewxy on Twitter

Slide 18

Slide 18 text

Programmers' Definition of Functions   May return a value (or not).   Return values are not mapped 1:1 with input values   Side effects!   Do this, then do that (aka procedures). @chewxy on Twitter

Slide 19

Slide 19 text

Functions and Pure Functions   Both take parameters*   Both return values*   Both are composable @chewxy on Twitter

Slide 20

Slide 20 text

Categories @chewxy on Twitter

Slide 21

Slide 21 text

Categories A collection of:   Things (values)   Have an identity function where I(x)  =  x     Arrows (functions)   Have domain (input) and range (output)   Can be composed   Obey some form of associativeness laws TL;DR – Pretty pictures to help us think about functions @chewxy on Twitter

Slide 22

Slide 22 text

Categories @chewxy on Twitter X Y Z f g gŸf

Slide 23

Slide 23 text

Categories* 1 2 3 4 2 3 4 5 f = (+1) 3 4 5 6 g = (+1) gŸf @chewxy on Twitter *The objects are expanded to show the values for visualization purposes

Slide 24

Slide 24 text

Big Deal…   Categories are cool… so what? @chewxy on Twitter

Slide 25

Slide 25 text

Big Deal…   Categories are cool… so what?   Category theory gives us the intuitions needed to think about issues that plague developers @chewxy on Twitter

Slide 26

Slide 26 text

Awkward Squad*   IO   Concurrency   Exception   FFI * With apologies to Simon Peyton Jones @chewxy on Twitter

Slide 27

Slide 27 text

Awkward Squad   IO   Permanent side effects   Concurrency   Non-determinism happens   Exception   Errors – What do we do with malformed/unexpected inputs?   FFI   Dependence on outside information Even the simple print can fail! @chewxy on Twitter

Slide 28

Slide 28 text

Print Fail! while  True:    print("Hello  World")   @chewxy on Twitter

Slide 29

Slide 29 text

Print Fail! while  True:    print("Hello  World")     >  python3  test.py  |  head  -­‐n  1   Hello  World   Traceback  (most  recent  call  last):      File  "test.py",  line  2,  in            print("Hello  World")   IOError:  [Errno  32]  Broken  pipe   @chewxy on Twitter

Slide 30

Slide 30 text

Dealing with Failure @chewxy on Twitter

Slide 31

Slide 31 text

Example def  div(num,  denom):    return  num/denom     def  sqrt(x):    return  math.sqrt(x)   @chewxy on Twitter

Slide 32

Slide 32 text

Domains (input)   The domain for div to function correctly:   For denom: Any numeric type (int, float, double … )   For num: Any numeric type   The domain for sqrt to function correctly:   For x: Any numeric type @chewxy on Twitter

Slide 33

Slide 33 text

Ranges (output)   Range of div:   Float   Range of sqrt:   Float @chewxy on Twitter

Slide 34

Slide 34 text

Example in Categories 2 4 5 0 50 25 20 ???? f = div(100, denom) 7.07 5.00 4.47 ????? g = sqrt(x) gŸf denom x @chewxy on Twitter

Slide 35

Slide 35 text

Extend the Range def  cleanDiv(num,  denom):    try:      return  num/denom    except  ZeroDivisionError:      return  None     def  cleanSqrt(x):    if  x  <  0:      return  None    return  math.sqrt(x)   @chewxy on Twitter

Slide 36

Slide 36 text

New Ranges (output)   Range of cleanDiv:   Float NoneType     Range of cleanSqrt:   Float NoneType   @chewxy on Twitter

Slide 37

Slide 37 text

With Extended Ranges 2 4 5 0 50 25 20 7.07 5.00 4.47 TypeError gŸf denom x None @chewxy on Twitter

Slide 38

Slide 38 text

Extend the Domain and Ranges def  cleanDiv(num,  denom):    try:      return  num/denom    except  ZeroDivisionError:      return  None     def  cleanSqrt(x):    #  special  check  for  x  is  None,  because  sqrt(0)  ==  0    if  x  <  0  or  x  is  None:      return  None    return  math.sqrt(x)   @chewxy on Twitter

Slide 39

Slide 39 text

Extended Domains + Ranges 2 4 5 0 50 25 20 f = cleanDiv(100, denom) 7.07 5.00 4.47 g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None

Slide 40

Slide 40 text

Tedious   To compose functions, you basically need to create new functions that have different ranges and domains   Or do it like what we do normally @chewxy on Twitter

Slide 41

Slide 41 text

How To Deal With It v  =  foo()  #  may  return  None   if  v:    v  =  bar(v)   else:    v  =  None         try:    v  =  bar(v)   except  Exception  as  pokémon:    v  =  None   @chewxy on Twitter

Slide 42

Slide 42 text

Tedious   To compose functions, you basically need to create new functions that have different ranges and domains   Or do it like what we do normally   Is there a way to generalize? @chewxy on Twitter

Slide 43

Slide 43 text

Map from This Category Numeric Float f = div(100, denom) Float g = sqrt(x) gŸf denom x @chewxy on Twitter

Slide 44

Slide 44 text

To This Category Numeric Float f = cleanDiv(100, denom) Float g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None

Slide 45

Slide 45 text

Recall Monads are just monoids in the category of endofunctors @chewxy on Twitter

Slide 46

Slide 46 text

Functor In mathematics, a functor[0] is a type of mapping between categories … [0] – More wikipedia citation?? This essay will fail if it's a uni essay @chewxy on Twitter

Slide 47

Slide 47 text

Implementing Monads Reality called. It wants its Python back @chewxy on Twitter

Slide 48

Slide 48 text

Interfaces Recap >>>  v  =  Value(1,2,3)   >>>  for  i  in  v:    print(i)   >>>  v2  =  ValueNoIter(1,2,3)   >>>  for  i  in  v2:    print(i)   Traceback  (most  recent  call  last):      File  "",  line  1,  in     TypeError:  'ValueNoIter'  object  is  not  iterable     @chewxy on Twitter

Slide 49

Slide 49 text

Interfaces Recap class  Value(object):    def  __init__(self,  *args):      self.value  =  args    def  __iter__(self):      for  v  in  self.value:        yield  v   @chewxy on Twitter

Slide 50

Slide 50 text

Interfaces*   Monads are anything with bind() and unit().   bind(): applies a function on a monadic value.   unit(): makes something a monadic value   There are different types of monads, each with different functionalities to their bind() and unit().   Monad Laws apply *Haskell typeclasses are like compile time type safe interfaces @chewxy on Twitter

Slide 51

Slide 51 text

Monadic Values Numeric Float f = cleanDiv(100, denom) Float g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None

Slide 52

Slide 52 text

Monadic Values Numeric Float f = cleanDiv(100, denom) Float g = cleanSqrt(x) gŸf denom x None None @chewxy on Twitter None

Slide 53

Slide 53 text

Maybe Monad   Deals with failure conditions   bind(): apply a function if the input is not None, else return None     unit(): return the value* * For clarity's sake, we're going to wrap the value in a Value class, which is strictly not necessary @chewxy on Twitter

Slide 54

Slide 54 text

Example Time def  bind(f):      def  inner(*args,  **kwargs):      a  =  [unbox(i)  for  i  in  args]      kw  =  {k:unbox(v)  for  k,  v  in  kwargs.items()}      if  None  in  a  or  None  in  kw:        return  None      return  f(*a,  **kw)    return  inner     def  unit(f):    def  inner(*args,  **kwargs):      return  Value(f(*args,  **kwargs))    return  inner     def  unbox(v):    try:      return  v.value    except  AttributeError:      return  v     @chewxy on Twitter

Slide 55

Slide 55 text

Example Time class  Value(object):    __slots__  =  ['value']    def  __init__(self,  v=None):      if  type(v)  is  Value:        self.value  =  v.value      else:        self.value  =  v    def  __repr__(self):      if  self.value  is  not  None:        return  "Just({})".format(self.value)      else:        return  "Nothing"   @chewxy on Twitter

Slide 56

Slide 56 text

Example Time @unit   @bind   def  div(num,  denom):    if  denom  ==  0:      return  None    return  num/denom     @unit   @bind   def  sqrt(x):    if  x  <  0:      return  None    return  math.sqrt(x)   @chewxy on Twitter

Slide 57

Slide 57 text

Sanity Check >>>  sqrt(div(100,  4))   Just(5.0)   >>>  sqrt(div(100,  0))   Nothing   >>>  div(sqrt(100),  2)   Just(5.0)   >>>  div(sqrt(100),  0)   Nothing   >>>  div(div(div(100,  0),2),10)   Nothing   @chewxy on Twitter

Slide 58

Slide 58 text

List Monad   Deals with collections   bind(): applies a function on the elements of the list   unit(): puts an element into a list @chewxy on Twitter

Slide 59

Slide 59 text

Example Time def  bind(f,  x):    return  [j  for  i  in  x  for  j  in  f(i)]     def  unit(x):    return  [x]   @chewxy on Twitter

Slide 60

Slide 60 text

Example Time def  sqrt(x):    if  x  <  0:      return  []    else:      s  =  math.sqrt(x)      return  [s,  -­‐s]     @chewxy on Twitter

Slide 61

Slide 61 text

Example Time >>>  bind(sqrt,  unit(100))   [10.0,  -­‐10.0]     >>>  bind(sqrt,  bind(sqrt,  [100]))   [3.1622776601683795,  -­‐3.1622776601683795]     >>>  bind(sqrt,  [4,  9,  16,  25,  36])   [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,   -­‐5.0,  6.0,  -­‐6.0]   @chewxy on Twitter

Slide 62

Slide 62 text

Seems Familiar… >>>  bind(sqrt,  [4,  9,  16,  25,  36])   [2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,   -­‐5.0,  6.0,  -­‐6.0]     >>>  [sqrt(x)  for  x  in  [4,  9,  16,  25,  36]]   >>>  list(map(sqrt,  [4,  9,  16,  25,  36]))   @chewxy on Twitter

Slide 63

Slide 63 text

The General Idea Monads help us compose functions that have different return types. Solution: 1.  Wrap the value with their contexts in something called a monadic value (which can be anything – you define unit()) 2.  Define a function bind() that defines out how to compose the functions that applies within that context @chewxy on Twitter

Slide 64

Slide 64 text

Opening Files def  unit(n):    return  open(n,  'r').__enter__()      #  type:    'file'     def  bind(f):    def  inner(x):      retVal  =  None      try:        retVal  =  f(x)      except  Exception  as  pokemon:              pass  #  do  something  with  pokemon        x.__exit__()      return  retVal    return  inner       @bind   def  doStuff(f):    return  f.readlines()       lines  =  doStuff(unit(fileName))     @chewxy on Twitter

Slide 65

Slide 65 text

Sounds Familiar? with  open('blah.txt',  'r')  as  f:    lines  =  doStuff(f)   @chewxy on Twitter

Slide 66

Slide 66 text

Other (Possible) Monads Twisted's Deferred (JS promises 10 years before Promises!) djangoORM's .query() and .filter()  *   Scikits-Learn's Pipeline  *   * Yet to check whether it's actually a monad @chewxy on Twitter

Slide 67

Slide 67 text

Why Does This Matter? @chewxy on Twitter

Slide 68

Slide 68 text

Why Does This Matter?   We compose functions all the time.   Programs should be easy for humans to reason about.   We haphazardly recreate solutions to solve certain classes of problems.   Understanding the underlying pattern to most of the issues programmers face makes us better (I hope?)   BYO paradigm @chewxy on Twitter

Slide 69

Slide 69 text

The End (for Realz) See you next year! @chewxy on Twitter

Slide 70

Slide 70 text

Appendix: Monad Laws #  left  bind   assert  bind(unit(v),  f)  ==  f(v)     #  right  bind   assert  bind(unit(v),  unit)  ==  unit(v)     #  associativeness   assert  bind(bind(unit(v),  f),  g)  ==  g(f(v))   @chewxy on Twitter

Slide 71

Slide 71 text

Appendix: Monad Transformers Build your own monad transformer @chewxy on Twitter

Slide 72

Slide 72 text

Appendix: Arrow Associativity @chewxy on Twitter hŸgŸf == (hŸg)Ÿf == hŸ(gŸf)