Xuanyi
August 01, 2015
1.8k

# Monads, In My Python? (PyConAU 2015)

The associated talk can be found here:
https://www.youtube.com/watch?v=WNwV3wR4JjA

August 01, 2015

## Transcript

1. MONADS, IN MY
PYTHON?
PYCON AU 2015
@chewxy on Twitter

2. What Are Monads
Monads are just monoids in the category of endofunctors.
@chewxy on Twitter

3. Example Monad
with  open('blah.txt',  r)  as  f:
lines  =  doSomething(f)
@chewxy on Twitter

4. The End
Thank you. See you next year.
@chewxy on Twitter

5. Real Life Code
def  anomalyDetection(data,  obsPerPeriod):

if  not  obsPerPeriod:
raise  Exception("errorMessageHere")

#  other  related  checks...

decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)

lessSeasonal  =  data  –  decomposed.seasonal

lessMedian  =  lessSeasonal  –  np.median(data)

#  and  so  on  and  so  forth...
@chewxy on Twitter

6. Real Life Code
def  anomalyDetection(data,  obsPerPeriod):

if  not  obsPerPeriod:
raise  Exception("errorMessageHere")

#  other  related  checks...

decomposed  =  seasonal_decompose(data,  freq=obsPerPeriod)

lessSeasonal  =  data  –  decomposed.seasonal

lessMedian  =  lessSeasonal  –  np.median(data)

#  and  so  on  and  so  forth...
@chewxy on Twitter

7. Composition
x  =  foo()
y  =  bar(x)
z  =  baz(y)

@chewxy on Twitter

8. Composition
x  =  foo()
y  =  bar(x)
z  =  baz(y)
Equivalent to
z  =  baz(bar(foo()))
@chewxy on Twitter

9. Composition
a  =  1  +  2  +  3
b  =  1  /  (3  –  sqrt(9))

@chewxy on Twitter

10. Composition
a  =  1  +  2  +  3
b  =  1  /  (3  –  sqrt(9))
Equivalent to
a  =  add(add(1,  2),  3)
b  =  div(1,  sub(3,  sqrt(9)))
@chewxy on Twitter

11. Then the universe implodes
@chewxy on Twitter

12. Functions
We know what functions are…
@chewxy on Twitter

13. Functions
We know what functions are…
def  foo(x):
#do  something
@chewxy on Twitter

14. Procedures
Back in the old days before C won, Pascal differentiated
procedures and functions.
SQL does the same.
@chewxy on Twitter

15. Procedures
Back in the old days before C won, Pascal differentiated
procedures and functions.
SQL does the same.
Functions returned values. Procedures didn't.
@chewxy on Twitter

16. Pure Functions
In mathematics, a function[1] is a relation between a set of
inputs and a set of permissible outputs with the property
that each input is related to exactly one output.
[1] – "Function" Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc. 2015-
ish
@chewxy on Twitter

17. Pure Functions
1
2
3
4
2
3
4
5
fn = (+1)
def fn(x: int) -> int: return x + 1
@chewxy on Twitter

18. Programmers' Definition of
Functions
May return a value (or not).
Return values are not mapped 1:1 with input values
Side effects!
Do this, then do that (aka procedures).
@chewxy on Twitter

19. Functions and Pure Functions
Both take parameters*
Both return values*
Both are composable
@chewxy on Twitter

20. Categories
@chewxy on Twitter

21. Categories
A collection of:
Things (values)
Have an identity function where I(x)  =  x
Arrows (functions)
Have domain (input) and range (output)
Can be composed
Obey some form of associativeness laws
TL;DR – Pretty pictures to help us think about functions
@chewxy on Twitter

22. Categories
@chewxy on Twitter
X Y
Z
f
g
gf

23. Categories*
1
2
3
4
2
3
4
5
f = (+1)
3
4
5
6
g = (+1)
gf
@chewxy on Twitter
*The objects are expanded to show the values for visualization purposes

24. Big Deal…
Categories are cool… so what?
@chewxy on Twitter

25. Big Deal…
Categories are cool… so what?
Category theory gives us the intuitions needed to think
about issues that plague developers
@chewxy on Twitter

26. Awkward Squad*
IO
Concurrency
Exception
FFI
* With apologies to Simon Peyton Jones
@chewxy on Twitter

27. Awkward Squad
IO
Permanent side effects
Concurrency
Non-determinism happens
Exception
Errors – What do we do with malformed/unexpected
inputs?
FFI
Dependence on outside information
Even the simple print can fail!
@chewxy on Twitter

28. Print Fail!
while  True:
print("Hello  World")
@chewxy on Twitter

29. Print Fail!
while  True:
print("Hello  World")

>  python3  test.py  |  head  -­‐n  1
Hello  World
Traceback  (most  recent  call  last):
File  "test.py",  line  2,  in
print("Hello  World")
IOError:  [Errno  32]  Broken  pipe
@chewxy on Twitter

30. Dealing with Failure
@chewxy on Twitter

31. Example
def  div(num,  denom):
return  num/denom

def  sqrt(x):
return  math.sqrt(x)
@chewxy on Twitter

32. Domains (input)
The domain for div to function correctly:
For denom: Any numeric type (int, float, double … )
For num: Any numeric type
The domain for sqrt to function correctly:
For x: Any numeric type
@chewxy on Twitter

33. Ranges (output)
Range of div:
Float
Range of sqrt:
Float
@chewxy on Twitter

34. Example in Categories
2
4
5
0
50
25
20
????
f = div(100, denom)
7.07
5.00
4.47
?????
g = sqrt(x)
gf
denom x
@chewxy on Twitter

35. Extend the Range
def  cleanDiv(num,  denom):
try:
return  num/denom
except  ZeroDivisionError:
return  None

def  cleanSqrt(x):
if  x  <  0:
return  None
return  math.sqrt(x)
@chewxy on Twitter

36. New Ranges (output)
Range of cleanDiv:
Float
NoneType
Range of cleanSqrt:
Float
NoneType
@chewxy on Twitter

37. With Extended Ranges
2
4
5
0
50
25
20 7.07
5.00
4.47
TypeError
gf
denom x
None
@chewxy on Twitter

38. Extend the Domain and Ranges
def  cleanDiv(num,  denom):
try:
return  num/denom
except  ZeroDivisionError:
return  None

def  cleanSqrt(x):
#  special  check  for  x  is  None,  because  sqrt(0)  ==  0
if  x  <  0  or  x  is  None:
return  None
return  math.sqrt(x)
@chewxy on Twitter

39. Extended Domains + Ranges
2
4
5
0
50
25
20
f = cleanDiv(100, denom)
7.07
5.00
4.47
g = cleanSqrt(x)
gf
denom x
None None
@chewxy on Twitter
None

40. Tedious
To compose functions, you basically need to create new
functions that have different ranges and domains
Or do it like what we do normally
@chewxy on Twitter

41. How To Deal With It
v  =  foo()  #  may  return  None
if  v:
v  =  bar(v)
else:
v  =  None

try:
v  =  bar(v)
except  Exception  as  pokémon:
v  =  None
@chewxy on Twitter

42. Tedious
To compose functions, you basically need to create new
functions that have different ranges and domains
Or do it like what we do normally
Is there a way to generalize?
@chewxy on Twitter

43. Map from This Category
Numeric Float
f = div(100, denom)
Float
g = sqrt(x)
gf
denom x
@chewxy on Twitter

44. To This Category
Numeric Float
f = cleanDiv(100, denom)
Float
g = cleanSqrt(x)
gf
denom x
None None
@chewxy on Twitter
None

45. Recall
Monads are just monoids in the category of endofunctors
@chewxy on Twitter

46. Functor
In mathematics, a functor[0] is a type of mapping between
categories …
[0] – More wikipedia citation?? This essay will fail if it's a uni essay
@chewxy on Twitter

47. Implementing Monads
Reality called. It wants its Python back
@chewxy on Twitter

48. Interfaces Recap
>>>  v  =  Value(1,2,3)
>>>  for  i  in  v:
print(i)
>>>  v2  =  ValueNoIter(1,2,3)
>>>  for  i  in  v2:
print(i)
Traceback  (most  recent  call  last):
File  "",  line  1,  in
TypeError:  'ValueNoIter'  object  is  not  iterable

@chewxy on Twitter

49. Interfaces Recap
class  Value(object):
def  __init__(self,  *args):
self.value  =  args
def  __iter__(self):
for  v  in  self.value:
yield  v
@chewxy on Twitter

50. Interfaces*
Monads are anything with bind() and unit().
bind(): applies a function on a monadic value.
unit(): makes something a monadic value
There are different types of monads, each with different
functionalities to their bind() and unit().
Monad Laws apply
*Haskell typeclasses are like compile time type safe interfaces
@chewxy on Twitter

51. Monadic Values
Numeric Float
f = cleanDiv(100, denom)
Float
g = cleanSqrt(x)
gf
denom x
None None
@chewxy on Twitter
None

52. Monadic Values
Numeric Float
f = cleanDiv(100, denom)
Float
g = cleanSqrt(x)
gf
denom x
None None
@chewxy on Twitter
None

53. Maybe Monad
Deals with failure conditions
bind(): apply a function if the input is not None, else return
None
unit(): return the value*
* For clarity's sake, we're going to wrap the value in a Value class, which is strictly not necessary
@chewxy on Twitter

54. Example Time
def  bind(f):
def  inner(*args,  **kwargs):
a  =  [unbox(i)  for  i  in  args]
kw  =  {k:unbox(v)  for  k,  v  in  kwargs.items()}
if  None  in  a  or  None  in  kw:
return  None
return  f(*a,  **kw)
return  inner

def  unit(f):
def  inner(*args,  **kwargs):
return  Value(f(*args,  **kwargs))
return  inner

def  unbox(v):
try:
return  v.value
except  AttributeError:
return  v

@chewxy on Twitter

55. Example Time
class  Value(object):
__slots__  =  ['value']
def  __init__(self,  v=None):
if  type(v)  is  Value:
self.value  =  v.value
else:
self.value  =  v
def  __repr__(self):
if  self.value  is  not  None:
return  "Just({})".format(self.value)
else:
return  "Nothing"
@chewxy on Twitter

56. Example Time
@unit
@bind
def  div(num,  denom):
if  denom  ==  0:
return  None
return  num/denom

@unit
@bind
def  sqrt(x):
if  x  <  0:
return  None
return  math.sqrt(x)
@chewxy on Twitter

57. Sanity Check
>>>  sqrt(div(100,  4))
Just(5.0)
>>>  sqrt(div(100,  0))
Nothing
>>>  div(sqrt(100),  2)
Just(5.0)
>>>  div(sqrt(100),  0)
Nothing
>>>  div(div(div(100,  0),2),10)
Nothing
@chewxy on Twitter

58. List Monad
Deals with collections
bind(): applies a function on the elements of the list
unit(): puts an element into a list
@chewxy on Twitter

59. Example Time
def  bind(f,  x):
return  [j  for  i  in  x  for  j  in  f(i)]

def  unit(x):
return  [x]
@chewxy on Twitter

60. Example Time
def  sqrt(x):
if  x  <  0:
return  []
else:
s  =  math.sqrt(x)
return  [s,  -­‐s]

@chewxy on Twitter

61. Example Time
>>>  bind(sqrt,  unit(100))
[10.0,  -­‐10.0]

>>>  bind(sqrt,  bind(sqrt,  [100]))
[3.1622776601683795,  -­‐3.1622776601683795]

>>>  bind(sqrt,  [4,  9,  16,  25,  36])
[2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,
-­‐5.0,  6.0,  -­‐6.0]
@chewxy on Twitter

62. Seems Familiar…
>>>  bind(sqrt,  [4,  9,  16,  25,  36])
[2.0,  -­‐2.0,  3.0,  -­‐3.0,  4.0,  -­‐4.0,  5.0,
-­‐5.0,  6.0,  -­‐6.0]

>>>  [sqrt(x)  for  x  in  [4,  9,  16,  25,  36]]
>>>  list(map(sqrt,  [4,  9,  16,  25,  36]))
@chewxy on Twitter

63. The General Idea
Monads help us compose functions that have different return
types.
Solution:
1.  Wrap the value with their contexts in something called a
monadic value (which can be anything – you define
unit())
2.  Define a function bind() that defines out how to compose
the functions that applies within that context
@chewxy on Twitter

64. Opening Files
def  unit(n):
return  open(n,  'r').__enter__()      #  type:    'file'

def  bind(f):
def  inner(x):
retVal  =  None
try:
retVal  =  f(x)
except  Exception  as  pokemon:
pass  #  do  something  with  pokemon
x.__exit__()
return  retVal
return  inner

@bind
def  doStuff(f):
return  f.readlines()

lines  =  doStuff(unit(fileName))

@chewxy on Twitter

65. Sounds Familiar?
with  open('blah.txt',  'r')  as  f:
lines  =  doStuff(f)
@chewxy on Twitter

66. Other (Possible) Monads
Twisted's Deferred (JS promises 10 years before
Promises!)
djangoORM's .query() and .filter()  *
Scikits-Learn's Pipeline  *
* Yet to check whether it's actually a monad
@chewxy on Twitter

67. Why Does This Matter?
@chewxy on Twitter

68. Why Does This Matter?
We compose functions all the time.
Programs should be easy for humans to reason about.
We haphazardly recreate solutions to solve certain
classes of problems.
Understanding the underlying pattern to most of the
issues programmers face makes us better (I hope?)
BYO paradigm
@chewxy on Twitter

69. The End (for Realz)
See you next year!
@chewxy on Twitter

70. Appendix: Monad Laws
#  left  bind
assert  bind(unit(v),  f)  ==  f(v)

#  right  bind
assert  bind(unit(v),  unit)  ==  unit(v)

#  associativeness
assert  bind(bind(unit(v),  f),  g)  ==  g(f(v))
@chewxy on Twitter

71. Appendix: Monad Transformers
Build your own monad transformer
@chewxy on Twitter

72. Appendix: Arrow Associativity
@chewxy on Twitter
hgf == (hg)f == h(gf)