MONADS, IN MY
PYTHON?
PYCON AU 2015
@chewxy on Twitter
Slide 2
Slide 2 text
What Are Monads
Monads are just monoids in the category of endofunctors.
@chewxy on Twitter
Slide 3
Slide 3 text
Example Monad
with
open('blah.txt',
r)
as
f:
lines
=
doSomething(f)
@chewxy on Twitter
Slide 4
Slide 4 text
The End
Thank you. See you next year.
@chewxy on Twitter
Slide 5
Slide 5 text
Real Life Code
def
anomalyDetection(data,
obsPerPeriod):
if
not
obsPerPeriod:
raise
Exception("errorMessageHere")
#
other
related
checks...
decomposed
=
seasonal_decompose(data,
freq=obsPerPeriod)
lessSeasonal
=
data
–
decomposed.seasonal
lessMedian
=
lessSeasonal
–
np.median(data)
#
and
so
on
and
so
forth...
@chewxy on Twitter
Slide 6
Slide 6 text
Real Life Code
def
anomalyDetection(data,
obsPerPeriod):
if
not
obsPerPeriod:
raise
Exception("errorMessageHere")
#
other
related
checks...
decomposed
=
seasonal_decompose(data,
freq=obsPerPeriod)
lessSeasonal
=
data
–
decomposed.seasonal
lessMedian
=
lessSeasonal
–
np.median(data)
#
and
so
on
and
so
forth...
@chewxy on Twitter
Slide 7
Slide 7 text
Composition
x
=
foo()
y
=
bar(x)
z
=
baz(y)
@chewxy on Twitter
Slide 8
Slide 8 text
Composition
x
=
foo()
y
=
bar(x)
z
=
baz(y)
Equivalent to
z
=
baz(bar(foo()))
@chewxy on Twitter
Slide 9
Slide 9 text
Composition
a
=
1
+
2
+
3
b
=
1
/
(3
–
sqrt(9))
@chewxy on Twitter
Slide 10
Slide 10 text
Composition
a
=
1
+
2
+
3
b
=
1
/
(3
–
sqrt(9))
Equivalent to
a
=
add(add(1,
2),
3)
b
=
div(1,
sub(3,
sqrt(9)))
@chewxy on Twitter
Slide 11
Slide 11 text
Then the universe implodes
@chewxy on Twitter
Slide 12
Slide 12 text
Functions
We know what functions are…
@chewxy on Twitter
Slide 13
Slide 13 text
Functions
We know what functions are…
def
foo(x):
#do
something
@chewxy on Twitter
Slide 14
Slide 14 text
Procedures
Back in the old days before C won, Pascal differentiated
procedures and functions.
SQL does the same.
@chewxy on Twitter
Slide 15
Slide 15 text
Procedures
Back in the old days before C won, Pascal differentiated
procedures and functions.
SQL does the same.
Functions returned values. Procedures didn't.
@chewxy on Twitter
Slide 16
Slide 16 text
Pure Functions
In mathematics, a function[1] is a relation between a set of
inputs and a set of permissible outputs with the property
that each input is related to exactly one output.
[1] – "Function" Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc. 2015-
ish
@chewxy on Twitter
Slide 17
Slide 17 text
Pure Functions
1
2
3
4
2
3
4
5
fn = (+1)
def fn(x: int) -> int: return x + 1
@chewxy on Twitter
Slide 18
Slide 18 text
Programmers' Definition of
Functions
May return a value (or not).
Return values are not mapped 1:1 with input values
Side effects!
Do this, then do that (aka procedures).
@chewxy on Twitter
Slide 19
Slide 19 text
Functions and Pure Functions
Both take parameters*
Both return values*
Both are composable
@chewxy on Twitter
Slide 20
Slide 20 text
Categories
@chewxy on Twitter
Slide 21
Slide 21 text
Categories
A collection of:
Things (values)
Have an identity function where I(x)
=
x
Arrows (functions)
Have domain (input) and range (output)
Can be composed
Obey some form of associativeness laws
TL;DR – Pretty pictures to help us think about functions
@chewxy on Twitter
Slide 22
Slide 22 text
Categories
@chewxy on Twitter
X Y
Z
f
g
gf
Slide 23
Slide 23 text
Categories*
1
2
3
4
2
3
4
5
f = (+1)
3
4
5
6
g = (+1)
gf
@chewxy on Twitter
*The objects are expanded to show the values for visualization purposes
Slide 24
Slide 24 text
Big Deal…
Categories are cool… so what?
@chewxy on Twitter
Slide 25
Slide 25 text
Big Deal…
Categories are cool… so what?
Category theory gives us the intuitions needed to think
about issues that plague developers
@chewxy on Twitter
Slide 26
Slide 26 text
Awkward Squad*
IO
Concurrency
Exception
FFI
* With apologies to Simon Peyton Jones
@chewxy on Twitter
Slide 27
Slide 27 text
Awkward Squad
IO
Permanent side effects
Concurrency
Non-determinism happens
Exception
Errors – What do we do with malformed/unexpected
inputs?
FFI
Dependence on outside information
Even the simple print can fail!
@chewxy on Twitter
Slide 28
Slide 28 text
Print Fail!
while
True:
print("Hello
World")
@chewxy on Twitter
Slide 29
Slide 29 text
Print Fail!
while
True:
print("Hello
World")
>
python3
test.py
|
head
-‐n
1
Hello
World
Traceback
(most
recent
call
last):
File
"test.py",
line
2,
in
print("Hello
World")
IOError:
[Errno
32]
Broken
pipe
@chewxy on Twitter
Slide 30
Slide 30 text
Dealing with Failure
@chewxy on Twitter
Slide 31
Slide 31 text
Example
def
div(num,
denom):
return
num/denom
def
sqrt(x):
return
math.sqrt(x)
@chewxy on Twitter
Slide 32
Slide 32 text
Domains (input)
The domain for div to function correctly:
For denom: Any numeric type (int, float, double … )
For num: Any numeric type
The domain for sqrt to function correctly:
For x: Any numeric type
@chewxy on Twitter
Slide 33
Slide 33 text
Ranges (output)
Range of div:
Float
Range of sqrt:
Float
@chewxy on Twitter
Slide 34
Slide 34 text
Example in Categories
2
4
5
0
50
25
20
????
f = div(100, denom)
7.07
5.00
4.47
?????
g = sqrt(x)
gf
denom x
@chewxy on Twitter
Slide 35
Slide 35 text
Extend the Range
def
cleanDiv(num,
denom):
try:
return
num/denom
except
ZeroDivisionError:
return
None
def
cleanSqrt(x):
if
x
<
0:
return
None
return
math.sqrt(x)
@chewxy on Twitter
Slide 36
Slide 36 text
New Ranges (output)
Range of cleanDiv:
Float
NoneType
Range of cleanSqrt:
Float
NoneType
@chewxy on Twitter
Slide 37
Slide 37 text
With Extended Ranges
2
4
5
0
50
25
20 7.07
5.00
4.47
TypeError
gf
denom x
None
@chewxy on Twitter
Slide 38
Slide 38 text
Extend the Domain and Ranges
def
cleanDiv(num,
denom):
try:
return
num/denom
except
ZeroDivisionError:
return
None
def
cleanSqrt(x):
#
special
check
for
x
is
None,
because
sqrt(0)
==
0
if
x
<
0
or
x
is
None:
return
None
return
math.sqrt(x)
@chewxy on Twitter
Slide 39
Slide 39 text
Extended Domains + Ranges
2
4
5
0
50
25
20
f = cleanDiv(100, denom)
7.07
5.00
4.47
g = cleanSqrt(x)
gf
denom x
None None
@chewxy on Twitter
None
Slide 40
Slide 40 text
Tedious
To compose functions, you basically need to create new
functions that have different ranges and domains
Or do it like what we do normally
@chewxy on Twitter
Slide 41
Slide 41 text
How To Deal With It
v
=
foo()
#
may
return
None
if
v:
v
=
bar(v)
else:
v
=
None
try:
v
=
bar(v)
except
Exception
as
pokémon:
v
=
None
@chewxy on Twitter
Slide 42
Slide 42 text
Tedious
To compose functions, you basically need to create new
functions that have different ranges and domains
Or do it like what we do normally
Is there a way to generalize?
@chewxy on Twitter
Slide 43
Slide 43 text
Map from This Category
Numeric Float
f = div(100, denom)
Float
g = sqrt(x)
gf
denom x
@chewxy on Twitter
Slide 44
Slide 44 text
To This Category
Numeric Float
f = cleanDiv(100, denom)
Float
g = cleanSqrt(x)
gf
denom x
None None
@chewxy on Twitter
None
Slide 45
Slide 45 text
Recall
Monads are just monoids in the category of endofunctors
@chewxy on Twitter
Slide 46
Slide 46 text
Functor
In mathematics, a functor[0] is a type of mapping between
categories …
[0] – More wikipedia citation?? This essay will fail if it's a uni essay
@chewxy on Twitter
Slide 47
Slide 47 text
Implementing Monads
Reality called. It wants its Python back
@chewxy on Twitter
Slide 48
Slide 48 text
Interfaces Recap
>>>
v
=
Value(1,2,3)
>>>
for
i
in
v:
print(i)
>>>
v2
=
ValueNoIter(1,2,3)
>>>
for
i
in
v2:
print(i)
Traceback
(most
recent
call
last):
File
"",
line
1,
in
TypeError:
'ValueNoIter'
object
is
not
iterable
@chewxy on Twitter
Slide 49
Slide 49 text
Interfaces Recap
class
Value(object):
def
__init__(self,
*args):
self.value
=
args
def
__iter__(self):
for
v
in
self.value:
yield
v
@chewxy on Twitter
Slide 50
Slide 50 text
Interfaces*
Monads are anything with bind() and unit().
bind(): applies a function on a monadic value.
unit(): makes something a monadic value
There are different types of monads, each with different
functionalities to their bind() and unit().
Monad Laws apply
*Haskell typeclasses are like compile time type safe interfaces
@chewxy on Twitter
Slide 51
Slide 51 text
Monadic Values
Numeric Float
f = cleanDiv(100, denom)
Float
g = cleanSqrt(x)
gf
denom x
None None
@chewxy on Twitter
None
Slide 52
Slide 52 text
Monadic Values
Numeric Float
f = cleanDiv(100, denom)
Float
g = cleanSqrt(x)
gf
denom x
None None
@chewxy on Twitter
None
Slide 53
Slide 53 text
Maybe Monad
Deals with failure conditions
bind(): apply a function if the input is not None, else return
None
unit(): return the value*
* For clarity's sake, we're going to wrap the value in a Value class, which is strictly not necessary
@chewxy on Twitter
Slide 54
Slide 54 text
Example Time
def
bind(f):
def
inner(*args,
**kwargs):
a
=
[unbox(i)
for
i
in
args]
kw
=
{k:unbox(v)
for
k,
v
in
kwargs.items()}
if
None
in
a
or
None
in
kw:
return
None
return
f(*a,
**kw)
return
inner
def
unit(f):
def
inner(*args,
**kwargs):
return
Value(f(*args,
**kwargs))
return
inner
def
unbox(v):
try:
return
v.value
except
AttributeError:
return
v
@chewxy on Twitter
Slide 55
Slide 55 text
Example Time
class
Value(object):
__slots__
=
['value']
def
__init__(self,
v=None):
if
type(v)
is
Value:
self.value
=
v.value
else:
self.value
=
v
def
__repr__(self):
if
self.value
is
not
None:
return
"Just({})".format(self.value)
else:
return
"Nothing"
@chewxy on Twitter
Slide 56
Slide 56 text
Example Time
@unit
@bind
def
div(num,
denom):
if
denom
==
0:
return
None
return
num/denom
@unit
@bind
def
sqrt(x):
if
x
<
0:
return
None
return
math.sqrt(x)
@chewxy on Twitter
Seems Familiar…
>>>
bind(sqrt,
[4,
9,
16,
25,
36])
[2.0,
-‐2.0,
3.0,
-‐3.0,
4.0,
-‐4.0,
5.0,
-‐5.0,
6.0,
-‐6.0]
>>>
[sqrt(x)
for
x
in
[4,
9,
16,
25,
36]]
>>>
list(map(sqrt,
[4,
9,
16,
25,
36]))
@chewxy on Twitter
Slide 63
Slide 63 text
The General Idea
Monads help us compose functions that have different return
types.
Solution:
1. Wrap the value with their contexts in something called a
monadic value (which can be anything – you define
unit())
2. Define a function bind() that defines out how to compose
the functions that applies within that context
@chewxy on Twitter
Sounds Familiar?
with
open('blah.txt',
'r')
as
f:
lines
=
doStuff(f)
@chewxy on Twitter
Slide 66
Slide 66 text
Other (Possible) Monads
Twisted's Deferred (JS promises 10 years before
Promises!)
djangoORM's .query() and .filter()
*
Scikits-Learn's Pipeline
*
* Yet to check whether it's actually a monad
@chewxy on Twitter
Slide 67
Slide 67 text
Why Does This Matter?
@chewxy on Twitter
Slide 68
Slide 68 text
Why Does This Matter?
We compose functions all the time.
Programs should be easy for humans to reason about.
We haphazardly recreate solutions to solve certain
classes of problems.
Understanding the underlying pattern to most of the
issues programmers face makes us better (I hope?)
BYO paradigm
@chewxy on Twitter
Slide 69
Slide 69 text
The End (for Realz)
See you next year!
@chewxy on Twitter
Slide 70
Slide 70 text
Appendix: Monad Laws
#
left
bind
assert
bind(unit(v),
f)
==
f(v)
#
right
bind
assert
bind(unit(v),
unit)
==
unit(v)
#
associativeness
assert
bind(bind(unit(v),
f),
g)
==
g(f(v))
@chewxy on Twitter
Slide 71
Slide 71 text
Appendix: Monad Transformers
Build your own monad transformer
@chewxy on Twitter