Slide 1
Slide 1 text
Getting deep in python features
Python Speleology
Slide 2
Slide 2 text
1. The Zen of Python
Slide 3
Slide 3 text
The Zen of Python
○ Beautiful is better than ugly.
○ Explicit is better than implicit.
○ Simple is better than complex.
○ Complex is better than complicated.
○ Flat is better than nested.
○ Sparse is better than dense.
○ Readability counts.
○ Special cases aren't special enough to break the rules.
○ Although practicality beats purity.
○ Errors should never pass silently.
○ Unless explicitly silenced.
○ In the face of ambiguity, refuse the temptation to guess.
○ There should be one -and preferably only one- obvious way to do it.
○ Although that way may not be obvious at first unless you're Dutch.
○ Now is better than never.
○ Although never is often better than right now.
○ If the implementation is hard to explain, it's a bad idea.
○ If the implementation is easy to explain, it may be a good idea.
○ Namespaces are one honking great idea -let's do more of those!
Slide 4
Slide 4 text
2. Types and objects
Slide 5
Slide 5 text
Types and objects: To be or not to be
a = 256
b = 256
print (a is b)
a = 257
b = 257
print (a is b)
Slide 6
Slide 6 text
Types and objects: To be or not to be
a = 256
b = 256
print (a is b)
a = 257
b = 257
print (a is b)
True
False
Slide 7
Slide 7 text
Types and objects: To be or not to be
a = 256
b = 256
print (a is b)
a = 257
b = 257
print (a is b)
True
False
a = 256
b = 256
print id(a), id(b)
a = 257
b = 257
print id(a), id(b)
22036112 22036112
22363568 22363640
Slide 8
Slide 8 text
Types and objects: To be or not to be
a = 256
b = 256
print (a is b)
a = 257
b = 257
print (a is b)
True
False
a = 256
b = 256
print id(a), id(b)
a = 257
b = 257
print id(a), id(b)
Slide 9
Slide 9 text
Types and objects: To be or not to be
a = 256
b = 256
print (a is b)
a = 257
b = 257
print (a is b)
True
False
a = 256
b = 256
print id(a), id(b)
a = 257
b = 257
print id(a), id(b)
22036112 22036112
22363568 22363640
Slide 10
Slide 10 text
Types and objects: functions are objects
def x(f):
return f
def y(f):
return x
print x(y)(8)(0)
Slide 11
Slide 11 text
Types and objects: functions are objects
def x(f):
return f
def y(f):
return x
print x(y)(8)(0)
0
Slide 12
Slide 12 text
Types and objects: functions are objects
def x(i):
if x.enabled:
return i
else:
return "disabled"
x.enabled = True
print x(8)
Slide 13
Slide 13 text
Types and objects: functions are objects
def x(i):
if x.enabled:
return i
else:
return "disabled"
x.enabled = True
print x(8)
What happened if not set x.enabled?
8
Slide 14
Slide 14 text
Types and objects: star operator
args = (3 , 6)
print range(*args)
args = { "name": "example" }
def f(name):
print name
f(**args)
def f(*args, **kwargs):
print args
print kwargs
Slide 15
Slide 15 text
Types and objects: star operator
args = (3 , 6)
print range(*args)
args = { "name": "example" }
def f(name):
print name
f(**args)
def f(*args, **kwargs):
print args
print kwargs
[3, 4, 5]
example
(1, 2, 3)
{'name': 'example'}
Slide 16
Slide 16 text
3. Reflection
Slide 17
Slide 17 text
Reflection: get & set
class Example(object):
num = 10
x = Example
dir(x)
hasattr(x, "num") == True
getattr(x, "num", 0) == 10
setattr(x, "num", 20)
Slide 18
Slide 18 text
Reflection: local & global
globals()
Return a dictionary representing the current global symbol table. This is always the dictionary of the
current module (inside a function or method, this is the module where it is defined, not the module from
which it is called).
locals()
Update and return a dictionary representing the current local symbol table. Free
variables are returned by locals() when it is called in function blocks, but not in
class blocks.
Slide 19
Slide 19 text
Reflection: __magic__
__name__
This is the name of the function. This only have a meaningful value is the function is defined with “def”.
__class__
This is a reference to the class a method belongs to.
__code__
This is a reference to the code object used in the implementation of python.
Slide 20
Slide 20 text
Reflection: inspector
from inspect import getcomments
# This is a comment
def f(x):
print x
print getcomments(f)
Slide 21
Slide 21 text
Reflection: inspector
from inspect import getcomments
# This is a comment
def f(x):
print x
print getcomments(f)
# This is a comment
Slide 22
Slide 22 text
Reflection: inspector
from inspect import getsource
# This is a comment
def f(x):
print x
print getsource(f)
Slide 23
Slide 23 text
Reflection: inspector
from inspect import getsource
# This is a comment
def f(x):
print x
print getsource(f)
def f(x):
print x
Slide 24
Slide 24 text
Reflection: let's more tricky
def f(x):
print x
print f.__code__.co_code
Slide 25
Slide 25 text
Reflection: let's more tricky
def f(x):
print x
print f.__code__.co_code
'd\x01\x00GHd\x00\x00S'
Slide 26
Slide 26 text
Reflection: let's more tricky
def f(x):
print x
print f.__code__.co_code
'd\x01\x00GHd\x00\x00S'
YEEES!!! THE BYTECODE!!!
Slide 27
Slide 27 text
4. Context manager
Slide 28
Slide 28 text
Context manager: in the beginning...
item = Item()
try:
item.open()
item.do()
finally:
item.close()
Slide 29
Slide 29 text
Context manager: nowadays...
with Item() as item:
item.do
class Item(object):
def __enter__(self):
self.open()
return self
def __exit__(self,exc_type,exc_value,exc_t):
self.close()
...
Slide 30
Slide 30 text
Context manager: the real world
with file("/tmp/test", "w") as f:
f.write("hello world")
with lock():
# do some concurrent
with sudo("root"):
# do some as root
Slide 31
Slide 31 text
5. Decorations
Slide 32
Slide 32 text
Decorations: bold & italic
def makebold(fn):
def wrapped():
return "" + fn() + ""
return wrapped
def makeitalic(fn):
def wrapped():
return "" + fn() + ""
return wrapped
@makebold
@makeitalic
def hello():
return "hello world"
Slide 33
Slide 33 text
Decorations: bold & italic
def makebold(fn):
def wrapped():
return "" + fn() + ""
return wrapped
def makeitalic(fn):
def wrapped():
return "" + fn() + ""
return wrapped
@makebold
@makeitalic
def hello():
return "hello world"
hello world
Slide 34
Slide 34 text
Decorations: complex decor
def makestyle(arg):
def decorator(f):
def wrapper(*args, **kw):
return "<" + arg + ">" + f() + "" + arg + ">"
return wrapper
return decorator
@makestyle("b")
def hello():
return "hello world"
print hello()
Slide 35
Slide 35 text
Decorations: syntax sugar
def makebold(fn):
def wrapped():
return "" + fn() + ""
return wrapped
def makestyle(arg):
def decorator(f):
def wrapper(*args, **kw):
return "<" + arg + ">" + f() + "" + arg + ">"
return wrapper
return decorator
makebold(hello)
Slide 36
Slide 36 text
6. Iterations
Slide 37
Slide 37 text
Iterations: comprehesions
squares = []
for x in range(10):
squares.append(x**2)
squares = [x**2 for x in range(10)]
[(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
{ (k,v) for k,v in [(1,2)] }
Slide 38
Slide 38 text
Iterations: comprehesions
squares = []
for x in range(10):
squares.append(x**2)
squares = [x**2 for x in range(10)]
[(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
{ (k,v) for k,v in [(1,2)] }
SET NOT DICT!
Slide 39
Slide 39 text
Iterations: comprehesions
squares = []
for x in range(10):
squares.append(x**2)
squares = [x**2 for x in range(10)]
[(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
{ (k,v) for k,v in [(1,2)] }
SET NOT DICT!
{ k:v for k,v in [(1,2)] }
Slide 40
Slide 40 text
Iterations: co-routine
def countdown(n):
print "Counting down from", n
while n > 0:
yield n
n -= 1
print "Done counting down"
for i in countdown(10):
print i
Slide 41
Slide 41 text
Iterations: co-routine
def countdown(n):
print "Counting down from", n
while n > 0:
yield n
n -= 1
print "Done counting down"
for i in countdown(10):
print i Counting down from 10
10
9
8
7
6
5
4
3
2
1
Done counting down
Slide 42
Slide 42 text
Iterations: co-routine
def countdown(n):
print "Counting down from", n
while n > 0:
yield n
n -= 1
print "Done counting down"
print countdown(10)
Slide 43
Slide 43 text
7. Overloading
Slide 44
Slide 44 text
Overloading: binary operations
__add__(self, other) x + y
__sub__(self, other) x - y
__mul__(self, other) x * y
__div__(self, other) x / y
__pow__(self, other) x ** y
Slide 45
Slide 45 text
Overloading: binary operations
__radd__(self, other) y + x
__rsub__(self, other) y - x
__rmul__(self, other) y * x
__rdiv__(self, other) y / x
__rpow__(self, other) y ** x
Slide 46
Slide 46 text
Overloading: binary operations
__radd__(self, other) 1 + x
__rsub__(self, other) 1 - x
__rmul__(self, other) 1 * x
__rdiv__(self, other) 1 / x
__rpow__(self, other) 1 ** x
Slide 47
Slide 47 text
Overloading: binary operations
__iadd__(self, other) x+=y
__isub__(self, other) x-=y
__imul__(self, other) x*=y
__idiv__(self, other) x/=y
__ipow__(self, other) x**=y
Slide 48
Slide 48 text
Overloading: unary operations
__neg__(self) -x
__pos__(self) +x
__abs__(self) abs(x)
__invert__(self) ~x
Slide 49
Slide 49 text
Overloading: conversion operations
__int__(self) int(x)
__float__(self) float(x)
__complex__(self) complex(x)
__str__(self) str(x)
__nonzero__(self) bool(x)
__unicode__(self) unicode(x)
Slide 50
Slide 50 text
Overloading: comparison operations
__eq__(self, other) x == y
__lt__(self, other) x < y
__le__(self, other) x <= y
__gt__(self, other) x > y
__ge__(self, other) x >= y
__ne__(self, other) x != y
Slide 51
Slide 51 text
Overloading: containers
__contains__(self, other) y in x
__getitem__(self, other) x[y]
__setitem__(self, other,value) x[y] = z
__delitem__(self, other) del x[y]
__len__(self) len(x)
__reversed__(self) reversed(x)
__iter__(self) iter(x)
Slide 52
Slide 52 text
8. The Class Factory
Slide 53
Slide 53 text
The Class Factory: class & objects
class Example(object):
attribute = "this is a class attribute"
def __init__(self):
self.attribute = "this is an obj attr override class one"
self.another = "this is another obj attr, no class"
print Example.attribute
print Example().attribute
print Example().another
print Example.another
Slide 54
Slide 54 text
The Class Factory: class & objects
class Example(object):
attribute = "this is a class attribute"
def __init__(self):
self.attribute = "this is an obj attr override class one"
self.another = "this is another obj attr, no class"
print Example.attribute
print Example().attribute
print Example().another
print Example.another
this is a class attribute
this is an object attribute and override class one
this is another object attribute, no class
Traceback (most recent call last):
Line 11, in
print Example.another
AttributeError: type object 'Example' has no attribute 'another'
Slide 55
Slide 55 text
The Class Factory: set & get
class Example(object):
def __init__(self):
self._name = x
@property
def name(self):
return self._name
@name.setter
def name(self, value):
self._name = value
Slide 56
Slide 56 text
The Class Factory: @property abuse
class Example(object):
def __init__(self, host, port):
self.host = host
self.port = port
@property
def connect(self):
lib.connect(self.host, self.port)
Slide 57
Slide 57 text
The Class Factory: @property abuse
class Example(object):
def __init__(self, host, port):
self.host = host
self.port = port
@property
def connect(self):
lib.connect(self.host, self.port)
NEVER TYPE METHODS AS
PROPERTIES
Slide 58
Slide 58 text
The Class Factory: methods and more methods
@staticmethod
Nothing more than a function defined inside a class.
It is callable without instantiating the class first. It’s
definition is immutable via inheritance.
@classmethod
Also callable without instantiating the class, but its
definition follows Sub class, not Parent class, via
inheritance. That’s because the first argument for
@classmethod function must always be cls (class).
Slide 59
Slide 59 text
The Class Factory: methods and more methods
@staticmethod
Nothing more than a function defined inside a class.
It is callable without instantiating the class first. It’s
definition is immutable via inheritance.
@classmethod
Also callable without instantiating the class, but its
definition follows Sub class, not Parent class, via
inheritance. That’s because the first argument for
@classmethod function must always be cls (class).
@staticmethod
def static_method():
print "I do not receive nothing :("
@classmethod
def class_method(cls):
print "I'm a class %s" % str(cls)
Slide 60
Slide 60 text
The Class Factory: methods and more methods
class Example(object):
def __init__(self, name):
self.name = name
@classmethod
def class_method(cls, name):
return cls(name)
x = Example.class_method("example")
print x
Slide 61
Slide 61 text
The Class Factory: methods and more methods
class Example(object):
def __init__(self, name):
self.name = name
@classmethod
def class_method(cls, name):
return cls(name)
x = Example.class_method("example")
print x
<__main__.Example object at 0x40358b2c>
Slide 62
Slide 62 text
The Class Factory: children and parents
class Example(object):
def __init__(self, name):
self.name = name
def do_something(self):
raise NotImplementedError()
class ChildExample(Example):
def do_something(self):
print self.name
Slide 63
Slide 63 text
The Class Factory: children and parents
class ExampleA(object):
def __init__(self, name):
self.name = name
def do_something(self):
raise NotImplementedError()
class ExampleB(object):
def do_otherthing(self):
raise NotImplementedError()
class ChildExample(ExampleB, ExampleA):
def do_something(self):
print self.name
def do_otherthing(self):
Slide 64
Slide 64 text
The Class Factory: τὰ μετὰ τὰ κλάση
class Example(object):
pass
x = Example()
y = Example
print x
print y
Slide 65
Slide 65 text
The Class Factory: τὰ μετὰ τὰ κλάση
class Example(object):
pass
x = Example()
y = Example
print x
print y
<__main__.Example object at 0x4035894c>
Slide 66
Slide 66 text
The Class Factory: τὰ μετὰ τὰ κλάση
def example():
class Example(object):
pass
return Example
x = example()
y = x()
print x
print y
Slide 67
Slide 67 text
The Class Factory: τὰ μετὰ τὰ κλάση
def example():
class Example(object):
pass
return Example
x = example()
y = x()
print x
print y
<__main__.Example object at 0x403589ec>
Slide 68
Slide 68 text
The Class Factory: τὰ μετὰ τὰ κλάση
class Example(object):
pass
print type(Example)
print type(Example())
Slide 69
Slide 69 text
The Class Factory: τὰ μετὰ τὰ κλάση
class Example(object):
pass
print type(Example)
print type(Example())
Slide 70
Slide 70 text
The Class Factory: τὰ μετὰ τὰ κλάση
class Example(object):
pass
print type(Example)
print type(Example())
Slide 71
Slide 71 text
The Class Factory: τὰ μετὰ τὰ κλάση
class Example(object):
pass
print type(Example)
print type(Example())
Slide 72
Slide 72 text
The Class Factory: τὰ μετὰ τὰ κλάση
class Example(object):
pass
x = Example
y = type('Example', (object,), {})
print x
print y
print (x == y)
Slide 73
Slide 73 text
The Class Factory: τὰ μετὰ τὰ κλάση
class Example(object):
pass
x = Example
y = type('Example', (object,), {})
print x
print y
print (x == y)
False
Slide 74
Slide 74 text
The Class Factory: __magic__
__new__(cls, *args, **kwargs)
Is the first method to get called in an object's instantiation, is a
@classmethod, and must return a new instance of type cls.
__init__(self, *args, **kwargs)
Is the initializer for the instance. It gets passed whatever the
primary constructor was called with.
__del__(self)
Is the destructor of the instance, will be invoked before clean
the reference to the instance.
Slide 75
Slide 75 text
The Class Factory: __metaclass__
def upper_attr(f_class_name, f_class_parents, f_class_attr):
attrs = ((name, value) \
for name, value in f_class_attr.items() \
if not name.startswith('__'))
uppercase_attr = dict((name.upper(), value) \
for name, value in attrs)
return type(f_class_name, f_class_parents, uppercase_attr)
class Foo(object):
__metaclass__ = upper_attr
bar = 'bip'
print hasattr(Foo, 'bar')
print hasattr(Foo, 'BAR')
Slide 76
Slide 76 text
The Class Factory: __metaclass__
def upper_attr(f_class_name, f_class_parents, f_class_attr):
attrs = ((name, value) \
for name, value in f_class_attr.items() \
if not name.startswith('__'))
uppercase_attr = dict((name.upper(), value) \
for name, value in attrs)
return type(f_class_name, f_class_parents, uppercase_attr)
class Foo(object):
__metaclass__ = upper_attr
bar = 'bip'
print hasattr(Foo, 'bar')
print hasattr(Foo, 'BAR')
False
True
Slide 77
Slide 77 text
9. Monkey Patching
Slide 78
Slide 78 text
Monkey patching: first try
class Person(object):
def speak(self):
print "hello"
def monkey(foo):
print "uh uh uh"
Person.speak = monkey
x = Person()
x.speak()
Slide 79
Slide 79 text
Monkey patching: be evil }:-)
import os
def monkey(*args, **kwargs):
print "no no no"
os.system = monkey
os.system("ls /tmp")
Slide 80
Slide 80 text
Monkey patching: be evil }:-)
import os
def monkey(*args, **kwargs):
print "no no no"
os.system = monkey
os.system("ls /tmp")
no no no
Slide 81
Slide 81 text
10. Profiling
Slide 82
Slide 82 text
Profiling: hand made
import time
class Timer(object):
def __init__(self, verbose=False):
self.verbose = verbose
def __enter__(self):
self.start = time.time()
return self
def __exit__(self, *args):
self.end = time.time()
self.secs = self.end - self.start
self.msecs = self.secs * 1000 # millisecs
if self.verbose:
print 'elapsed time: %f ms' % self.msecs
Slide 83
Slide 83 text
Profiling: hand made
import time
class Timer(object):
def __init__(self, verbose=False):
self.verbose = verbose
def __enter__(self):
self.start = time.time()
return self
def __exit__(self, *args):
self.end = time.time()
self.secs = self.end - self.start
self.msecs = self.secs * 1000 # millisecs
if self.verbose:
print 'elapsed time: %f ms' % self.msecs
from redis import Redis
rdb = Redis()
with Timer() as t:
rdb.lpush("foo", "bar")
print "=> elasped lpush: %s s" % t.
secs
with Timer as t:
rdb.lpop("foo")
print "=> elasped lpop: %s s" % t.secs
Slide 84
Slide 84 text
Profiling: profile & cProfile
try:
import cProfile as profile
except ImportError:
import profile
def fib(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fib(n-1) + fib(n-2)
def fib_seq(n):
seq = [ ]
if n > 0:
seq.extend(fib_seq(n-1))
seq.append(fib(n))
return seq
Slide 85
Slide 85 text
Profiling: profile & cProfile
try:
import cProfile as profile
except ImportError:
import profile
def fib(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fib(n-1) + fib(n-2)
def fib_seq(n):
seq = [ ]
if n > 0:
seq.extend(fib_seq(n-1))
seq.append(fib(n))
return seq
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765]
57356 function calls (66 primitive calls) in 0.746 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
21 0.000 0.000 0.000 0.000 :0(append)
20 0.000 0.000 0.000 0.000 :0(extend)
1 0.001 0.001 0.001 0.001 :0(setprofile)
1 0.000 0.000 0.744 0.744 :1()
1 0.000 0.000 0.746 0.746 profile:0(print fib_seq(20); print)
0 0.000 0.000 profile:0(profiler)
57291/21 0.743 0.000 0.743 0.035 profile_fibonacci_raw.py:13(fib)
21/1 0.001 0.000 0.744 0.744 profile_fibonacci_raw.py:22(fib_seq)
Slide 86
Slide 86 text
11. Documentation
Slide 87
Slide 87 text
Documentation: Zen
Don't create documentation for your code.
Code your documentation.
def elements(n):
"""Return a list of n numbers from 0 to n-
1.
"""
return range(0,n)
Slide 88
Slide 88 text
Documentation: everything is an object
def elements(n):
"""Return a list of n numbers from 0 to n-
1.
"""
return range(0,n)
print elements.__doc__
Slide 89
Slide 89 text
Documentation: everything is an object
def elements(n):
"""Return a list of n numbers from 0 to n-
1.
"""
return range(0,n)
print elements.__doc__
Return a list of n numbers from 0 to n-1
Slide 90
Slide 90 text
Documentation: style is important
def elements(n):
"""Return a list of n numbers from 0 to n-1.
:type n: int
:param n: the limit upper for the elements to be created (not
included).
:return: a class:`list` with the items.
"""
return range(0,n)
class Example(object):
"""This is the documentation of the class.
Usually you do not need it :)
"""
def __init__(self, param):
"""This is the documentation of the instance type.
"""
Slide 91
Slide 91 text
Documentation: do the work
$ sphinx-quickstart
Slide 92
Slide 92 text
Documentation: Bonus... abuse the doc
class Command(object):
"""Undocumented command
"""
class ListCommand(Command):
"""List command.
"""
def show_help(command):
cmd = getattr(globals()[__name__], "%sCommand" % command,
None)
if cmd is None:
return "Invalid command"
else:
return cmd.__doc__
print show_help("List")
Slide 93
Slide 93 text
12. Future
Slide 94
Slide 94 text
Future: pypy
Just in time compiler FAST!
With sandboxing
Best concurrency support
Slide 95
Slide 95 text
Future: python3
def hello(name: str, age: int) -> str:
return name
print hello.__anotations__
Slide 96
Slide 96 text
Future: python3
def hello(name: str, age: int) -> str:
return name
print hello.__anotations__
{'return':,'name':,'age':}
Slide 97
Slide 97 text
Future: python3
from functools import lru_cache
@lru_cache(maxsize=None)
def fib(n):
if n < 2:
return n
return fib(n-1)+fib(n-2)
Slide 98
Slide 98 text
Applauses & questions
Not necessarily in that order.