Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Python Speleology

Python Speleology

A deep overview about some tricky python features

Avatar for Andrés J. Díaz

Andrés J. Díaz

September 18, 2014
Tweet

More Decks by Andrés J. Díaz

Other Decks in Programming

Transcript

  1. The Zen of Python ◦ Beautiful is better than ugly.

    ◦ Explicit is better than implicit. ◦ Simple is better than complex. ◦ Complex is better than complicated. ◦ Flat is better than nested. ◦ Sparse is better than dense. ◦ Readability counts. ◦ Special cases aren't special enough to break the rules. ◦ Although practicality beats purity. ◦ Errors should never pass silently. ◦ Unless explicitly silenced. ◦ In the face of ambiguity, refuse the temptation to guess. ◦ There should be one -and preferably only one- obvious way to do it. ◦ Although that way may not be obvious at first unless you're Dutch. ◦ Now is better than never. ◦ Although never is often better than right now. ◦ If the implementation is hard to explain, it's a bad idea. ◦ If the implementation is easy to explain, it may be a good idea. ◦ Namespaces are one honking great idea -let's do more of those!
  2. Types and objects: To be or not to be a

    = 256 b = 256 print (a is b) a = 257 b = 257 print (a is b)
  3. Types and objects: To be or not to be a

    = 256 b = 256 print (a is b) a = 257 b = 257 print (a is b) True False
  4. Types and objects: To be or not to be a

    = 256 b = 256 print (a is b) a = 257 b = 257 print (a is b) True False a = 256 b = 256 print id(a), id(b) a = 257 b = 257 print id(a), id(b) 22036112 22036112 22363568 22363640
  5. Types and objects: To be or not to be a

    = 256 b = 256 print (a is b) a = 257 b = 257 print (a is b) True False a = 256 b = 256 print id(a), id(b) a = 257 b = 257 print id(a), id(b)
  6. Types and objects: To be or not to be a

    = 256 b = 256 print (a is b) a = 257 b = 257 print (a is b) True False a = 256 b = 256 print id(a), id(b) a = 257 b = 257 print id(a), id(b) 22036112 22036112 22363568 22363640
  7. Types and objects: functions are objects def x(f): return f

    def y(f): return x print x(y)(8)(0) 0
  8. Types and objects: functions are objects def x(i): if x.enabled:

    return i else: return "disabled" x.enabled = True print x(8)
  9. Types and objects: functions are objects def x(i): if x.enabled:

    return i else: return "disabled" x.enabled = True print x(8) What happened if not set x.enabled? 8
  10. Types and objects: star operator args = (3 , 6)

    print range(*args) args = { "name": "example" } def f(name): print name f(**args) def f(*args, **kwargs): print args print kwargs
  11. Types and objects: star operator args = (3 , 6)

    print range(*args) args = { "name": "example" } def f(name): print name f(**args) def f(*args, **kwargs): print args print kwargs [3, 4, 5] example (1, 2, 3) {'name': 'example'}
  12. Reflection: get & set class Example(object): num = 10 x

    = Example dir(x) hasattr(x, "num") == True getattr(x, "num", 0) == 10 setattr(x, "num", 20)
  13. Reflection: local & global globals() Return a dictionary representing the

    current global symbol table. This is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called). locals() Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks.
  14. Reflection: __magic__ __name__ This is the name of the function.

    This only have a meaningful value is the function is defined with “def”. __class__ This is a reference to the class a method belongs to. __code__ This is a reference to the code object used in the implementation of python.
  15. Reflection: inspector from inspect import getcomments # This is a

    comment def f(x): print x print getcomments(f)
  16. Reflection: inspector from inspect import getcomments # This is a

    comment def f(x): print x print getcomments(f) # This is a comment
  17. Reflection: inspector from inspect import getsource # This is a

    comment def f(x): print x print getsource(f)
  18. Reflection: inspector from inspect import getsource # This is a

    comment def f(x): print x print getsource(f) def f(x): print x
  19. Reflection: let's more tricky def f(x): print x print f.__code__.co_code

    'd\x01\x00GHd\x00\x00S' YEEES!!! THE BYTECODE!!!
  20. Context manager: nowadays... with Item() as item: item.do class Item(object):

    def __enter__(self): self.open() return self def __exit__(self,exc_type,exc_value,exc_t): self.close() ...
  21. Context manager: the real world with file("/tmp/test", "w") as f:

    f.write("hello world") with lock(): # do some concurrent with sudo("root"): # do some as root
  22. Decorations: bold & italic def makebold(fn): def wrapped(): return "<b>"

    + fn() + "</b>" return wrapped def makeitalic(fn): def wrapped(): return "<i>" + fn() + "</i>" return wrapped @makebold @makeitalic def hello(): return "hello world"
  23. Decorations: bold & italic def makebold(fn): def wrapped(): return "<b>"

    + fn() + "</b>" return wrapped def makeitalic(fn): def wrapped(): return "<i>" + fn() + "</i>" return wrapped @makebold @makeitalic def hello(): return "hello world" <b><i>hello world</i></b>
  24. Decorations: complex decor def makestyle(arg): def decorator(f): def wrapper(*args, **kw):

    return "<" + arg + ">" + f() + "</" + arg + ">" return wrapper return decorator @makestyle("b") def hello(): return "hello world" print hello()
  25. Decorations: syntax sugar def makebold(fn): def wrapped(): return "<b>" +

    fn() + "</b>" return wrapped def makestyle(arg): def decorator(f): def wrapper(*args, **kw): return "<" + arg + ">" + f() + "</" + arg + ">" return wrapper return decorator makebold(hello)
  26. Iterations: comprehesions squares = [] for x in range(10): squares.append(x**2)

    squares = [x**2 for x in range(10)] [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y] { (k,v) for k,v in [(1,2)] }
  27. Iterations: comprehesions squares = [] for x in range(10): squares.append(x**2)

    squares = [x**2 for x in range(10)] [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y] { (k,v) for k,v in [(1,2)] } SET NOT DICT!
  28. Iterations: comprehesions squares = [] for x in range(10): squares.append(x**2)

    squares = [x**2 for x in range(10)] [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y] { (k,v) for k,v in [(1,2)] } SET NOT DICT! { k:v for k,v in [(1,2)] }
  29. Iterations: co-routine def countdown(n): print "Counting down from", n while

    n > 0: yield n n -= 1 print "Done counting down" for i in countdown(10): print i
  30. Iterations: co-routine def countdown(n): print "Counting down from", n while

    n > 0: yield n n -= 1 print "Done counting down" for i in countdown(10): print i Counting down from 10 10 9 8 7 6 5 4 3 2 1 Done counting down
  31. Iterations: co-routine def countdown(n): print "Counting down from", n while

    n > 0: yield n n -= 1 print "Done counting down" print countdown(10) <generator object at 0x4035874c>
  32. Overloading: binary operations __add__(self, other) x + y __sub__(self, other)

    x - y __mul__(self, other) x * y __div__(self, other) x / y __pow__(self, other) x ** y
  33. Overloading: binary operations __radd__(self, other) y + x __rsub__(self, other)

    y - x __rmul__(self, other) y * x __rdiv__(self, other) y / x __rpow__(self, other) y ** x
  34. Overloading: binary operations __radd__(self, other) 1 + x __rsub__(self, other)

    1 - x __rmul__(self, other) 1 * x __rdiv__(self, other) 1 / x __rpow__(self, other) 1 ** x
  35. Overloading: binary operations __iadd__(self, other) x+=y __isub__(self, other) x-=y __imul__(self,

    other) x*=y __idiv__(self, other) x/=y __ipow__(self, other) x**=y
  36. Overloading: comparison operations __eq__(self, other) x == y __lt__(self, other)

    x < y __le__(self, other) x <= y __gt__(self, other) x > y __ge__(self, other) x >= y __ne__(self, other) x != y
  37. Overloading: containers __contains__(self, other) y in x __getitem__(self, other) x[y]

    __setitem__(self, other,value) x[y] = z __delitem__(self, other) del x[y] __len__(self) len(x) __reversed__(self) reversed(x) __iter__(self) iter(x)
  38. The Class Factory: class & objects class Example(object): attribute =

    "this is a class attribute" def __init__(self): self.attribute = "this is an obj attr override class one" self.another = "this is another obj attr, no class" print Example.attribute print Example().attribute print Example().another print Example.another
  39. The Class Factory: class & objects class Example(object): attribute =

    "this is a class attribute" def __init__(self): self.attribute = "this is an obj attr override class one" self.another = "this is another obj attr, no class" print Example.attribute print Example().attribute print Example().another print Example.another this is a class attribute this is an object attribute and override class one this is another object attribute, no class Traceback (most recent call last): Line 11, in <module> print Example.another AttributeError: type object 'Example' has no attribute 'another'
  40. The Class Factory: set & get class Example(object): def __init__(self):

    self._name = x @property def name(self): return self._name @name.setter def name(self, value): self._name = value
  41. The Class Factory: @property abuse class Example(object): def __init__(self, host,

    port): self.host = host self.port = port @property def connect(self): lib.connect(self.host, self.port)
  42. The Class Factory: @property abuse class Example(object): def __init__(self, host,

    port): self.host = host self.port = port @property def connect(self): lib.connect(self.host, self.port) NEVER TYPE METHODS AS PROPERTIES
  43. The Class Factory: methods and more methods @staticmethod Nothing more

    than a function defined inside a class. It is callable without instantiating the class first. It’s definition is immutable via inheritance. @classmethod Also callable without instantiating the class, but its definition follows Sub class, not Parent class, via inheritance. That’s because the first argument for @classmethod function must always be cls (class).
  44. The Class Factory: methods and more methods @staticmethod Nothing more

    than a function defined inside a class. It is callable without instantiating the class first. It’s definition is immutable via inheritance. @classmethod Also callable without instantiating the class, but its definition follows Sub class, not Parent class, via inheritance. That’s because the first argument for @classmethod function must always be cls (class). @staticmethod def static_method(): print "I do not receive nothing :(" @classmethod def class_method(cls): print "I'm a class %s" % str(cls)
  45. The Class Factory: methods and more methods class Example(object): def

    __init__(self, name): self.name = name @classmethod def class_method(cls, name): return cls(name) x = Example.class_method("example") print x
  46. The Class Factory: methods and more methods class Example(object): def

    __init__(self, name): self.name = name @classmethod def class_method(cls, name): return cls(name) x = Example.class_method("example") print x <__main__.Example object at 0x40358b2c>
  47. The Class Factory: children and parents class Example(object): def __init__(self,

    name): self.name = name def do_something(self): raise NotImplementedError() class ChildExample(Example): def do_something(self): print self.name
  48. The Class Factory: children and parents class ExampleA(object): def __init__(self,

    name): self.name = name def do_something(self): raise NotImplementedError() class ExampleB(object): def do_otherthing(self): raise NotImplementedError() class ChildExample(ExampleB, ExampleA): def do_something(self): print self.name def do_otherthing(self):
  49. The Class Factory: τὰ μετὰ τὰ κλάση class Example(object): pass

    x = Example() y = Example print x print y <__main__.Example object at 0x4035894c> <class '__main__.Example'>
  50. The Class Factory: τὰ μετὰ τὰ κλάση def example(): class

    Example(object): pass return Example x = example() y = x() print x print y
  51. The Class Factory: τὰ μετὰ τὰ κλάση def example(): class

    Example(object): pass return Example x = example() y = x() print x print y <class '__main__.Example'> <__main__.Example object at 0x403589ec>
  52. The Class Factory: τὰ μετὰ τὰ κλάση class Example(object): pass

    print type(Example) print type(Example()) <type 'type'> <class '__main__.Example'>
  53. The Class Factory: τὰ μετὰ τὰ κλάση class Example(object): pass

    print type(Example) print type(Example()) <type 'type'> <class '__main__.Example'>
  54. The Class Factory: τὰ μετὰ τὰ κλάση class Example(object): pass

    print type(Example) print type(Example()) <type 'type'> <class '__main__.Example'>
  55. The Class Factory: τὰ μετὰ τὰ κλάση class Example(object): pass

    x = Example y = type('Example', (object,), {}) print x print y print (x == y)
  56. The Class Factory: τὰ μετὰ τὰ κλάση class Example(object): pass

    x = Example y = type('Example', (object,), {}) print x print y print (x == y) <class '__main__.Example'> <class '__main__.Example'> False
  57. The Class Factory: __magic__ __new__(cls, *args, **kwargs) Is the first

    method to get called in an object's instantiation, is a @classmethod, and must return a new instance of type cls. __init__(self, *args, **kwargs) Is the initializer for the instance. It gets passed whatever the primary constructor was called with. __del__(self) Is the destructor of the instance, will be invoked before clean the reference to the instance.
  58. The Class Factory: __metaclass__ def upper_attr(f_class_name, f_class_parents, f_class_attr): attrs =

    ((name, value) \ for name, value in f_class_attr.items() \ if not name.startswith('__')) uppercase_attr = dict((name.upper(), value) \ for name, value in attrs) return type(f_class_name, f_class_parents, uppercase_attr) class Foo(object): __metaclass__ = upper_attr bar = 'bip' print hasattr(Foo, 'bar') print hasattr(Foo, 'BAR')
  59. The Class Factory: __metaclass__ def upper_attr(f_class_name, f_class_parents, f_class_attr): attrs =

    ((name, value) \ for name, value in f_class_attr.items() \ if not name.startswith('__')) uppercase_attr = dict((name.upper(), value) \ for name, value in attrs) return type(f_class_name, f_class_parents, uppercase_attr) class Foo(object): __metaclass__ = upper_attr bar = 'bip' print hasattr(Foo, 'bar') print hasattr(Foo, 'BAR') False True
  60. Monkey patching: first try class Person(object): def speak(self): print "hello"

    def monkey(foo): print "uh uh uh" Person.speak = monkey x = Person() x.speak()
  61. Monkey patching: be evil }:-) import os def monkey(*args, **kwargs):

    print "no no no" os.system = monkey os.system("ls /tmp")
  62. Monkey patching: be evil }:-) import os def monkey(*args, **kwargs):

    print "no no no" os.system = monkey os.system("ls /tmp") no no no
  63. Profiling: hand made import time class Timer(object): def __init__(self, verbose=False):

    self.verbose = verbose def __enter__(self): self.start = time.time() return self def __exit__(self, *args): self.end = time.time() self.secs = self.end - self.start self.msecs = self.secs * 1000 # millisecs if self.verbose: print 'elapsed time: %f ms' % self.msecs
  64. Profiling: hand made import time class Timer(object): def __init__(self, verbose=False):

    self.verbose = verbose def __enter__(self): self.start = time.time() return self def __exit__(self, *args): self.end = time.time() self.secs = self.end - self.start self.msecs = self.secs * 1000 # millisecs if self.verbose: print 'elapsed time: %f ms' % self.msecs from redis import Redis rdb = Redis() with Timer() as t: rdb.lpush("foo", "bar") print "=> elasped lpush: %s s" % t. secs with Timer as t: rdb.lpop("foo") print "=> elasped lpop: %s s" % t.secs
  65. Profiling: profile & cProfile try: import cProfile as profile except

    ImportError: import profile def fib(n): if n == 0: return 0 elif n == 1: return 1 else: return fib(n-1) + fib(n-2) def fib_seq(n): seq = [ ] if n > 0: seq.extend(fib_seq(n-1)) seq.append(fib(n)) return seq
  66. Profiling: profile & cProfile try: import cProfile as profile except

    ImportError: import profile def fib(n): if n == 0: return 0 elif n == 1: return 1 else: return fib(n-1) + fib(n-2) def fib_seq(n): seq = [ ] if n > 0: seq.extend(fib_seq(n-1)) seq.append(fib(n)) return seq [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765] 57356 function calls (66 primitive calls) in 0.746 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 21 0.000 0.000 0.000 0.000 :0(append) 20 0.000 0.000 0.000 0.000 :0(extend) 1 0.001 0.001 0.001 0.001 :0(setprofile) 1 0.000 0.000 0.744 0.744 <string>:1(<module>) 1 0.000 0.000 0.746 0.746 profile:0(print fib_seq(20); print) 0 0.000 0.000 profile:0(profiler) 57291/21 0.743 0.000 0.743 0.035 profile_fibonacci_raw.py:13(fib) 21/1 0.001 0.000 0.744 0.744 profile_fibonacci_raw.py:22(fib_seq)
  67. Documentation: Zen Don't create documentation for your code. Code your

    documentation. def elements(n): """Return a list of n numbers from 0 to n- 1. """ return range(0,n)
  68. Documentation: everything is an object def elements(n): """Return a list

    of n numbers from 0 to n- 1. """ return range(0,n) print elements.__doc__
  69. Documentation: everything is an object def elements(n): """Return a list

    of n numbers from 0 to n- 1. """ return range(0,n) print elements.__doc__ Return a list of n numbers from 0 to n-1
  70. Documentation: style is important def elements(n): """Return a list of

    n numbers from 0 to n-1. :type n: int :param n: the limit upper for the elements to be created (not included). :return: a class:`list` with the items. """ return range(0,n) class Example(object): """This is the documentation of the class. Usually you do not need it :) """ def __init__(self, param): """This is the documentation of the instance type. """
  71. Documentation: Bonus... abuse the doc class Command(object): """Undocumented command """

    class ListCommand(Command): """List command. """ def show_help(command): cmd = getattr(globals()[__name__], "%sCommand" % command, None) if cmd is None: return "Invalid command" else: return cmd.__doc__ print show_help("List")
  72. Future: python3 def hello(name: str, age: int) -> str: return

    name print hello.__anotations__ {'return':<class'int'>,'name':<class'str'>,'age':<class'int'>}