Slide 1

Slide 1 text

Python Gotchas

Slide 2

Slide 2 text

Adrián Matellanes Lead API Developer at @EburyES & Málaga Python Organizer twitter.com/_amatellanes github.com/amatellanes

Slide 3

Slide 3 text

We’re hiring! https://careers.ebury.com/

Slide 4

Slide 4 text

Join us! https://www.meetup.com/malaga-python/

Slide 5

Slide 5 text

La Térmica - Málaga Saturday, 6 de May, 2017 Call For Papers www.opensouthcode.org

Slide 6

Slide 6 text

What is a “gotcha”? A gotcha is a valid construct in a programming language that works as documented but is counter-intuitive and almost invites mistakes.

Slide 7

Slide 7 text

Learning the Zen of Python

Slide 8

Slide 8 text

Learning the Zen of Python >>> import this https://www.python.org/dev/peps/pep-0020/

Slide 9

Slide 9 text

Learning the Zen of Python >>> import this Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! https://www.python.org/dev/peps/pep-0020/

Slide 10

Slide 10 text

Division Operator

Slide 11

Slide 11 text

Division Operator >>> 5 / 2 # Python 2 >>> 5 // 2 # Python 2 >>> 5.0 / 2 # Python 2 >>> 5.0 // 2 # Python 2

Slide 12

Slide 12 text

Division Operator >>> 5 / 2 # Python 2 2 >>> 5 // 2 # Python 2 2 >>> 5.0 / 2 # Python 2 2.5 >>> 5.0 // 2 # Python 2 2.0

Slide 13

Slide 13 text

Division Operator >>> 5 / 2 # Python 3 >>> 5 // 2 # Python 3 >>> 5.0 / 2 # Python 3 >>> 5.0 // 2 # Python 3

Slide 14

Slide 14 text

Division Operator >>> 5 / 2 # Python 3 2.5 >>> 5 // 2 # Python 3 2 >>> 5.0 / 2 # Python 3 2.5 >>> 5.0 // 2 # Python 3 2.0

Slide 15

Slide 15 text

Division Operator >>> from __future__ import division >>> 5 / 2 # Python 2 or 3 2.5 https://www.python.org/dev/peps/pep-0238/

Slide 16

Slide 16 text

Maths?

Slide 17

Slide 17 text

Maths? >>> True + 3 >>> False * 4 >>> True / False

Slide 18

Slide 18 text

Maths? >>> True + 3 4 >>> False * 4 0 >>> True / False ZeroDivisionError: integer division or modulo by zero

Slide 19

Slide 19 text

Maths? >>> 'a' * 5 >>> 'b' + 4

Slide 20

Slide 20 text

Maths? >>> 'a' * 5 'aaaaa' >>> 'b' + 4 TypeError: cannot concatenate 'str' and 'int' objects

Slide 21

Slide 21 text

Forgetting parentheses

Slide 22

Slide 22 text

Forgetting parentheses >>> def foo(): ... return None ... >>> foo is None >>> foo() is None

Slide 23

Slide 23 text

>>> def foo(): ... return None ... >>> foo is None False >>> foo() is None True Forgetting parentheses

Slide 24

Slide 24 text

Chaining comparisons

Slide 25

Slide 25 text

Chaining comparisons >>> (42 > 23) == True >>> 42 > (23 == True) >>> 42 > 23 == True

Slide 26

Slide 26 text

Chaining comparisons >>> (42 > 23) == True True >>> 42 > (23 == True) True >>> 42 > 23 == True False https://docs.python.org/3/reference/expressions.html#operator-precedence

Slide 27

Slide 27 text

Chaining comparisons >>> 42 > 23 == True False >>> 42 > 23 and 23 == True False https://docs.python.org/3/reference/expressions.html#comparisons

Slide 28

Slide 28 text

is not Identity Operator

Slide 29

Slide 29 text

is not Identity Operator >>> 2 is not None >>> 2 is (not None)

Slide 30

Slide 30 text

is not Identity Operator >>> 2 is not None True >>> 2 is (not None) False

Slide 31

Slide 31 text

Lazy Binding

Slide 32

Slide 32 text

Lazy Binding >>> x = 23 >>> f = lambda: x >>> x = 42 >>> print(f())

Slide 33

Slide 33 text

Lazy Binding >>> x = 23 >>> f = lambda: x >>> x = 42 >>> print(f()) 42

Slide 34

Slide 34 text

Lazy Binding >>> x = 23 >>> f = lambda x=x: x >>> x = 42 >>> print(f()) 23

Slide 35

Slide 35 text

Lazy Binding >>> funcs = [] >>> for i in range(10): ... funcs.append(lambda: i ** 2) ... >>> print([f() for f in funcs])

Slide 36

Slide 36 text

Lazy Binding >>> funcs = [] >>> for i in range(10): ... funcs.append(lambda: i ** 2) ... >>> print([f() for f in funcs]) [81, 81, 81, 81, 81, 81, 81, 81, 81, 81]

Slide 37

Slide 37 text

Lazy Binding >>> funcs = [] >>> for i in range(10): ... funcs.append(lambda i=i: i ** 2) ... >>> print([f() for f in funcs]) [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Slide 38

Slide 38 text

Evaluation Time Discrepancy

Slide 39

Slide 39 text

Evaluation Time Discrepancy >>> seq = [4, 8, 15, 16, 16, 23, 42] >>> g = (x for x in seq if x in seq) >>> seq = [16] >>> print(list(g)) https://www.python.org/dev/peps/pep-0289/

Slide 40

Slide 40 text

Evaluation Time Discrepancy >>> seq = [4, 8, 15, 16, 16, 23, 42] >>> g = (x for x in seq if x in seq) >>> seq = [16] >>> print(list(g)) [16, 16] https://www.python.org/dev/peps/pep-0289/

Slide 41

Slide 41 text

Memory Management

Slide 42

Slide 42 text

Memory Management >>> list1 = [1, 2, 3] >>> list2 = list1 >>> list1[1] = 5 >>> print(list1) >>> print(list2)

Slide 43

Slide 43 text

Memory Management >>> list1 = [1, 2, 3] >>> list2 = list1 >>> list1[1] = 0 >>> print(list1) [1, 0, 3] >>> print(list2) [1, 0, 3]

Slide 44

Slide 44 text

Memory Management >>> list1 = [1, 2, 3] >>> list2 = list1[:] >>> list1[1] = 0 >>> print(list1) [1, 0, 3] >>> print(list2) [1, 2, 3]

Slide 45

Slide 45 text

Memory Management >>> from copy import copy >>> list1 = [1, 2, 3] >>> list2 = copy(list1) >>> list1[1] = 5 >>> print(list1) [1, 0, 3] >>> print(list2) [1, 2, 3]

Slide 46

Slide 46 text

Memory Management >>> list1 = [{'a': [1, 2, 3]}, {'b': [4, 5, 6]}] >>> list2 = list1 >>> list1[0]['a'][1] = 0 >>> print(list1) >>> print(list2)

Slide 47

Slide 47 text

Memory Management >>> list1 = [{'a': [1, 2, 3]}, {'b': [4, 5, 6]}] >>> list2 = list1 >>> list1[0]['a'][1] = 0 >>> print(list1) [{'a': [1, 0, 3]}, {'b': [4, 5, 6]}] >>> print(list2) [{'a': [1, 0, 3]}, {'b': [4, 5, 6]}]

Slide 48

Slide 48 text

Memory Management >>> from copy import copy >>> list1 = [{'a': [1, 2, 3]}, {'b': [4, 5, 6]}] >>> list2 = copy(list1) >>> list1[0]['a'][1] = 0 >>> print(list1) [{'a': [1, 0, 3]}, {'b': [4, 5, 6]}] >>> print(list2) [{'a': [1, 0, 3]}, {'b': [4, 5, 6]}]

Slide 49

Slide 49 text

Memory Management >>> from copy import deepcopy >>> list1 = [{'a': [1, 2, 3]}, {'b': [4, 5, 6]}] >>> list2 = deepcopy(list1) >>> list1[0]['a'][1] = 0 >>> print(list1) [{'a': [1, 0, 3]}, {'b': [4, 5, 6]}] >>> print(list2) [{'a': [1, 2, 3]}, {'b': [4, 5, 6]}]

Slide 50

Slide 50 text

Default Mutable Argument

Slide 51

Slide 51 text

Default Mutable Argument >>> def foo(x=[]): ... x.append(1) ... print(x) ... >>> foo() >>> foo()

Slide 52

Slide 52 text

Default Mutable Argument >>> def foo(x=[]): ... x.append(1) ... print(x) ... >>> foo() [1] >>> foo() [1, 1]

Slide 53

Slide 53 text

Default Mutable Argument >>> def foo(x=None): ... if x is None: ... x = [] ... x.append(1) ... print(x) ... >>> foo() [1] >>> foo() [1]

Slide 54

Slide 54 text

Default Mutable Argument >>> def foo(x=None): ... x = x or [] ... x.append(1) ... print(x) ... >>> foo() [1] >>> foo() [1]

Slide 55

Slide 55 text

Unpacking

Slide 56

Slide 56 text

Unpacking >>> a, b, c = 1, 2, 3 >>> a >>> b >>> c

Slide 57

Slide 57 text

Unpacking >>> a, b, c = 1, 2, 3 >>> a 1 >>> b 2 >>> c 3

Slide 58

Slide 58 text

Unpacking >>> a, b, c = [1, 2, 3] >>> a >>> b >>> c

Slide 59

Slide 59 text

Unpacking >>> a, b, c = [1, 2, 3] >>> a 1 >>> b 2 >>> c 3

Slide 60

Slide 60 text

Unpacking >>> a, b, c = (2 * i + 1 for i in range(3)) >>> a >>> b >>> c

Slide 61

Slide 61 text

Unpacking >>> a, b, c = (2 * i + 1 for i in range(3)) >>> a 1 >>> b 2 >>> c 3

Slide 62

Slide 62 text

Unpacking >>> a, b, c = [1, (2, 3), 4] >>> a >>> b >>> c

Slide 63

Slide 63 text

Unpacking >>> a, b, c = [1, (2, 3), 4] >>> a 1 >>> b (2, 3) >>> c 3

Slide 64

Slide 64 text

Unpacking >>> a, b, c = [1, 2, 3, 4] # Python 2 >>> a >>> b >>> c

Slide 65

Slide 65 text

Unpacking >>> a, b, c = [1, 2, 3, 4] # Python 2 ValueError: too many values to unpack

Slide 66

Slide 66 text

Unpacking >>> a, b, c = [1, 2, 3, 4] # Python 3 >>> a >>> b >>> c

Slide 67

Slide 67 text

Unpacking >>> a, b, c = [1, 2, 3, 4] # Python 3 ValueError: too many values to unpack (expected 3)

Slide 68

Slide 68 text

Unpacking >>> a, *b, c = [1, 2, 3, 4] # Only Python 3 >>> a >>> b >>> c

Slide 69

Slide 69 text

Unpacking >>> a, *b, c = [1, 2, 3, 4] # Only Python 3 >>> a 1 >>> b [2, 3] >>> c 4

Slide 70

Slide 70 text

Slicing list

Slide 71

Slide 71 text

Slicing list >>> a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] >>> a[::2] >>> a[::3] >>> a[2:8:2]

Slide 72

Slide 72 text

Slicing list >>> a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] >>> a[::2] [0, 2, 4, 6, 8, 10] >>> a[::3] [0, 3, 6, 9] >>> a[2:8:2] [2, 4, 6]

Slide 73

Slide 73 text

Dictionary and Set Comprehensions

Slide 74

Slide 74 text

>>> {x: x ** 2 for x in range(5)} >>> {x for x in range(5)} Dictionary and Set Comprehensions

Slide 75

Slide 75 text

>>> {x: x ** 2 for x in range(5)} {0: 0, 1: 1, 2: 4, 3: 9, 4: 16} >>> {x for x in range(5)} {0, 1, 2, 3, 4} Dictionary and Set Comprehensions

Slide 76

Slide 76 text

>>> type({}) Dictionary and Set Comprehensions

Slide 77

Slide 77 text

>>> type({}) dict Dictionary and Set Comprehensions

Slide 78

Slide 78 text

Bytecode Files Everywhere!

Slide 79

Slide 79 text

Disabling bytecode (.pyc) files $ export PYTHONDONTWRITEBYTECODE=1 Bytecode Files Everywhere! https://www.python.org/dev/peps/pep-0304/

Slide 80

Slide 80 text

Removing bytecode (.pyc) files $ find . -type f -name "*.py[co]" -delete -or \ -type d -name "__pycache__" -delete $ pyclean . $ py3clean . Bytecode Files Everywhere! http://manpages.ubuntu.com/manpages/xenial/man1/pyclean.1.html http://manpages.ubuntu.com/manpages/xenial/man1/py3clean.1.html

Slide 81

Slide 81 text

Using class variables incorrectly

Slide 82

Slide 82 text

>>> class A(object): ... x = 1 >>> class B(A): ... pass >>> class C(A): ... pass >>> print(A.x, B.x, C.x) Using class variables incorrectly

Slide 83

Slide 83 text

>>> class A(object): ... x = 1 >>> class B(A): ... pass >>> class C(A): ... pass >>> print(A.x, B.x, C.x) 1 1 1 Using class variables incorrectly

Slide 84

Slide 84 text

>>> class A(object): ... x = 1 >>> class B(A): ... pass >>> class C(A): ... pass >>> B.x = 2 >>> A.x = 3 >>> print(A.x, B.x, C.x) Using class variables incorrectly

Slide 85

Slide 85 text

>>> class A(object): ... x = 1 >>> class B(A): ... pass >>> class C(A): ... pass >>> B.x = 2 >>> A.x = 3 >>> print(A.x, B.x, C.x) 3 2 3 Using class variables incorrectly

Slide 86

Slide 86 text

Catching multiple exceptions

Slide 87

Slide 87 text

>>> try: ... x = int('a') ... except ValueError, IndexError: # In Python 2 ... print 'Error' Catching multiple exceptions

Slide 88

Slide 88 text

>>> try: ... x = int('a') ... except ValueError, IndexError: # In Python 2 ... print 'Error' 'Error' Catching multiple exceptions

Slide 89

Slide 89 text

>>> try: ... x = int('a') ... except ValueError, IndexError: # In Python 3 ... print('Error') SyntaxError: invalid syntax Catching multiple exceptions

Slide 90

Slide 90 text

>>> try: ... # do something that may fail ... except (ValueError, IndexError) as e: # In Python 2 and 3 ... # do this if ANYTHING goes wrong Catching multiple exceptions

Slide 91

Slide 91 text

Misunderstanding scope rules

Slide 92

Slide 92 text

>>> x = 10 >>> def foo(): ... x += 1 ... print(x) ... >>> foo() Misunderstanding scope rules

Slide 93

Slide 93 text

>>> x = 10 >>> def foo(): ... x += 1 ... print(x) ... >>> foo() UnboundLocalError: local variable 'x' referenced before assignment Misunderstanding scope rules

Slide 94

Slide 94 text

>>> l = [1, 2, 3] >>> def foo(): ... l.append(5) ... >>> foo() >>> l Misunderstanding scope rules

Slide 95

Slide 95 text

>>> l = [1, 2, 3] >>> def foo(): ... l.append(5) ... >>> foo() >>> l [1, 2, 3, 5] Misunderstanding scope rules

Slide 96

Slide 96 text

>>> l = [1, 2, 3] >>> def foo(): ... l += [5] ... >>> foo() >>> l Misunderstanding scope rules

Slide 97

Slide 97 text

>>> l = [1, 2, 3] >>> def foo(): ... l += [5] ... >>> foo() >>> l UnboundLocalError: local variable 'l' referenced before assignment Misunderstanding scope rules

Slide 98

Slide 98 text

Modifying a list while iterating over it

Slide 99

Slide 99 text

>>> odd = lambda x : bool(x % 2) >>> numbers = [n for n in range(10)] >>> for i in range(len(numbers)): ... if odd(numbers[i]): ... del numbers[i] ... Modifying a list while iterating over it

Slide 100

Slide 100 text

>>> odd = lambda x : bool(x % 2) >>> numbers = [n for n in range(10)] >>> for i in range(len(numbers)): ... if odd(numbers[i]): ... del numbers[i] ... IndexError: list index out of range Modifying a list while iterating over it

Slide 101

Slide 101 text

>>> odd = lambda x : bool(x % 2) >>> numbers = [n for n in range(10)] >>> numbers[:] = [n for n in numbers if odd(n)] >>> numbers [1, 3, 5, 7, 9] Modifying a list while iterating over it

Slide 102

Slide 102 text

Name clashing with modules and keywords

Slide 103

Slide 103 text

>>> list = [1, 2] >>> list() Name clashing with modules and keywords

Slide 104

Slide 104 text

>>> list = [1, 2] >>> list() TypeError: 'list' object is not callable Name clashing with modules and keywords

Slide 105

Slide 105 text

# email.py a = 1 Name clashing with modules and keywords

Slide 106

Slide 106 text

>>> import email >>> email.parser() AttributeError: 'module' object has no attribute 'parser' Name clashing with modules and keywords

Slide 107

Slide 107 text

Obscuring import statements

Slide 108

Slide 108 text

# module1.py x = 1 # module2.py x = 2 # module3.py x = 3 Obscuring import statements

Slide 109

Slide 109 text

>>> from module1 import * >>> from module2 import * >>> from module3 import * >>> print(x) Obscuring import statements

Slide 110

Slide 110 text

>>> from module1 import * >>> from module2 import * >>> from module3 import * >>> print(x) 3 Obscuring import statements

Slide 111

Slide 111 text

Creating Circular Module Dependencies

Slide 112

Slide 112 text

# recur1.py x = 1 import recur2 y = 2 Creating Circular Module Dependencies

Slide 113

Slide 113 text

# recur2.py from recur1 import x from recur1 import y Creating Circular Module Dependencies

Slide 114

Slide 114 text

>>> import recur1 Creating Circular Module Dependencies

Slide 115

Slide 115 text

>>> import recur1 ... File "recur1.py", line 2, in import recur2 File "recur2.py", line 2, in from recur1 import y ImportError: cannot import name y Creating Circular Module Dependencies

Slide 116

Slide 116 text

# recur1.py from recur2 import x def f(): return x print(f()) Creating Circular Module Dependencies

Slide 117

Slide 117 text

# recur2.py import recur1 x = 1 def g(): print(recur1.f()) Creating Circular Module Dependencies

Slide 118

Slide 118 text

>>> import recur1 Creating Circular Module Dependencies

Slide 119

Slide 119 text

>>> import recur1 1 Creating Circular Module Dependencies

Slide 120

Slide 120 text

>>> import recur2 Creating Circular Module Dependencies

Slide 121

Slide 121 text

>>> import recur2 ... File "/home/miusuario/tes/recur2.py", line 1, in import recur1 File "/home/miusuario/tes/recur1.py", line 1, in from recur2 import x ImportError: cannot import name 'x' Creating Circular Module Dependencies

Slide 122

Slide 122 text

# recur2.py x = 1 import recur1 def g(): print(recur1.f()) Creating Circular Module Dependencies

Slide 123

Slide 123 text

>>> import recur2 1 Creating Circular Module Dependencies

Slide 124

Slide 124 text

# recur2.py x = 1 def g(): import recur1 print(recur1.f()) Creating Circular Module Dependencies

Slide 125

Slide 125 text

Thank you! Questions? Slides available at https://speakerdeck.com/amatellanes/python-gotchas