Stdlib Safari - Exotic Animal Edition

Little-known Features and Modules of Python's Standard Library

Transcript

1. Stdlib Safari Exotic Animal Edition AKA "Seeking Strange Serpentine Specimens"
2. About me Baptiste Mispelon (bmispelon) I've been using python for over 5 years. Currently doing web development with Django at M2BPO.
3. About this talk Builtin functions Little-known usage of common builtins Little-known modules
4. Part 1: Builtins Python2 has 80 builtin functions. Python3 has 69. That's not a lot, especially compared to a certain scriPting language whose name sHall not be sPoken.
5. Builtins Table (python2) abs complex getattr locals range super all delattr globals long raw_input tuple any dict hasattr map reduce type dir hash max reload unichr basestring divmod help memoryview repr unicode bin enumerate hex min reversed vars bool eval id next round xrange bytearray input object set zip execfile int oct setattr __import__ callable file isinstance open slice apply chr filter issubclass ord sorted buffer classmethod float iter pow staticmethod coerce cmp format len print str intern compile frozenset list property sum
7. Builtins Table (python3) abs complex getattr locals range super all delattr globals tuple any dict hasattr map type ascii dir hash max unichr divmod help memoryview repr bin enumerate hex min reversed vars bool eval id next round bytearray exec input object set zip bytes int oct setattr __import__ callable isinstance open slice chr filter issubclass ord sorted classmethod float iter pow staticmethod
8. format len print str compile frozenset list property sum
9. Short Selection dir divmod slice set / frozenset
10. dir Quick and dirty introspection. List of all the attributes/methods of an object (as strings).
11. Example What are all the public attributes of this object? obj = 'hello world' [attr for attr in dir(obj) if not attr.startswith('_')]
12. Example What are all the public attributes of this object? obj = 'hello world' [attr for attr in dir(obj) if not attr.startswith('_')] # ['capitalize', 'casefold', 'center', # 'count', 'encode', 'endswith', ...]
13. divmod Given two numbers, return a pair of quotient, remainder.
14. Example a, b = 42, 5 q, r = a // b, a % b q, r # ?
15. Example a, b = 42, 5 q, r = a // b, a % b q, r # 8, 2
16. Example a, b = 42, 5 q, r = divmod(a, b) q, r # 8, 2
17. slice Create slice objects.
18. Example s = 'abcdefghi' s[ :8] # ? s[1:8] # ? s[1:8:2] # ?
19. Example s = 'abcdefghi' s[ :8] # 'abcdefgh' s[1:8] # ? s[1:8:2] # ?
20. Example s = 'abcdefghi' s[ :8] # 'abcdefgh' s[1:8] # 'bcdefgh' s[1:8:2] # ?
21. Example s = 'abcdefghi' s[ :8] # 'abcdefgh' s[1:8] # 'bcdefgh' s[1:8:2] # 'bdfh'
22. With slice s = 'abcdefghi' s[ :8] == s[slice(8)] s[1:8] == s[slice(1, 8)] s[1:8:2] == s[slice(1, 8, 2)]
23. set Unordered collection of distinct objects.
24. Advantages Behave like lists: iter(set), len(set), x in set Elements are unique x in set is O(1)
25. Restrictions Elements must be hashable (like keys of a dict) Not indexable: some_set[0] does not work. No order (like a dict)
26. Set literals Since python 2.7: set([1, 2, 3]) # Is the same as doing {1, 2, 3}
27. Set comprehensions Since python 2.7 (too): set(w.lower() for w in wordlist) # Is the same as doing {w.lower() for w in wordlist}
28. Operators intersection: s1 & s2 union: s1 | s2 difference: s1 - s2 symmetric difference: s1 ^ s2 Subset/Superset: s1 < s2
29. Examples set('abababc') # {'a', 'c', 'b'} {1, 2} & {2, 3} {1, 2} | {2, 3} {1, 2} - {2, 3} {1, 2} ^ {2, 3} {1, 2} < {2, 3} {1, 2} < {3, 2, 1}
30. Examples set('abababc') # {'a', 'c', 'b'} {1, 2} & {2, 3} # {2} {1, 2} | {2, 3} {1, 2} - {2, 3} {1, 2} ^ {2, 3} {1, 2} < {2, 3} {1, 2} < {3, 2, 1}
31. Examples set('abababc') # {'a', 'c', 'b'} {1, 2} & {2, 3} # {2} {1, 2} | {2, 3} # {1, 2, 3} {1, 2} - {2, 3} {1, 2} ^ {2, 3} {1, 2} < {2, 3} {1, 2} < {3, 2, 1}
32. Examples set('abababc') # {'a', 'c', 'b'} {1, 2} & {2, 3} # {2} {1, 2} | {2, 3} # {1, 2, 3} {1, 2} - {2, 3} # {1} {1, 2} ^ {2, 3} {1, 2} < {2, 3} {1, 2} < {3, 2, 1}
33. Examples set('abababc') # {'a', 'c', 'b'} {1, 2} & {2, 3} # {2} {1, 2} | {2, 3} # {1, 2, 3} {1, 2} - {2, 3} # {1} {1, 2} ^ {2, 3} # {1, 3} {1, 2} < {2, 3} {1, 2} < {3, 2, 1}
34. Examples set('abababc') # {'a', 'c', 'b'} {1, 2} & {2, 3} # {2} {1, 2} | {2, 3} # {1, 2, 3} {1, 2} - {2, 3} # {1} {1, 2} ^ {2, 3} # {1, 3} {1, 2} < {2, 3} # False {1, 2} < {3, 2, 1}
35. Examples set('abababc') # {'a', 'c', 'b'} {1, 2} & {2, 3} # {2} {1, 2} | {2, 3} # {1, 2, 3} {1, 2} - {2, 3} # {1} {1, 2} ^ {2, 3} # {1, 3} {1, 2} < {2, 3} # False {1, 2} < {3, 2, 1} # True
36. frozenset Like a set, but immutable.
37. Why? Making it immutable makes it hashable. This means we can use it as the key of a dictionnary. Or put it in another set/frozenset.
38. Why? Making it immutable makes it hashable. This means we can use it as the key of a dictionnary. Or put it in another set/frozenset. (Yo Dawg, I heard you like sets...)
39. Part 2: Builtins with a twist next's second parameter iter's second parameter Imaginary friends
40. iter Make an iterator out of an iterable. Iterable An object that can be iterated over (for x in y) Iterator A type of iterable that can only go forward.
41. Example l = list('abc') ''.join(c.upper() for c in l) # ?
42. Example l = list('abc') ''.join(c.upper() for c in l) # 'ABC'
43. Example l = list('abc') ''.join(c.upper() for c in l) # 'ABC' ''.join(c.upper() for c in l) # 'ABC'
44. Example (2) i = iter('abc') ''.join(c.upper() for c in i) # ?
45. Example (2) i = iter('abc') ''.join(c.upper() for c in i) # 'ABC'
46. Example (2) i = iter('abc') ''.join(c.upper() for c in i) # 'ABC' ''.join(c.upper() for c in i) # ?
47. Example (2) i = iter('abc') ''.join(c.upper() for c in i) # 'ABC' ''.join(c.upper() for c in i) # ''
48. The Twist If you pass a second argument, iter behaves quite differently. iter(callable, sentinel)
49. iter(callable, sentinel) Call the first argument yielding the return value and stopping when the sentinel value is reached (or when the call raises StopIteration).
50. Example fp = open('mydata.txt') as fp: for line in iter(fp.readline, 'STOP\n'): print line
51. Example (2) from random import choice roll = lambda: choice(range(1, 7)) list(iter(roll, 6)) # [] list(iter(roll, 6)) # [4, 4, 1]
52. ne×t Advance an iterator and return the value. Raise StopIteration if empty.
53. Example i = iter('abc') next(i) # ? next(i) # ? next(i) # ? next(i) # ?
54. Example i = iter('abc') next(i) # 'a' next(i) # ? next(i) # ? next(i) # ?
55. Example i = iter('abc') next(i) # 'a' next(i) # 'b' next(i) # ? next(i) # ?
56. Example i = iter('abc') next(i) # 'a' next(i) # 'b' next(i) # 'c' next(i) # ?
57. Example i = iter('abc') next(i) # 'a' next(i) # 'b' next(i) # 'c' next(i) # StopIteration
58. The Twist next takes a second argument. Similar to dict.get, it's a value to return if the iterator is empty (instead of raising StopIteration).
59. Example (part 2) i = iter('abc') next(i, 'x') # 'a' next(i, 'x') # 'b' next(i, 'x') # 'c' next(i, 'x') # ?
60. Example (part 2) i = iter('abc') next(i, 'x') # 'a' next(i, 'x') # 'b' next(i, 'x') # 'c' next(i, 'x') # 'x'
61. Imaginary friends Did you know that python has first class support for complex numbers? a = complex(real=4, imag=1) b = complex(real=1, imag=2) a + b # ?
62. Imaginary friends Did you know that python has first class support for complex numbers? a = complex(real=4, imag=1) b = complex(real=1, imag=2) a + b # complex(5, 3)
63. Imaginary friends Did you know that python has first class support for complex numbers? a = complex(real=4, imag=1) b = complex(real=1, imag=2) a + b # complex(5, 3) a + b + 1 # ?
64. Imaginary friends Did you know that python has first class support for complex numbers? a = complex(real=4, imag=1) b = complex(real=1, imag=2) a + b # complex(5, 3) a + b + 1 # complex(6, 3)
65. The Twist It also has imaginary number litterals: 1 + 2j # complex(1, 2)
66. The Twist It also has imaginary number litterals: 1 + 2j # complex(1, 2) 3j # complex(0, 3)
67. The Twist It also has imaginary number litterals: 1 + 2j # complex(1, 2) 3j # complex(0, 3) 1j ** 2 # ?
68. The Twist It also has imaginary number litterals: 1 + 2j # complex(1, 2) 3j # complex(0, 3) 1j ** 2 # -1
69. Fun with maths from math import e, pi e ** (1j * pi) # ?
70. Fun with maths from math import e, pi e ** (1j * pi) # -1
71. Part 3: Exotic Stdlib Modules Python ships with more than 300 modules in its standard library. Here's a quick tour of some of its less known corners.
72. python -m FOO Basically the same as doing python FOO.py where FOO is on the python path. Executes the content of the module.
73. Easter eggs python -m this python -m __hello__ python -m antigravity
74. Utilities Some standard modules are written in a way that they can be used as standalone programs (not just imported). Uses the if __name__ == '__main__' trick.
75. Share a directory over the network?
76. Share a directory over the network? python2 -m SimpleHTTPServer python3 -m http.server Browse and dowload files from the current directory (on port 8000).
77. Check a file for bad identation?
78. Check a file for bad identation? python -m tabnanny your_file.py
79. Print a calendar of the current year?
80. Print a calendar of the current year? python -m calendar
81. Print a calendar of the current year? python -m calendar python -m calendar 2014
82. Print a calendar of the current year? python -m calendar python -m calendar 2014 10
83. Encode a file in base64?
84. Encode a file in base64? python -m base64 some_file
85. Open a URL in the browser?
86. Open a URL in the browser? python -m webbrowser "http://google.hu"
87. Get documentation for a function?
88. Get documentation for a function? python -m pydoc str.find python -m pydoc yourmodule.yourfunction
89. List of functions in a file?
90. List of functions in a file? python -m pyclbr your_file.py python -m pyclbr modulename
91. List of functions in a file? python -m pyclbr your_file.py python -m pyclbr modulename Safe to run on arbitrary files.
92. Batteries Included An arbitrary selection of useful modules, for use in python code.
93. Parsing CSV? with open('file.csv') as f: for line in f: row = line.split(',') print row
94. import csv import csv with open('file.csv') as f: for row in csv.reader(f): print row
95. Leap Years def is_leap(year): return (year % 4) == 0
96. Leap Years def is_leap(year): return (year % 4) == 0 # Hmm... wait
97. Leap Years def is_leap(year): return (year % 4) == 0 \ and (year % 100) != 0 # Aah, that's better
98. Leap Years def is_leap(year): return (year % 4) == 0 \ and (year % 100) != 0 # ... or is it?
99. import calendar import calendar calendar.isleap(2013)
100. Prompting for a password password = raw_input("Password: ")
101. import getpass from getpass import getpass password = getpass()
102. Alphabet s = 'abcdefghijklmnoprstuvwxyz' s = '0123456789'
103. import string import string string.ascii_lowercase string.ascii_uppercase string.digits string.punctuation string.whitespace
104. Temporary Files from random import choice as ch S = 'abcdefghjklmnopqrstuvwxyz' name = ''.join(ch(S) for _ in range(8)) temp = open('/tmp/%s' % name, 'w')
105. import tempfile import tempfile temp = tempfile.TemporaryFile()
106. Passing files to a program import sys for fname in sys.argv[1:]: f = open(fname) for line in f: process(line)
107. import fileinput import fileinput for line in fileinput.fileinput(): process(line)
108. import fileinput import fileinput for line in fileinput.fileinput(): process(line) # It even supports '-' for STDIN
109. Wrapping text for i in range(len(text) // 80): print text[i*80: (i+1)*80]
110. import textwrap import textwrap lines = textwrap.wrap(text, width=80) print '\n'.join(lines)
111. import textwrap import textwrap print textwrap.fill(text, width=80)
112. Python Keywords kw = ['class', 'for', 'if', ...]
113. import keyword import keyword keyword.kwlist
114. import keyword import keyword keyword.kwlist keyword.iskeyword('class')
115. Unicode print u'Csá'
116. Unicode print u'Csá' print u'Cs\xe1'
117. Unicode print u'Csá' print u'Cs\xe1' print u'Cs\u00e1'
118. Unicode print u'Csá' print u'Cs\xe1' print u'Cs\u00e1' print u'Cs\U000000e1'
119. Unicode print u'Csá' print u'Cs\xe1' print u'Cs\u00e1' print u'Cs\U000000e1' print u'Cs\N{LATIN SMALL LETTER A\ WITH ACUTE}'
120. import unicodedata from unicodedata import name, lookup name(u'á') # LATIN SMALL LETTER A WITH ACUTE lookup('LATIN SMALL LETTER A WITH ACUTE') # á
121. Listing Files import os for fname in os.listdir('.'): if fname.endswith('.txt'): print fname
122. import glob import glob for fname in glob.glob('*.txt'): print fname
123. And many more colorsys, difflib, fractions, fnmatch, robotparser, wave, shlex, contextlib, inspect, ...
124. TLDR; Python is awesome. Go look in your /usr/lib/python/, see what you can find.
125. Thanks for listening! Questions? Kérdes?