Preparing for the future - Python 3 is here to stay

A look at the history and current adoption of Python 2 and 3, major differences between the two version, hurdles to porting and finally a set of tips and recommendations for writing code to run on both Python 2 and 3.

This talk was presented at the Amsterdam Python meetup group on 2015-03-24 (http://www.meetup.com/Amsterdam-Python-Meetup-Group/events/220704819/).

Transcript

1. Preparing for the future Python 3 is here to stay #py020
2. Hi, I'm Nick https://nick.groenen.me/about/
3. Are you using Python 3 yet?
4. ➔ Python 3.0 final was released on December 3rd, 2008. ➔ The final 2.x version (2.7) release came out mid-2010 ➔ Python 2.7 support extended by 5 years, to 2020 A little bit of history
5. Today Django Flask NumPy SciPy Boto Pandas Tornado Celery SQLAlchemy PyPy
6. Tomorrow ➔ OpenStack ➔ Debian ➔ Ubuntu ➔ Fedora
7. Stragglers ➔ Twisted ➔ gevent ➔ Jython ➔ IronPython
8. Like it or not The future is Python 3000
9. What's so different about Python 3?
10. ➔ Cleans up some warts in the language ➔ More readable, consistent & explicit Intentionally backward-incompatible
11. The unicode "issue"
12. PEP 3100 Miscellaneous Python 3.0 Plans
13. Python 2 Unicode vs. 8-bit Python 3 Text vs. data
14. Python 2 Unicode vs. 8-bit ASCII str type Separate unicode type No byte type Python 3 Text vs. data str is unicode text byte and bytearray for encoded text
15. Python 2 allows mixing of str and unicode objects >>> u'This is unicode.' + " This isn't" u"This is unicode. This isn't"
16. Python 3 does not >>> "This is unicode." + b' These are bytes.' TypeError: Can't convert 'bytes' object to str implicitly >>> "This is unicode." + b' These are bytes.'.decode('us-ascii') 'This is unicode. These are bytes.'
17. Renames cookielib Cookie htmlentitydefs HTMLParser httplib BaseHTTPServer CGIHTTPServer SimpleHTTPServer pipes.quote SocketServer UserDict.UserDict UserList.UserList UserString.UserString xmlrpclib SimpleXMLRPCServer http.cookiejar http.cookies html.entities html.parser http.client http.server http.server http.server shlex.quote socketserver collections.UserDict collections.UserList collections.UserString xmlrpc.client xmlrpc.server
18. Print is a function (as are a few others)
19. Generators ➔ zip() ➔ map() ➔ filter() ➔ dictionary's .keys() method ➔ dictionary's .values() method ➔ dictionary's .items() method
20. Python 2 3/2 = 1 3.0/2.0 = 1.5 Integer division
21. Python 2 3/2 = 1 3.0/2.0 = 1.5 Integer division Python 3 3/2 = 1.5 3.0/2.0 = 1.5 3//2 = 1
22. Comparing unorderable types [1, 2] > 'foo' = False (1, 2) > 'foo' = True [1, 2] > (1, 2) = False
23. Comparing unorderable types [1, 2] > 'foo' = False (1, 2) > 'foo' = True [1, 2] > (1, 2) = False TypeError: unorderable types: list() > str()
24. Improved variable unpacking a, b, c = range(3) a = 0 b = 1 c = 2
25. Improved variable unpacking a, b, c = range(3) a = 0 b = 1 c = 2 a, *rest, b = range(10) a = 0 rest = [2, 3, 4, 5, 6, 7, 8, 9] b = 9
26. Keyword-only arguments >>> def f(a, b, *args, option=True): ... print(repr(args)) ... print(repr(option)) ...
27. Keyword-only arguments >>> def f(a, b, *args, option=True): ... print(repr(args)) ... print(repr(option)) ... >>> f(1, 2, False) (False,) True
28. Keyword-only arguments >>> def f(a, b, *args, option=True): ... print(repr(args)) ... print(repr(option)) ... >>> f(1, 2, False) (False,) True >>> f(1, 2, option=False) () False
29. Keyword-only arguments >>> def f(a, b, *, option=True): ... print(repr(option)) ...
30. Keyword-only arguments >>> def f(a, b, *, option=True): ... print(repr(option)) ... >>> f(1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: f() takes 2 positional arguments but 3 were given
31. Keyword-only arguments >>> def f(a, b, *, option=True): ... print(repr(option)) ... >>> f(1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: f() takes 2 positional arguments but 3 were given >>> f(1, 2, option=False) False
32. PEP 3134 Exception Chaining and Embedded Tracebacks
33. Chained exceptions >>> import socket >>> >>> class IPLookupError(Exception): ... """Unable to resolve hostname to IP address""" ... >>> >>> def hostname_to_ip(hostname): ... try: ... return socket.gethostbyname(hostname) ... except socket.gaierror: ... raise IPLookupError("Unable to resolve '%s'" % hostname) ...
34. Chained exceptions >>> print(hostname_to_ip("nick.groenen.me")) 104.28.24.37
35. Chained exceptions >>> print(hostname_to_ip("nick.groenen.me")) 104.28.24.37 >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost'
36. Chained exceptions >>> print(hostname_to_ip("nick.groenen.me")) 104.28.24.37 >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost' Where did socket.gaierror go?!?
37. Chained exceptions >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File "<stdin>", line 3, in hostname_to_ip socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost'
38. raise from ..
39. Chained exceptions >>> def hostname_to_ip(hostname): ... try: ... return socket.gethostbyname(hostname) ... except socket.gaierror: ... raise IPLookupError("Unable to resolve '%s'" % hostname) ...
40. Chained exceptions >>> def hostname_to_ip(hostname): ... try: ... return socket.gethostbyname(hostname) ... except socket.gaierror as e: ... raise IPLookupError("Unable to resolve '%s'" % hostname) from e ...
41. Chained exceptions >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File "<stdin>", line 3, in hostname_to_ip socket.gaierror: [Errno -2] Name or service not known The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost'
42. OSError subclasses try: shutil.copy2(source, dest) except OSError as e: if e.errno in [errno.EPERM, errno.EACCES]: sudo(shutil.copy2(source, dest)) else: raise
43. OSError subclasses try: shutil.copy2(source, dest) except PermissionError: sudo(shutil.copy2(source, dest))
44. asyncio (formerly Tulip)
45. >>> def f(x: int) -> float: ... pass … >>> f.__annotations__ {'return': <class 'float'>, 'x': <class 'int'>} Function annotations
46. Pyc files No more `find . -name '*.pyc' -delete`
47. venv
48. Python 3 looks awesome! Should I stop caring about Python 2 now?
49. Target both Python 2 and Python 3
50. Start with Python 3 only
51. Target Python 3.3 and up
52. Get your character {en,de} codings/unicode right
53. The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) http://www.joelonsoftware.com/articles/Unicode.html
54. ➔ Encoded text comes in (socket, pipe, file, etc) ➔ .decode() ➔ Unicode internally ➔ .encode() ➔ Encoded text goes out Unicode sandwich
55. Start porting code
56. Have extensive test coverage You already have this, right? :-)
57. Target Python 2.7 and 3.3+ from __future__ import absolute_import, division, print_function PY2 = sys.version_info[0] < 3 PY3 = not PY2 if PY3: from urllib.parse import urljoin from collections import MutableMapping as DictMixin else: from urlparse import urljoin from UserDict import DictMixin
58. Python-Future from __future__ import ( absolute_import, division, print_function, ) from builtins import (bytes, str, open, super, range, zip, round, input, int, pow, object)
59. Python-Future # Backported Py3 bytes object b = bytes(b'ABCD') assert repr(b) == "b'ABCD'" # These raise TypeErrors: # b + u'EFGH' # Backported Py3 str object s = str(u'ABCD') assert s != bytes(b'ABCD') # These raise TypeErrors: # bytes(b'B') in s # input() replaces Py2's raw_input() (with no eval()): name = input('What is your name? ') print('Hello ' + name)
60. Python-Future # Many Py3 module names are supported directly on both Py2.x and 3.x: from http.client import HttpConnection import html.parser import queue import xmlrpc.client # Refactored modules with clashing names on Py2 and Py3 are supported # as follows: from future import standard_library standard_library.install_aliases() # Then, for example: from itertools import filterfalse, zip_longest from urllib.request import urlopen from collections import Counter, OrderedDict # backported to Py2.6 from collections import UserDict, UserList, UserString from subprocess import getoutput, getstatusoutput
61. Python-Future ➔ Py2-only -> Py2/Py3 with futurize ➔ Py3-only -> Py2/Py3 with pasturize
62. Six ➔ Uglier (but less magicky) ➔ Supports Python 2.5 and Python < 3.3
63. 2to3 $ 2to3 -w my_python_2_script.py ➔ There's also a separate 3to2 to go the other way around
64. 2to3 from setuptools import setup setup( name='your.module', version = '1.0', description='A description of your module', packages = ['your', 'you.module'], use_2to3 = True, convert_2to3_doctests = ['src/your/module/README.txt'], )
65. Thank you!
66. Resources ➔ Porting to Python 3 - The Book Site http://python3porting.com/bookindex.html ➔ Can I use Python 3? https://caniusepython3.com/ ➔ Python 3 Readiness http://py3readiness.org/ ➔ Porting to Python 3 (Django) https://docs.djangoproject.com/en/1.7/topics/python3/ ➔ PortingToPy3k https://wiki.python.org/moin/PortingToPy3k/ ➔ The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) http://www.joelonsoftware.com/articles/Unicode.html