Slide 1

Slide 1 text

Preparing for the future Python 3 is here to stay #py020

Slide 2

Slide 2 text

Hi, I'm Nick https://nick.groenen.me/about/

Slide 3

Slide 3 text

Are you using Python 3 yet?

Slide 4

Slide 4 text

➔ Python 3.0 final was released on December 3rd, 2008. ➔ The final 2.x version (2.7) release came out mid-2010 ➔ Python 2.7 support extended by 5 years, to 2020 A little bit of history

Slide 5

Slide 5 text

Today Django Flask NumPy SciPy Boto Pandas Tornado Celery SQLAlchemy PyPy

Slide 6

Slide 6 text

Tomorrow ➔ OpenStack ➔ Debian ➔ Ubuntu ➔ Fedora

Slide 7

Slide 7 text

Stragglers ➔ Twisted ➔ gevent ➔ Jython ➔ IronPython

Slide 8

Slide 8 text

Like it or not The future is Python 3000

Slide 9

Slide 9 text

What's so different about Python 3?

Slide 10

Slide 10 text

➔ Cleans up some warts in the language ➔ More readable, consistent & explicit Intentionally backward-incompatible

Slide 11

Slide 11 text

The unicode "issue"

Slide 12

Slide 12 text

PEP 3100 Miscellaneous Python 3.0 Plans

Slide 13

Slide 13 text

Python 2 Unicode vs. 8-bit Python 3 Text vs. data

Slide 14

Slide 14 text

Python 2 Unicode vs. 8-bit ASCII str type Separate unicode type No byte type Python 3 Text vs. data str is unicode text byte and bytearray for encoded text

Slide 15

Slide 15 text

Python 2 allows mixing of str and unicode objects >>> u'This is unicode.' + " This isn't" u"This is unicode. This isn't"

Slide 16

Slide 16 text

Python 3 does not >>> "This is unicode." + b' These are bytes.' TypeError: Can't convert 'bytes' object to str implicitly >>> "This is unicode." + b' These are bytes.'.decode('us-ascii') 'This is unicode. These are bytes.'

Slide 17

Slide 17 text

Renames cookielib Cookie htmlentitydefs HTMLParser httplib BaseHTTPServer CGIHTTPServer SimpleHTTPServer pipes.quote SocketServer UserDict.UserDict UserList.UserList UserString.UserString xmlrpclib SimpleXMLRPCServer http.cookiejar http.cookies html.entities html.parser http.client http.server http.server http.server shlex.quote socketserver collections.UserDict collections.UserList collections.UserString xmlrpc.client xmlrpc.server

Slide 18

Slide 18 text

Print is a function (as are a few others)

Slide 19

Slide 19 text

Generators ➔ zip() ➔ map() ➔ filter() ➔ dictionary's .keys() method ➔ dictionary's .values() method ➔ dictionary's .items() method

Slide 20

Slide 20 text

Python 2 3/2 = 1 3.0/2.0 = 1.5 Integer division

Slide 21

Slide 21 text

Python 2 3/2 = 1 3.0/2.0 = 1.5 Integer division Python 3 3/2 = 1.5 3.0/2.0 = 1.5 3//2 = 1

Slide 22

Slide 22 text

Comparing unorderable types [1, 2] > 'foo' = False (1, 2) > 'foo' = True [1, 2] > (1, 2) = False

Slide 23

Slide 23 text

Comparing unorderable types [1, 2] > 'foo' = False (1, 2) > 'foo' = True [1, 2] > (1, 2) = False TypeError: unorderable types: list() > str()

Slide 24

Slide 24 text

Improved variable unpacking a, b, c = range(3) a = 0 b = 1 c = 2

Slide 25

Slide 25 text

Improved variable unpacking a, b, c = range(3) a = 0 b = 1 c = 2 a, *rest, b = range(10) a = 0 rest = [2, 3, 4, 5, 6, 7, 8, 9] b = 9

Slide 26

Slide 26 text

Keyword-only arguments >>> def f(a, b, *args, option=True): ... print(repr(args)) ... print(repr(option)) ...

Slide 27

Slide 27 text

Keyword-only arguments >>> def f(a, b, *args, option=True): ... print(repr(args)) ... print(repr(option)) ... >>> f(1, 2, False) (False,) True

Slide 28

Slide 28 text

Keyword-only arguments >>> def f(a, b, *args, option=True): ... print(repr(args)) ... print(repr(option)) ... >>> f(1, 2, False) (False,) True >>> f(1, 2, option=False) () False

Slide 29

Slide 29 text

Keyword-only arguments >>> def f(a, b, *, option=True): ... print(repr(option)) ...

Slide 30

Slide 30 text

Keyword-only arguments >>> def f(a, b, *, option=True): ... print(repr(option)) ... >>> f(1, 2, 3) Traceback (most recent call last): File "", line 1, in TypeError: f() takes 2 positional arguments but 3 were given

Slide 31

Slide 31 text

Keyword-only arguments >>> def f(a, b, *, option=True): ... print(repr(option)) ... >>> f(1, 2, 3) Traceback (most recent call last): File "", line 1, in TypeError: f() takes 2 positional arguments but 3 were given >>> f(1, 2, option=False) False

Slide 32

Slide 32 text

PEP 3134 Exception Chaining and Embedded Tracebacks

Slide 33

Slide 33 text

Chained exceptions >>> import socket >>> >>> class IPLookupError(Exception): ... """Unable to resolve hostname to IP address""" ... >>> >>> def hostname_to_ip(hostname): ... try: ... return socket.gethostbyname(hostname) ... except socket.gaierror: ... raise IPLookupError("Unable to resolve '%s'" % hostname) ...

Slide 34

Slide 34 text

Chained exceptions >>> print(hostname_to_ip("nick.groenen.me")) 104.28.24.37

Slide 35

Slide 35 text

Chained exceptions >>> print(hostname_to_ip("nick.groenen.me")) 104.28.24.37 >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File "", line 1, in File "", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost'

Slide 36

Slide 36 text

Chained exceptions >>> print(hostname_to_ip("nick.groenen.me")) 104.28.24.37 >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File "", line 1, in File "", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost' Where did socket.gaierror go?!?

Slide 37

Slide 37 text

Chained exceptions >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File "", line 3, in hostname_to_ip socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 1, in File "", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost'

Slide 38

Slide 38 text

raise from ..

Slide 39

Slide 39 text

Chained exceptions >>> def hostname_to_ip(hostname): ... try: ... return socket.gethostbyname(hostname) ... except socket.gaierror: ... raise IPLookupError("Unable to resolve '%s'" % hostname) ...

Slide 40

Slide 40 text

Chained exceptions >>> def hostname_to_ip(hostname): ... try: ... return socket.gethostbyname(hostname) ... except socket.gaierror as e: ... raise IPLookupError("Unable to resolve '%s'" % hostname) from e ...

Slide 41

Slide 41 text

Chained exceptions >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File "", line 3, in hostname_to_ip socket.gaierror: [Errno -2] Name or service not known The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 1, in File "", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost'

Slide 42

Slide 42 text

OSError subclasses try: shutil.copy2(source, dest) except OSError as e: if e.errno in [errno.EPERM, errno.EACCES]: sudo(shutil.copy2(source, dest)) else: raise

Slide 43

Slide 43 text

OSError subclasses try: shutil.copy2(source, dest) except PermissionError: sudo(shutil.copy2(source, dest))

Slide 44

Slide 44 text

asyncio (formerly Tulip)

Slide 45

Slide 45 text

>>> def f(x: int) -> float: ... pass … >>> f.__annotations__ {'return': , 'x': } Function annotations

Slide 46

Slide 46 text

Pyc files No more `find . -name '*.pyc' -delete`

Slide 47

Slide 47 text

venv

Slide 48

Slide 48 text

Python 3 looks awesome! Should I stop caring about Python 2 now?

Slide 49

Slide 49 text

Target both Python 2 and Python 3

Slide 50

Slide 50 text

Start with Python 3 only

Slide 51

Slide 51 text

Target Python 3.3 and up

Slide 52

Slide 52 text

Get your character {en,de} codings/unicode right

Slide 53

Slide 53 text

The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) http://www.joelonsoftware.com/articles/Unicode.html

Slide 54

Slide 54 text

➔ Encoded text comes in (socket, pipe, file, etc) ➔ .decode() ➔ Unicode internally ➔ .encode() ➔ Encoded text goes out Unicode sandwich

Slide 55

Slide 55 text

Start porting code

Slide 56

Slide 56 text

Have extensive test coverage You already have this, right? :-)

Slide 57

Slide 57 text

Target Python 2.7 and 3.3+ from __future__ import absolute_import, division, print_function PY2 = sys.version_info[0] < 3 PY3 = not PY2 if PY3: from urllib.parse import urljoin from collections import MutableMapping as DictMixin else: from urlparse import urljoin from UserDict import DictMixin

Slide 58

Slide 58 text

Python-Future from __future__ import ( absolute_import, division, print_function, ) from builtins import (bytes, str, open, super, range, zip, round, input, int, pow, object)

Slide 59

Slide 59 text

Python-Future # Backported Py3 bytes object b = bytes(b'ABCD') assert repr(b) == "b'ABCD'" # These raise TypeErrors: # b + u'EFGH' # Backported Py3 str object s = str(u'ABCD') assert s != bytes(b'ABCD') # These raise TypeErrors: # bytes(b'B') in s # input() replaces Py2's raw_input() (with no eval()): name = input('What is your name? ') print('Hello ' + name)

Slide 60

Slide 60 text

Python-Future # Many Py3 module names are supported directly on both Py2.x and 3.x: from http.client import HttpConnection import html.parser import queue import xmlrpc.client # Refactored modules with clashing names on Py2 and Py3 are supported # as follows: from future import standard_library standard_library.install_aliases() # Then, for example: from itertools import filterfalse, zip_longest from urllib.request import urlopen from collections import Counter, OrderedDict # backported to Py2.6 from collections import UserDict, UserList, UserString from subprocess import getoutput, getstatusoutput

Slide 61

Slide 61 text

Python-Future ➔ Py2-only -> Py2/Py3 with futurize ➔ Py3-only -> Py2/Py3 with pasturize

Slide 62

Slide 62 text

Six ➔ Uglier (but less magicky) ➔ Supports Python 2.5 and Python < 3.3

Slide 63

Slide 63 text

2to3 $ 2to3 -w my_python_2_script.py ➔ There's also a separate 3to2 to go the other way around

Slide 64

Slide 64 text

2to3 from setuptools import setup setup( name='your.module', version = '1.0', description='A description of your module', packages = ['your', 'you.module'], use_2to3 = True, convert_2to3_doctests = ['src/your/module/README.txt'], )

Slide 65

Slide 65 text

Thank you!

Slide 66

Slide 66 text

Resources ➔ Porting to Python 3 - The Book Site http://python3porting.com/bookindex.html ➔ Can I use Python 3? https://caniusepython3.com/ ➔ Python 3 Readiness http://py3readiness.org/ ➔ Porting to Python 3 (Django) https://docs.djangoproject.com/en/1.7/topics/python3/ ➔ PortingToPy3k https://wiki.python.org/moin/PortingToPy3k/ ➔ The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) http://www.joelonsoftware.com/articles/Unicode.html