Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Preparing for the future - Python 3 is here to stay

Preparing for the future - Python 3 is here to stay

A look at the history and current adoption of Python 2 and 3, major differences between the two version, hurdles to porting and finally a set of tips and recommendations for writing code to run on both Python 2 and 3.

This talk was presented at the Amsterdam Python meetup group on 2015-03-24 (http://www.meetup.com/Amsterdam-Python-Meetup-Group/events/220704819/).

18bdcd2f189a9b8fe2f836dde8db09fc?s=128

Nick Groenen

March 24, 2015
Tweet

More Decks by Nick Groenen

Other Decks in Programming

Transcript

  1. Preparing for the future Python 3 is here to stay

    #py020
  2. Hi, I'm Nick https://nick.groenen.me/about/

  3. Are you using Python 3 yet?

  4. ➔ Python 3.0 final was released on December 3rd, 2008.

    ➔ The final 2.x version (2.7) release came out mid-2010 ➔ Python 2.7 support extended by 5 years, to 2020 A little bit of history
  5. Today Django Flask NumPy SciPy Boto Pandas Tornado Celery SQLAlchemy

    PyPy
  6. Tomorrow ➔ OpenStack ➔ Debian ➔ Ubuntu ➔ Fedora

  7. Stragglers ➔ Twisted ➔ gevent ➔ Jython ➔ IronPython

  8. Like it or not The future is Python 3000

  9. What's so different about Python 3?

  10. ➔ Cleans up some warts in the language ➔ More

    readable, consistent & explicit Intentionally backward-incompatible
  11. The unicode "issue"

  12. PEP 3100 Miscellaneous Python 3.0 Plans

  13. Python 2 Unicode vs. 8-bit Python 3 Text vs. data

  14. Python 2 Unicode vs. 8-bit ASCII str type Separate unicode

    type No byte type Python 3 Text vs. data str is unicode text byte and bytearray for encoded text
  15. Python 2 allows mixing of str and unicode objects >>>

    u'This is unicode.' + " This isn't" u"This is unicode. This isn't"
  16. Python 3 does not >>> "This is unicode." + b'

    These are bytes.' TypeError: Can't convert 'bytes' object to str implicitly >>> "This is unicode." + b' These are bytes.'.decode('us-ascii') 'This is unicode. These are bytes.'
  17. Renames cookielib Cookie htmlentitydefs HTMLParser httplib BaseHTTPServer CGIHTTPServer SimpleHTTPServer pipes.quote

    SocketServer UserDict.UserDict UserList.UserList UserString.UserString xmlrpclib SimpleXMLRPCServer http.cookiejar http.cookies html.entities html.parser http.client http.server http.server http.server shlex.quote socketserver collections.UserDict collections.UserList collections.UserString xmlrpc.client xmlrpc.server
  18. Print is a function (as are a few others)

  19. Generators ➔ zip() ➔ map() ➔ filter() ➔ dictionary's .keys()

    method ➔ dictionary's .values() method ➔ dictionary's .items() method
  20. Python 2 3/2 = 1 3.0/2.0 = 1.5 Integer division

  21. Python 2 3/2 = 1 3.0/2.0 = 1.5 Integer division

    Python 3 3/2 = 1.5 3.0/2.0 = 1.5 3//2 = 1
  22. Comparing unorderable types [1, 2] > 'foo' = False (1,

    2) > 'foo' = True [1, 2] > (1, 2) = False
  23. Comparing unorderable types [1, 2] > 'foo' = False (1,

    2) > 'foo' = True [1, 2] > (1, 2) = False TypeError: unorderable types: list() > str()
  24. Improved variable unpacking a, b, c = range(3) a =

    0 b = 1 c = 2
  25. Improved variable unpacking a, b, c = range(3) a =

    0 b = 1 c = 2 a, *rest, b = range(10) a = 0 rest = [2, 3, 4, 5, 6, 7, 8, 9] b = 9
  26. Keyword-only arguments >>> def f(a, b, *args, option=True): ... print(repr(args))

    ... print(repr(option)) ...
  27. Keyword-only arguments >>> def f(a, b, *args, option=True): ... print(repr(args))

    ... print(repr(option)) ... >>> f(1, 2, False) (False,) True
  28. Keyword-only arguments >>> def f(a, b, *args, option=True): ... print(repr(args))

    ... print(repr(option)) ... >>> f(1, 2, False) (False,) True >>> f(1, 2, option=False) () False
  29. Keyword-only arguments >>> def f(a, b, *, option=True): ... print(repr(option))

    ...
  30. Keyword-only arguments >>> def f(a, b, *, option=True): ... print(repr(option))

    ... >>> f(1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: f() takes 2 positional arguments but 3 were given
  31. Keyword-only arguments >>> def f(a, b, *, option=True): ... print(repr(option))

    ... >>> f(1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: f() takes 2 positional arguments but 3 were given >>> f(1, 2, option=False) False
  32. PEP 3134 Exception Chaining and Embedded Tracebacks

  33. Chained exceptions >>> import socket >>> >>> class IPLookupError(Exception): ...

    """Unable to resolve hostname to IP address""" ... >>> >>> def hostname_to_ip(hostname): ... try: ... return socket.gethostbyname(hostname) ... except socket.gaierror: ... raise IPLookupError("Unable to resolve '%s'" % hostname) ...
  34. Chained exceptions >>> print(hostname_to_ip("nick.groenen.me")) 104.28.24.37

  35. Chained exceptions >>> print(hostname_to_ip("nick.groenen.me")) 104.28.24.37 >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent

    call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost'
  36. Chained exceptions >>> print(hostname_to_ip("nick.groenen.me")) 104.28.24.37 >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent

    call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost' Where did socket.gaierror go?!?
  37. Chained exceptions >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File

    "<stdin>", line 3, in hostname_to_ip socket.gaierror: [Errno -2] Name or service not known During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost'
  38. raise from ..

  39. Chained exceptions >>> def hostname_to_ip(hostname): ... try: ... return socket.gethostbyname(hostname)

    ... except socket.gaierror: ... raise IPLookupError("Unable to resolve '%s'" % hostname) ...
  40. Chained exceptions >>> def hostname_to_ip(hostname): ... try: ... return socket.gethostbyname(hostname)

    ... except socket.gaierror as e: ... raise IPLookupError("Unable to resolve '%s'" % hostname) from e ...
  41. Chained exceptions >>> print(hostname_to_ip("nosuchhost")) Traceback (most recent call last): File

    "<stdin>", line 3, in hostname_to_ip socket.gaierror: [Errno -2] Name or service not known The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in hostname_to_ip __main__.IPLookupError: Unable to resolve 'nosuchhost'
  42. OSError subclasses try: shutil.copy2(source, dest) except OSError as e: if

    e.errno in [errno.EPERM, errno.EACCES]: sudo(shutil.copy2(source, dest)) else: raise
  43. OSError subclasses try: shutil.copy2(source, dest) except PermissionError: sudo(shutil.copy2(source, dest))

  44. asyncio (formerly Tulip)

  45. >>> def f(x: int) -> float: ... pass … >>>

    f.__annotations__ {'return': <class 'float'>, 'x': <class 'int'>} Function annotations
  46. Pyc files No more `find . -name '*.pyc' -delete`

  47. venv

  48. Python 3 looks awesome! Should I stop caring about Python

    2 now?
  49. Target both Python 2 and Python 3

  50. Start with Python 3 only

  51. Target Python 3.3 and up

  52. Get your character {en,de} codings/unicode right

  53. The Absolute Minimum Every Software Developer Absolutely, Positively Must Know

    About Unicode and Character Sets (No Excuses!) http://www.joelonsoftware.com/articles/Unicode.html
  54. ➔ Encoded text comes in (socket, pipe, file, etc) ➔

    .decode() ➔ Unicode internally ➔ .encode() ➔ Encoded text goes out Unicode sandwich
  55. Start porting code

  56. Have extensive test coverage You already have this, right? :-)

  57. Target Python 2.7 and 3.3+ from __future__ import absolute_import, division,

    print_function PY2 = sys.version_info[0] < 3 PY3 = not PY2 if PY3: from urllib.parse import urljoin from collections import MutableMapping as DictMixin else: from urlparse import urljoin from UserDict import DictMixin
  58. Python-Future from __future__ import ( absolute_import, division, print_function, ) from

    builtins import (bytes, str, open, super, range, zip, round, input, int, pow, object)
  59. Python-Future # Backported Py3 bytes object b = bytes(b'ABCD') assert

    repr(b) == "b'ABCD'" # These raise TypeErrors: # b + u'EFGH' # Backported Py3 str object s = str(u'ABCD') assert s != bytes(b'ABCD') # These raise TypeErrors: # bytes(b'B') in s # input() replaces Py2's raw_input() (with no eval()): name = input('What is your name? ') print('Hello ' + name)
  60. Python-Future # Many Py3 module names are supported directly on

    both Py2.x and 3.x: from http.client import HttpConnection import html.parser import queue import xmlrpc.client # Refactored modules with clashing names on Py2 and Py3 are supported # as follows: from future import standard_library standard_library.install_aliases() # Then, for example: from itertools import filterfalse, zip_longest from urllib.request import urlopen from collections import Counter, OrderedDict # backported to Py2.6 from collections import UserDict, UserList, UserString from subprocess import getoutput, getstatusoutput
  61. Python-Future ➔ Py2-only -> Py2/Py3 with futurize ➔ Py3-only ->

    Py2/Py3 with pasturize
  62. Six ➔ Uglier (but less magicky) ➔ Supports Python 2.5

    and Python < 3.3
  63. 2to3 $ 2to3 -w my_python_2_script.py ➔ There's also a separate

    3to2 to go the other way around
  64. 2to3 from setuptools import setup setup( name='your.module', version = '1.0',

    description='A description of your module', packages = ['your', 'you.module'], use_2to3 = True, convert_2to3_doctests = ['src/your/module/README.txt'], )
  65. Thank you!

  66. Resources ➔ Porting to Python 3 - The Book Site

    http://python3porting.com/bookindex.html ➔ Can I use Python 3? https://caniusepython3.com/ ➔ Python 3 Readiness http://py3readiness.org/ ➔ Porting to Python 3 (Django) https://docs.djangoproject.com/en/1.7/topics/python3/ ➔ PortingToPy3k https://wiki.python.org/moin/PortingToPy3k/ ➔ The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) http://www.joelonsoftware.com/articles/Unicode.html