Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Preparing for the future - Python 3 is here to stay

Preparing for the future - Python 3 is here to stay

A look at the history and current adoption of Python 2 and 3, major differences between the two version, hurdles to porting and finally a set of tips and recommendations for writing code to run on both Python 2 and 3.

This talk was presented at the Amsterdam Python meetup group on 2015-03-24 (http://www.meetup.com/Amsterdam-Python-Meetup-Group/events/220704819/).

Nick Groenen

March 24, 2015
Tweet

More Decks by Nick Groenen

Other Decks in Programming

Transcript

  1. Preparing for the future
    Python 3 is here to stay
    #py020

    View full-size slide

  2. Hi, I'm Nick
    https://nick.groenen.me/about/

    View full-size slide

  3. Are you using Python 3 yet?

    View full-size slide

  4. ➔ Python 3.0 final was released on December 3rd, 2008.
    ➔ The final 2.x version (2.7) release came out mid-2010
    ➔ Python 2.7 support extended by 5 years, to 2020
    A little bit of history

    View full-size slide

  5. Today
    Django
    Flask
    NumPy
    SciPy
    Boto
    Pandas
    Tornado
    Celery
    SQLAlchemy
    PyPy

    View full-size slide

  6. Tomorrow
    ➔ OpenStack
    ➔ Debian
    ➔ Ubuntu
    ➔ Fedora

    View full-size slide

  7. Stragglers
    ➔ Twisted
    ➔ gevent
    ➔ Jython
    ➔ IronPython

    View full-size slide

  8. Like it or not
    The future is Python 3000

    View full-size slide

  9. What's so different about
    Python 3?

    View full-size slide

  10. ➔ Cleans up some warts in the language
    ➔ More readable, consistent & explicit
    Intentionally backward-incompatible

    View full-size slide

  11. The unicode "issue"

    View full-size slide

  12. PEP 3100
    Miscellaneous Python 3.0 Plans

    View full-size slide

  13. Python 2
    Unicode vs. 8-bit
    Python 3
    Text vs. data

    View full-size slide

  14. Python 2
    Unicode vs. 8-bit
    ASCII str type
    Separate unicode type
    No byte type
    Python 3
    Text vs. data
    str is unicode text
    byte and bytearray for
    encoded text

    View full-size slide

  15. Python 2 allows mixing of
    str and unicode objects
    >>> u'This is unicode.' + " This isn't"
    u"This is unicode. This isn't"

    View full-size slide

  16. Python 3 does not
    >>> "This is unicode." + b' These are bytes.'
    TypeError: Can't convert 'bytes' object to str implicitly
    >>> "This is unicode." + b' These are bytes.'.decode('us-ascii')
    'This is unicode. These are bytes.'

    View full-size slide

  17. Renames
    cookielib
    Cookie
    htmlentitydefs
    HTMLParser
    httplib
    BaseHTTPServer
    CGIHTTPServer
    SimpleHTTPServer
    pipes.quote
    SocketServer
    UserDict.UserDict
    UserList.UserList
    UserString.UserString
    xmlrpclib
    SimpleXMLRPCServer
    http.cookiejar
    http.cookies
    html.entities
    html.parser
    http.client
    http.server
    http.server
    http.server
    shlex.quote
    socketserver
    collections.UserDict
    collections.UserList
    collections.UserString
    xmlrpc.client
    xmlrpc.server

    View full-size slide

  18. Print is a function
    (as are a few others)

    View full-size slide

  19. Generators
    ➔ zip()
    ➔ map()
    ➔ filter()
    ➔ dictionary's .keys() method
    ➔ dictionary's .values() method
    ➔ dictionary's .items() method

    View full-size slide

  20. Python 2
    3/2 = 1
    3.0/2.0 = 1.5
    Integer division

    View full-size slide

  21. Python 2
    3/2 = 1
    3.0/2.0 = 1.5
    Integer division
    Python 3
    3/2 = 1.5
    3.0/2.0 = 1.5
    3//2 = 1

    View full-size slide

  22. Comparing unorderable types
    [1, 2] > 'foo' = False
    (1, 2) > 'foo' = True
    [1, 2] > (1, 2) = False

    View full-size slide

  23. Comparing unorderable types
    [1, 2] > 'foo' = False
    (1, 2) > 'foo' = True
    [1, 2] > (1, 2) = False
    TypeError: unorderable types: list() > str()

    View full-size slide

  24. Improved variable unpacking
    a, b, c = range(3)
    a = 0
    b = 1
    c = 2

    View full-size slide

  25. Improved variable unpacking
    a, b, c = range(3)
    a = 0
    b = 1
    c = 2
    a, *rest, b = range(10)
    a = 0
    rest = [2, 3, 4, 5, 6, 7, 8, 9]
    b = 9

    View full-size slide

  26. Keyword-only arguments
    >>> def f(a, b, *args, option=True):
    ... print(repr(args))
    ... print(repr(option))
    ...

    View full-size slide

  27. Keyword-only arguments
    >>> def f(a, b, *args, option=True):
    ... print(repr(args))
    ... print(repr(option))
    ...
    >>> f(1, 2, False)
    (False,)
    True

    View full-size slide

  28. Keyword-only arguments
    >>> def f(a, b, *args, option=True):
    ... print(repr(args))
    ... print(repr(option))
    ...
    >>> f(1, 2, False)
    (False,)
    True
    >>> f(1, 2, option=False)
    ()
    False

    View full-size slide

  29. Keyword-only arguments
    >>> def f(a, b, *, option=True):
    ... print(repr(option))
    ...

    View full-size slide

  30. Keyword-only arguments
    >>> def f(a, b, *, option=True):
    ... print(repr(option))
    ...
    >>> f(1, 2, 3)
    Traceback (most recent call last):
    File "", line 1, in
    TypeError: f() takes 2 positional arguments but 3 were given

    View full-size slide

  31. Keyword-only arguments
    >>> def f(a, b, *, option=True):
    ... print(repr(option))
    ...
    >>> f(1, 2, 3)
    Traceback (most recent call last):
    File "", line 1, in
    TypeError: f() takes 2 positional arguments but 3 were given
    >>> f(1, 2, option=False)
    False

    View full-size slide

  32. PEP 3134
    Exception Chaining and Embedded Tracebacks

    View full-size slide

  33. Chained exceptions
    >>> import socket
    >>>
    >>> class IPLookupError(Exception):
    ... """Unable to resolve hostname to IP address"""
    ...
    >>>
    >>> def hostname_to_ip(hostname):
    ... try:
    ... return socket.gethostbyname(hostname)
    ... except socket.gaierror:
    ... raise IPLookupError("Unable to resolve '%s'" % hostname)
    ...

    View full-size slide

  34. Chained exceptions
    >>> print(hostname_to_ip("nick.groenen.me"))
    104.28.24.37

    View full-size slide

  35. Chained exceptions
    >>> print(hostname_to_ip("nick.groenen.me"))
    104.28.24.37
    >>> print(hostname_to_ip("nosuchhost"))
    Traceback (most recent call last):
    File "", line 1, in
    File "", line 5, in hostname_to_ip
    __main__.IPLookupError: Unable to resolve 'nosuchhost'

    View full-size slide

  36. Chained exceptions
    >>> print(hostname_to_ip("nick.groenen.me"))
    104.28.24.37
    >>> print(hostname_to_ip("nosuchhost"))
    Traceback (most recent call last):
    File "", line 1, in
    File "", line 5, in hostname_to_ip
    __main__.IPLookupError: Unable to resolve 'nosuchhost'
    Where did socket.gaierror go?!?

    View full-size slide

  37. Chained exceptions
    >>> print(hostname_to_ip("nosuchhost"))
    Traceback (most recent call last):
    File "", line 3, in hostname_to_ip
    socket.gaierror: [Errno -2] Name or service not known
    During handling of the above exception, another exception occurred:
    Traceback (most recent call last):
    File "", line 1, in
    File "", line 5, in hostname_to_ip
    __main__.IPLookupError: Unable to resolve 'nosuchhost'

    View full-size slide

  38. raise from ..

    View full-size slide

  39. Chained exceptions
    >>> def hostname_to_ip(hostname):
    ... try:
    ... return socket.gethostbyname(hostname)
    ... except socket.gaierror:
    ... raise IPLookupError("Unable to resolve '%s'" % hostname)
    ...

    View full-size slide

  40. Chained exceptions
    >>> def hostname_to_ip(hostname):
    ... try:
    ... return socket.gethostbyname(hostname)
    ... except socket.gaierror as e:
    ... raise IPLookupError("Unable to resolve '%s'" % hostname) from e
    ...

    View full-size slide

  41. Chained exceptions
    >>> print(hostname_to_ip("nosuchhost"))
    Traceback (most recent call last):
    File "", line 3, in hostname_to_ip
    socket.gaierror: [Errno -2] Name or service not known
    The above exception was the direct cause of the following exception:
    Traceback (most recent call last):
    File "", line 1, in
    File "", line 5, in hostname_to_ip
    __main__.IPLookupError: Unable to resolve 'nosuchhost'

    View full-size slide

  42. OSError subclasses
    try:
    shutil.copy2(source, dest)
    except OSError as e:
    if e.errno in [errno.EPERM, errno.EACCES]:
    sudo(shutil.copy2(source, dest))
    else:
    raise

    View full-size slide

  43. OSError subclasses
    try:
    shutil.copy2(source, dest)
    except PermissionError:
    sudo(shutil.copy2(source, dest))

    View full-size slide

  44. asyncio
    (formerly Tulip)

    View full-size slide

  45. >>> def f(x: int) -> float:
    ... pass

    >>> f.__annotations__
    {'return': , 'x': }
    Function annotations

    View full-size slide

  46. Pyc files
    No more `find . -name '*.pyc' -delete`

    View full-size slide

  47. Python 3 looks awesome!
    Should I stop caring about
    Python 2 now?

    View full-size slide

  48. Target both Python 2 and
    Python 3

    View full-size slide

  49. Start with Python 3 only

    View full-size slide

  50. Target Python 3.3 and up

    View full-size slide

  51. Get your character {en,de}
    codings/unicode right

    View full-size slide

  52. The Absolute Minimum Every Software
    Developer Absolutely, Positively Must Know
    About Unicode and Character Sets (No
    Excuses!)
    http://www.joelonsoftware.com/articles/Unicode.html

    View full-size slide

  53. ➔ Encoded text comes in (socket, pipe, file, etc)
    ➔ .decode()
    ➔ Unicode internally
    ➔ .encode()
    ➔ Encoded text goes out
    Unicode sandwich

    View full-size slide

  54. Start porting code

    View full-size slide

  55. Have extensive test
    coverage
    You already have this, right? :-)

    View full-size slide

  56. Target Python 2.7 and 3.3+
    from __future__ import absolute_import, division, print_function
    PY2 = sys.version_info[0] < 3
    PY3 = not PY2
    if PY3:
    from urllib.parse import urljoin
    from collections import MutableMapping as DictMixin
    else:
    from urlparse import urljoin
    from UserDict import DictMixin

    View full-size slide

  57. Python-Future
    from __future__ import (
    absolute_import,
    division,
    print_function,
    )
    from builtins import (bytes, str, open, super, range,
    zip, round, input, int, pow, object)

    View full-size slide

  58. Python-Future
    # Backported Py3 bytes object
    b = bytes(b'ABCD')
    assert repr(b) == "b'ABCD'"
    # These raise TypeErrors:
    # b + u'EFGH'
    # Backported Py3 str object
    s = str(u'ABCD')
    assert s != bytes(b'ABCD')
    # These raise TypeErrors:
    # bytes(b'B') in s
    # input() replaces Py2's raw_input() (with no eval()):
    name = input('What is your name? ')
    print('Hello ' + name)

    View full-size slide

  59. Python-Future
    # Many Py3 module names are supported directly on both Py2.x and 3.x:
    from http.client import HttpConnection
    import html.parser
    import queue
    import xmlrpc.client
    # Refactored modules with clashing names on Py2 and Py3 are supported
    # as follows:
    from future import standard_library
    standard_library.install_aliases()
    # Then, for example:
    from itertools import filterfalse, zip_longest
    from urllib.request import urlopen
    from collections import Counter, OrderedDict # backported to Py2.6
    from collections import UserDict, UserList, UserString
    from subprocess import getoutput, getstatusoutput

    View full-size slide

  60. Python-Future
    ➔ Py2-only -> Py2/Py3 with futurize
    ➔ Py3-only -> Py2/Py3 with pasturize

    View full-size slide

  61. Six
    ➔ Uglier (but less magicky)
    ➔ Supports Python 2.5 and Python < 3.3

    View full-size slide

  62. 2to3
    $ 2to3 -w my_python_2_script.py
    ➔ There's also a separate 3to2 to go the other
    way around

    View full-size slide

  63. 2to3
    from setuptools import setup
    setup(
    name='your.module',
    version = '1.0',
    description='A description of your module',
    packages = ['your', 'you.module'],
    use_2to3 = True,
    convert_2to3_doctests = ['src/your/module/README.txt'],
    )

    View full-size slide

  64. Resources
    ➔ Porting to Python 3 - The Book Site
    http://python3porting.com/bookindex.html
    ➔ Can I use Python 3?
    https://caniusepython3.com/
    ➔ Python 3 Readiness
    http://py3readiness.org/
    ➔ Porting to Python 3 (Django)
    https://docs.djangoproject.com/en/1.7/topics/python3/
    ➔ PortingToPy3k
    https://wiki.python.org/moin/PortingToPy3k/
    ➔ The Absolute Minimum Every Software Developer Absolutely, Positively Must Know
    About Unicode and Character Sets (No Excuses!)
    http://www.joelonsoftware.com/articles/Unicode.html

    View full-size slide