Making code compatible with Python 2 and 3

A presentation about writing future-proof code that is compatible with Python 2 and 3.

Transcript

1. Making code
   compatible with
   Python 2 and 3
   

   View Slide

2. Adrián Matellanes
   Lead API Developer at @EburyES & Málaga Python Organizer
   twitter.com/_amatellanes
   github.com/amatellanes
   

   View Slide

3. La Térmica - Málaga
   Saturday, 6 de May, 2017
   Call For Papers
   www.opensouthcode.org
   

   View Slide

4. Python 2 Release Schedule
   

   View Slide

5. The End Of Life date for Python 2.7
   Python 2.7 will retire in 2020
   https://pythonclock.org/
   https://www.python.org/dev/peps/pep-0373/
   

   View Slide

6. 2.8
   Python 2.8
   https://www.python.org/dev/peps/pep-0404/
   

   View Slide

7. Why upgrade to Python 3?
   

   View Slide

8. Why upgrade to Python 3?
   Python 2.x is legacy
   Python 3.x is the present and future of
   the language
   

   View Slide

9. Why upgrade to Python 3?
   “Python 2 is the next COBOL.”
   —Alex Gaynor, at PyCon AU 2013
   

   View Slide

10. Why upgrade to Python 3?
    ● Python 3 support for 360 most downloaded packages on PyPI
    http://py3readiness.org/
    ● Python 3 "Wall of Superpowers"
    https://python3wos.appspot.com/
    ● caniusepython3
    https://pypi.python.org/pypi/caniusepython3
    $ caniusepython3 -r requirements.txt test-requirement.txt
    

    View Slide

11. __future__
    Future statement definitions
    

    View Slide

12. __future__
    nested_scopes PEP 227: Statically Nested Scopes
    generators PEP 255: Simple Generators
    division PEP 238: Changing the Division Operator
    absolute_import PEP 328: Imports: Multi-Line and Absolute/Relative
    with_statement PEP 343: The “with” Statement
    print_function PEP 3105: Make print a function
    unicode_literals PEP 3112: Bytes literals in Python 3000
    https://docs.python.org/2.7/library/__future__.html
    

    View Slide

13. six
    Python 2 and 3 compatibility utilities
    

    View Slide

14. six
    Constants
    ● six.PY2, six.PY3
    ● six.class_types
    ● six.integer_types
    ● six.string_types
    ● six.text_type
    ● six.binary_type
    ● six.MAXSIZE
    https://pypi.python.org/pypi/six
    

    View Slide

15. six
    Object model compatibility
    ● six.next(it)
    ● six.callable(obj)
    ● six.iterkeys(dict, **kwargs)
    ● six.itervalues(dict, **kwargs)
    ● six.viewkeys(dict)
    ● six.viewvalues(dict)
    ● …
    https://pypi.python.org/pypi/six
    

    View Slide

16. six
    Renamed modules and attributes compatibility
    ● six.moves
    Name Python 2 name Python 3 name
    html_parser HTMLParser html.parser
    >>> from six.moves import html_parser
    https://pypi.python.org/pypi/six
    

    View Slide

17. six
    Syntax compatibility
    ● six.exec_(code, globals=None, locals=None)
    ● six.print_(*args, *, file=sys.stdout, end="\n", sep="
    ", flush=False)
    ● six.with_metaclass(metaclass, *bases),
    @six.add_metaclass(metaclass)
    ● …
    https://pypi.python.org/pypi/six
    

    View Slide

18. six
    Binary and text data
    ● six.b(data)
    ● six.u(text)
    ● six.StringIO
    ● six.BytesIO
    ● @six.python_2_unicode_compatible
    ● ...
    https://pypi.python.org/pypi/six
    

    View Slide

19. six
    unittest assertions
    ● six.assertCountEqual()
    ● six.assertRaisesRegex()
    ● six.assertRegex()
    https://pypi.python.org/pypi/six
    

    View Slide

20. six
    Django Support
    ● A customized version of six is bundled with Django as of version 1.4.2.
    ● django.utils.six
    https://docs.djangoproject.com/en/1.10/topics/python3/#writing-compatible-code-with-six
    

    View Slide

21. 2to3
    Automated Python 2 to 3 code
    translation
    

    View Slide

22. 2to3
    # example.py
    def greet(name):
    print 'Hello, {0}!'.format(name)
    print 'What's your name?'
    name = raw_input()
    greet(name)
    https://docs.python.org/2.7/library/2to3.html
    

    View Slide

23. 2to3
    $ 2to3 example.py
    https://docs.python.org/2.7/library/2to3.html
    

    View Slide

24. 2to3
    # example.py
    def greet(name):
    print('Hello, {0}!'.format(name))
    print('What's your name?')
    name = input()
    greet(name)
    https://docs.python.org/2.7/library/2to3.html
    

    View Slide

25. future
    Clean single-source support for Python 3
    and 2
    

    View Slide

26. future
    ● future.builtins
    ● past.builtins
    ● future.standard_library and future.moves
    ● futurize and pasteurize
    ● Support for directly importing 30 standard library modules under their
    Python 3 names on Python 2
    https://pypi.python.org/pypi/future
    

    View Slide

27. Writing Python 2-3 compatible
    code
    

    View Slide

28. Writing Python 2-3 compatible code
    ● Only worry about supporting Python 2.7
    ● Using __future__
    ● Make sure you have good test coverage
    ● Use pylint to help make sure you don’t regress on your Python 3 support
    ● Use caniusepython3 to find out which of your dependencies are
    blocking Python 3
    ● tox can help test against multiple versions of Python
    ● Use 2to3 to rewrite your code to run only under Python 3
    

    View Slide

29. Examples
    

    View Slide

30. print
    # Python 2 only:
    print 'Hello'
    # Python 2 and 3:
    print('Hello')
    # Python 2 and 3:
    from __future__ import print_function # at top of module
    print('Hello', 'Guido')
    

    View Slide

31. print
    # Python 2 only:
    print >> sys.stderr, 'Hello'
    # Python 2 and 3:
    from __future__ import print_function
    print('Hello', file=sys.stderr)
    

    View Slide

32. Raising exceptions
    # Python 2 only:
    raise ValueError, 'dodgy value'
    # Python 2 and 3:
    raise ValueError('dodgy value')
    

    View Slide

33. Catching exceptions
    # Python 2 only:
    Try:
    ...
    except ValueError, e:
    ...
    # Python 2 and 3:
    try:
    ...
    except ValueError as e:
    ...
    

    View Slide

34. Division
    # Integer division (rounding down):
    # Python 2 only:
    assert 2 / 3 == 0
    # Python 2 and 3:
    assert 2 // 3 == 0
    

    View Slide

35. Division
    # True division (float division):
    # Python 3 only:
    assert 3 / 2 == 1.5
    # Python 2 and 3:
    from __future__ import division # at top of module
    assert 3 / 2 == 1.5
    

    View Slide

36. Unicode string literals
    # Python 2 and 3
    s1 = u'The Zen of Python'
    s2 = u'きたないのよりきれいな方がいい \n'
    # Python 2 and 3
    from __future__ import unicode_literals
    http://python-future.org/unicode_literals
    

    View Slide

37. Byte-string literals
    # Python 2 only
    s = 'This must be a byte-string'
    # Python 2 and 3
    s = b'This must be a byte-string'
    

    View Slide

38. StringIO
    # Python 2 only:
    from StringIO import StringIO
    # or:
    from cStringIO import StringIO
    # Python 2 and 3:
    from io import BytesIO # for handling byte strings
    from io import StringIO # for handling unicode strings
    

    View Slide

39. Dictionaries
    # Python 2 only:
    for key in heights.iterkeys():
    ...
    # Python 2 and 3:
    for key in heights:
    ...
    

    View Slide

40. Dictionaries
    # Python 2 only:
    for value in heights.itervalues():
    ...
    # Idiomatic Python 3
    for value in heights.values(): # extra memory overhead on Py2
    ...
    # Python 2 and 3: option 2
    from six import itervalues
    for key in itervalues(heights):
    ...
    

    View Slide

41. Dictionaries
    # Python 2 only:
    for (key, value) in heights.iteritems():
    ...
    # Python 2 and 3: option 1
    for (key, value) in heights.items(): # inefficient on Py2
    ...
    # Python 2 and 3: option 3
    from six import iteritems
    for (key, value) in iteritems(heights):
    ...
    

    View Slide

42. imap
    # Python 2 only:
    from itertools import imap
    myiter = imap(func, mylist)
    # Python 3 only:
    myiter = map(func, mylist)
    # Python 2 and 3: option 2
    try:
    import itertools.imap as map
    except ImportError:
    pass
    

    View Slide

43. Metaclasses
    # Python 2 only:
    class Form(BaseForm):
    __metaclass__ = FormType
    # Python 3 only:
    class Form(BaseForm, metaclass=FormType):
    pass
    # Python 2 and 3:
    from six import with_metaclass
    class Form(with_metaclass(FormType, BaseForm)):
    pass
    

    View Slide

44. Custom __str__ methods
    # Python 2 only:
    class MyClass(object):
    def __unicode__(self):
    return 'Unicode string: \u5b54\u5b50'
    def __str__(self):
    return unicode(self).encode('utf-8')
    

    View Slide

45. Custom __str__ methods
    # Python 2 and 3:
    from future.utils import python_2_unicode_compatible
    @python_2_unicode_compatible
    class MyClass(object):
    def __str__(self):
    return u'Unicode string: \u5b54\u5b50'
    

    View Slide

46. reduce
    # Python 2 only:
    from __builtin__ import reduce
    # Python 2 and 3:
    from functools import reduce
    

    View Slide

47. Custom iterators
    # Python 2 only
    class Upper(object):
    def next(self): # Py2-style
    return self._iter.next().upper()
    assert itr.next() == 'H' # Py2-style
    

    View Slide

48. Custom iterators
    # Python 2 and 3:
    from future.utils import implements_iterator
    @implements_iterator
    class Upper(object):
    def __next__(self): # Py3-style iterator interface
    return next(self._iter).upper()
    assert next(itr) == 'H'
    

    View Slide

49. urllib module
    # Python 2 only:
    from urllib import urlencode
    # Python 3 only:
    from urllib.parse import urlencode
    # Python 2 and 3: easiest option
    from future.standard_library import install_aliases
    install_aliases()
    from urllib.parse import urlencode
    

    View Slide

50. References
    ● “Supporting Python 3: An in-depth guide”
    Lennart Regebro
    ● “Dive Into Python 3”
    Appendix A: Porting Code to Python 3 using 2to3
    Mark Pilgrim
    

    View Slide

51. Thank you!
    Questions?
    

    View Slide