Things I learned porting code to Python 3

English version of slides from my talk about Python 3 porting at PythonRio meetup

Transcript

1. Things I learned porting code
   to Python 3
   Elias Dorneles
   @eliasdorneles
   

   View Slide

2. There is no shame
   in being using
   exclusively
   Python 2!
   

   View Slide

3. Despite what
   the Internet is
   saying at the
   moment :)
   

   View Slide

4. Why Python 3?
   ● It’s a better language
   ○ tuple unpacking
   ○ more consistent APIs
   ○ async/await
   ○ syntax for gradual typing
   ○ Easier for beginners
   ● It’s the future of the ecosystem
   ○ your users want PY3
   

   View Slide

5. Migration strategies
   

   View Slide

6. Migration strategies
   ● Migrate at once to Python 3
   ○ for code with low risks/deps
   ● Fork for Python 3 version
   ○ for highly stable code, that don’t get modifications
   ● Python 2 & 3 in the same codebase
   ○ for code being maintained and used by others
   

   View Slide

7. Let’s focus in Python 2 & 3 in the
   same codebase
   

   View Slide

8. Important to have a good test coverage
   ● Doesn’t need to be 100%, but it’s good to have enough to
   catch important regressions
   ● Run the tests in both Python 2 & Python 3
   ○ tip: ignore the ones aren’t passing in Python 3 yet, try to
   keep passing the ones that already do
   ● Remember the tests may be incorrect in Python 3 and
   needing to change too
   

   View Slide

9. Focus in Python 2.7
   and 3.3+
   

   View Slide

10. Focus in 2.7 and 3.3+
    ● If you’re still on 2.6 or previous, first migrate to 2.7+
    ● Ignore 3.x lesser than 3.3
    ○ didn’t had good enough support (syntax & stdlib) to
    write code compatible with 2
    ○ vendors (distros) generally support 3.3+
    ● Tip for installing several distinct versions of Python:
    ○ http://saghul.github.io/pythonz (CPython, PyPy, etc)
    

    View Slide

11. Use six! https://pypi.python.org/pypi/six
    ● Best library for Python 2 & 3 in the same codebase, when
    coming from Python 2
    ● Everything in one file, easy to copy and paste inside the
    project if needed
    ● Helper functions, constants,
    shims, aliases
    ● Vocabulary for porting
    ● Learn it, love it! <3
    

    View Slide

12. Backwards compatibility makes everything harder
    ● In some cases, it might be necessary to break
    ○ unicode vs bytes force you to fix some issues
    ○ when it’s really a bug, it’s acceptable to break
    ● Be ready to do some deprecation cycles
    ○ emit warnings for obsolete code, telling how to fix
    ○ tip: DeprecationWarning is ignored by default, create
    a new class MyDeprecationWarning(Warning)
    

    View Slide

13. Syntax is easy to fix!
    The real challenge
    is unicode vs bytes.
    

    View Slide

14. Unicode vs bytes: the challenge
    For each “string”, you need to decide if:
    ● should it be only text? (unicode in PY2, str in PY3)
    ● should it be only binary? (str/bytes in PY2, bytes in PY3)
    ● should it be native string? (str in PY2 & PY3)
    It’s simpler when you can choose either text or binary.
    It’s important a good coverage, and checking correctness of
    tests too.
    

    View Slide

15. Comparison of different types fail in PY3
    >>> max(1, 2, '3')
    '3'
    >>> max(1, 2, '3')
    Traceback (most recent call last):
    File "", line 1, in
    TypeError: unorderable types: str() > int()
    Python 2
    Python 3
    

    View Slide

16. Comparison of different types fail in PY3
    >>> '' == b''
    True
    >>> '' == b''
    False
    Python 2
    Python 3
    

    View Slide

17. foo == 'bar'
    Careful about tests with
    assertions like:
    It may pass in Python 3 but still be incorrect!
    

    View Slide

18. foo == b'bar'
    Perhaps the test should
    be written as:
    

    View Slide

19. Use the imports from __future__
    ● Exception: avoid unicode_literals
    ○ stdlib functions in PY2 expecting bytes end up getting
    unicode
    ○ but it may be nice when porting from PY3 to PY2
    from __future__ import (
    print_function, absolute_import, division
    )
    

    View Slide

20. For big projects, start from the leaves
    ● Start porting the bits with less dependencies
    ● Example:
    ○ pick a module under utils
    ○ review the code and the tests
    ○ fix the tests, add some more
    ○ port
    ● It’s nice to document some decisions on bytes vs unicode
    ○ Scrapy: started with utils, then Request/Response
    

    View Slide

21. ● Free book: http://python3porting.com
    ● Opinionated guide: https://docs.python.
    org/dev/howto/pyporting.html
    ● Reasons to use Python 3:
    http://asmeurer.github.io/python3-presentation/slides.html
    ● About unicode_literals: http://python-future.org/imports.
    html#should-i-import-unicode-literals
    Online resources
    

    View Slide

22. Now, go forth
    and port!
    

    View Slide

23. Thanks.
    Questions?
    Elias Dorneles
    @eliasdorneles
    

    View Slide