Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Things I learned porting code to Python 3

Things I learned porting code to Python 3

English version of slides from my talk about Python 3 porting at PythonRio meetup

Elias Dorneles

June 25, 2016
Tweet

More Decks by Elias Dorneles

Other Decks in Programming

Transcript

  1. Things I learned porting code
    to Python 3
    Elias Dorneles
    @eliasdorneles

    View full-size slide

  2. There is no shame
    in being using
    exclusively
    Python 2!

    View full-size slide

  3. Despite what
    the Internet is
    saying at the
    moment :)

    View full-size slide

  4. Why Python 3?
    ● It’s a better language
    ○ tuple unpacking
    ○ more consistent APIs
    ○ async/await
    ○ syntax for gradual typing
    ○ Easier for beginners
    ● It’s the future of the ecosystem
    ○ your users want PY3

    View full-size slide

  5. Migration strategies

    View full-size slide

  6. Migration strategies
    ● Migrate at once to Python 3
    ○ for code with low risks/deps
    ● Fork for Python 3 version
    ○ for highly stable code, that don’t get modifications
    ● Python 2 & 3 in the same codebase
    ○ for code being maintained and used by others

    View full-size slide

  7. Let’s focus in Python 2 & 3 in the
    same codebase

    View full-size slide

  8. Important to have a good test coverage
    ● Doesn’t need to be 100%, but it’s good to have enough to
    catch important regressions
    ● Run the tests in both Python 2 & Python 3
    ○ tip: ignore the ones aren’t passing in Python 3 yet, try to
    keep passing the ones that already do
    ● Remember the tests may be incorrect in Python 3 and
    needing to change too

    View full-size slide

  9. Focus in Python 2.7
    and 3.3+

    View full-size slide

  10. Focus in 2.7 and 3.3+
    ● If you’re still on 2.6 or previous, first migrate to 2.7+
    ● Ignore 3.x lesser than 3.3
    ○ didn’t had good enough support (syntax & stdlib) to
    write code compatible with 2
    ○ vendors (distros) generally support 3.3+
    ● Tip for installing several distinct versions of Python:
    ○ http://saghul.github.io/pythonz (CPython, PyPy, etc)

    View full-size slide

  11. Use six! https://pypi.python.org/pypi/six
    ● Best library for Python 2 & 3 in the same codebase, when
    coming from Python 2
    ● Everything in one file, easy to copy and paste inside the
    project if needed
    ● Helper functions, constants,
    shims, aliases
    ● Vocabulary for porting
    ● Learn it, love it! <3

    View full-size slide

  12. Backwards compatibility makes everything harder
    ● In some cases, it might be necessary to break
    ○ unicode vs bytes force you to fix some issues
    ○ when it’s really a bug, it’s acceptable to break
    ● Be ready to do some deprecation cycles
    ○ emit warnings for obsolete code, telling how to fix
    ○ tip: DeprecationWarning is ignored by default, create
    a new class MyDeprecationWarning(Warning)

    View full-size slide

  13. Syntax is easy to fix!
    The real challenge
    is unicode vs bytes.

    View full-size slide

  14. Unicode vs bytes: the challenge
    For each “string”, you need to decide if:
    ● should it be only text? (unicode in PY2, str in PY3)
    ● should it be only binary? (str/bytes in PY2, bytes in PY3)
    ● should it be native string? (str in PY2 & PY3)
    It’s simpler when you can choose either text or binary.
    It’s important a good coverage, and checking correctness of
    tests too.

    View full-size slide

  15. Comparison of different types fail in PY3
    >>> max(1, 2, '3')
    '3'
    >>> max(1, 2, '3')
    Traceback (most recent call last):
    File "", line 1, in
    TypeError: unorderable types: str() > int()
    Python 2
    Python 3

    View full-size slide

  16. Comparison of different types fail in PY3
    >>> '' == b''
    True
    >>> '' == b''
    False
    Python 2
    Python 3

    View full-size slide

  17. foo == 'bar'
    Careful about tests with
    assertions like:
    It may pass in Python 3 but still be incorrect!

    View full-size slide

  18. foo == b'bar'
    Perhaps the test should
    be written as:

    View full-size slide

  19. Use the imports from __future__
    ● Exception: avoid unicode_literals
    ○ stdlib functions in PY2 expecting bytes end up getting
    unicode
    ○ but it may be nice when porting from PY3 to PY2
    from __future__ import (
    print_function, absolute_import, division
    )

    View full-size slide

  20. For big projects, start from the leaves
    ● Start porting the bits with less dependencies
    ● Example:
    ○ pick a module under utils
    ○ review the code and the tests
    ○ fix the tests, add some more
    ○ port
    ● It’s nice to document some decisions on bytes vs unicode
    ○ Scrapy: started with utils, then Request/Response

    View full-size slide

  21. ● Free book: http://python3porting.com
    ● Opinionated guide: https://docs.python.
    org/dev/howto/pyporting.html
    ● Reasons to use Python 3:
    http://asmeurer.github.io/python3-presentation/slides.html
    ● About unicode_literals: http://python-future.org/imports.
    html#should-i-import-unicode-literals
    Online resources

    View full-size slide

  22. Now, go forth
    and port!

    View full-size slide

  23. Thanks.
    Questions?
    Elias Dorneles
    @eliasdorneles

    View full-size slide