Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Things I learned porting code to Python 3

Things I learned porting code to Python 3

English version of slides from my talk about Python 3 porting at PythonRio meetup

Elias Dorneles

June 25, 2016

More Decks by Elias Dorneles

Other Decks in Programming


  1. Why Python 3? • It’s a better language ◦ tuple

    unpacking ◦ more consistent APIs ◦ async/await ◦ syntax for gradual typing ◦ Easier for beginners • It’s the future of the ecosystem ◦ your users want PY3
  2. Migration strategies • Migrate at once to Python 3 ◦

    for code with low risks/deps • Fork for Python 3 version ◦ for highly stable code, that don’t get modifications • Python 2 & 3 in the same codebase ◦ for code being maintained and used by others
  3. Important to have a good test coverage • Doesn’t need

    to be 100%, but it’s good to have enough to catch important regressions • Run the tests in both Python 2 & Python 3 ◦ tip: ignore the ones aren’t passing in Python 3 yet, try to keep passing the ones that already do • Remember the tests may be incorrect in Python 3 and needing to change too
  4. Focus in 2.7 and 3.3+ • If you’re still on

    2.6 or previous, first migrate to 2.7+ • Ignore 3.x lesser than 3.3 ◦ didn’t had good enough support (syntax & stdlib) to write code compatible with 2 ◦ vendors (distros) generally support 3.3+ • Tip for installing several distinct versions of Python: ◦ http://saghul.github.io/pythonz (CPython, PyPy, etc)
  5. Use six! https://pypi.python.org/pypi/six • Best library for Python 2 &

    3 in the same codebase, when coming from Python 2 • Everything in one file, easy to copy and paste inside the project if needed • Helper functions, constants, shims, aliases • Vocabulary for porting • Learn it, love it! <3
  6. Backwards compatibility makes everything harder • In some cases, it

    might be necessary to break ◦ unicode vs bytes force you to fix some issues ◦ when it’s really a bug, it’s acceptable to break • Be ready to do some deprecation cycles ◦ emit warnings for obsolete code, telling how to fix ◦ tip: DeprecationWarning is ignored by default, create a new class MyDeprecationWarning(Warning)
  7. Unicode vs bytes: the challenge For each “string”, you need

    to decide if: • should it be only text? (unicode in PY2, str in PY3) • should it be only binary? (str/bytes in PY2, bytes in PY3) • should it be native string? (str in PY2 & PY3) It’s simpler when you can choose either text or binary. It’s important a good coverage, and checking correctness of tests too.
  8. Comparison of different types fail in PY3 >>> max(1, 2,

    '3') '3' >>> max(1, 2, '3') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unorderable types: str() > int() Python 2 Python 3
  9. Comparison of different types fail in PY3 >>> '' ==

    b'' True >>> '' == b'' False Python 2 Python 3
  10. foo == 'bar' Careful about tests with assertions like: It

    may pass in Python 3 but still be incorrect!
  11. Use the imports from __future__ • Exception: avoid unicode_literals ◦

    stdlib functions in PY2 expecting bytes end up getting unicode ◦ but it may be nice when porting from PY3 to PY2 from __future__ import ( print_function, absolute_import, division )
  12. For big projects, start from the leaves • Start porting

    the bits with less dependencies • Example: ◦ pick a module under utils ◦ review the code and the tests ◦ fix the tests, add some more ◦ port • It’s nice to document some decisions on bytes vs unicode ◦ Scrapy: started with utils, then Request/Response
  13. • Free book: http://python3porting.com • Opinionated guide: https://docs.python. org/dev/howto/pyporting.html •

    Reasons to use Python 3: http://asmeurer.github.io/python3-presentation/slides.html • About unicode_literals: http://python-future.org/imports. html#should-i-import-unicode-literals Online resources