Things I learned porting code to Python 3

English version of slides from my talk about Python 3 porting at PythonRio meetup

Transcript

1. Things I learned porting code to Python 3 Elias Dorneles @eliasdorneles
2. There is no shame in being using exclusively Python 2!
3. Despite what the Internet is saying at the moment :)
4. Why Python 3? • It’s a better language ◦ tuple unpacking ◦ more consistent APIs ◦ async/await ◦ syntax for gradual typing ◦ Easier for beginners • It’s the future of the ecosystem ◦ your users want PY3
5. Migration strategies
6. Migration strategies • Migrate at once to Python 3 ◦ for code with low risks/deps • Fork for Python 3 version ◦ for highly stable code, that don’t get modifications • Python 2 & 3 in the same codebase ◦ for code being maintained and used by others
7. Let’s focus in Python 2 & 3 in the same codebase
8. Important to have a good test coverage • Doesn’t need to be 100%, but it’s good to have enough to catch important regressions • Run the tests in both Python 2 & Python 3 ◦ tip: ignore the ones aren’t passing in Python 3 yet, try to keep passing the ones that already do • Remember the tests may be incorrect in Python 3 and needing to change too
9. Focus in Python 2.7 and 3.3+
10. Focus in 2.7 and 3.3+ • If you’re still on 2.6 or previous, first migrate to 2.7+ • Ignore 3.x lesser than 3.3 ◦ didn’t had good enough support (syntax & stdlib) to write code compatible with 2 ◦ vendors (distros) generally support 3.3+ • Tip for installing several distinct versions of Python: ◦ http://saghul.github.io/pythonz (CPython, PyPy, etc)
11. Use six! https://pypi.python.org/pypi/six • Best library for Python 2 & 3 in the same codebase, when coming from Python 2 • Everything in one file, easy to copy and paste inside the project if needed • Helper functions, constants, shims, aliases • Vocabulary for porting • Learn it, love it! <3
12. Backwards compatibility makes everything harder • In some cases, it might be necessary to break ◦ unicode vs bytes force you to fix some issues ◦ when it’s really a bug, it’s acceptable to break • Be ready to do some deprecation cycles ◦ emit warnings for obsolete code, telling how to fix ◦ tip: DeprecationWarning is ignored by default, create a new class MyDeprecationWarning(Warning)
13. Syntax is easy to fix! The real challenge is unicode vs bytes.
14. Unicode vs bytes: the challenge For each “string”, you need to decide if: • should it be only text? (unicode in PY2, str in PY3) • should it be only binary? (str/bytes in PY2, bytes in PY3) • should it be native string? (str in PY2 & PY3) It’s simpler when you can choose either text or binary. It’s important a good coverage, and checking correctness of tests too.
15. Comparison of different types fail in PY3 >>> max(1, 2, '3') '3' >>> max(1, 2, '3') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unorderable types: str() > int() Python 2 Python 3
16. Comparison of different types fail in PY3 >>> '' == b'' True >>> '' == b'' False Python 2 Python 3
17. foo == 'bar' Careful about tests with assertions like: It may pass in Python 3 but still be incorrect!
18. foo == b'bar' Perhaps the test should be written as:
19. Use the imports from __future__ • Exception: avoid unicode_literals ◦ stdlib functions in PY2 expecting bytes end up getting unicode ◦ but it may be nice when porting from PY3 to PY2 from __future__ import ( print_function, absolute_import, division )
20. For big projects, start from the leaves • Start porting the bits with less dependencies • Example: ◦ pick a module under utils ◦ review the code and the tests ◦ fix the tests, add some more ◦ port • It’s nice to document some decisions on bytes vs unicode ◦ Scrapy: started with utils, then Request/Response
21. • Free book: http://python3porting.com • Opinionated guide: https://docs.python. org/dev/howto/pyporting.html • Reasons to use Python 3: http://asmeurer.github.io/python3-presentation/slides.html • About unicode_literals: http://python-future.org/imports. html#should-i-import-unicode-literals Online resources
22. Now, go forth and port!
23. Thanks. Questions? Elias Dorneles @eliasdorneles