Share
Embed
Transcript
51 slides
-
By Your Bootstraps: Porting your Application to Python3 PyCon 2014,
Montréal Tres Seaver @tresseaver [email protected] -
Objectives “Straddling” Python 2 / 3 in single codebase Choosing
target Python versions -
Objectives “Straddling” Python 2 / 3 in single codebase Choosing
target Python versions Porting as an iterative process Ordering components by dependencies -
Objectives “Straddling” Python 2 / 3 in single codebase Choosing
target Python versions Porting as an iterative process Ordering components by dependencies Adding test coverage to reduce risk -
Objectives “Straddling” Python 2 / 3 in single codebase Choosing
target Python versions Porting as an iterative process Ordering components by dependencies Adding test coverage to reduce risk Covering C extensions -
Objectives “Straddling” Python 2 / 3 in single codebase Choosing
target Python versions Porting as an iterative process Ordering components by dependencies Adding test coverage to reduce risk Covering C extensions Porting Hygeine -
Background Experience report / best practices summary Based on porting
~180 kLOC Python, ~25 kLOC c codebase Zope3 component architecture ZODB WebOb Pyramid Other dependencies -
Porting Strategies Port once, abandon Python2 Not the subject of
this talk Customers / users still need Python2 More feasible for applications than libraries 2to3 may be useful starting point -
Porting Strategies Port once, abandon Python2 Not the subject of
this talk Customers / users still need Python2 More feasible for applications than libraries 2to3 may be useful starting point -
Porting Strategies “Fix up” at installation using 2to3 Python2 users
unaffected Python3 source “drifts” from canonical version Bug reports don't match 2to3 painfully slow on large codebases -
Porting Strategies “Fix up” at installation using 2to3 Python2 users
unaffected Python3 source “drifts” from canonical version Bug reports don't match 2to3 painfully slow on large codebases -
Porting Strategies “Straddling” in a single codebase Use “compatible subset”
of Python syntax Conditional imports mask stdlib changes six module can help (but you might not need it) -
Porting Strategies “Straddling” in a single codebase WIN! Use “compatible
subset” of Python syntax Conditional imports mask stdlib changes six module can help (but you might not need it) -
Targeting Python Versions Syntax changes make Python2 < 2.6 hard
No b'' literals No except Exception as e: -
Targeting Python Versions Syntax changes make Python2 < 2.6 hard
No b'' literals No except Exception as e: Much more cruft / pain -
Targeting Python Versions Syntax changes make Python2 < 2.6 hard
No b'' literals No except Exception as e: Much more cruft / pain 2.4 / 2.5 are long past EOL But some folks need system Python in “enterprisey” systems -
Targeting Python Versions Incompatibilities make Python3 < 3.2 hard PEP
3333 fixes WSGI in Py3k callable() restored in 3.2 3.3 restores u'' literals 3.2 is “system Python3” on some LTS systems -
Managing Porting Risks Ports are great opportunities for bug injection
Fear of breaking working software is the barrier Even more than effort -
Managing Porting Risks Some mitigations also improve your software Improved
testing Modernized idioms in Python2 Clarity in text vs. bytes -
Bottom-Up Porting Port packages with no dependencies first Then port
packages with already-ported dependencies Note Python versions supported by dependencies -
Bottom-Up Porting Port packages with no dependencies first Then port
packages with already-ported dependencies Note Python versions supported by dependencies Lather, rinse, repeat... -
Bottom-Up Porting Port packages with no dependencies first Then port
packages with already-ported dependencies Note Python versions supported by dependencies Lather, rinse, repeat... Finally, port the application -
“Common subset” idioms Lennart Regebro's book, http://python3porting.com/noconv.html python2.7 -3 can
point out problem areas “Modernize” idioms in Python2 code -
“Common subset” idioms Lennart Regebro's book, http://python3porting.com/noconv.html python2.7 -3 can
point out problem areas “Modernize” idioms in Python2 code Distinguish bytes vs. text Use b'' / u'' for all literals -
“Common subset” idioms Adopt new syntax E.g., except … as
… print()(function-like) -
“Common subset” idioms Adopt new syntax E.g., except … as
… print()(function-like) Use new stdlib facilities E.g., io.BytesIO vs StringIO.StringIO -
Testing Avoids Hair-Loss Untested code is where the bugs go
to hide 100% coverage is ideal before porting Unit testing preferable for libraries Functional testing best for applications -
Testing Avoids Hair-Loss Untested code is where the bugs go
to hide 100% coverage is ideal before porting Unit testing preferable for libraries Functional testing best for applications Measure coverage: https://pypi.python.org/pypi/coverage -
Testing Avoids Hair-Loss Work to improve assertions as well as
coverage Assert contracts, not implementation details -
Testing Avoids Hair-Loss Doctests make straddling really hard Replace doctests
with unit / functional tests If at all feasible, convert doctests to Sphinx examples -
Testing Avoids Hair-Loss Automate running tests tox helps ensure that
tests pass under all supported versions Testing with pypy! Example -
Considerations for C Extensions Python reference implementation Easier to test
Supports PyPy Design for same API as C -
Considerations for C Extensions Python reference implementation Easier to test
Supports PyPy Design for same API as C 100% coverage for Python -
Considerations for C Extensions Python reference implementation Easier to test
Supports PyPy Design for same API as C 100% coverage for Python Ensure C version passes same tests -
Hygeine Issues Signal supported versions using Trove classifiers Consider bumping
major version Allow users to stick with “safe” versions as you iterates -
Hygeine Issues Signal supported versions using Trove classifiers Consider bumping
major version Allow users to stick with “safe” versions as you iterate Apply continuous integration Travis-CI Jenkins Shining Panda for Windows