Python Packaging for Humans

Turn your project into a good Python citizen by leveraging Distribute (the successor to setuptools) to write re-usable, installable, testable, and well-documented Python packages.

PyGotham II, June 8, 2012.

1. PYTHON PACKAGING FOR HUMANS by Dan Crosta @lazlofruvous
2. JUST DO THIS: from setuptools import setup, find_packages from distutils.core import Extension setup( name='mypackage', version='0.1', description='My New Packaging Technique is Unstoppable', long_description=__doc__, author='Dan Crosta', author_email='dcrosta@late.am', license='BSD', classifiers=[ 'Development Status :: 3 - Alpha', 'Programming Language :: Python :: 2'], packages=find_packages(), ext_modules=[Extension('_cmypackage', ['_cmypackage.c'])], setup_requires=['nose'], tests_require=['coverage'], )
3. BACK IN MY DAY... • Google "python <thing I need to do>" • Search for download link • Extract tarball • python setup.py install • ... • Ugh.
4. BACK IN MY DAY... • PyPI launches in 2003 • setuptools/easy_install • distribute/pip • Py3k packaging/distutils2
5. BACK IN MY DAY... • Most of this has existed for a decade. • Why isn't *everything* packaged?
6. WHY PACKAGE? • Dead simple for users • Good for package authors • Really not hard
7. GOOD FOR USERS: • Single interface to package management • Lots of "do what I mean" baked in • Makes deployment way easier
8. GOOD FOR AUTHORS: • Development helpers • Plays nice with others • Know what users are getting
9. This is how you do it.
10. REPOSITORY LAYOUT + mypackage/ + README + LICENSE + distribute_setup.py + setup.py + setup.cfg + mypackage/ | + __init__.py | + foo.py | + bar.py + _cmypackage.c + tests/ + test_foo.py + test_bar.py
11. SETUP.PY #1: JUST ENOUGH from setuptools import setup setup( name='mypackage', version='0.1', description='My New Packaging Technique is Unstoppable', author='Dan Crosta', author_email='dcrosta@late.am', license='BSD', packages=['mypackage'], )
12. SETUP.PY #1: JUST ENOUGH from setuptools import setup, find_packages setup( name='mypackage', version='0.1', description='My New Packaging Technique is Unstoppable', author='Dan Crosta', author_email='dcrosta@late.am', license='BSD', packages=find_packages(), )
13. SETUP.PY #2: DEPENDENCIES from setuptools import setup setup( # ... install_requires=[ 'somepackage == 0.1', 'otherpackage >= 0.2', 'yetanother != 0.3'], )
14. SETUP.PY #3: CLASSIFIERS from setuptools import setup setup( # ... classifiers=[ 'Development Status :: 3 - Alpha', 'Programming Language :: Python :: 2'], ) • See http://pypi.python.org/pypi for full list
15. SETUP.PY #4: SCRIPTS from setuptools import setup setup( # ... entry_points={ 'console_scripts': [ 'packagescript=mypackage.scripts:main'], }, )
16. SETUP.PY #5: PACKAGE DATA from setuptools import setup setup( # ... include_package_data=True, )
17. SETUP.PY #6: C EXTENSIONS from setuptools import setup setup( # ... ext_modules=[ Extension('_cmypackage', ['_cmypackage.c']) ], )
18. This is how you make sure it works.
19. VIRTUALENV • Packaging's best friend • Use --no-site-packages
20. VIRTUALENV # In your source tree $ python setup.py sdist $ ls dist/ mypackage-0.1.tar.gz # In a clean virtualenv $ pip install /path/to/mypackage-0.1.tar.gz
21. This is how you share it with everyone.
22. PYPI • Python Package Index • aka "Cheeseshop" • pip, easy_install, buildout all look here • You want your package here.
23. PYPI • Create an account at http://pypi.python.org/pypi
24. PYPI $ python setup.py register We need to know who you are, so please choose either: 1. use your existing login, 2. register as a new user, 3. have the server generate a new password for you, or 4. quit Your selection [default 1]: 1 Username: dcrosta Password: ******* Registering mypackage to http://pypi.python.org/pypi Server response (200): OK
25. PYPI $ python setup.py sdist upload running upload Submitting dist/mypackage-0.1.tar.gz to pypi.python.org/pypi Server response (200): OK
26. This is how you get the most out of setup.py.
27. NOSETESTS from setuptools import setup setup( # ... setup_requires=['nose'], tests_require=['coverage'], )
28. NOSETESTS # setup.cfg, read by setup.py [nosetests] verbosity = 0 detailed-errors = 1 with-coverage = 1 cover-package = mypackage cover-inclusive = 1
29. NOSETESTS $ python setup.py nosetests running nosetests Name Stmts Miss Cover Missing -------------------------------------------------- mypackage 2 0 100% mypackcage.foo 19 4 79% 13, 31-33 mypackage.bar 16 6 63% 13, 22-26 -------------------------------------------------- TOTAL 37 10 73% -------------------------------------------------- Ran 12 tests in 0.027s
30. TRAVIS-CI # .travis.yml language: python python: - 2.5 - 2.6 - 2.7 install: python setup.py -q install script: python setup.py -q nosetests
31. TRAVIS-CI
32. This is how you extend it.
33. BUILDING DOCS class doc(Command): description = "generate sphinx documentation" user_options = [] boolean_options = [] def initialize_options(self): pass def finalize_options(self): pass def run(self): status = subprocess.call(["make", "-C", "docs", "html"]) if status: raise RuntimeError("documentation build failed")