Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Taming the Wild West of Python Packaging

Taming the Wild West of Python Packaging

Sharing your Python package with the world is easy! Copy-paste your setup.py, update a few fields, and upload, right? But choosing the correct metadata isn’t as simple, and can have serious consequences. With a few simple tips, you can graduate from packaging cowboy to packaging pro.

9c98621c87f4f1333e0dcbb5282fa65b?s=128

John Reese

May 04, 2019
Tweet

Transcript

  1. @n7cmdr - jreese.sh Taming the Wild West
 Of Python Packaging

    John Reese
 Production Engineer, Facebook
  2. @n7cmdr - jreese.sh Not every user uses `pip install`

  3. @n7cmdr - jreese.sh Not every user is a human

  4. @n7cmdr - jreese.sh • Internal mirror of PyPI packages •

    Build wheels from source distributions • Use clean virtualenv and chroot • Support multiple versions of Python • Track packages for vulnerability management
  5. @n7cmdr - jreese.sh Not every maintainer is a packager

  6. @n7cmdr - jreese.sh • No dependencies • No Python version

    requirements • No source distribution (wheels only) • No setup.py
  7. @n7cmdr - jreese.sh • setuptools is available • Dependencies are

    already installed • install_requires is good enough
  8. @n7cmdr - jreese.sh Not every setup.py sets up everything

  9. @n7cmdr - jreese.sh • Packaging guide ignores setup_requires, test_requires •

    Setuptools has no way to specify C/C++ dependencies • `pip wheel .` fails when `pip install .` doesn’t
  10. @n7cmdr - jreese.sh Not every package is difficult

  11. @n7cmdr - jreese.sh • Include setup.py • Include long description

    with markdown • Include tests, inside module namespace • Specify python/setup/test/install requirements
  12. @n7cmdr - jreese.sh from setuptools import setup.py setup( name="package", versions="1.0",

    long_description="<readme contents>", long_description_content_type="text/markdown", author="Your Name", author_email="you@gmail.com", url="https://github.com/team/package", license="MIT", classifiers=[ "Development Status :: 1 - Planning", "License :: OSI Approved :: MIT License", ], python_requires=">=3.6", setup_requires=["setuptools>=38.6.0"], install_requires=[...], tests_require=[...], packages=["package", "package.tests"], test_suite="package.tests", )
  13. @n7cmdr - jreese.sh from setuptools import setup.py setup( name="package", versions="1.0",

    long_description="<readme contents>", long_description_content_type="text/markdown", author="Your Name", author_email="you@gmail.com", url="https://github.com/team/package", license="MIT", classifiers=[ "Development Status :: 1 - Planning", "License :: OSI Approved :: MIT License", ], python_requires=">=3.6", setup_requires=["setuptools>=38.6.0"], install_requires=[...], tests_require=[...], packages=["package", "package.tests"], test_suite="package.tests", )
  14. @n7cmdr - jreese.sh • Run `setup.py test sdist wheel` from

    fresh virtualenv • Always upload a source distribution to PyPI • Consider uploading wheels as well
  15. @n7cmdr - jreese.sh Not every setup tool is standard

  16. @n7cmdr - jreese.sh Not every package should execute arbitrary code

    to install
  17. @n7cmdr - jreese.sh Thanks! John Reese
 Production Engineer, Facebook @n7cmdr

    github.com/jreese