Packaging in packaging: dh-virtualenv

Deploying your software can become a tricky task, regardless of the language. In the spirit of the Python conferences, every conference needs at least one packaging talk.

This talk is about dh-virtualenv. It's a Python packaging tool aimed for Debian-based systems and for deployment flows that already take advantage of Debian packaging with Python virtualenvs

Transcript

1. Packaging in packaging dh-virtualenv
2. @nailor Jyrki Pulliainen jyrki@spotify.com Photo credit: Pontus Persson
3. What’s in this talk? •  Look at different, existing deployment strategies •  What is dh-virtualenv? •  Packaging sentry with dh-virtualenv
4. virtualenv
5. +  Stable +  Non-Python Dependencies +  Existing infrastructure +  Scripting
7. -  Out of date -  Complex -  Global state
8. +  Has the new stuff +  De facto +  Battle tested +  Contained virtualenv
10. -  Native dependency handling -  Source installs -  Random scripts sh: mysql_config: command not found
11. What is dh-virtualenv?
12. Virtualenv inside a Debian package
13. It’s Open Source! •  GPL’d •  Documented •  Comes with a simple tutorial!
14. It’s a DebHelper Extension •  Idea stolen adapted from @hynek * https://hynek.me/articles/python-app-deployment-with-native-packages/
15. In practice?
16. Package builder that… 1.  Creates a virtualenv 2. Installs requirements.txt into that virtualenv 3. Installs your project 4. Fixes the virtualenv to match production
17. Packaging with dh-virtualenv Case in point: sentry
18. 1.  Install dh-virtualenv apt-get install dh-virtualenv 2.  Create ‘debian’ directory 3.  Insert the minimal boilerplate files in there
19. Source: sentry Section: python Priority: extra Maintainer: Jyrki Pulliainen <jyrki@spotify.com> Build-Depends: debhelper (>= 9), python, dh-virtualenv, python-dev Standards-Version: 3.9.5 Package: sentry Architecture: any Depends: ${python:Depends}, ${misc:Depends} Description: dh-virtualenv-packaged sentry Amazing tool, now packaged with dh-virtualenv! debian/control:
20. sentry (6.4.4-1) unstable; urgency=low * Initial packaging with dh-virtualenv -- Jyrki Pulliainen <jyrki@spotify.com> Wed, 23 Jul 2014 12:02:20 +0200 debian/changelog:
21. echo 9 > debian/compat
22. #!/usr/bin/make -f %: dh $@ --with python-virtualenv debian/rules:
23. 5.  Build your package! dpkg-buildpackage
24. dpkg-buildpackage: source package dh-virtualenv dpkg-buildpackage: source version 0.7-2 dpkg-buildpackage: source distribution unstable dpkg-buildpackage: source changed by Jyrki Pulliainen <jyrki@spotify.com> dpkg-buildpackage: host architecture i386 dpkg-source --before-build dh-virtualenv fakeroot debian/rules clean dh clean --with python2 --with sphinxdoc dh_testdir debian/rules override_dh_auto_clean make[1]: Entering directory `/vagrant_src/dh-virtualenv' rm -rf doc/_build rm -f doc/dh_virtualenv.1 rm -rf dh_virtualenv.egg-info dh_auto_clean running clean 'build/lib.linux-i686-2.7' does not exist -- can't clean it 'build/bdist.linux-i686' does not exist -- can't clean it removing 'build/scripts-2.7' (and everything under it) make[1]: Leaving directory `/vagrant_src/dh-virtualenv' dh_clean dpkg-source -b dh-virtualenv dpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../
25. dh_virtualenv Running virtualenv with interpreter /usr/bin/python2 New python executable in debian/sentry/usr/share/python/sentry/bin/python2 Also creating executable in debian/sentry/usr/share/python/sentry/bin/python Installing setuptools, pip...done. Unpacking /vagrant_src/sentry-with-dh-virtualenv Running setup.py (path:/tmp/pip-ttX1Ip-build/setup.py) egg_info for package from file:/// vagrant_src/sentry-with-dh-virtualenv warning: no previously-included files matching '*~' found anywhere in distribution Downloading/unpacking BeautifulSoup>=3.2.1,<3.3.0 (from sentry==6.4.4) Downloading BeautifulSoup-3.2.1.tar.gz Running setup.py (path:/vagrant_src/sentry-with-dh-virtualenv/debian/sentry/usr/share/python/ sentry/build/BeautifulSoup/setup.py) egg_info for package BeautifulSoup Downloading/unpacking celery>=3.0.15,<3.1.0 (from sentry==6.4.4) Downloading celery-3.0.25.tar.gz (1.1MB): 1.1MB downloaded Running setup.py (path:/vagrant_src/sentry-with-dh-virtualenv/debian/sentry/usr/share/python/ sentry/build/celery/setup.py) egg_info for package celery Upgrade: no old version found. no previously-included directories found matching '*.pyc' no previously-included directories found matching '*.sw*’
26. 5.  Install the shiny Debian package in production!
28. +  Non-Python dependencies +  Existing Infrastructure +  The new hotness +  Contained dh-virtualenv
29. -  Can be slow to build -  Requires still digging for requirements -  Build system needs to have the exactly same Python dh-virtualenv
30. Thanks! github.com/spotify/dh-virtualenv dh-virtualenv.readthedocs.org labs.spotify.com/2013/10/10/packaging-in- your-packaging-dh-virtualenv/