DevOps Karlsruhe Meetup · 2018-02-20

DevOps Tool Bazaar · https://www.meetup.com/devops-karlsruhe/events/246690416/

* Packaging Python Software with dh-virtualenv and fpm
* Diagnosing Client-Side Errors with Sentry.io

1. Architecture Board Architecture Board Application Monitoring Service Application Monitoring Service Phase 1 – Metrics Gateway Phase 1 – Metrics Gateway Jürgen Hermann DevOps Karlsruhe Meetup · 2018-02-20 DevOps Tool Bazaar DevOps Tool Bazaar dh-virtualenv · fpm · Sentry.io dh-virtualenv · fpm · Sentry.io jhermann jh@web.de
2. Python Packaging Overview Python Packaging Overview • Distribution packages ≠ Python packages • Source and binary distributions • sdist (zip, tar.gz, tar.bz) • bdist_wheel (whl) • Repositories: PyPI · Artifactory · DevPI • Distribution packages ≠ Python packages • Source and binary distributions • sdist (zip, tar.gz, tar.bz) • bdist_wheel (whl) • Repositories: PyPI · Artifactory · DevPI
3. Core Python Packaging & Deployment Tools Core Python Packaging & Deployment Tools • “setuptools” to build packages • “twine” to upload packages to a repository • “pip” to install packages from a repository • “virtualenv” or “pyvenv” (python3 -m venv) to isolate dependencies • “setuptools” to build packages • “twine” to upload packages to a repository • “pip” to install packages from a repository • “virtualenv” or “pyvenv” (python3 -m venv) to isolate dependencies
4. Linux Considerations (Binary Packages) Linux Considerations (Binary Packages) • Wheel filenames include platform tags (PEP425) • Python version, Python ABI, platform ABI (OS) • Example: numpy-1.14.0-cp36-cp36m-macosx_10_10_x86_64.whl • Linux presents a very diverse ABI spectrum • Windows and MacOS have way less variation • Thus PEP513 – manylinux1_{x86_64,i686} • Restricted set of kernel / userspace ABIs • Build Python binary packages in old CentOS images ☺ • Mostly works, with some quirks (e.g. stripping for packaging) • Wheel filenames include platform tags (PEP425) • Python version, Python ABI, platform ABI (OS) • Example: numpy-1.14.0-cp36-cp36m-macosx_10_10_x86_64.whl • Linux presents a very diverse ABI spectrum • Windows and MacOS have way less variation • Thus PEP513 – manylinux1_{x86_64,i686} • Restricted set of kernel / userspace ABIs • Build Python binary packages in old CentOS images ☺ • Mostly works, with some quirks (e.g. stripping for packaging)
5. Omnibus Omnibus Packages Packages
6. Omnibus Packages · Pros and Cons Omnibus Packages · Pros and Cons ⊕ Dependencies not predetermined (by target OS) ⊕ Multi-platform deployment way easier ⊕ Much simpler release process ⊖ Maintainer is responsible for security ⊖ Possibly more version variation in production ⓘ Security updates might be slower – or faster ⓘ Efficiency often dominates other concerns (esp. with deploying in-house tools) ⊕ Dependencies not predetermined (by target OS) ⊕ Multi-platform deployment way easier ⊕ Much simpler release process ⊖ Maintainer is responsible for security ⊖ Possibly more version variation in production ⓘ Security updates might be slower – or faster ⓘ Efficiency often dominates other concerns (esp. with deploying in-house tools)
7. Tooling: dh-virtualenv Tooling: dh-virtualenv • Native packaging of Python applications • Debhelper extension • Combines debhelper + virtualenv + pip • Keeps “look & feel” of both Debian and Python tooling • Minimal additional effort • DEB Package Virtualenv Wheels (Dependencies) ⊃ ⊃ • Native packaging of Python applications • Debhelper extension • Combines debhelper + virtualenv + pip • Keeps “look & feel” of both Debian and Python tooling • Minimal additional effort • DEB Package Virtualenv Wheels (Dependencies) ⊃ ⊃
8. Installing dh-virtualenv 1.0 Installing dh-virtualenv 1.0 • Basic DEB tooling: • apt-get install build-essential debhelper devscripts equivs • Stretch: A simple APT install • Jessie: • apt-get install -t jessie-backports dh-virtualenv • Ubuntu LTS (Xenial): • dpkg -i dh-virtualenv_1.0-1_all.deb # download from zesty • Basic DEB tooling: • apt-get install build-essential debhelper devscripts equivs • Stretch: A simple APT install • Jessie: • apt-get install -t jessie-backports dh-virtualenv • Ubuntu LTS (Xenial): • dpkg -i dh-virtualenv_1.0-1_all.deb # download from zesty
9. Example: "sentry.io" Debian Packaging Example: "sentry.io" Debian Packaging https://github.com/1and1/debianized-sentry • Package a release from PyPI • Add commonly used plugins • Add systemd services • Add default configuration https://github.com/1and1/debianized-sentry • Package a release from PyPI • Add commonly used plugins • Add systemd services • Add default configuration
10. $ /opt/venvs/sentry/bin/pip freeze | column $ /opt/venvs/sentry/bin/pip freeze | column hiredis==0.1.6 hiredis==0.1.6 honcho==0.7.1 honcho==0.7.1 httplib2==0.10.3 httplib2==0.10.3 idna==2.6 idna==2.6 ipaddress==1.0.19 ipaddress==1.0.19 jmespath==0.9.3 jmespath==0.9.3 kafka-python==1.3.5 kafka-python==1.3.5 kombu==3.0.35 kombu==3.0.35 loremipsum==1.0.5 loremipsum==1.0.5 lxml==4.1.1 lxml==4.1.1 milksnake==0.1.1 milksnake==0.1.1 mistune==0.8.3 mistune==0.8.3 mmh3==2.3.1 mmh3==2.3.1 mock==1.0.1 mock==1.0.1 oauth2==1.9.0.post1 oauth2==1.9.0.post1 oauthlib==2.0.6 oauthlib==2.0.6 olefile==0.44 olefile==0.44 percy==1.0.8 percy==1.0.8 petname==2.0 petname==2.0 Pillow==4.2.1 Pillow==4.2.1 progressbar2==3.10.1 progressbar2==3.10.1 psycopg2==2.6.2 psycopg2==2.6.2 py==1.5.2 py==1.5.2 pycparser==2.18 pycparser==2.18 PyJWT==1.5.3 PyJWT==1.5.3 pyOpenSSL==17.5.0 pyOpenSSL==17.5.0 pytest==3.1.3 pytest==3.1.3 pytest-django==2.9.1 pytest-django==2.9.1 pytest-html==1.9.0 pytest-html==1.9.0 python-dateutil==2.6.1 python-dateutil==2.6.1 python-ldap==2.5.2 python-ldap==2.5.2 amqp==1.4.9 amqp==1.4.9 anyjson==0.3.3 anyjson==0.3.3 asn1crypto==0.24.0 asn1crypto==0.24.0 BeautifulSoup==3.2.1 BeautifulSoup==3.2.1 billiard==3.3.0.23 billiard==3.3.0.23 boto3==1.4.5 boto3==1.4.5 botocore==1.5.70 botocore==1.5.70 cached-property==1.3.1 cached-property==1.3.1 celery==3.1.18 celery==3.1.18 certifi==2017.11.5 certifi==2017.11.5 cffi==1.11.2 cffi==1.11.2 chardet==3.0.4 chardet==3.0.4 click==6.7 click==6.7 contextlib2==0.5.5 contextlib2==0.5.5 cryptography==2.1.4 cryptography==2.1.4 cssselect==1.0.1 cssselect==1.0.1 cssutils==0.9.10 cssutils==0.9.10 debianized-sentry==8.22.0.2 debianized-sentry==8.22.0.2 Django==1.6.11 Django==1.6.11 django-auth-ldap==1.2.16 django-auth-ldap==1.2.16 django-crispy-forms==1.4.0 django-crispy-forms==1.4.0 django-jsonfield==0.9.13 django-jsonfield==0.9.13 django-picklefield==0.3.2 django-picklefield==0.3.2 django-sudo==2.1.0 django-sudo==2.1.0 django-templatetag-sugar==1.0 django-templatetag-sugar==1.0 djangorestframework==2.4.8 djangorestframework==2.4.8 docutils==0.14 docutils==0.14 email-reply-parser==0.2.0 email-reply-parser==0.2.0 enum34==1.1.6 enum34==1.1.6 exam==0.10.6 exam==0.10.6 futures==3.2.0 futures==3.2.0 python-memcached==1.59 python-memcached==1.59 python-openid==2.2.5 python-openid==2.2.5 python-u2flib-server==4.0.1 python-u2flib-server==4.0.1 python-utils==2.2.0 python-utils==2.2.0 pytz==2017.3 pytz==2017.3 PyYAML==3.11 PyYAML==3.11 qrcode==5.3 qrcode==5.3 raven==5.33.0 raven==5.33.0 rb==1.7 rb==1.7 redis==2.10.5 redis==2.10.5 redis-py-cluster==1.3.4 redis-py-cluster==1.3.4 requests==2.18.4 requests==2.18.4 requests-oauthlib==0.8.0 requests-oauthlib==0.8.0 s3transfer==0.1.12 s3transfer==0.1.12 selenium==3.4.3 selenium==3.4.3 sentry==8.22.0 sentry==8.22.0 sentry-kafka==1.1 sentry-kafka==1.1 sentry-ldap-auth==2.5 sentry-ldap-auth==2.5 sentry-plugins==>8.22.0 sentry-plugins==>8.22.0 setproctitle==1.1.10 setproctitle==1.1.10 simplejson==3.8.2 simplejson==3.8.2 six==1.10.0 six==1.10.0 sqlparse==0.1.19 sqlparse==0.1.19 statsd==3.1 statsd==3.1 structlog==16.1.0 structlog==16.1.0 symbolic==1.1.5 symbolic==1.1.5 toronado==0.0.11 toronado==0.0.11 ua-parser==0.7.3 ua-parser==0.7.3 unidiff==0.5.4 unidiff==0.5.4 urllib3==1.22 urllib3==1.22 uWSGI==2.0.15 uWSGI==2.0.15
11. Binary Extension Packages Trouble-Shooting Binary Extension Packages Trouble-Shooting If stuff doesn‘t build OOTB… • “dh_strip” and “dh_shlibdeps” exclusions • Disable “manylinux1” wheels • Pass the “--no-binary=:all:” option to pip • … or add “no-manylinux1” to your build environment • Increases build times (no wheels, or no extension ones) • Often requires more build dependencies (“*-dev” packages) If stuff doesn‘t build OOTB… • “dh_strip” and “dh_shlibdeps” exclusions • Disable “manylinux1” wheels • Pass the “--no-binary=:all:” option to pip • … or add “no-manylinux1” to your build environment • Increases build times (no wheels, or no extension ones) • Often requires more build dependencies (“*-dev” packages)
12. debian/control debian/control
13. debian/rules debian/rules
14. debian/prerm debian/prerm
15. Springerle Project Templates Springerle Project Templates https://github.com/Springerle https://github.com/Springerle
16. References References • Python Packaging User Guide https://packaging.python.org/ • dh-virtualenv https://github.com/spotify/dh-virtualenv • fpm (Effing Package Management) https://github.com/jordansissel/fpm • How We Deploy Python Code (Nylas) https://www.nylas.com/blog/packaging-deploying-python/ • Python Packaging User Guide https://packaging.python.org/ • dh-virtualenv https://github.com/spotify/dh-virtualenv • fpm (Effing Package Management) https://github.com/jordansissel/fpm • How We Deploy Python Code (Nylas) https://www.nylas.com/blog/packaging-deploying-python/
17. Questions? Questions? Thank you! Thank you!
18. Tool for simplified builds of native packages Written in Ruby Tool for simplified builds of native packages Written in Ruby fpm (Effing Package Management) fpm (Effing Package Management)
19. fpm Project Goals fpm Project Goals • Simple way to create packages • … without the need to memorize too much • Help deliver software with minimal steps or training • Easily build platform-native packages • Do many things with one tool, including: • Creating packages easily (deb, rpm, freebsd, etc) • Tweaking existing packages (removing files, changing metadata/dependencies) • Stripping pre/post/maintainer scripts from packages • Simple way to create packages • … without the need to memorize too much • Help deliver software with minimal steps or training • Easily build platform-native packages • Do many things with one tool, including: • Creating packages easily (deb, rpm, freebsd, etc) • Tweaking existing packages (removing files, changing metadata/dependencies) • Stripping pre/post/maintainer scripts from packages
20. Sources and Targets Sources and Targets • Package types: dir · gem · deb · npm · rpm · tar · cpan · pear · empty · puppet · python · osxpkg · solaris · p5p · pkgin · freebsd · apk · zip · sh · virtualenv · pacman · pleaserun • Sources: dir, gem, npm, python, … • Targets: deb, apk, osxpkg, rpm, pacman, … • Package types: dir · gem · deb · npm · rpm · tar · cpan · pear · empty · puppet · python · osxpkg · solaris · p5p · pkgin · freebsd · apk · zip · sh · virtualenv · pacman · pleaserun • Sources: dir, gem, npm, python, … • Targets: deb, apk, osxpkg, rpm, pacman, …
21. Wrap Python package into Debian package Wrap Python package into Debian package • Results conform to Debian policies • Usually more work than omnibus packages • Build main application or library package • Hunt down the dependency tree • If no standard package exists, build it • Iterate (possibly a lot) • OS dictates (some) dependency versions • Results conform to Debian policies • Usually more work than omnibus packages • Build main application or library package • Hunt down the dependency tree • If no standard package exists, build it • Iterate (possibly a lot) • OS dictates (some) dependency versions → →
22. fpm and Python3 fpm and Python3 fpm --input-type python \ --python-bin python3 \ --python-pip pip3 \ --python-package-name-prefix python3 \ … ¯\_( ツ )_/¯ Vote for https://github.com/jordansissel/fpm/issues/1361 fpm --input-type python \ --python-bin python3 \ --python-pip pip3 \ --python-package-name-prefix python3 \ … ¯\_( ツ )_/¯ Vote for https://github.com/jordansissel/fpm/issues/1361
23. Questions? Questions? Thank you! Thank you!
25. Motivation & Requirements Motivation & Requirements Mission: Monitor a Single Page App (SPA) in a browser. • Main goals: • Observability for SPAs / PWAs that run outside of your data center in the browser (and thus outside of classical instrumentation and monitoring) • Early detection of problems of the user application (catch/report browser-side exceptions) • Technical infrastructure to support these goals: • Collection of raw data like metrics, errors / exceptions, and logging events • Transformation of that data into different forms, and persist it to different targets as needed • Correlation of client-side interactions with resulting server-side requests Mission: Monitor a Single Page App (SPA) in a browser. • Main goals: • Observability for SPAs / PWAs that run outside of your data center in the browser (and thus outside of classical instrumentation and monitoring) • Early detection of problems of the user application (catch/report browser-side exceptions) • Technical infrastructure to support these goals: • Collection of raw data like metrics, errors / exceptions, and logging events • Transformation of that data into different forms, and persist it to different targets as needed • Correlation of client-side interactions with resulting server-side requests
26. Solution Strategy – Errors & Exceptions Solution Strategy – Errors & Exceptions • Sentry is a “modern realtime error logging and aggregation platform” • Specialist system tailored to the problem domain (with lots of plugins and extensions) • Excellent and mature client API libraries, e.g. raven.js • Supports many client runtime environments (→ backend use) • SaaS + On-Premise · Forum · Optional Paid Support • Inception 2008 · Python · Django · Redis · PostgreSQL • 17948 commits · 322 releases · 292 contributors · BSD 3-Clause • Sentry is a “modern realtime error logging and aggregation platform” • Specialist system tailored to the problem domain (with lots of plugins and extensions) • Excellent and mature client API libraries, e.g. raven.js • Supports many client runtime environments (→ backend use) • SaaS + On-Premise · Forum · Optional Paid Support • Inception 2008 · Python · Django · Redis · PostgreSQL • 17948 commits · 322 releases · 292 contributors · BSD 3-Clause
28. Weekly Report Email Weekly Report Email
29. Sentry.io On-Premise Deployment Sentry.io On-Premise Deployment
30. Questions? Questions? Thank you! Thank you!