Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Deployability of Python Web Applications
Bruno Renié
July 02, 2013
Programming
17
2.1k
Deployability of Python Web Applications
Bruno Renié
July 02, 2013
Tweet
Share
More Decks by Bruno Renié
See All by Bruno Renié
brutasse
3
390
brutasse
1
130
brutasse
21
2.3k
brutasse
9
1.5k
brutasse
4
510
brutasse
4
490
brutasse
3
360
Other Decks in Programming
See All in Programming
fr0gger
2
2.4k
inoue2002
1
280
kubode
1
160
adarapata
1
400
kaonash
2
1.8k
syucream
4
1.4k
progate
2
420
deepu105
0
230
o0h
PRO
2
1.5k
amakza
0
160
patorash
0
110
hyodol2513
2
970
Featured
See All Featured
smashingmag
283
47k
cherdarchuk
71
260k
shpigford
370
42k
davidbonilla
70
3.6k
trallard
15
780
bkeepers
PRO
54
4.3k
eitanlees
115
10k
jnunemaker
PRO
40
4.7k
morganepeng
19
1.3k
imathis
479
150k
holman
288
130k
trishagee
24
3k
Transcript
Bruno Renié — EuroPython 2013 Deployability of Python web applications
Deployability, n The extent to which something is deployable
Disclaimer Most of this isn't python-specific or even web-specific Oriented
at custom infrastructures Some things still apply if you're on PaaS
How easy it is to install, configure and operate your
software?
Mostly about devs and ops working together
12factor.net
installation configuration operation
Installation
Installing postgres sudo apt-get install postgresql
Installing a python webapp sudo apt-get install build-essential python-virtualenv git
clone https://deadbeef@github.com/corp/repo cd repo virtualenv env env/bin/pip install -r requirements.txt # Figure out PYTHONPATH
Installing a python webapp sudo apt-get install build-essential python-virtualenv git
clone https://deadbeef@github.com/corp/repo cd repo virtualenv env env/bin/pip install -r requirements.txt # Figure out PYTHONPATH
Installing software is a solved problem Just use packaging Yep,
python packaging
Why python packaging? Release process Dependency management Trivial rollbacks Easy
system packaging
Packaging in 30 seconds # setup.py from distutils.core import setup
from setuptools import find_packages with open('requirements.txt') as reqs: install_requires = reqs.read().split('\n') setup( name='project', version=__import__('project').__version__, packages=find_packages(), include_package_data=True, zip_safe=False, install_requires=install_requires, ) # MANIFEST.in include requirements.txt recursive-include project *
Private package hosting Local filesystem Network-based, ala pypi.python.org python setup.py
sdist pip install --download dist -r requirements.txt rsync -avz -e ssh dist/ host.corp.com:/srv/pypi pip install --no-index --find-links=/srv/pypi myproject HTML directory index (apache / nginx / SimpleHTTPServer) pip install --no-index --find-links=http://host myproject
System packages fpm -s python -t deb setup.py awk -F=
'{printf "fpm -s python -t deb -v %s %s\n", $3, $1}' \ requirements.txt | sh https://github.com/rcrowley/freight https://github.com/jordansissel/fpm Sign, upload to your private repository sudo apt-get install python-myproject sudo apt-get install python-myproject=1.2.3
Pin your dependencies Bad Good Django Django>=1.4,<1.5 Django==1.4.5 http://nvie.com/posts/pin-your-packages/ This
is for end products, not libraries
Configuration
Configuring postgres $EDITOR /etc/postgresql/9.2/main/postgresql.conf service postgresql restart
Does your app have a config file? settings.py, production_settings.py are
not config files Configuration != code
Problems with configuration as code Incompatible with packaging Environment-specific code
Production-specific code will break production. Code shouldn't be tied to environments Code shouldn't be generated (salt / puppet / fabric)
Define your configuration What changes between environments? Database Secret key
Host / port Credentials to external services (AWS, Sentry…) Read configuration from your code .ini files yaml environment variables …
Code changes ↓ release, deploy app Infrastructure changes ↓ write
config, reload app
Config as environment variables Trivial to set with $PROCESS_MANAGER Native
to every programming language De-facto standard (PaaS). Interoperability! Shared hosting Apache Pros Cons
Case study: Django settings Before After DATABASES = {'default': {'HOST':
'localhost', …}} settings_local.py DATABASES = {'default': {'HOST': 'prod', …}} settings_prod.py DATABASES = {'default': {'HOST': 'staging', …}} settings_staging.py DATABASES = {'default': dj_database_url.config()} settings.py DATABASE_URL="postgres://host:5432/db" env
Config patterns SECRET_KEY = os.environ['SECRET_KEY'] KeyError: 'SECRET_KEY' PORT = int(os.environ.get('PORT',
8000)) Sane defaults when possible Prevent the app from booting if something critical is missing Use *_URL and parsers to reduce the number of variables EMAIL_URL DATABASE_URL REDIS_URL
In development django-dotenv virtualenvwrapper postactivate hooks custom manage.py envdir …
Operation
WSGI Have a WSGI entry point gunicorn myapp.wsgi -b 0.0.0.0:$PORT
Stateless processes Persistence via external services Database Caching Storage …
Scale out with processes More traffic? Spawn more processes. Caveat:
backend services rarely scale horizontally
Maximize dev/prod parity Same software Same people Same versions If
you use postgres in production, use postgres in development PostgreSQL 9.1 and 9.2 do not perform equally Developers should know about infrastructure
Continuous integration/deployment CI != green badge on your github page
CD != always running master in production Having shippable code Deploying it whenever your want CI CD tested packaged installable
Example workflow Commit Run tests Package Staging Production
Example workflow Staging Production Run tests Commit Package manual or
automated Jenkins, SaltStack, IRC bots are your automation friends
use packaging to manage software clearly define the configuration contract
automate as much as possible to minimize deployment friction
@brutasse bruno@renie.fr ?