Applied Buildout Gary Walker 

Overview What is buildout? What problems can it solve? How do I use it? How do I get it running in production? 

What is Buildout? 

buildout Write code Install dependencies Freeze dependencies Distribute list of dependencies Write code Configure dependencies Run buildout Distribute config file virtualenv/pip virtualenv vs buildout - development 

virtualenv vs buildout - installation buildout Clone repository Create virtualenv pip install packages Download buildout.cfg Run bootstrap.py Run bin/buildout virtualenv/pip 

Example Buildout Config [buildout] parts = myapp [myapp] recipe = zc.recipe.egg eggs = something-from-pypi 

Getting a Clean Environment 

Bootstrapping - Windows http://downloads.buildout.org/2/bootstrap.py Inside the domain: \\share\buildout\bootstrap.bat Uses a version of Python installed by Group Policy Outside the domain: powershell -command (new-object System.Net.WebClient). Downloadfile(\"http://shapeset/buildout/bootstrap/bootstrap. py\", \"bootstrap.py\") 

Bootstrapping - POSIX curl http://share/buildout/bootstrap.py > bootstrap.py 

Further Information Brandon Rhodes, “A Brief Introduction to Buildout” http://www.youtube.com/watch?v=HXvzzK9m2IA Benoit Bryan, “From pip+virtualenv to buildout” http://www.marmelune.net/en/python/buildout/from- virtualenv-to-buildout/ 

What Problems Can it Solve? 

Build-System Code in Every Branch 

Semantic Versioning Given a version number MAJOR.MINOR. PATCH, increment the: 1. MAJOR version a. when you make incompatible API changes, 2. MINOR version a. when you add functionality in a backwards- compatible manner, and 3. PATCH version a. when you make backwards-compatible bug fixes. b. 

Semantic Versioning in Practice [buildtools] recipe = zc.recipe.egg eggs = hurf < 1 durf < 2 

Extending Buildout with Recipes 

Artifact Repository Recipe [clientdata] recipe = artifactrepo.recipe project = clientdatapackage version = 2.3 destination = ../../clientdata_directory 

Artifact Repository Recipe def update(self): new_file_info = self.get_download_info() existing_file_info = self.get_update_info() if new_file_info != existing_file_info: self.install() 

Artifact Repository Recipe def get_download_info(self): url_params = { 'project': self.options['project'], 'version': self.options.get('version', ''), url = REPO_URL + urlencode(url_params) return urlopen(url).read() 

Artifact Repository Recipe def install(self): file_info = self.get_download_info() destination_dir = self.options.get('destination') if not os.path.isdir(destination_dir): os.makedirs(destination_dir) # cache info about file so we can check in update() json.dump(file_info, open(self.update_info_path, 'w')) self.download_file(file_info['file_path']) 

Production Infrastructure 

Config-file Inheritance 

Config Variable Substitution ${sectionname:optionname} Works up and down inheritance chain 

Config-file Inheritance: base.cfg [buildout] show-picked-versions = true find-links = http://egg/repository 

Config-file Inheritance: base-dev.cfg [buildout] extends = http://configs/base.cfg eggs = ${buildout.package} develop = . parts = app nose [nose] recipe = pbp.recipe.noserunner eggs = ${buildout.eggs} [app] recipe = zc.recipe.egg eggs = ${buildout.eggs} 

Config-file Inheritance: app.cfg [buildout] package = myapp extends = http://configs/base-dev.cfg 

Buildout bootstrap.py -v 1.2.3 distribute/setuptools ● point bootstrap.py to custom distribute setup script ● hardcode distribute version in that setup script Resilience Against Buildout/Distribute Code Changes 

Resilience against PyPI downtime https://pypi.python.org/pypi/proxypypi $ pip install proxypypi $ cd cachedir $ proxypypi run [buildout] index = http://pypi-proxy-machine/simple/