Tim Penhey: Deploying a Django application using Juju

Slide 1

Slide 1 text

Deploying a Django based application with Juju Tim Penhey @howbazaar

Slide 2

Slide 2 text

or perhaps

Slide 3

Slide 3 text

How I used Juju to deploy our Django based website to Amazon EC2 Tim Penhey @howbazaar

Slide 4

Slide 4 text

● Not a Django talk ● Not about how Juju works ● Not lots of python code What this talk is not

Slide 5

Slide 5 text

Backstory ● Rachel had this cool idea… ● I said “I know how to do that…”

Slide 6

Slide 6 text

Backstory ● Co-founded “know about when and where” (kawaw) with Rachel Saunders and Brenton Hall ● Web site and mobile application combo ● MVP ready… now what?

Slide 7

Slide 7 text

● Django ● PostgreSQL ● jQuery, backbone.js ● PhoneGap The basics

Slide 8

Slide 8 text

Developing on my laptop We did what all developers did… ./manage.py runserver … but then, after a while … ./manage.py runserver_plus

Slide 9

Slide 9 text

Making it publicly visible Now we had real issues ● keeping the site alive ● backups ● secure connections ● scalability

Slide 10

Slide 10 text

Disclaimer I work for Canonical on the Juju project

Slide 11

Slide 11 text

What is Juju? ● Service orchestration in the cloud Configure, manage, maintain, deploy and scale efficiently with best-practice Charms on any public, private or hybrid cloud from a powerful GUI or the command-line.

Slide 12

Slide 12 text

Charms give Juju its power ● encapsulate application configurations ● define how services get deployed ● define how they connect to other services ● know how to scale

Slide 13

Slide 13 text

but really… why?

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

Existing charms postgresql python-django gunicorn

Slide 16

Slide 16 text

Test deployment locally ● deploy into LXC containers on my laptop ● same charms are used ● tests interaction

Slide 17

Slide 17 text

Juju can help with scaling later ● Add ha-proxy, memcached ● Multiple database servers ● Scale out the application servers

Slide 18

Slide 18 text

Simple repeatable deployments ● Repeatability is a key ● New code as easy as upgrading a charm

Slide 19

Slide 19 text

however private codebase

Slide 20

Slide 20 text

Writing a subordinate charm ● Simplest way to get code on to the same machine as the python-django charm was to write a subordinate charm ● a.k.a. payload charm ● puts the python code and resources into the right place on the disk

Slide 21

Slide 21 text

Hooks ● install ● config-changed ● upgrade-charm ● python-django-relation-joined ● python-django-relation-changed

Slide 22

Slide 22 text

Hooks - continued ● Hooks are just scripts ● Juju is hook language agnostic ● Just needs to be executable ● I use python (with charm-helpers)

Slide 23

Slide 23 text

Hooks - reality ● Install packages ○ apt, pip ○ install, upgrade-charm ● Everything else ○ copy code across ○ config-changed, relationship hooks ○ config-changed also gets called after upgrade- charm

Slide 24

Slide 24 text

Charm settings.py import glob from os.path import abspath, dirname, join PROJECT_DIR = abspath(dirname(__file__)) conffiles = glob.glob(join(PROJECT_DIR, '{{ dir }}', '*.py')) conffiles.sort() if not 'INSTALLED_APPS' in locals(): from django.conf.global_settings import * for f in conffiles: exec(open(abspath(f)).read())

Slide 25

Slide 25 text

First real code change ● break up settings.py ● charm makes files based on relationships or configuration settings ○ config options ■ 10-secret.py ■ 30-allowed.py ■ 40-debug.py ■ 50-extra-config.py ○ relationships - e.g. postgresql ■ 20-engine-pgsql.py

Slide 26

Slide 26 text

Mirror the charm settings ● Very similar base settings.py ● Added settings dir ○ 10-secret.py - SECRET_KEY ○ 15-defaults.py - template loaders, base apps ○ 20-database.py - local db settings ○ 25-static-files.py - STATIC* settings ○ 30-debug.py - DEBUG and TEMPLATE_DEBUG ○ 40-project.py - root urls, wsgi app ○ 45-debug-toolbar-juju.py ○ 50-kawaw-juju.py ○ 60-allauth-juju.py ○ 70-logging-juju.py ○ 80-email-juju.py

Slide 27

Slide 27 text

urls.py was next ● Similar loading pattern for urls ● Kinda makes sense for a framework charm

Slide 28

Slide 28 text

First deployment

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

Error ● Improperly configured ● hmm… more info needed ● increased logging ● turned on debug

Slide 31

Slide 31 text

Second deployment

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Error ● debug toolbar not configured ● manual registration needed for urls if not using ./manage.py

Slide 34

Slide 34 text

Third deployment

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

Error ● allauth not configured ● hadn’t set up the google and facebook entries ● oh yeah… ● off to the admin interface

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No CSS, no images ● default static file serving not enabled with gunicorn ● google suggests nginx

Slide 39

Slide 39 text

nginx ● no promoted nginx charm ● I wanted it to run beside the django charm ● simplest approach was to roll it into the kawaw subordinate ● added SSL ● ./manage.py collectstatic --noinput

Slide 40

Slide 40 text

Fourth deployment

Slide 41

Slide 41 text

OMG - it worked!

Slide 42

Slide 42 text

kawaw charm makefile update-payload: @mkdir -p payload/ssl @cat ../../../requirements.txt | grep -v "^pytz" | grep -v "^psycopg2" > payload/requirements.txt @cp -r ../../../wit payload @cp ../../ssl/server.key payload/ssl @cp ../../ssl/kawaw.pem payload/ssl @rm -r payload/wit/fixtures

Slide 43

Slide 43 text

Deploying updated code $ juju upgrade-charm kawaw That’s it

Slide 44

Slide 44 text

Lessons learned

Slide 45

Slide 45 text

Lessons learned ● insufficient documentation around using the python-django charm, had to read the code ● insufficient documentation around how to write a payload charm for python-django, very much work it out as you go

Slide 46

Slide 46 text

Lessons learned ● python-django and gunicorn still needs nginx to be really useful ● should have a full stack django charm that includes gunicorn and nginx by default ○ that just knows how to enable SSL for apps

Slide 47

Slide 47 text

Future work ● PostgreSQL charm has daily backups by default, need to test restore ● Database still lives on the ephemeral instance, need to move it elsewhere ● Automated staging site based of production that has a copy of production database

Slide 48

Slide 48 text

Juju commands cd ~/src/wit/working/juju export JUJU_REPOSITORY=$(pwd) juju bootstrap --constraints 'mem=2048 root-disk=20G' juju deploy --to 0 cs:trusty/postgresql --config kawaw-prod.yaml juju deploy --to 0 local:python-django --config kawaw-prod.yaml juju deploy local:gunicorn juju add-relation postgresql:db python-django juju add-relation python-django gunicorn juju deploy local:kawaw juju add-relation python-django kawaw juju expose kawaw

Slide 49

Slide 49 text

Questions?