Tim Penhey: Deploying a Django application using Juju

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Tim Penhey:
Deploying a Django application using Juju
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2014 - Sunday, 14 Sep 2014 - Track 2
http://kiwi.pycon.org/

**Audience level**

Intermediate

**Description**

Juju is a service orchestration tool for deploying services into public or private clouds. I will talk about how I deployed my Django application using Juju.

**Abstract**

Juju is a service orchestration tool for deploying services into public or private clouds. The services that Juju deploys are described in Charms. A Charm should use the "best practice" for deploying the service. In the case of Django, this would be using gunicorn and nginx. I explain how the postgresql charm communicates with the django charm, and how I had to change my application's settings and urls to work with the django charm. The application that is deployed is a "proprietary" app, and as such, the deployment of the application itself is done with another charm. The interaction between the subordinate charms and the principal django charm are explained, and how the data is exchanged between the charms. As well as the deployment story, I also explain how application updates are deployed very simply.

**YouTube**

https://www.youtube.com/watch?v=6hc8VkFeHjA

Transcript

1. Deploying a Django
   based application
   with Juju
   Tim Penhey
   @howbazaar
   

   View Slide

2. or perhaps
   

   View Slide

3. How I used Juju to deploy
   our Django based website
   to Amazon EC2
   Tim Penhey
   @howbazaar
   

   View Slide

4. ● Not a Django talk
   ● Not about how Juju works
   ● Not lots of python code
   What this talk is not
   

   View Slide

5. Backstory
   ● Rachel had this cool idea…
   ● I said “I know how to do that…”
   

   View Slide

6. Backstory
   ● Co-founded “know about when and where”
   (kawaw) with Rachel Saunders and Brenton
   Hall
   ● Web site and mobile application combo
   ● MVP ready… now what?
   

   View Slide

7. ● Django
   ● PostgreSQL
   ● jQuery, backbone.js
   ● PhoneGap
   The basics
   

   View Slide

8. Developing on my laptop
   We did what all developers did…
   ./manage.py runserver
   … but then, after a while …
   ./manage.py runserver_plus
   

   View Slide

9. Making it publicly visible
   Now we had real issues
   ● keeping the site alive
   ● backups
   ● secure connections
   ● scalability
   

   View Slide

10. Disclaimer
    I work for Canonical on the Juju project
    

    View Slide

11. What is Juju?
    ● Service orchestration in the cloud
    Configure, manage, maintain, deploy and
    scale efficiently with best-practice Charms on
    any public, private or hybrid cloud from a
    powerful GUI or the command-line.
    

    View Slide

12. Charms give Juju its power
    ● encapsulate application configurations
    ● define how services get deployed
    ● define how they connect to other services
    ● know how to scale
    

    View Slide

13. but really…
    why?
    

    View Slide

14. View Slide

15. Existing charms
    postgresql
    python-django
    gunicorn
    

    View Slide

16. Test deployment locally
    ● deploy into LXC containers on my laptop
    ● same charms are used
    ● tests interaction
    

    View Slide

17. Juju can help with scaling later
    ● Add ha-proxy, memcached
    ● Multiple database servers
    ● Scale out the application servers
    

    View Slide

18. Simple repeatable deployments
    ● Repeatability is a key
    ● New code as easy as upgrading a charm
    

    View Slide

19. however
    private codebase
    

    View Slide

20. Writing a subordinate charm
    ● Simplest way to get code on to the same
    machine as the python-django charm was to
    write a subordinate charm
    ● a.k.a. payload charm
    ● puts the python code and resources into
    the right place on the disk
    

    View Slide

21. Hooks
    ● install
    ● config-changed
    ● upgrade-charm
    ● python-django-relation-joined
    ● python-django-relation-changed
    

    View Slide

22. Hooks - continued
    ● Hooks are just scripts
    ● Juju is hook language agnostic
    ● Just needs to be executable
    ● I use python (with charm-helpers)
    

    View Slide

23. Hooks - reality
    ● Install packages
    ○ apt, pip
    ○ install, upgrade-charm
    ● Everything else
    ○ copy code across
    ○ config-changed, relationship hooks
    ○ config-changed also gets called after upgrade-
    charm
    

    View Slide

24. Charm settings.py
    import glob
    from os.path import abspath, dirname, join
    PROJECT_DIR = abspath(dirname(__file__))
    conffiles = glob.glob(join(PROJECT_DIR, '{{ dir }}', '*.py'))
    conffiles.sort()
    if not 'INSTALLED_APPS' in locals():
    from django.conf.global_settings import *
    for f in conffiles:
    exec(open(abspath(f)).read())
    

    View Slide

25. First real code change
    ● break up settings.py
    ● charm makes files based on relationships or
    configuration settings
    ○ config options
    ■ 10-secret.py
    ■ 30-allowed.py
    ■ 40-debug.py
    ■ 50-extra-config.py
    ○ relationships - e.g. postgresql
    ■ 20-engine-pgsql.py
    

    View Slide

26. Mirror the charm settings
    ● Very similar base settings.py
    ● Added settings dir
    ○ 10-secret.py - SECRET_KEY
    ○ 15-defaults.py - template loaders, base apps
    ○ 20-database.py - local db settings
    ○ 25-static-files.py - STATIC* settings
    ○ 30-debug.py - DEBUG and TEMPLATE_DEBUG
    ○ 40-project.py - root urls, wsgi app
    ○ 45-debug-toolbar-juju.py
    ○ 50-kawaw-juju.py
    ○ 60-allauth-juju.py
    ○ 70-logging-juju.py
    ○ 80-email-juju.py
    

    View Slide

27. urls.py was next
    ● Similar loading pattern for urls
    ● Kinda makes sense for a framework charm
    

    View Slide

28. First deployment
    

    View Slide

29. View Slide

30. Error
    ● Improperly configured
    ● hmm… more info needed
    ● increased logging
    ● turned on debug
    

    View Slide

31. Second deployment
    

    View Slide

32. View Slide

33. Error
    ● debug toolbar not configured
    ● manual registration needed for urls if not
    using ./manage.py
    

    View Slide

34. Third deployment
    

    View Slide

35. View Slide

36. Error
    ● allauth not configured
    ● hadn’t set up the google and facebook
    entries
    ● oh yeah…
    ● off to the admin interface
    

    View Slide

37. View Slide

38. No CSS, no images
    ● default static file serving not enabled with
    gunicorn
    ● google suggests nginx
    

    View Slide

39. nginx
    ● no promoted nginx charm
    ● I wanted it to run beside the django charm
    ● simplest approach was to roll it into the
    kawaw subordinate
    ● added SSL
    ● ./manage.py collectstatic --noinput
    

    View Slide

40. Fourth deployment
    

    View Slide

41. OMG - it worked!
    

    View Slide

42. kawaw charm makefile
    update-payload:
    @mkdir -p payload/ssl
    @cat ../../../requirements.txt | grep -v "^pytz"
    | grep -v "^psycopg2" > payload/requirements.txt
    @cp -r ../../../wit payload
    @cp ../../ssl/server.key payload/ssl
    @cp ../../ssl/kawaw.pem payload/ssl
    @rm -r payload/wit/fixtures
    

    View Slide

43. Deploying updated code
    $ juju upgrade-charm kawaw
    That’s it
    

    View Slide

44. Lessons learned
    

    View Slide

45. Lessons learned
    ● insufficient documentation around using
    the python-django charm, had to read the
    code
    ● insufficient documentation around how to
    write a payload charm for python-django,
    very much work it out as you go
    

    View Slide

46. Lessons learned
    ● python-django and gunicorn still needs
    nginx to be really useful
    ● should have a full stack django charm that
    includes gunicorn and nginx by default
    ○ that just knows how to enable SSL for apps
    

    View Slide

47. Future work
    ● PostgreSQL charm has daily backups by
    default, need to test restore
    ● Database still lives on the ephemeral
    instance, need to move it elsewhere
    ● Automated staging site based of
    production that has a copy of production
    database
    

    View Slide

48. Juju commands
    cd ~/src/wit/working/juju
    export JUJU_REPOSITORY=$(pwd)
    juju bootstrap --constraints 'mem=2048 root-disk=20G'
    juju deploy --to 0 cs:trusty/postgresql --config kawaw-prod.yaml
    juju deploy --to 0 local:python-django --config kawaw-prod.yaml
    juju deploy local:gunicorn
    juju add-relation postgresql:db python-django
    juju add-relation python-django gunicorn
    juju deploy local:kawaw
    juju add-relation python-django kawaw
    juju expose kawaw
    

    View Slide

49. Questions?
    

    View Slide