Magic Beans - Deploying Django on Elastic Beanstalk

Slide 1

Slide 1 text

Magic Beans Deploying Django on Elastic Beanstalk with Docker Sebastian Vetter @elbaschid github.com/elbaschid

Slide 2

Slide 2 text

For those who don't know me • Sebastian Vetter • Backend Developer @ Snowball One • You can ﬁnd me on: • twitter: @elbaschid • github: github.com/elbaschid

Slide 3

Slide 3 text

Deploying Websites Can Be Ugly

Slide 4

Slide 4 text

Deploying Websites Can Be Ugly 1. Set up server infrastructure. 2. Develop a magical application. 3. Use continuous integration, e.g. Travis 4. On success, manually deploy via scripts

Slide 5

Slide 5 text

Let's try and make it more beautiful

Slide 6

Slide 6 text

And we'll use beans for that

Slide 7

Slide 7 text

Not This One!

Slide 8

Slide 8 text

That's better!

Slide 9

Slide 9 text

AWS Elastic Beanstalk (EB) • Amazon's PaaS solution • Heroku-style deployments • Combines various parts of AWS And now with Docker support

Slide 10

Slide 10 text

EB Architecture (I)

Slide 11

Slide 11 text

EB Architecture (II)

Slide 12

Slide 12 text

Elastic Beanstalk Console (I)

Slide 13

Slide 13 text

Elastic Beanstalk Console (II)

Slide 14

Slide 14 text

What is Docker?

Slide 15

Slide 15 text

What is Docker? • Isolated processes in userspace • Immutable containers • Lightweight images • Git-style container distribution • More on https://www.docker.com/ whatisdocker/

Slide 16

Slide 16 text

Let's look at the magic

Slide 17

Slide 17 text

The ideal solution ...

Slide 18

Slide 18 text

... was a failure!

Slide 19

Slide 19 text

The new approach

Slide 20

Slide 20 text

The main steps 1. Push to github 2. Triggers a test build on Travis 3. On success: 1. Create deployment artefact 2. Store in S3 3. Beanstalk creates Docker container 4. Deploy to EC2

Slide 21

Slide 21 text

How it works

Slide 22

Slide 22 text

Options to run/deploy a docker container on EB. • Using Dockerrun.aws.json to pull from docker registry • Using Dockerﬁle to build on the EC2 instance • Using zip archive including both ﬁles + more

Slide 23

Slide 23 text

The build artefact my-magic-app.zip + Dockerfile + Dockerrun.aws.json + scripts/ + .ebextensions/ ....

Slide 24

Slide 24 text

Dockerfile FROM stackbrew/ubuntu:14.04 RUN apt-get -qq update && \ apt-get install -y -q all-the-things && \ curl https://bootstrap.pypa.io/get-pip.py -o /tmp/get-pip.py && \ python /tmp/get-pip.py ADD www /app/ WORKDIR /app RUN pip install -r deploy/requirements/test.txt EXPOSE 8000 VOLUME ['/var/log'] ADD scripts/start.sh /app/start.sh CMD /app/start.sh

Slide 25

Slide 25 text

Dockerrun.aws.json { "AWSEBDockerrunVersion": "1", "Volumes": [ { "HostDirectory": "/var/log/my-app", "ContainerDirectory": "/var/log" } ], "Ports": [ { "ContainerPort": "8000" } ], "Logging": "/var/log" }

Slide 26

Slide 26 text

Setting up Travis to release docker image on success. • Run the full test suite • After success, build the Docker container • Deploy the new container incl. static ﬁles

Slide 27

Slide 27 text

After successful tests after_success: - cd .. - ./scripts/deploy_to_beanstalk.sh

Slide 28

Slide 28 text

Putting the magic into beans • No easy to use deployment scripts from Amazon • eb: useless and broken • awscli: easy to setup but lots of cli ﬂags

Slide 29

Slide 29 text

It's Roll-Your-Own Time • Small script beanstalk • Uses beans.yml for conﬁguration • Inspired by tools like ﬁg

Slide 30

Slide 30 text

Settings in beans.yml app_name: my-eb-app bucket_name: my-s3-bucket my-app-env: environment: AWS_ACCESS_KEY_ID: AWS_SECRET_KEY: DJANGO_SECRET_KEY: DJANGO_DATABASE_HOST: DJANGO_DATABASE_PORT: DJANGO_DATABASE_NAME:

Slide 31

Slide 31 text

Using Beansstalk $ python beanstalk.py create_archive $ python beanstalk.py release $ python beanstalk.py deploy As an example: export RELEASE_VERSION=$TRAVIS_JOB_ID-$GIT_COMMIT $ python beanstalk.py create_archive ${RELEASE_VERSION} $ python beanstalk.py release ${RELEASE_VERSION} $ python beanstalk.py deploy my-app-env ${RELEASE_VERSION}

Slide 32

Slide 32 text

Versioning and deployment to EB environment. • Semantic versioning doesn't work in CD (..) • How to generate meaningful versions?

Slide 33

Slide 33 text

What we do: • Use the Travis Job ID • And the git commit GIT_COMMIT=$(git rev-parse --short HEAD) RELEASE_VERSION=$TRAVIS_JOB_$GIT_COMMIT

Slide 34

Slide 34 text

Running the Docker container #!/bin/bash set -e python manage.py migrate /usr/local/bin/uwsgi --http :8000 \ --wsgi-file deploy/wsgi/test.py \ --logto /var/log/uwsgi.log

Slide 35

Slide 35 text

Ideas and improvements • Building docker images on Travis CI and alternatives (Circle CI, Wercker). • Running browser tests against the production container instead of live server testcase. • Handling migrations in continiuous deployment. • Deploying with zero downtime.