Magic Beans - Deploying Django on Elastic Beanstalk

Transcript

1. Magic Beans Deploying Django on Elastic Beanstalk with Docker Sebastian Vetter @elbaschid github.com/elbaschid
2. For those who don't know me • Sebastian Vetter • Backend Developer @ Snowball One • You can find me on: • twitter: @elbaschid • github: github.com/elbaschid
3. Deploying Websites Can Be Ugly
4. Deploying Websites Can Be Ugly 1. Set up server infrastructure. 2. Develop a magical application. 3. Use continuous integration, e.g. Travis 4. On success, manually deploy via scripts
5. Let's try and make it more beautiful
6. And we'll use beans for that
7. Not This One!
8. That's better!
9. AWS Elastic Beanstalk (EB) • Amazon's PaaS solution • Heroku-style deployments • Combines various parts of AWS And now with Docker support
10. EB Architecture (I)
11. EB Architecture (II)
12. Elastic Beanstalk Console (I)
13. Elastic Beanstalk Console (II)
14. What is Docker?
15. What is Docker? • Isolated processes in userspace • Immutable containers • Lightweight images • Git-style container distribution • More on https://www.docker.com/ whatisdocker/
16. Let's look at the magic
17. The ideal solution ...
18. ... was a failure!
19. The new approach
20. The main steps 1. Push to github 2. Triggers a test build on Travis 3. On success: 1. Create deployment artefact 2. Store in S3 3. Beanstalk creates Docker container 4. Deploy to EC2
21. How it works
22. Options to run/deploy a docker container on EB. • Using Dockerrun.aws.json to pull from docker registry • Using Dockerfile to build on the EC2 instance • Using zip archive including both files + more
23. The build artefact my-magic-app.zip + Dockerfile + Dockerrun.aws.json + scripts/ + .ebextensions/ ....
24. Dockerfile FROM stackbrew/ubuntu:14.04 RUN apt-get -qq update && \ apt-get install -y -q all-the-things && \ curl https://bootstrap.pypa.io/get-pip.py -o /tmp/get-pip.py && \ python /tmp/get-pip.py ADD www /app/ WORKDIR /app RUN pip install -r deploy/requirements/test.txt EXPOSE 8000 VOLUME ['/var/log'] ADD scripts/start.sh /app/start.sh CMD /app/start.sh
25. Dockerrun.aws.json { "AWSEBDockerrunVersion": "1", "Volumes": [ { "HostDirectory": "/var/log/my-app", "ContainerDirectory": "/var/log" } ], "Ports": [ { "ContainerPort": "8000" } ], "Logging": "/var/log" }
26. Setting up Travis to release docker image on success. • Run the full test suite • After success, build the Docker container • Deploy the new container incl. static files
27. After successful tests after_success: - cd .. - ./scripts/deploy_to_beanstalk.sh
28. Putting the magic into beans • No easy to use deployment scripts from Amazon • eb: useless and broken • awscli: easy to setup but lots of cli flags
29. It's Roll-Your-Own Time • Small script beanstalk • Uses beans.yml for configuration • Inspired by tools like fig
30. Settings in beans.yml app_name: my-eb-app bucket_name: my-s3-bucket my-app-env: environment: AWS_ACCESS_KEY_ID: AWS_SECRET_KEY: DJANGO_SECRET_KEY: DJANGO_DATABASE_HOST: <RDS host> DJANGO_DATABASE_PORT: <RDS port> DJANGO_DATABASE_NAME: <RDS DB name DJANGO_CONFIGURATION: Test settings: 'command': Timeout: 1000
31. Using Beansstalk $ python beanstalk.py create_archive <version> $ python beanstalk.py release <version> $ python beanstalk.py deploy <env> <version> As an example: export RELEASE_VERSION=$TRAVIS_JOB_ID-$GIT_COMMIT $ python beanstalk.py create_archive ${RELEASE_VERSION} $ python beanstalk.py release ${RELEASE_VERSION} $ python beanstalk.py deploy my-app-env ${RELEASE_VERSION}
32. Versioning and deployment to EB environment. • Semantic versioning doesn't work in CD (<major>.<minor>.<patch>) • How to generate meaningful versions?
33. What we do: • Use the Travis Job ID • And the git commit GIT_COMMIT=$(git rev-parse --short HEAD) RELEASE_VERSION=$TRAVIS_JOB_$GIT_COMMIT
34. Running the Docker container #!/bin/bash set -e python manage.py migrate /usr/local/bin/uwsgi --http :8000 \ --wsgi-file deploy/wsgi/test.py \ --logto /var/log/uwsgi.log
35. Ideas and improvements • Building docker images on Travis CI and alternatives (Circle CI, Wercker). • Running browser tests against the production container instead of live server testcase. • Handling migrations in continiuous deployment. • Deploying with zero downtime.
36. Looks great from up here
37. But still not Paradise