Докеризация веб-приложения на Python Антон Егоров, Ostrovok.ru

Выступление на конференции PyCon Russia 2016

1. by Anton Egorov July 2016 Deliver Python Apps with Docker
2. Who am I? • Software Engineer, 10 years in web development • Scripting mostly with Python • 2 years using Docker, dockerized about 50 apps, 80% of them were Python apps • Ex Lead Developer at ostrovok.ru • CTO and Founder at sabaka.io
3. Plan • Quick introduction to Docker • App dockerization example • Production deployment notes • Treasure Goblin lives on special slides
4. Docker in a few slides
5. What is Docker? • It’s an open-source project that automates application deployment using containers • Initial release was in March 2013 • Written in Go • Works on Linux, OS X, Windows
6. Why Docker? • The best tool for software packaging • Easy to build, just follow best practices • Fast at runtime, overhead is close to nothing • You have identical installation at any environment (dev, stage, prod) • Predictable, easy to test, makes QA happy
7. Image • An image is a container basis • It’s a filesystem and a set of parameters to use at runtime (libs, bins, your software) • It doesn’t have a state and never changes • Dockerfile contains instructions to build the image
8. Container • A container is a running instance of the image • It consists of • Docker image • Execution environment • Set of instructions
9. Data Volume • Directory within one or more containers that bypasses the Union File System • Designed to persist data, independent of the container’s life cycle
10. Notes • Treat containers as read-only instances • Use data volumes to take care of persistent data
11. Layers
12. Images are layered • Docker store images in a layered file system • Each instruction from Dockerfile is a distinct read only layer • When you create a new container, you add a new, thin, writable layer
13. pull base image…
14. pull base image…
15. pull base image, done
16. build: add new layers
17. build: add new layers
18. build: add new layers
19. build: done
20. run container
21. Practice
22. Application • We have a demo Django app written in Python 3 • We are going to serve static along with the app • We don't care about HTTPS inside the container • Logging and monitoring is out of the scope of this talk
23. ls -la • Dockerfile — build instructions • .dockerignore — excludes files from the build context • demo — app source code • etc — runtime configuration files
24. Dockerfile
25. Dockerfile: libs, bins FROM ubuntu:16.04 RUN export DEBIAN_FRONTEND=noninteractive \ && apt-get update -y && apt-get upgrade -y \ && apt-get install -y nginx-light \ python3 python3-pip python3-psycopg2 \ && BUILD_DEPS='build-essential python3-dev' \ && apt-get install -y ${BUILD_DEPS} \ && pip3 install --no-cache-dir circus==0.13.0 gunicorn==19.5.0 \ && apt-get autoremove -y ${BUILD_DEPS} \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
26. Dockerfile best practices • Join commands sequence into one RUN instruction to reduce the number of layers • Clean build dependencies • Plan your layers to take advantage of the Docker build cache
27. Dockerfile: pip install • Container is an isolated environment, so you don't have to use python-virtualenv • We do python packages installation in a separate step to achieve caching COPY requirements.txt /opt/demo/app/
 RUN pip3 install --no-cache-dir \ -r /opt/demo/app/requirements.txt
28. Dockerfile: prepare the app • Copy the app source code and configuration files • Fix files permissions • Run validations (e.g. nginx -t) • Execute management scripts to prepare the app
29. Dockerfile: prepare the app COPY etc/ /etc/ COPY demo/ /opt/demo/app/ WORKDIR /opt/demo/app ENV STATIC_ROOT=/opt/demo/static RUN nginx -t \ && ./manage.py collectstatic \ --settings=demo.settings --no-input -v0 CMD ["circusd", "/etc/circus/web.ini"]
30. etc
31. etc/circus/web.ini [watcher:web]
 cmd=/usr/local/bin/gunicorn \ demo.wsgi:application -c /etc/gunicorn.py
 working_dir = /opt/demo/app
 copy_env = True
 user = www-data [watcher:nginx]
 cmd = /usr/sbin/nginx
 stop_signal = QUIT
 user = root
32. etc/nginx/sites-available/default server {
 listen 80;
 location / {
 include proxy_params;
 proxy_pass http://127.0.0.1:8000;
 }
 location /static/ {
 root /opt/demo/;
 access_log off;
 }
 }
33. /etc/gunicorn.py import multiprocessing
 import os
 
 bind = '127.0.0.1:8000'
 default_workers = multiprocessing.cpu_count() * 2 + 1
 workers = os.environ.get( 'WEB_CONCURRENCY', default_workers)
 worker_class = 'sync'
 max_requests = 300 max_requests_jitter = 300
 errorlog = '-'
34. demo/settings.py The best way to configure your app is environment (e.g. database connection) DATABASES = {
 'default': dj_database_url.config(
 env='DATABASE_URL',
 default='postgres://localhost/demo')}
35. Demo
36. docker-compose.yml web: build: . container_name: web environment: DJANGO_SETTINGS_MODULE: demo.settings_prod DATABASE_URL: postgres://postgres:secret@db/demo ALLOWED_HOSTS: "*" ports: - "80:80" db: image: postgres:9.5 environment: POSTGRES_PASSWORD: "secret" POSTGRES_DB: "demo" volumes: - /var/lib/postgresql/data
37. Build and run We can build an image using docker-compose docker-compose build Run containers docker-compose up -d Run database migrations docker exec -it web \
 /opt/demo/app/manage.py migrate
38. Production
39. Docker Registry • You can use any SaaS solution (e.g. Docker Hub, Quay, Amazon ECR, Google Container Registry) • Or deploy and configure your own Registry, it's easy and fully automated
40. Orchestration • Docker Compose is a good choice for small projects • You should automate your deployment with tools like Ansible or have a look at Docker Swarm if you want to manage a medium-sized cluster • Kubernetes and Mesosphere are complex tools with advanced features • Alternatively you can use SaaS solutions like Docker Cloud, Amazon ECS, Google Container Engine
41. Good luck!
42. Links • Demo https://github.com/satyrius/paid • Docs https://docs.docker.com/ • Best practices https://docs.docker.com/engine/ userguide/eng-image/dockerfile_best-practices/ • Circus https://circus.readthedocs.io/en/latest/for-ops/ configuration/ • Gunicorn http://docs.gunicorn.org/en/stable/ settings.html
43. Jobs • ostrovok.ru is looking for Python/Golang/iOS/ Android developers and much more
 https://jobs.lever.co/ostrovok.ru • sabaka.io is looking for DevOps/SysOps engineers and Python/Golang talents
 https://angel.co/sabaka/jobs
44. Questions? anton.egoroff@gmail.com