Ринат Хабибиев (Redmadrobot) - Автоматизация деплоя Docker контейнеров при помощи Fabric

Доклад с Moscow Python Conf 2016 (http://conf.python.ru)
Видео: https://conf.python.ru/avtomatizaciya-deploya-docker-kontejnerov-pri-pomoshi-fabric/

Контейнеризация приложений сегодня является не просто модным трендом. Объективно такой подход позволяет во многом оптимизировать процесс серверной разработки путем унификации поддерживаемых окружений (dev, test, staging, production). Запуск Docker контейнеров на произвольной инфраструктуре, применение миграций и бэкап БД, а также откат к предыдущему состоянию - как эти задачи решаются при помощи Fabric - тема моего доклада.

Transcript

1. Docker deploy automation 
 with Fabric AUTHOR: Rinat Khabibiev MOSCOW, October 2016 $ fab
2. RINAT KHABIBIEV 2 https://github.com/renskiy https://www.facebook.com/rinat.khabibiev
3. Since 2008 http://www.redmadrobot.com
4. 4 https://www.docker.com/company SOME DOCKER STATISTICS
5. 5 DOCKER, CAVEATS • Open source • Network overhead https://domino.research.ibm.com/library/cyberdig.nsf/papers/0929052195DD819C85257D2300681E7B/$File/rc25482.pdf
6. 6 DOCKER, BENEFITS • Simple requirements • CI/CD integration • Doesn’t require Internet connection to update App • The 12-factor App (12factor.net)
7. 7 REDMADROBOT: TOTAL DOCKERIZATION • Python (Django) • Cron • PostgreSQL • RabbitMQ • Nginx • Elasticsearch • Redis
8. 8 AVAILABLE DEPLOY TOOLS •Vagrant •Fabric •Ansible •Capistrano •Docker compose •Kubernetes •?
9. 9 DEPLOY TOOL REQUIREMENTS • Arbitrary environment support • Easy adaptation (copy/paste) • Easy customization • No special education or experience requirements • Docker support • DB migrations apply and rollback • Support of private registry and complex networks
10. FABRICIO 10 https://github.com/renskiy/fabricio DOCKER DEPLOY AUTOMATION TOOL
11. 11 FABRICIO DEPLOY CONFIG EXAMPLE # fabfile.py
 from fabricio import docker, tasks
 
 
 class NginxContainer(docker.Container):
 
 image = docker.Image('nginx:1.9')
 
 ports = '80:80'
 
 nginx = tasks.DockerTasks(
 container=NginxContainer(name='web'),
 hosts=['user@example.com'],
 )

12. 12 FABRICIO DEPLOY PROCESS $ fab --list Available commands: nginx backup -> pull -> migrate -> update nginx.deploy backup -> pull -> migrate -> update nginx.pull pull Docker image from registry nginx.rollback rollback Docker container to previous version nginx.update start new Docker container if necessary $ fab nginx [user@example.com] Executing task ‘pull’ [user@example.com] run: docker pull nginx:stable [user@example.com] out: 1.9: Pulling from library/nginx ... $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ec44b023adf0 nginx:1.9 "nginx -g" 2 minutes ago Up 2 minutes 0.0.0.0:80->80/tcp web
13. 13 FABRICIO FEATURES • Build Docker images • Create containers from images with provided tags • Unlimited infrastructures • Parallel execution (Fabric feature) • Rollback containers to previous version • Work with public and private Docker registries • DB migrations and rollback, backup and restore
14. 14 FABRICIO COMMAND PARAMS # show detailed information about command $ fab --display nginx Displaying detailed information for task 'nginx': backup -> pull -> migrate -> update Arguments: self, tag=None, force=False, migrate=True, backup=False # deploy container using image with provided tag $ fab nginx:1.10 # force new container $ fab nginx:force=yes # combo! $ fab nginx:1.10,force=yes # rollback container to previous version $ fab nginx.rollback
15. 15 FABRICIO DOCKER CONTAINER DEFINITION class Container(object): 
 image = None # type: Image
 cmd = None stop_timeout = 10
 user = None
 ports = None
 env = None
 volumes = None
 links = None
 hosts = None
 network = None
 restart_policy = None
 stop_signal = None
 
 def __init__(self, name, **options):
 self.name = name
 ...
16. 16 FABRICIO AND PRIVATE DOCKER REGISTRY # start local Docker registry $ docker run -d -p 5000:5000 --name registry -v /data/registry:/var/lib/registry registry:2 nginx = tasks.PullDockerTasks(
 container=NginxContainer('web'),
 hosts=['user@example.com'],
 ) $ fab --list Available commands: nginx prepare -> push -> backup -> pull -> migrate -> update nginx.deploy prepare -> push -> backup -> pull -> migrate -> update nginx.prepare prepare Docker image nginx.pull pull Docker image from registry nginx.push push Docker image to registry nginx.rollback rollback Docker container to previous version nginx.update start new Docker container if necessary
17. 17 BUILD DOCKER IMAGES WITH FABRICIO class MyContainer(docker.Container):
 
 image = docker.Image('my_image')
 
 app = tasks.BuildDockerTasks(
 container=MyContainer('my_service'),
 hosts=['user@example.com'],
 build_path='.',
 ) $ fab --list Available commands: app prepare -> push -> backup -> pull -> migrate -> update app.deploy prepare -> push -> backup -> pull -> migrate -> update app.prepare prepare Docker image app.pull pull Docker image from registry app.push push Docker image to registry app.rollback rollback Docker container to previous version app.update start new Docker container if necessary
18. 18 FABRICIO ROLES AND INFRASTRUCTURES from fabric import api as fab @tasks.infrastructure
 def production():
 fab.env.update(roledefs={'front': ['user@example.com']}) nginx = tasks.DockerTasks(
 container=NginxContainer(name='web'),
 roles=['front'],
 ) $ fab --list Available commands: production select production infrastructure to run task(s) on production.confirm automatically confirm production infrastructure selection ... $ fab production nginx Are you sure you want to select production infrastructure to run task(s) on? [y/N]
19. 19 FABRICIO: DEPLOYING DJANGO PROJECTS from fabric import api as fab
 from fabricio import docker, tasks
 from fabricio.apps.python.django import DjangoContainer
 
 class BaseDjangoContainer(DjangoContainer):
 image = docker.Image('my_django')
 
 @property
 def env(self):
 return 'DJANGO_SETTINGS_MODULE=settings.{}'.format(
 fab.env.infrastructure,
 )
 
 django = tasks.BuildDockerTasks(
 container=BaseDjangoContainer('api'),
 hosts=['user@example.com'],
 migrate_commands=True,
 )
20. 20 FABRICIO: DEPLOYING DJANGO PROJECTS $ fab --list Available commands: django prepare -> push -> backup -> pull -> migrate -> update django.deploy prepare -> push -> backup -> pull -> migrate -> update django.migrate apply migrations django.migrate_back remove previously applied migrations if any django.prepare prepare Docker image django.pull pull Docker image from registry django.push push Docker image to registry django.rollback rollback Docker container to previous version django.update start new Docker container if necessary
21. 21 FABRICIO: DATA BACKUP AND RESTORE from fabricio.apps.db.postgres import PostgresqlBackupMixin
 
 class BackupDjangoContainer(BaseDjangoContainer, PostgresqlBackupMixin):
 volumes = '/data/backup/postgres:/backup'
 db_backup_dir = '/backup'
 
 django = tasks.BuildDockerTasks(
 container=BackupDjangoContainer('api'),
 hosts=['user@example.com'],
 backup_commands=True, ) $ fab --list Available commands: django.backup backup data django.restore restore data ...
22. 22 FABRICIO REQUIREMENTS AND INSTALL • Python 2.6 or 2.7 • Docker CLI (Linux/Mac/Windows) • Docker 1.9 or greater recommended (remote side) # virtualenv install $ pip install --upgrade fabricio # macOS system-wide install $ sudo pip install --upgrade fabricio six==1.4.1
23. 23 FABRICIO ROADMAP •Master-Slave configurations for PostgreSQL •Docker Swarm support •docker-py integration •MySQL? •Non-Django frameworks?
25. 25 USEFUL LINKS & QUESTIONS • Fabricio: https://github.com/renskiy/fabricio • Author of Fabricio: https://www.facebook.com/rinat.khabibiev • Хабра-блог Redmadrobot: https://habrahabr.ru/company/redmadrobot • Redmadrobot on Facebook: https://www.facebook.com/redmadrobot • The 12-factor App (SaaS dev patterns): https://12factor.net • Docker image with cron: https://hub.docker.com/r/renskiy/cron Questions