Ринат Хабибиев (Redmadrobot) - Автоматизация деплоя Docker контейнеров при помощи Fabric

Доклад с Moscow Python Conf 2016 (http://conf.python.ru)
Видео: https://conf.python.ru/avtomatizaciya-deploya-docker-kontejnerov-pri-pomoshi-fabric/

Контейнеризация приложений сегодня является не просто модным трендом. Объективно такой подход позволяет во многом оптимизировать процесс серверной разработки путем унификации поддерживаемых окружений (dev, test, staging, production). Запуск Docker контейнеров на произвольной инфраструктуре, применение миграций и бэкап БД, а также откат к предыдущему состоянию - как эти задачи решаются при помощи Fabric - тема моего доклада.

Transcript

1. Docker deploy automation 

   with Fabric
   AUTHOR: Rinat Khabibiev
   MOSCOW, October 2016
   $ fab
   

   View Slide

2. RINAT KHABIBIEV
   2
   https://github.com/renskiy https://www.facebook.com/rinat.khabibiev
   

   View Slide

3. Since 2008
   http://www.redmadrobot.com
   

   View Slide

4. 4
   https://www.docker.com/company
   SOME DOCKER STATISTICS
   

   View Slide

5. 5
   DOCKER, CAVEATS
   • Open source
   • Network overhead
   https://domino.research.ibm.com/library/cyberdig.nsf/papers/0929052195DD819C85257D2300681E7B/$File/rc25482.pdf
   

   View Slide

6. 6
   DOCKER, BENEFITS
   • Simple requirements
   • CI/CD integration
   • Doesn’t require Internet connection to update App
   • The 12-factor App (12factor.net)
   

   View Slide

7. 7
   REDMADROBOT: TOTAL DOCKERIZATION
   • Python (Django)
   • Cron
   • PostgreSQL
   • RabbitMQ
   • Nginx
   • Elasticsearch
   • Redis
   

   View Slide

8. 8
   AVAILABLE DEPLOY TOOLS
   •Vagrant
   •Fabric
   •Ansible
   •Capistrano
   •Docker compose
   •Kubernetes
   •?
   

   View Slide

9. 9
   DEPLOY TOOL REQUIREMENTS
   • Arbitrary environment support
   • Easy adaptation (copy/paste)
   • Easy customization
   • No special education or experience requirements
   • Docker support
   • DB migrations apply and rollback
   • Support of private registry and complex networks
   

   View Slide

10. FABRICIO
    10
    https://github.com/renskiy/fabricio
    DOCKER DEPLOY AUTOMATION TOOL
    

    View Slide

11. 11
    FABRICIO DEPLOY CONFIG EXAMPLE
    # fabfile.py

    from fabricio import docker, tasks

    

    

    class NginxContainer(docker.Container):

    

    image = docker.Image('nginx:1.9')

    

    ports = '80:80'

    

    nginx = tasks.DockerTasks(

    container=NginxContainer(name='web'),

    hosts=['user@example.com'],

    )

    

    View Slide

12. 12
    FABRICIO DEPLOY PROCESS
    $ fab --list
    Available commands:
    nginx backup -> pull -> migrate -> update
    nginx.deploy backup -> pull -> migrate -> update
    nginx.pull pull Docker image from registry
    nginx.rollback rollback Docker container to previous version
    nginx.update start new Docker container if necessary
    $ fab nginx
    [user@example.com] Executing task ‘pull’
    [user@example.com] run: docker pull nginx:stable
    [user@example.com] out: 1.9: Pulling from library/nginx
    ...
    $ docker ps
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
    ec44b023adf0 nginx:1.9 "nginx -g" 2 minutes ago Up 2 minutes 0.0.0.0:80->80/tcp web
    

    View Slide

13. 13
    FABRICIO FEATURES
    • Build Docker images
    • Create containers from images with provided tags
    • Unlimited infrastructures
    • Parallel execution (Fabric feature)
    • Rollback containers to previous version
    • Work with public and private Docker registries
    • DB migrations and rollback, backup and restore
    

    View Slide

14. 14
    FABRICIO COMMAND PARAMS
    # show detailed information about command
    $ fab --display nginx
    Displaying detailed information for task 'nginx':
    backup -> pull -> migrate -> update
    Arguments: self, tag=None, force=False, migrate=True, backup=False
    # deploy container using image with provided tag
    $ fab nginx:1.10
    # force new container
    $ fab nginx:force=yes
    # combo!
    $ fab nginx:1.10,force=yes
    # rollback container to previous version
    $ fab nginx.rollback
    

    View Slide

15. 15
    FABRICIO DOCKER CONTAINER DEFINITION
    class Container(object):
    

    image = None # type: Image

    cmd = None
    stop_timeout = 10

    user = None

    ports = None

    env = None

    volumes = None

    links = None

    hosts = None

    network = None

    restart_policy = None

    stop_signal = None

    

    def __init__(self, name, **options):

    self.name = name

    ...
    

    View Slide

16. 16
    FABRICIO AND PRIVATE DOCKER REGISTRY
    # start local Docker registry
    $ docker run -d -p 5000:5000 --name registry -v /data/registry:/var/lib/registry registry:2
    nginx = tasks.PullDockerTasks(

    container=NginxContainer('web'),

    hosts=['user@example.com'],

    )
    $ fab --list
    Available commands:
    nginx prepare -> push -> backup -> pull -> migrate -> update
    nginx.deploy prepare -> push -> backup -> pull -> migrate -> update
    nginx.prepare prepare Docker image
    nginx.pull pull Docker image from registry
    nginx.push push Docker image to registry
    nginx.rollback rollback Docker container to previous version
    nginx.update start new Docker container if necessary
    

    View Slide

17. 17
    BUILD DOCKER IMAGES WITH FABRICIO
    class MyContainer(docker.Container):

    

    image = docker.Image('my_image')

    

    app = tasks.BuildDockerTasks(

    container=MyContainer('my_service'),

    hosts=['user@example.com'],

    build_path='.',

    )
    $ fab --list
    Available commands:
    app prepare -> push -> backup -> pull -> migrate -> update
    app.deploy prepare -> push -> backup -> pull -> migrate -> update
    app.prepare prepare Docker image
    app.pull pull Docker image from registry
    app.push push Docker image to registry
    app.rollback rollback Docker container to previous version
    app.update start new Docker container if necessary
    

    View Slide

18. 18
    FABRICIO ROLES AND INFRASTRUCTURES
    from fabric import api as fab
    @tasks.infrastructure

    def production():

    fab.env.update(roledefs={'front': ['user@example.com']})
    nginx = tasks.DockerTasks(

    container=NginxContainer(name='web'),

    roles=['front'],

    )
    $ fab --list
    Available commands:
    production select production infrastructure to run task(s) on
    production.confirm automatically confirm production infrastructure selection
    ...
    $ fab production nginx
    Are you sure you want to select production infrastructure to run task(s) on? [y/N]
    

    View Slide

19. 19
    FABRICIO: DEPLOYING DJANGO PROJECTS
    from fabric import api as fab

    from fabricio import docker, tasks

    from fabricio.apps.python.django import DjangoContainer

    

    class BaseDjangoContainer(DjangoContainer):

    image = docker.Image('my_django')

    

    @property

    def env(self):

    return 'DJANGO_SETTINGS_MODULE=settings.{}'.format(

    fab.env.infrastructure,

    )

    

    django = tasks.BuildDockerTasks(

    container=BaseDjangoContainer('api'),

    hosts=['user@example.com'],

    migrate_commands=True,

    )
    

    View Slide

20. 20
    FABRICIO: DEPLOYING DJANGO PROJECTS
    $ fab --list
    Available commands:
    django prepare -> push -> backup -> pull -> migrate -> update
    django.deploy prepare -> push -> backup -> pull -> migrate -> update
    django.migrate apply migrations
    django.migrate_back remove previously applied migrations if any
    django.prepare prepare Docker image
    django.pull pull Docker image from registry
    django.push push Docker image to registry
    django.rollback rollback Docker container to previous version
    django.update start new Docker container if necessary
    

    View Slide

21. 21
    FABRICIO: DATA BACKUP AND RESTORE
    from fabricio.apps.db.postgres import PostgresqlBackupMixin

    

    class BackupDjangoContainer(BaseDjangoContainer,
    PostgresqlBackupMixin):

    volumes = '/data/backup/postgres:/backup'

    db_backup_dir = '/backup'

    

    django = tasks.BuildDockerTasks(

    container=BackupDjangoContainer('api'),

    hosts=['user@example.com'],

    backup_commands=True,
    )
    $ fab --list
    Available commands:
    django.backup backup data
    django.restore restore data
    ...
    

    View Slide

22. 22
    FABRICIO REQUIREMENTS AND INSTALL
    • Python 2.6 or 2.7
    • Docker CLI (Linux/Mac/Windows)
    • Docker 1.9 or greater recommended (remote side)
    # virtualenv install
    $ pip install --upgrade fabricio
    # macOS system-wide install
    $ sudo pip install --upgrade fabricio six==1.4.1
    

    View Slide

23. 23
    FABRICIO ROADMAP
    •Master-Slave configurations for PostgreSQL
    •Docker Swarm support
    •docker-py integration
    •MySQL?
    •Non-Django frameworks?
    

    View Slide

24. View Slide

25. 25
    USEFUL LINKS & QUESTIONS
    • Fabricio: https://github.com/renskiy/fabricio
    • Author of Fabricio: https://www.facebook.com/rinat.khabibiev
    • Хабра-блог Redmadrobot: https://habrahabr.ru/company/redmadrobot
    • Redmadrobot on Facebook: https://www.facebook.com/redmadrobot
    • The 12-factor App (SaaS dev patterns): https://12factor.net
    • Docker image with cron: https://hub.docker.com/r/renskiy/cron
    Questions
    

    View Slide