Scaling with Ansible

Scaling with Ansible

A quick introduction to Ansible followed up with presentation of the most interesting bits and how we use those. Presentation from Europython 2014 (Berlin)


Federico Marani

July 23, 2014


  1. Scaling with Ansible Federico Marani - Europython 2014

  2. Me! Coder, open-source enthusiast, command- line addict. Head of Engineering

  3. None
  4. Devops - infrastructure as code - people leave but knowledge

    stays - automate everything!
  5. Devops should not require programming experience

  6. Ansible - Quick to get started - Builds on Python,

    SSH and YAML… - Pushes updates (instead of pull) - Playbooks, tasks and inventories
  7. Configuration management

  8. Playbook

  9. task order is important think “imperative” programming

  10. tasks are idempotent* *can be applied multiple times without changing

    the result beyond the initial application
  11. handlers commands flagged for later execution

  12. Inventory

  13. Use host groups, roles/includes Use the action with least side

    effects (e.g. prefer “copy” to “template”)
  14. - include: install_web_packages.yml - include: config_nginx.yml - include: config_supervisor.yml -

    include: database_daily_restore.yml when: BUILD_ENVIRONMENT == "staging" - include: app_deploy.yml tags=code Includes
  15. Code deployment

  16. Deployments are really custom --- # ... - name: create

    virtualenv for Django web application shell: virtualenv --python=/usr/bin/python3.4 {{virtualenv_dir}} when: venv_dir.stat.isdir is not defined - name: install web application dependencies listed in requirements.txt pip: requirements={{app_code_dir}}-{{build_date.stdout}} /requirements.txt virtualenv={{virtualenv_dir}} - name: Install Bower packages shell: bower install chdir={{app_code_dir}}-{{build_date.stdout}} /assets/js/vendor/ - name: Install NPM packages npm: path={{app_code_dir}}-{{build_date.stdout}}/assets - name: Run grunt make (sass/requirejs/uglify/etc) shell: grunt make chdir={{app_code_dir}}-{{build_date.stdout}} /assets/ - name: Django collectstatic django_manage: command=collectstatic app_path={{app_code_dir}}-{{build_date.stdout}} virtualenv={{virtualenv_dir}} settings={{django_settings}} environment: django_env_vars - name: Testing if there are new migrations django_manage: command="migrate --list" app_path={{app_code_dir}}-{{build_date.stdout}} virtualenv={{virtualenv_dir}} settings={{django_settings}} environment: django_env_vars register: migration_test - name: Stopping Celery before applying migrations supervisorctl: name=trialreach_celery state=stopped when: "'( )' in migration_test.out" - name: Django migrate django_manage: command=migrate app_path={{app_code_dir}}-{{build_date.stdout}} virtualenv={{virtualenv_dir}} settings={{django_settings}} environment: django_env_vars when: "'( )' in migration_test.out" - name: Symlink release directory to new one file: src={{app_code_dir}}-{{build_date.stdout}}/ dest= {{app_code_dir}} state=link - name: Restart uwsgi supervisorctl: name="uwsgi" state=restarted - name: Send deployment notifications to Hipchat hipchat: token=X room="Notifications" msg_from="Web Warden" msg="Code deployed to {{ ansible_hostname }}"
  17. - fail: msg="Please define environment to deploy" when: app_environment is

    not defined Conditionals* * uses Jinja2 expressions
  18. - shell: git tag chdir="{{app_code_dir}}" register: git_tags - fail: msg="Please

    define a tag when deploying to production" when: app_environment == "production" and tag not in git_tags. stdout_lines register
  19. - name: ensure pip packages are installed pip: name={{ item

    }} state=present with_items: - virtualenv==1.11.4 - supervisor with_items
  20. - name: send notifications to Hipchat hipchat: room="Eng" msg="Deployed to

    {{ansible_hostname}}" changed_when: False facts
  21. Frequently used actions apt: install packages service: interface to init

    scripts pip: install pip packages git: checkout repositories file: check file state copy/template: copy files to server django_manage: run management commands supervisorctl: run supervisor commands
  22. Current state - 1000+ lines of playbooks - 4 environments

    - 10 production-type servers with Postgresql, Neo4J, Nginx, Solr. - Dev machines, branch machines. - AWS, DigitalOcean, Vagrant
  23. - Keep server stateless - Do things in the right

    place, either code-level or infrastructure-level. ...and remember
  24. None
  25. Thanks! Federico Marani @flagZ pics from: - -

    - Matrix