Scaling with Ansible

A quick introduction to Ansible followed up with presentation of the most interesting bits and how we use those. Presentation from Europython 2014 (Berlin)

1. Scaling with Ansible Federico Marani - Europython 2014
2. Me! Coder, open-source enthusiast, command- line addict. Head of Engineering @TrialReach
4. Devops - infrastructure as code - people leave but knowledge stays - automate everything!
5. Devops should not require programming experience
6. Ansible - Quick to get started - Builds on Python, SSH and YAML… - Pushes updates (instead of pull) - Playbooks, tasks and inventories
7. Configuration management
8. Playbook
9. task order is important think “imperative” programming
10. tasks are idempotent* *can be applied multiple times without changing the result beyond the initial application
11. handlers commands flagged for later execution
12. Inventory
13. Use host groups, roles/includes Use the action with least side effects (e.g. prefer “copy” to “template”)
14. - include: install_web_packages.yml - include: config_nginx.yml - include: config_supervisor.yml - include: database_daily_restore.yml when: BUILD_ENVIRONMENT == "staging" - include: app_deploy.yml tags=code Includes
15. Code deployment
16. Deployments are really custom --- # ... - name: create virtualenv for Django web application shell: virtualenv --python=/usr/bin/python3.4 {{virtualenv_dir}} when: venv_dir.stat.isdir is not defined - name: install web application dependencies listed in requirements.txt pip: requirements={{app_code_dir}}-{{build_date.stdout}} /requirements.txt virtualenv={{virtualenv_dir}} - name: Install Bower packages shell: bower install chdir={{app_code_dir}}-{{build_date.stdout}} /assets/js/vendor/ - name: Install NPM packages npm: path={{app_code_dir}}-{{build_date.stdout}}/assets - name: Run grunt make (sass/requirejs/uglify/etc) shell: grunt make chdir={{app_code_dir}}-{{build_date.stdout}} /assets/ - name: Django collectstatic django_manage: command=collectstatic app_path={{app_code_dir}}-{{build_date.stdout}} virtualenv={{virtualenv_dir}} settings={{django_settings}} environment: django_env_vars - name: Testing if there are new migrations django_manage: command="migrate --list" app_path={{app_code_dir}}-{{build_date.stdout}} virtualenv={{virtualenv_dir}} settings={{django_settings}} environment: django_env_vars register: migration_test - name: Stopping Celery before applying migrations supervisorctl: name=trialreach_celery state=stopped when: "'( )' in migration_test.out" - name: Django migrate django_manage: command=migrate app_path={{app_code_dir}}-{{build_date.stdout}} virtualenv={{virtualenv_dir}} settings={{django_settings}} environment: django_env_vars when: "'( )' in migration_test.out" - name: Symlink release directory to new one file: src={{app_code_dir}}-{{build_date.stdout}}/ dest= {{app_code_dir}} state=link - name: Restart uwsgi supervisorctl: name="uwsgi" state=restarted - name: Send deployment notifications to Hipchat hipchat: token=X room="Notifications" msg_from="Web Warden" msg="Code deployed to {{ ansible_hostname }}"
17. - fail: msg="Please define environment to deploy" when: app_environment is not defined Conditionals* * uses Jinja2 expressions
18. - shell: git tag chdir="{{app_code_dir}}" register: git_tags - fail: msg="Please define a tag when deploying to production" when: app_environment == "production" and tag not in git_tags. stdout_lines register
19. - name: ensure pip packages are installed pip: name={{ item }} state=present with_items: - virtualenv==1.11.4 - supervisor with_items
20. - name: send notifications to Hipchat hipchat: room="Eng" msg="Deployed to {{ansible_hostname}}" changed_when: False facts
21. Frequently used actions apt: install packages service: interface to init scripts pip: install pip packages git: checkout repositories file: check file state copy/template: copy files to server django_manage: run management commands supervisorctl: run supervisor commands
22. Current state - 1000+ lines of playbooks - 4 environments - 10 production-type servers with Postgresql, Neo4J, Nginx, Solr. - Dev machines, branch machines. - AWS, DigitalOcean, Vagrant
23. - Keep server stateless - Do things in the right place, either code-level or infrastructure-level. ...and remember
25. Thanks! Federico Marani @flagZ pics from: - http://xkcd.com/838/ - https://flic.kr/p/7AGXT6 - Matrix