Introduction to Ansible: From Playbook to Production

An introduction to Ansible along with a step-by-step guide for creating a testing process with Serverspec.

Transcript

1. Introduction to Ansible: From Playbook to Production Kevin Day DevOps CLE April 9th, 2015
2. Target Audience Developers & sys admins that develop or maintain a web service
3. Objective Teach you how you can use Ansible for the entire lifecycle of server development and deployment: • Flawless production deployments • Repeatable environment builds • Infrastructure testing & validation • Seamless flow between developers and ops
4. What You’ll Learn Part 1: Ansible Basics • What is Ansible? • How to write an Ansible playbook Part 2: Ansible in the DevOps Lifecycle • Local Development w/ Vagrant & Serverspec • Continuous Integration w/ Circle CI & AWS • Environment build-out & testing w/ AWS AMI & ASG
5. About Me • Kevin Day @kday • Lead Developer at SparkBase • Fantasy sports websites ◦ Manage 25 servers ◦ 100,000 uniques per day ◦ 700 requests per second • Been using Ansible for last 8 months • Developing a fully tested DevOps process • Building production servers this month with this approach
6. Why This Talk? • Ansible is awesome • Everyone does DevOps differently ◦ Ansible doesn’t have good built-in testing1 ◦ CI is largely ignored ◦ Differences in building vs. provisioning servers • Present and receive feedback on my approach • Curious how others are using Ansible 1 http://docs.ansible.com/test_strategies.html “Ansible resources are models of desired-state. As such, it should not be necessary to test that services are running, packages are installed, or other such things.”
7. Code Online Example repo: https://github.com/kday/ansible-from-playbook-to-production
8. Ansible Basics
9. What is Ansible? • Configuration management and orchestration tool • Uses YAML syntax to define the state a server should have • Converts YAML statements into bash commands that are run on remote server via SSH • Adhoc and playbooks
10. How does it compare to other tools? • Competitor to Puppet, Chef, and Salt • Different from Puppet and Chef because it: ◦ is agentless ◦ push instead of pull ◦ can also provision servers
11. Ansible Playbook • Basis for a simple configuration management and multi-machine deployment system • Contains one or more plays • Requires a host to be specified --- - hosts: webservers vars: http_port: 80 max_clients: 200 remote_user: root tasks: - name: ensure apache is at the latest version yum: pkg=httpd state=latest - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache - name: ensure apache is running (and enable it at boot) service: name=httpd state=started enabled=yes handlers: - name: restart apache service: name=httpd state=restarted webservers.yml
12. Roles • Core method to reuse configuration • One playbook can execute multiple roles against a server • Ansible Galaxy is the public repository of shared roles --- - hosts: webservers roles: - common - webservers webservers.yml
13. Directory Convention site.yml webservers.yml fooservers.yml roles/ common/ files/ templates/ tasks/ handlers/ vars/ defaults/ meta/ webservers/ files/ templates/ tasks/ handlers/ vars/ defaults/ meta/
14. Variables • Command Line Params • Role Params • Defaults • Host Vars • Group Vars Vault: • Password-protect secrets while keeping them in source control ansible-playbook test.yml --extra-vars=”foo=bar” - hosts: webservers vars: http_port: 80 group_vars/webservers/main.yml --- - http_port: 80
15. Inventory Static Inventory • Create a list of host names and their addresses in a file • Must be manually updated Dynamic Inventory • Uses APIs from hosting provider (AWS) to generate a list of servers • Add meta data to each server, such as tags, that can be referenced in playbooks • Highly recommended when using a cloud provider
16. Ansible in the DevOps Lifecycle
17. Tech Stack • Ansible • Amazon AWS • Amazon AMI-based deployments • Vagrant & Virtualbox • Packer (for only building one base image) • Serverspec
18. Goals 1. Use TDD for local infrastructure development 2. Continuous Integration for testing Ansible roles 3. Local & production environments as similar as possible 4. Build any new environment quickly
19. Local Development 1. Write failing test 2. Update playbook 3. Run playbook against Vagrant 4. Re-run tests 5. Commit & push spec/flask_web/flask_web_spec.rb
20. Local Development 1. Write failing test 2. Update playbook 3. Run playbook against Vagrant 4. Re-run tests 5. Commit & push roles/machines/flask-web/tasks/main.yml
21. Local Development 1. Write failing test 2. Update playbook 3. Run playbook against Vagrant 4. Re-run tests 5. Commit & push
22. Local Development 1. Write failing test 2. Update playbook 3. Run playbook against Vagrant 4. Re-run tests 5. Commit & push
23. Local Development 1. Write failing test 2. Update playbook 3. Run playbook against Vagrant 4. Re-run tests 5. Commit & push
24. Continuous Integration
25. Continuous Integration On every git push... For each server type: 1. Spin up EC2 instance 2. Execute playbook 3. Run server spec tests 4. Pass/fail the build
26. Continuous Integration On every git push... For each server type: 1. Spin up EC2 instance 2. Execute playbook 3. Run server spec tests 4. Pass/fail the build
27. Continuous Integration On every git push... For each server type: 1. Spin up EC2 instance 2. Execute playbook 3. Run server spec tests 4. Pass/fail the build
28. Continuous Integration On every git push... For each server type: 1. Spin up EC2 instance 2. Execute playbook 3. Run server spec tests 4. Pass/fail the build
29. Building Environments DRY: What to reuse? • Reuse roles, not playbooks • Tailor each playbook to the environment • Specs for shared machine role • TODO: Specs for environment-specific differences
30. Unit of Deployment = ASG AWS Auto Scaling Group • Defined by an AMI • Rules for scaling up and down (optional) • Allows for multi-AZ redundancy • Netflix model • Useful even for a single server ◦ max instances = 1 ◦ min instances = 1
31. ASG Playbook 1. Launch temp instance 2. Run Ansible roles against temp instance 3. Register AMI 4. Terminate temp instance 5. Create AWS Launch Configuration 6. Create AWS Auto Scaling Group environments/prod/flask-web-asg.yml
32. ASG Playbook 1. Launch temp instance 2. Run Ansible roles against temp instance 3. Register AMI 4. Terminate temp instance 5. Create AWS Launch Configuration 6. Create AWS Auto Scaling Group environments/prod/flask-web-asg.yml
33. ASG Playbook 1. Launch temp instance 2. Run Ansible roles against temp instance 3. Register AMI 4. Terminate temp instance 5. Create AWS Launch Configuration 6. Create AWS Auto Scaling Group environments/prod/flask-web-asg.yml
34. Playbooks For Each Environment environments/prod/flask-web-asg.yml environments/staging/flask-web-asg.yml
35. TODO: Rolling Deployments Ansible playbook for each type of deployment deployments/prod/db-migrations-and-web.yml Hard (impossible?) to write good specs for this though.
36. Resources Ansible Testing https://servercheck.in/blog/testing-ansible-roles-travis-ci-github http://www.nickhammond.com/testing-ansible-playbook-semaphore/ Ansible Roles https://github.com/theonion/ansible-roles https://github.com/ansible/ansible-examples
37. Conclusion • Ansible basics • Ansible in the DevOps lifecycle ◦ Vagrant ◦ Serverspec ◦ Circle CI ◦ AWS • Do as much in Ansible as possible https://github.com/kday/ansible-from-playbook-to-production kday33@gmail.com @kday