Ansible: Your first step into server provisioning

An introduction to the problems that server provisioning tools addresses, and how to implement this using the Ansible IT automation tool.

Transcript

1. Your first step into server provisioning
2. Ben Turner Ben Turner @phantomwhale @phantomwhale
4. The Story So Far The Story So Far
6. Fred
7. Documentation is a lie waiting to happen !!!
8. Snowflake Server
9. Provisioning tools Provisioning tools
10. Provisioning tools
11. Provisioning tools - Elements Contain similar elements: – Directive – Directive Script – Master Node – Slave Nodes
12. Ansible elements The specfic elements in Ansible are known as... Directive => Task Directive Script => Playbook Master Node => Your own machine Client Node => Any remote server
13. Ansible Ansible
14. Master node installation Pip - Yum - Apt - $ sudo pip install ansible $ sudo yum install ansible $ sudo apt-add-repository \ ppa:rquillo/ansible $ sudo apt-get update $ sudo apt-get install ansible
15. Client node requirements SSH (that's it !) (OK, and a bit of Python)
16. Inventory File Tells the master node about the child nodes Contains list of all servers, and defines groups of servers Defaults to /etc/ansible/hosts but you can use your own
17. Example Inventory File [webservers] my.server.com another.server.com [databases] one.example.com Two.example.com [primary] my.server.com one.example.com Groups Servers
18. Command Line Command Line
19. Ping the machines Check basic connectivity and everything is functioning $ ansible all -m ping my.server.com | success >> { "changed": false, "ping": "pong" }
20. Execute commands Lets you run arbitrary command line statements $ ansible all -m command -a “/sbin/reboot” $ ansible all -a “/sbin/reboot” Module name Arguments
21. Connectivitiy options What if you don't want to SSH in as your current user ? --user use different username for SSH --ask-pass ask for SSH password --sudo run commands through sudo --ask-sudo-pass ask for sudo password --sudo-user sudo to different non-root user ansible -u bwayne --sudo --sudo-user batman ...
22. Modules Modules
23. Create user $ ansible all -a 'useradd -p cryptpwd adminuser' $ ansible all -a 'groupadd admins' $ ansible all -a 'usermod -aG admins adminuser' Using the basic 'Command' module my.server.com | success | rc=0 >> my.server.com | success | rc=0 >> my.server.com | success | rc=0 >> my.server.com | FAILED | rc=9 >> useradd: user 'adminuser' already exists my.server.com | FAILED | rc=9 >> groupadd: group 'admins' already exists my.server.com | success | rc=0 >>
24. Create user with modules $ ansible all -m group -a 'name=admins' $ ansible all -m user \ -a 'name=adminuser groups=admins' Using the 'User' and 'Group' modules my.server.com | success >> { "changed": true, "gid": 1001, "name": "admins", "state": "present", "system": false } my.server.com | success >> { "changed": false, "gid": 1001, "name": "admins", "state": "present", "system": false }
25. Core modules ✔ Cloud ✔ Database ✔ Files ✔ Messaging ✔ Monitoring ✔ Network ✔ And more... ✔ Notifitcation ✔ Packaging ✔ Source Control ✔ System ✔ Utilities ✔ Web Infrastructure
26. Modules – DIY idempotency What if there is no module for what you need? $ ansible all -a 'dpkg -i elasticsearch.deb \ creates=/usr/share/elasticsearch' my.server.com | success | rc=0 >> Selecting previously unselected package elasticsearch (Reading database ... 191433 files currently installed) Unpacking elasticsearch (from elasticsearch.deb) Setting up elasticsearch (0.90.10) ... * Starting ElasticSearch Server ...done. my.server.com | skipped
27. Modules – roll your own Or roll your own ! • Can be written in any language, just needs to – accept File IO – output to Stdout • Help is out there for Rubyists: https://github.com/ansible/ansible-for-rubyists
28. Playbooks Playbooks
29. Playbooks If modules are the tools, then playbooks are the design plans • Written in YAML • Intended to be configuration, not a progamming script • Consists of one or more 'plays' $ ansible-playbook playbook.yml
30. Example playbook --- - hosts: webservers remote_user: root vars: http_port: 80 max_clients: 200 tasks: - name: ensure apache is at the latest version apt: pkg=apache2 state=latest - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache handlers: - name: restart apache service: name=httpd state=restarted
31. Playbooks - tasks Tasks are a call to an ansible module tasks: - name: ensure apache is at the latest version apt: pkg=apache2 state=latest Description Module $ ansible -m apt -a 'pkg=apache2 state=latest' Arguments
32. Playbooks - variables Variables define system specific settings / behaviour Substitute into tasks with Jinja2 templating language --- - hosts: searchservers vars: es_heap_size: 4g tasks: - name: update elasticsearch heap size lineinfile: dest=/etc/init.d/elasticsearch regexp=/ES_HEAP_SIZE/ line=ES_HEAP_SIZE={{ es_heap_size }}
33. Playbooks - inventory variables Can also define per host or group in the inventory file [small] devhost1 devhost2 [small:vars] es_heap_size: 1g [small] devhost1 es_heap_size: 1g devhost2 es_heap_size: 1g
34. Playbooks – Template files Variables can be substituted into template files as well ./templates/elasticsearch.j2 # Elasticsearch configuration # (abridged) ES_USER=elasticsearch ES_GROUP=elasticsearch ES_HEAP_SIZE={{ es_heap_size }} MAX_OPEN_FILES={{ max_open_files }} ...
35. Playbooks - Handlers Some tasks require a change to a service to happen at some point, not neccasarily immediatley. To do this they notify a handler, which acts after the play --- - hosts: webservers tasks: - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache # … other things that modify apache setup … handlers: - name: restart apache service: name=httpd state=restarted
36. Putting them all together --- - hosts: webservers remote_user: root vars: http_port: 80 max_clients: 200 tasks: - name: ensure apache is at the latest version apt: pkg=httpd state=latest - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache handlers: - name: restart apache service: name=httpd state=restarted
37. Running the playbook PLAY [webservers] ********************************** GATHERING FACTS ************************************ ok: [my.server.com] TASK: [ensure apache is at the latest version] ***** changed: [my.server.com] TASK: [write the apache config file] *************** changed: [my.server.com] NOTIFIED: [restart apache] ************************* changed: [my.server.com] PLAY RECAP ***************************************** my.server.com: ok=4 changed=3 unreachable=0 failed=0
38. Running with cows ___________________ < PLAY [webservers] > ------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || _________________ < GATHERING FACTS > ----------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || Got cowsay ?
39. Advanced Plays Advanced Plays
40. Playbooks - Loops - name: required for compiling Ruby from source apt: pkg=build-essential state=present - name: oh we also need git-core as well apt: pkg=git-core state=present - name: must not forget curl too apt: pkg=curl state=present - name: How many more things are there ?!? ... Sometimes, loops can really help - name: required for compiling Ruby from source apt: pkg={{ item }} state=present with_items: - build-essential - git-core - curl - ...
41. Playbooks - Conditionals Can use conditionals to adjust what gets executed against the server based on current variables vars: awesome: true tasks: - shell: echo “This machine is awesome !” when: awesome tasks: - name: "shutdown Debian flavored systems" command: /sbin/shutdown -t now when: ansible_os_family == "Debian"
42. Playbooks – Machine Facts ... "ansible_architecture": "x86_64", "ansible_bios_date": "09/20/2012", "ansible_bios_version": "6.00", "ansible_os_family": "Debian", "ansible_pkg_mgr": "apt", "ansible_processor": [ "Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz" ], "ansible_processor_cores": 1, "ansible_swapfree_mb": 665, "ansible_swaptotal_mb": 1021, ... Ansible provides a ginormous amount of machine facts as variables during the play
43. Structure Structure
44. Playbooks - structure As playbooks grow, it becomes helpful to package up the various components • Tasks • Variables • Inventory variables • Files / Templates • Handlers playbook.yml files/ license.txt templates/ httpd.j2
45. Moving tasks Can move tasks into their own file... ./tasks/apache.yml tasks: - name: ensure apache is at the latest version apt: pkg=httpd state=latest - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache …
46. Including tasks ...and then include them into our Playbook --- - hosts: webservers remote_user: root vars: http_port: 80 max_clients: 200 tasks: - include: tasks/apache.yml handlers: - name: restart apache service: name=httpd state=restarted
47. Including variables / handlers ...can also do this with variables and handlers --- - hosts: webservers remote_user: root var_files: - vars/apache.yml tasks: - include: tasks/apache.yml handlers: - include: handlers/apache.yml
48. Moving inventory variables Can move inventory variables into host or group specific files --- # file: group_vars/small es_heap_size: 1g [small] devhost1 foo: bar devhost2 [small:vars] es_heap_size: 1g --- # file: host_vars/devhost1 foo: bar
49. Roles Roles
50. Roles “Roles are ways of automatically loading certain variables, tasks, and handlers based on a known file structure.” playbook.yml files/ handlers/ apache.yml tasks/ apache.yml templates/ httpd.j2 vars/ apache.yml playbook.yml roles/ apache/ files/ handlers/ main.yml tasks/ main.yml templates/ httpd.j2 vars/ main.yml
51. How does the playbook look now? With roles, your playbook is now as simple as: --- - hosts: webservers remote_user: root roles: - apache --- - hosts: webservers remote_user: root roles: - common - { name: deb, when: os_family == 'debian' } - apache - java - elasticsearch - ruby - rails - redis
52. Best Practice site.yml webserver.yml database.yml production_inventory staging_inventory group_vars/ group1 group2 host_vars/ host1 host2 roles/ ... Site.yml - referencing all other playbooks Seperate production inventory file Group and host specific variables in files named by group / host Split everything else into roles
53. That was your That was your first first step into Ansible... step into Ansible... ...so what about your ...so what about your next next step? step?
54. “In the category of DevOps orchestration engines, Ansible has nearly universal acclaim within ThoughtWorks projects. It has useful tools and abstractions at a useful level of granularity.” - Thoughtworks Radar Jan '14
55. The Documentation is GREAT
56. Ansible Weekly - devopsu.com Matt Jaynes
59. Ben Turner – about.me/benturner Ben Turner – about.me/benturner @phantomwhale @phantomwhale Ansible – Your first step into server provisioning Ansible – Your first step into server provisioning Any Questions ? Any Questions ?