Multi-Stage Provisioning with Ansible

devops days Berlin 2015

http://www.devopsdays.org/events/2015-berlin/proposals/Multi_Stage_Provisioning_With_Ansible/

How to configure your server landscape is always difficult. You have servers of different kinds, like web servers and database servers on the one hand and you have different stages, like "development" and "production" on the other. Puppet, Chef and Ansible simplify such configuration, but you really need a clear model of what you want to achieve and how to map that to the capabilities of your toolbox. Ansible is easy to learn and your playbooks can become powerful very fast. But it is not so obvious how to model multiple "stages" with Ansible.

In this talk a very quick overview of Ansible is given and different solutions to handle multi-stage provisioning are explained and contrasted with the mechanisms of Puppet. It is not a comparison, in the sense: what is best -- but simply a comparison of the models and mechanisms of these tools in a certain (important) dimension.

Transcript

1. Multi-Stage Ansible Victor Volle DevOps Days Berlin 2015
2. 2 From 10 servers to 1000 https://www.flickr.com/photos/mariachily/4110399647
3. configuration management 3 “configuration” proc web ram: 1 GB ram: 2 GB db: h2@localhost
4. configuration management 4 “configuration” mango01:3000
 
 ram=2GB mango01:4000
 ram=2GB mango03-06
 ram=4GB mango07-10 ram=4GB dattel01
 ram=2GB db=h2@127.0… dattel02
 ram=4GB db=h2@127.0… dattel02, dattel03 ram=8GB db=pg@10.1… dattel07, dattel08 ram=8GB db=pg@10.1… Dev Test PreProd Prod web proc
5. Dr. Victor Volle IT-Something “Senior IT Consultant” 2015- codecentric “Architect” 2008- Senacor Technologies AG “Chief Architect JEE” 2004- ING DiBa Developer, Architect, Head of … 1996- develop group, Erlangen Intro 5 Who me?
6. Ansible:
 Intro 6
7. Configuration Management Tools 7 Ansible: Core Concepts 1. Inventory 2. Facts & Variables 3. Playbooks - which hosts? - “groups” of hosts? db:
 ram: 2GB "ansible_distribution_version":
 "14.04" - “facts”: the current state of my hosts - “variables”: properties of hosts (future state, config files, …) - hosts: a00 tasks: - apt: name=nginx - service: ... - what should be “done” on my hosts (future state)
8. Ansible 8 Inventory # Inventory [web]
 mango[01-10] [proc]
 dattel[01-08] group hosts
9. Ansible 9 Playbooks # jetty.yml --- - name: Install Jetty hosts: web tasks: - include: java8-include.yml - name: download jetty get_url: url={{jetty_download_url}} dest=/tmp/{{jetty_file}} - name: extract jetty archive unarchive: src="/tmp/{{ jetty_file }}" dest={{ jetty_dir }} - name: install init script template: >
 src=templates/jetty.j2
 dest=/etc/init.d/jetty mode=0744 - name: enable service service: name=jetty enabled=yes A “script” that describes the target state of the machine for which “hosts” (from the inventory) shall the Playbook be applied the list of target states (tasks/ changes) each task has a name (documentation) what to do
 (target state)
10. Ansible 10 Playbooks ❯ ansible-playbook init00.yml PLAY [Initialise “Ansible host” for workshop] ***************************** GATHERING FACTS *********************************************************** ok: [a00] TASK: [user vivo] ********************************************************* changed: [a00] TASK: [install python-pip and python-dev] ********************************* changed: [a00] => (item=python-dev,python-pip) TASK: [install Ansible] *************************************************** changed: [a00] PLAY RECAP **************************************************************** a00 : ok=4 changed=3 unreachable=0 failed=0 each task (should have) has a name A playbook starts with gathering facts no problems and nothing has changed when executing Did anything change? (yes)
12. Configuration for Environments 12
13. configuration management 13 “configuration” mango01
 
 ram=2GB mango02
 ram=2GB mango03-06
 ram=4GB mango07-10 ram=4GB dattel01
 ram=4GB db=h2@127.0… dattel02
 ram=4GB db=h2@127.0… dattel02, dattel03 ram=8GB db=pg@10.1… dattel07, dattel08 ram=8GB db=pg@10.1… Dev Test PreProd Prod web proc
14. Ansible 14 multi-stage: possible solutions 1. group variables 2. multiple inventories 3. exploiting variable precedence 4. creating your own vars plugin
15. Ansible 15 group variables (inventory) # Inventory [web]
 mango[01-10] [proc]
 dattel[01-08] [dev]
 mango01
 dattel01 ... [prod]
 mango[07-10]
 dattel[07-08] ... [web:vars]
 ram=2G [proc:vars]
 ram=2G [dev:vars]
 ram=2G group
 variables it is not possible to set different values for “ram” for web and proc we need a “namespace”
16. dev prod procram ?G ?G 4G 2G Ansible 16 group variables (inventory) # Inventory [web]
 mango[01-10] [proc]
 dattel[01-08] [dev]
 mango01
 dattel01 ... [prod]
 mango[07-10]
 dattel[07-08] ... [web:vars]
 webram=2G [proc:vars]
 procram=2G [dev:vars]
 webram=2G
 procram=4G ... [prod:vars]
 webram=4G
 procram=8G groups are alphabetically sorted "proc"
 < "prod"
17. group variables Ansible 17 Default Workspace structure workspace/ ├── files/ ├── group_vars/ ├── host_vars/ ├── roles/ ├── templates/ ├── ansible.cfg ├── ansible_inventory ├── java.yml … groups are still
 sorted alphabetically
18. Ansible 18 Variables »Avoid defining the variable “x” in 47 places and then ask the question “which x gets used”. Why? Because that’s not Ansible’s Zen philosophy of doing things.
 
 There is only one Empire State Building. One Mona Lisa, etc. Figure out where to define a variable, and don’t make it complicated.
19. Ansible 19 multi-stage: possible solutions 1. group variables 2. multiple inventories 3. exploiting variable precedence 4. creating your own vars plugin
20. Ansible 20 inventories per environment # dev_inventory [web]
 mango01 [proc]
 dattel01 [dev:children]
 web
 proc [dev:vars]
 webram=2G
 procram=4G # prod_inventory [web]
 mango02 [proc]
 dattel02 [prod:children]
 web
 proc [prod:vars]
 webram=4G
 procram=8G … - “good enough” in most cases! Use it as long as possible - (you should place the variables in the group_vars folder though) - you cannot invoke a playbook on all hosts — which you probably do not want anyway - there is some duplication in the inventories, but not too much - get’s very unwieldy when you add another dimension like ‘location’ see e.g. : http://rosstuck.com/multistage-environments-with-ansible/
21. Ansible 21 multi-stage: possible solutions 1. group variables 2. multiple inventories 3. exploiting variable precedence 4. creating your own vars plugin
22. Ansible 22 Variable Precedence ❯ ansible-playbook -e “ram=16G" ... # Inventory mango01 ansible_ssh_host=10.0.1.16 # Inventory ... mango10 ram=8G [web:vars]
 ram=4G connection variables extra vars role defaults facts inventory variables most everything else most everything else high low
23. configuration management 23 Variable Precedence connection variables extra vars role defaults facts inventory variables most everything else most everything else most everything else include_vars: role vars vars_files: vars: ./host_vars/* ./group_vars/* ./group_vars/all Not documented Changed from
 1.8 to 1.9 will be defined for Ansible 2.0
24. Use include_vars task or vars_files (That’s what we are currently doing) Problem: you have to add that to every Playbook Inventory per Environment Problem: inventory variables have a low priority configuration management 24 variable precedence connection variables extra vars role defaults facts inventory variables most everything else most everything else most everything else include_vars: role vars vars_files: vars: ./host_vars/* ./group_vars/* ./group_vars/all role defaults Write your own vars plugin (That’s what I am currently planning)
25. Ansible 25 Variables per stage/envrionment # varprecedence.yml --- - name: Check Var precedence hosts: "{{ lookup('env','STAGE') }}:&web" 
 vars_files: - "{{ lookup('env','STAGE') }}_vars.yml"
 tasks: - include_vars: include_vars{{ lookup('env','STAGE') }}.yml ...
 
 
 
 servers, which are in group “web” and in group “{{STAGE}}” (1) load variables from environment/stage specific file (2) load variables from environment/stage specific file
26. configuration management 26 “exploiting variable precedence” connection variables extra vars role defaults facts inventory variables most everything else most everything else most everything else include_vars: role vars vars_files: vars: ./host_vars/* ./group_vars/* ./group_vars/all role defaults 3 2 1
27. Ansible 27 multi-stage: possible solutions 1. group variables 2. multiple inventories 3. exploiting variable precedence 4. creating your own vars plugin
28. :hierarchy: - "locations/%{::location}" - "envs/%{::stage}" - "roles/%{::role}" - common Puppet 28 Puppet Hiera: a simple Hierarchical Database a puppet “role” is roughly equivalent to an Ansible “group”
29. Puppet 29 Puppet Hiera: a simple Hierarchical Database :hierarchy: - "locations/%{::location}" - "envs/%{::stage}" - "roles/%{::role}" - common "## locations/ $ "## ams.yaml $ "## asia.yaml $ "## berlin.yaml $ "## praha.yaml $ %## timb.yaml "## envs/ $ "## dev.yaml $ "## preprod.yaml $ "## prod.yaml $ %## test.yaml %## roles/ "## proc.yaml %## web.yaml
30. Ansible: vars_plugin 30 https://github.com/kontrafiktion/ansible-plugins nothing to see there, yet