CloudOpen 2014 - Extending Cloud Automation: When OpenStack Meets Ansible

Slide 1

Slide 1 text

COPYRIGHT © 2012 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 1 Benjamin Zores, Alcatel-Lucent Enterprise CloudOpen 2014 – 14th October 2014 – Dusseldorf, Germany Extending Cloud Automation: When OpenStack Meets Ansible

Slide 2

Slide 2 text

2 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible A Few Things About Me … ALCATEL LUCENT ENTERPRISE TECHNICAL DIRECTOR, PERSONAL CLOUD SOLUTIONS • Cloud Architecture and Application/Infrastructure Design • R&D Development, Automation, Operations … OPEN SOURCE PROJECT FOUNDER, LEADER AND/OR CONTRIBUTOR FOR: • OpenBricks & GeeXboX : Embedded Linux cross-build tool and distribution. • uShare UPnP A/V & DLNA Media Server, FFMpeg, MPlayer … LINUX FOUNDATION CONFERENCES REGULAR LINUX FOUNDATION’S EVENTS SPEAKER • Various talks on: • Linux Embedded Systems at Embedded Linux Conference (Europe) • Android Architecture and Device Porting at Android Builder Summit GNU/LINUX MAGAZINE FRANCE RECURRENT TECHNICAL WRITER • Various publications on: • Android Architecture Internals • Cloud (OpenStack, Ansible …)

Slide 3

Slide 3 text

3 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Self-Promotion Time ! Series of articles published in GNU/Linux Magazine France Android 4: Fondements Internes Benjamin Zores, Ed. Diamond – Sept. 2014

Slide 4

Slide 4 text

Slide 5

Slide 5 text

5 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. A Good Developer Is A Lazy Developer ! Don’t do over and over again things that someone else can do for you. (especially if that someone is a bot) Extending Cloud Automation: When OpenStack Meets Ansible Why we’re here ?

Slide 6

Slide 6 text

6 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible How one feels after deploying its servers through shell scripts … #!/bin/sh echo “net.core.rmem_default=16384" | sudo tee -a /etc/sysctl.conf echo “deb http://nwps.ws/pub/mariadb/repo/5.5/debian wheezy main” | \ sudo tee –a /etc/apt/conf.d/mariadb.conf sudo apt-get -y install mariadb-server […] AWESOME !!

Slide 7

Slide 7 text

7 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Shell Scripts + Money = ? ( © John Lynch, http://goo.gl/gkmKGN ) “Model-driven orchestration frameworks for complex infrastructure management and automation”

Slide 8

Slide 8 text

Slide 9

Slide 9 text

9 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible State of the Art Original Rockstars ! - Great Tools - Field Pioneers But just incredibly complex to start with, even for simple cases. Not everyone is managing 500 servers in the cloud after all …

Slide 10

Slide 10 text

Slide 11

Slide 11 text

11 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Introducing Ansible … A fictional machine capable of instantaneous or superluminal communication (© Wikipedia). Design Goals: 1.  Minimal in nature: Python based with no dependencies on the environment. 2.  Consistent. 3.  Secure: relies on OpenSSH only, with no vulnerable remote agents. 4.  Highly-Reliable: N re-deployments provide the same result. 5.  Low Learning Curve. -  SW to manage and configure computers. -  Python + Jinja2 + YAML + SSH (that’s it !). -  Manages nodes over SSH. -  Does not require additional remote dependencies. -  First Release: February 20th 2012.

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

15 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Introduction to Ansible [lan1] 192.168.0.1 192.168.0.2 # ansible -i inventory.txt lan1 -m shell -a "/bin/echo Hello World" 192.168.0.1 | success | rc=0 >> Hello World 192.168.0.2 | success | rc=0 >> Hello World # ansible -i inventory.txt lan1 -u ben --sudo -m shell -a "/bin/echo Hello World" Ansible Inventory File Module Arguments Module Name

Slide 16

Slide 16 text

16 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Ansible Inventory [europe] foo.domain.com [usa] 192.168.0.1 [world:children] europe usa [webservers] www[01:50].domain.com [databases] db-[a:f].domain.com Hosts can be described by FQDN or IP With support for infinite depth inheritance [targets] localhost ansible_connection=local web1.domain.com ansible_connection=ssh ansible_ssh_user=user1 web2.domain.com ansible_connection=ssh ansible_ssh_user=user2 With support for wildcards And per-host tuning variables.

Slide 17

Slide 17 text

17 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Ansible Variables [lan1] 192.168.0.1 msg="Hey !" 192.168.0.2 msg= "What’s up ?" # ansible -i inventory.txt lan1 -m shell \ -a "/bin/echo {{msg}}" 192.168.0.1 | success | rc=0 >> Hey ! 192.168.0.2 | success | rc=0 >> What’s up ? Inventory.txt: [europe:vars] domain=my.domain.eu [usa:vars] domain=my.domain.com ./group_vars/europe : -- domain: my.domain.eu ./group_vars/usa : -- domain: my.domain.com YAML File Format

Slide 18

Slide 18 text

18 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Ansible Modules (235 in Ansible v1.7.1) # ansible-doc -l acl Sets and retrieves file ACL information. add_host Add a host (and alternatively a group) to the ansible-playbook alternatives Manages alternative programs for common commands apache2_module Enables/disables a module of the Apache2 webserver apt Manages apt-packages apt_key Add or remove an apt key apt_repository Add and remove APT repositories [...] xattr Set/retrieve extended attributes yum Manages packages with the `yum' package manager zfs Manage zfs zypper Manage packages on SuSE and openSuSE

Slide 19

Slide 19 text

19 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Ansible Module How-To # ansible-doc shell > SHELL The [shell] module takes the command name followed by a list of space-delimited arguments. It is almost exactly like the [command] module but runs the command through a shell (`/bin/sh') on the remote node. Options (= is mandatory): - chdir cd into this directory before running the command - creates a filename, when it already exists, this step will *not* be run. - executable change the shell used to execute the command. Should be an absolute path to the executable. = free_form The shell module takes a free form command to run, as a string. There's not an actual option named "free form". See the examples! - removes a filename, when it does not exist, this step will *not* be run. Notes: If you want to execute a command securely and predictably, it may be better to use the [command] module instead. Best practices when writing playbooks will follow the trend of using [command] unless [shell] is explicitly required. When running ad-hoc commands, use your best judgement. # Execute the command in remote shell; stdout goes to the specified # file on the remote - shell: somescript.sh >> somelog.txt

Slide 20

Slide 20 text

Slide 21

Slide 21 text

21 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Ansible Playbooks --- - hosts: lan1 remote_user: ben sudo: no vars: name: ”My First Playbook” tasks: - name: verify servers activity ping: - name: say hello shell: /bin/echo "{{name}} {{msg}}” notify: - we are done handlers: - name: we are done shell: /bin/echo ”That’s it !" YAML File Format

Slide 22

Slide 22 text

22 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Ansible Playbooks # ansible-playbook -i inventory.txt playbook.yml -v PLAY [lan1] *********************************************************** GATHERING FACTS ***************************************************** ok: [192.168.0.1] ok: [192.168.0.2] TASK: [verify servers activity] ********************************************* ok: [192.168.0.1] => {"changed": false, "ping": "pong”} ok: [192.168.0.2] => {"changed": false, "ping": "pong"} TASK: [say hello] ******************************************************* changed: [192.168.0.1] => {"changed": true, "cmd": "/bin/echo \”My First Playbook Hey !\" ", "delta": "0:00:00.005264", "end": "2014-07-06 16:42:54.115860", "rc": 0, "start": "2014-07-06 16:42:54.110596", "stderr": "", "stdout": ”My First Playbook Hey !"} changed: [192.168.0.2] => {"changed": true, "cmd": "/bin/echo \”My First Playbook What’s up ?\" ", "delta": "0:00:00.002732", "end": "2014-07-06 16:42:54.078013", "rc": 0, "start": "2014-07-06 16:42:54.075281", "stderr": "", "stdout": ”My First Playbook What’s up ?"} […]

Slide 23

Slide 23 text

23 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Ansible Playbooks NOTIFIED: [we are done] **************************************** changed: [192.168.0.1] => {"changed": true, "cmd": "/bin/echo \"That’s it !\" ", "delta": "0:00:00.005559", "end": "2014-07-06 16:42:54.312184", "rc": 0, "start": "2014-07-06 16:42:54.306625", "stderr": "", "stdout": "That’s it !"} changed: [192.168.0.2] => {"changed": true, "cmd": "/bin/echo \"That’s it !\" ", "delta": "0:00:00.002824", "end": "2014-07-06 16:42:54.306878", "rc": 0, "start": "2014-07-06 16:42:54.304054", "stderr": "", "stdout": "That’s it !"} PLAY RECAP ************************************************** 192.168.0.1 : ok=4 changed=2 unreachable=0 failed=0 192.168.0.2 : ok=4 changed=2 unreachable=0 failed=0

Slide 24

Slide 24 text

24 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Ansible Playbook vars_file: - “vars/common.yml” - “vars/production.yml” tasks: - include: tasks/nginx.yml - include: tasks/php-fpm.yml - include: tasks/mariadb.yml

Slide 25

Slide 25 text

25 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Example: (Parts of) MariaDB Cluster Automation [mariadb] 192.168.0.1 name=my-db-maria1 192.168.0.2 name=my-db-maria2 192.168.0.3 name=my-db-maria3 tasks/hosts.yml: - name: declare hosts lineinfile: dest=/etc/hosts regexp='.*{{ item }} .*$' line="{{item}} {{ hostvars[item]['name'] }} {{ hostvars[item]['name'] }}.{{domain}}" state=present with_items: groups['{{hosts_list}}'] - hosts: mariadb vars: domain : domain.com hosts_list: mariadb tasks: - include: tasks/hosts.yml Targets /etc/hosts: 192.168.0.1 my-db-maria1 my-db-maria1.domain.com 192.168.0.2 my-db-maria2 my-db-maria2.domain.com 192.168.0.3 my-db-maria3 my-db-maria3.domain.com

Slide 26

Slide 26 text

26 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Example: (Parts of) MariaDB Cluster Automation tasks/mariadb-galera.yml: - name: Add MariaDB APT key apt_key: url=http://keyserver.ubuntu.com/pks/lookup? op=get&fingerprint=on&search=0xcbcb082a1bb943db - name: Add MariaDB APT repository apt_repository: repo='deb http://ftp.igh.cnrs.fr/pub/mariadb/repo/5.5/debian wheezy main' - name: APT pinning for MariaDB action: copy src=files/mariadb.pref dest=/etc/apt/preferences.d/mariadb.pref \ owner=root group=root mode=0644 vars: mariadb_debian_password: my_debian_password mariadb_root_password: my_root_password mariadb_cluster_name: my_cluster mariadb_cluster_list: "{{ groups['mariadb'] }}” mariadb_gcache_size: 4G mariadb_extra_cfg: skip-external-locking: ~ skip-name-resolve: ~ # Force no DNS resolution tasks: - include: tasks/mariadb-galera.yml

Slide 27

Slide 27 text

27 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Example: (Parts of) MariaDB Cluster Automation tasks/mariadb-galera.yml: - name: install mariadb-galera-server action: apt name=mariadb-galera-server update_cache=yes - name: copy mysql_secure_installation credentials when: mariadb_root_password is defined action: template src=templates/mariadb_secure_installation.j2 \ dest=/tmp/mariadb_secure_installation owner=root group=root mode=0600 templates/mariadb_secure_installation.j2: {{ mariadb_root_password }} {{ mariadb_root_password }}

Slide 28

Slide 28 text

28 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Example: (Parts of) MariaDB Cluster Automation tasks/mariadb-galera.yml: - name: write conf.d/mariadb.cnf action: template src=templates/mariadb.cnf.j2 dest=/etc/mysql/conf.d/mariadb.cnf templates/mariadb.cnf.j2: [mysqld] wsrep_provider=/usr/lib/galera/libgalera_smm.so {% if mariadb_cluster_name is defined %} wsrep_cluster_name='{{ mariadb_cluster_name }}’ {% endif %} {% if mariadb_cluster_list is defined %} wsrep_cluster_address=gcomm:// {{ mariadb_cluster_list[0] }}{% for node in mariadb_cluster_list[1:] %},{{ node }}{% endfor %}{% endif %} Targets /etc/mysql/mariadb.cnf: wsrep_cluster_name=‘my_cluster’ wsrep_cluster_address=gcomm://192.168.0.1,192.168.0.2,192.168.0.3 Jinja2 Syntax

Slide 29

Slide 29 text

29 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible Example: (Parts of) MariaDB Cluster Automation templates/mariadb.cnf.j2: [mysqld] wsrep_node_address={{ ansible_eth0.ipv4.address }} wsrep_node_name='{{ ansible_hostname }}’ {% if mariadb_extra_cfg is defined %} {% for key, value in mariadb_extra_cfg.iteritems() %} {{ key }}{% if value is not none %}={{ value }}{% endif %} {% endfor %} {% endif %} Target /etc/mysql/mariadb.cnf: wsrep_node_address=192.168.0.1 wsrep_node_name=my-db-maria1 skip-external-locking skip-name-resolve

Slide 30

Slide 30 text

Slide 31

Slide 31 text

31 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible OpenStack in a Nutshell - #1 (most widely adopted) Open-Source IaaS project. - Awesome REST Management API. - Perfect for instant spawning of new Virtual Machines (VMs) - But VMs yet to be configured ... What if I could connect OpenStack VMs with Ansible for nightly CI ?

Slide 32

Slide 32 text

32 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible (One of my) Cloud application: OpenTouch TeamShare -  Online collaboration tool for SMBs. -  Provides multi-projects file storage and sharing for enterprises. -  With project management, chat and collaboration capabilities.

Slide 33

Slide 33 text

33 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible (One of my) Cloud application network topology -  2x HAProxy Load-Balancers -  2x NGINX frontal Web Servers -  2x NGINX frontal Web File Servers -  2x PHP Backends -  1x SMTP Server -  3x MariaDB Master-Master Galera Cluster + 2x Galera Arbiters -  2x MongoDB Master-Slave Cluster + 1x MongoDB Arbiter -  3x RabbitMQ Master-Master Clusters -  1x LibreOffice Server -  1x NFS Server Now let’s say that I want to test both my application and my infrastructure every single night for non-regression !

Slide 34

Slide 34 text

Slide 35

Slide 35 text

35 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible AvOID: Ansible OpenStack Instances Deployer -  Open Source software by Alcatel-Lucent Enterprise. -  Written in Python with dependencies to OpenStack Nova/Neutron APIs. -  Relies on Ansible with IaaS bindings: -  Currently OpenStack only -  But wide open to support many much more … who knows … -  Comes as a library with both CLI and Web clients. How it works: 1.  Describe your infrastructure topology in a YAML file once and for all. 2.  Run avoid-cli. Grab a coffee, that’s it. How it (internally) works: 1.  Parses your topology file. 2.  Optionally terminates (all) OpenStack VMs and spawn new ones and build Ansible inventory file. 3.  Creates VMs dependency graph for parallelized post-configuration by Ansible. 4.  Post-configure VMs through Ansible playbooks (continuous-integration style).

Slide 36

Slide 36 text

36 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible AvOID: Ansible OpenStack Instances Deployer Check it out on GitHub: https://github.com/OpenTouch/AvOID

Slide 37

Slide 37 text

37 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible AvOID Topology File - globals: env: dev ssh_key: /path/to/ssh_private_key.pem ssh_user: remote_user os_user: openstack_user os_passwd: openstack_password os_tenant: openstack_tenant os_auth_url: http://my.private.cloud.com:5000/v2.0 os_image: Debian – Wheezy os_network: My OpenStack Tenant Network os_ssh_key: My OpenStack Tenant SSH Key Name ansible_inventory_template: /path/to/ansible/inventory_template.txt ansible_playbooks_directory: /path/to/ansible/playbooks

Slide 38

Slide 38 text

38 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible AvOID Topology File nodes: - node: name: web-server-1 flavor: m1.small ansible_config_keys: webserver playbook: webserver additional_network: LAN Network 2, LAN Network 3 security: http, https depends: file-server-1 floating_ips: 1.2.3.4, 5.6.7.8, 10.20.30.40 vips: 10.0.1.2, 10.0.2.2, 10.0.3.2 - node: name: file-server-1 flavor: m3.medium ansible_config_keys: fileserver playbook: fileserver volumes: - { name: web-volume1, size: 1 } - { name: web-volume2, size: 10 }

Slide 39

Slide 39 text

39 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible AvOID command-line # avoid-cli Usage: avoid-cli [opts] where is: - status: list topology status - redeploy : terminate VM, spawn a new one and restart playbook - redeployall: redeploy all nodes - runplaybook : restart playbook as it - runallplaybooks: restart all playbooks - geninventory: generate Ansible inventory file based on topology.yml

Slide 40

Slide 40 text

40 COPYRIGHT © 2014 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Extending Cloud Automation: When OpenStack Meets Ansible AvOID Web UI # avoid-web my-topo-dev.yml Read of my-dev.yml done: 32 VMs and 22 playbooks Now go to http://localhost:8888/ !