Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Juergen Brendel: Deploying test and production systems with Ansible

Juergen Brendel: Deploying test and production systems with Ansible

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Juergen Brendel:
Deploying test and production systems with Ansible
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2014 - Sunday, 14 Sep 2014 - Track 1
http://kiwi.pycon.org/

**Audience level**

Intermediate

**Description**

This talk presents Ansible as a light-weight, simple and effective configuration management tool. We will see why configuration management is important and how we can use Ansible to automatically deploy and test full production clusters - showcased with a sample Django application - with just a single command and in a number of different environments.

**Abstract**

With modern configuration, deployment and orchestration systems, software developers can easily maintain, bring up and tear down local test systems, staging, demonstration and production systems, which are 100% identical to each other.

This presentation will give an introduction to Ansible, a modern, simple and efficient configuration management system. We will see how we can deploy complete clusters (with load balancers, database and some Django application servers), reliably and fully automated, either on dedicated servers, on local virtual machines or in the cloud.

If all goes well, there will even be a live demo in which we will bring up a complete cluster or two in different environments, to illustrate the power of this approach.

**YouTube**

https://www.youtube.com/watch?v=Dg5TSfLHW3Q

6b880a0b67fac54c42c77fe70d97334d?s=128

New Zealand Python User Group

September 14, 2014
Tweet

Transcript

  1. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Automated deployment with Ansible

    Juergen Brendel juergen@brendel.com @BrendelConsult
  2. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Summary • Configuration management

    background • Ansible intro • Unified development, test and deployment environments • Ansible and the cloud • Contributing to Ansible
  3. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Configuration Management: Why and

    how
  4. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult What configuration? • System

    packages • Settings • Users/groups • Applications
  5. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Arcane magic Configuring servers

    How do you configure a server? Manual instructions Scripts CM tools Automation!
  6. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Ensure all system packages

    are updated. CM Tools Describe the desired state Ensure that user “xyz” exists. Ensure package “apache” is installed. Have latest sources: Clone repo, update if it exists already. Ensure package “postgres” v9.1 is installed. Ensure DB “app_data” exists with password “****”.
  7. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult CM Tools variety •

    Puppet (2005) • Chef (2009) • Salt (2011) • Ansible (2012) • Fabric • Scripts “powerful, feature-rich, enterprisy” “simple, fast, good for most things” “not really CM tools”
  8. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Ansible: Intro and key

    concepts
  9. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Ansible overview • “Orchestration

    engine” for CM and deployment • Written in Python • Uses YAML • “Playbooks” • Config specs or explicit commands • Linux and Windows
  10. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Ansible simplicity Key points:

    – No central configuration server – No key management – No agent to install on target machine – Explicit order Requirements: – Need SSH access (with key or password) – Need Python installed on target machine
  11. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Ansible architecture Server 1

    Server 2 Server 3 Server 4 Server 5 Your laptop
  12. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Modules Hundreds of them.

    They know how to do stuff... - Command - Shell - Script - Copy - Sync - Templates - Line ops - Install packages - Users and groups - Networking - Services - Repositories - Message queues - Monitoring - Notification - Web servers - Database servers - Cloud infrastructure
  13. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult How does it work?

    Server Your laptop Python module: “install apache” Run module Delete module on server
  14. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Inventory and groups Define

    hosts, organized in groups • by function • by location • by hosting provider • ...
  15. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Inventory and groups Define

    hosts, organized in groups [europe] server1.somehoster.co.uk server2.otherhoster.de [north­america] host­a.serverhost.com host­b.serverhost.com [frontend] server1.somehoster.co.uk host­b.serverhost.com [backend] server2.otherhoster.de host­a.serverhost.com
  16. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Adhoc commands Single commands,

    applied to groups $ ansible ­i hosts europe ­a “uname ­a” $ ansible ­i hosts frontend ­a “/sbin/reboot” ­f 3
  17. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Playbooks ­­­ ­ hosts:

    frontend sudo: yes tasks: ­ name: Update the system apt: pkg=nginx state=latest ­ name: Create the user account user: name=appuser shell=/bin/bash state=present ­ name: Copy files to remote user's home copy: > src=files/names.txt dst=/home/appuser owner=appuser mode=0644
  18. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Variables ­­­ ­ hosts:

    all sudo: yes vars: username: appuser tasks: ­ name: Create the user account user: > name={{ username }} shell=/bin/bash state=present
  19. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Project layout 1 /

    my_hosts group_vars/ all frontend backend europe north­america site.yml
  20. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Project layout 2 /

    ansible.cfg deploy_hosts staging_hosts group_vars/ all frontend backend europe north­america host_vars/ server1.somehoster.co.uk host­b.serverhost.com site.yml roles/ common/ tasks/ main.yml handlers/ main.yml templates/ sshd_config.j2 files/ my_script.sh vars/ main.yml web/ ... db/ ...
  21. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Playbooks with roles ­­­

    ­ hosts: frontend sudo: yes roles: ­ common ­ web
  22. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Local environments

  23. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Dev, Test, Deploy Common

    issues
  24. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Dev, Test, Deploy Why

    didn't we catch this bug in testing? How do I setup my development environment? We can't reproduce the issues. I don't have access to our test server. “It works for me...” (shrug)
  25. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Dev, Test, Deploy Why

    didn't we catch this bug in testing? How do I setup my development environment? We can't reproduce the issues. I don't have access to our test server. “It works for me...” (shrug) Wouldn't this be nice instead? Single command: Dev environment created Single command: Test environment created
  26. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Vagrant • Use Vagrant

    to spin-up VMs • local (VirtualBox, VMware, etc.) • cloud (EC2) • Use Ansible as 'provisioner' • Make an inventory file with just your VM • Point at same playbook as before
  27. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Ansible and “the cloud”

  28. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult The rise and rise

    of APIs APIs Local Infrastructure Services This is really cool!
  29. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Ansible 'cloud' modules Public

    cloud • Amazon AWS • Google Compute • Azure • Digital Ocean • Rackspace • Linode Public cloud • Amazon AWS • Google Compute • Azure • Digital Ocean • Rackspace • Linode Private cloud • OpenStack • Eucalyptus • Vsphere • Docker • libvirt Private cloud • OpenStack • Eucalyptus • Vsphere • Docker • libvirt
  30. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Example: AWS modules EC2

    / infrastructure • Instances • Images • VPCs • Load balancers Services • S3 • Route 53 • Databases • Cache
  31. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Example: Create instances #

    My hostfile: Empty groups [frontend] [backend] [my­server­group]
  32. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Example: Create instances ­

    local_action: module: ec2 key_name: my­key group: my­security­group instance_type: t2.micro image: ami­120abc90 region: us­east­1 count: 3 register: ec2results
  33. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Example: Create instances ­

    local_action: module: ec2 key_name: my­key group: my­security­group instance_type: t2.micro image: ami­120abc90 region: us­east­1 count: 3 register: ec2results { "ec2results": { "changed": true, "instance_ids": [ "i­da9fbbe4", ... ], "instances": [ { "id": "i­da9fbbe4", "instance_type": "t1.micro", "image_id": "ami­0729b73d", "region": "us­east­1", "public_ip": "54.253.65.41", "private_ip": "10.248.39.121", ... }, ...
  34. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Example: Create instances ­

    local_action: module: ec2 key_name: my­key group: my­security­group instance_type: t2.micro image: ami­120abc90 region: us­east­1 count: 3 register: ec2results ­ local_action: module: add_host hostname: {{ item.public_ip }} groupname: my­server­group with_items: ec2results.instances
  35. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Contributing to Ansible

  36. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Contributing • The perfect

    FOSS project? • Lots of small, independent modules • Written in Python For yourself: Any language, even bash. For contributions: Python preferred
  37. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Getting started $ git

    clone git@github.com:ansible/ansible.git $ source ansible/hacking/env­setup … edit your module … $ ansible/hacking/test­module ­m ./my_module
  38. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Sample module #!/usr/bin/python import

    datetime import json date = str(datetime.datetime.now()) print json.dumps({ “changed” : True, "time" : date }) Catch exceptions All output to stdout JSON
  39. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Sample module #!/usr/bin/python DOCUMENTATION

    = ''' ­­­ module: filewrite short_description: write data to a file description: ­ Write data to start of file. Create if necessary. options: path: description: ­ The full path of the file/object to write to required: true default: null data: description: ­ The data to be written required: false default: some test data '''
  40. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Sample module import os

    from ansible.module_utils.basic import * def main(): module = AnsibleModule( argument_spec = dict( path = dict(required=True), data = dict(default='some test', type='str'), ), ) path = module.params.get('path') data = module.params.get('data') try: if os.path.isfile(path): with open(path, "r") as f: d = f.read(len(data)) if d == data: module.exit_json(changed=False, other_data="Not changed.") with open(path, "w") as f: f.write(data) module.exit_json(changed=True, other_data="Wrote %s characters." % len(data)) except Exception as e: module.fail_json(msg=str(e)) main()
  41. Automated Deployment with Ansible juergen@brendel.com @BrendelConsult Thank you very much!

    Questions: juergen@brendel.com Ansible docs: http://docs.ansible.com/ Ansible source: https://github.com/ansible/ansible Vagrant: http://www.vagrantup.com/ Example project: https://github.com/jbrendel/ansible_cluster_setup