Deploying with Vagrant and Ansible

85ac50a701799ef4e3ce34ed441c11a8?s=47 yeukhon
April 17, 2013

Deploying with Vagrant and Ansible

April 17, 2013
Adv Tps Intrnet Prog: CSC I4330 3GG [2337] (CCNY )

85ac50a701799ef4e3ce34ed441c11a8?s=128

yeukhon

April 17, 2013
Tweet

Transcript

  1. 11.

    Vagrant Box • A Vagrant box is a packaged virtual

    machine. If you extract it you get a vagrant file, ovf and and a vmdk file. • Vagrant has official boxes for download (distribute with a universal SSH key, so do not use it in production)
  2. 12.

    Official Boxes Ubuntu Lucid 32 Bit http://files.vagrantup.com/lucid32.box Ubuntu Lucid 64

    Bit http://files.vagrantup.com/lucid64.box Ubuntu Precise 32 Bit http://files.vagrantup.com/precise32.box Ubuntu Precise 64 Bit http://files.vagrantup.com/precise64.box
  3. 14.

    Step 1: Create a box 1. Download a box 2.

    Initialize a Vagrant workspace $ mkdir vagrant-example $ cd vagrant-example $ vagrant init precise32 $ vagrant box add precise32 http://files.vagrantup.com/precise32.box
  4. 15.

    Step 2: Vagrantfile Vagrantfile Vagrantfile is a ruby configuration file

    used to configure a vagrant instance. Vagrant.configure("2") do |config| config.vm.box = "precise32" config.vm.network :public_network config.vm.provider :virtualbox do |vb| vb.customize ["modifyvm", :id, "--memory", "400"] end end
  5. 16.

    Step 3: vagrant up vagrant up vagrant up will create

    a new VM or boot up an existing Vagrant virtual machine. $ vagrant up $ vagrant up Bringing machine 'default' up with 'virtualbox' provider... Bringing machine 'default' up with 'virtualbox' provider... [default] Importing base box 'precise32'... [default] Importing base box 'precise32'...
  6. 17.

    Step 3: vagrant ssh vagrant ssh vagrant ssh will allow

    you to automatically ssh into your vagrant instance if you are using an official box. $ vagrant ssh $ vagrant ssh Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic-pae i686) Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic-pae i686) * Documentation: https://help.ubuntu.com/ * Documentation: https://help.ubuntu.com/ Welcome to your Vagrant-built virtual machine. Welcome to your Vagrant-built virtual machine. Last login: Fri Sep 14 06:22:31 2012 from 10.0.2.2 Last login: Fri Sep 14 06:22:31 2012 from 10.0.2.2
  7. 18.

    Vagrant Commands Usage: vagrant [-v] [-h] command [<args>] Usage: vagrant

    [-v] [-h] command [<args>] -v, --version Print the version and exit. -v, --version Print the version and exit. -h, --help Print this help. -h, --help Print this help. Available subcommands: Available subcommands: box box destroy destroy halt halt init init package package plugin plugin provision provision reload reload resume resume ssh ssh ssh-config ssh-config status status suspend suspend up up For help on any individual command run `vagrant COMMAND -h` For help on any individual command run `vagrant COMMAND -h`
  8. 19.

    I don't want an official box You can use any

    ISO local or over the wire. $ mkdir vagrant-example $ mkdir vagrant-example $ cd vagrant-example $ cd vagrant-example $ vagrant init $ vagrant init $ vim Vagrantfile config.vm.box_url = "http://domain.com/path/to/above.box"
  9. 20.

    I want my own SSH key Even for vagrant official

    boxes, you should consider changing to your own SSH key.
  10. 23.

    Treat your infrastructure like your software. O Version control your

    configuration and specs O Readable O Automated
  11. 24.

    REQUIREMENTS: O Strong community support O Medium to little entry

    to barrier (installation, tutorial, customizability) O Readable configuration O Version control O As lightweight as possible
  12. 28.

    STRONG COMMUNITY Started in 2012, has over 423 forks, 1,429

    watcher, 2000+ tickets solved, weekly clearance, active IRC & mailing list
  13. 30.

    5 REQUIRED DEPENDENCIES Python for running Python Simplejson/json for results

    Jinja2 for template Pyyaml for configuration Paramiko for ssh2 protocol
  14. 31.

    #TODO We will configure a machine for repoapi live. O

    bootstrap machine with dev pkgs O setup nginx O setup database O clone and install master repo O run server
  15. 32.

    Getting Started • Install AnVsible http://ansible.cc/docs/ $ sudo pip install

    pyyaml jinja2 paramiko $ sudo pip install pyyaml jinja2 paramiko $ git clone git://github.com/ansible/ansible.git $ git clone git://github.com/ansible/ansible.git $ cd ansible $ cd ansible $ sudo make install $ sudo make install
  16. 33.

    Modules • Ansible community has a lot of modules that

    you can used to configure a server. Modules could be in ruby, python, perl, etc. Just executable. http://ansible.cc/docs/modules.html apt, yum, pip, gem, homebrew, user apt, yum, pip, gem, homebrew, user file, copy, template, django_manage file, copy, template, django_manage postgres_user, postgres_db, mysql*, mongodb*, rabbitmq*, Riak postgres_user, postgres_db, mysql*, mongodb*, rabbitmq*, Riak ec2, ec2_facts, s3, ec2, ec2_facts, s3, hg, git, svn hg, git, svn many more... many more...
  17. 34.

    Hosts / Inventory An inventory file contains a list of

    machine addresses. http://ansible.cc/docs/patterns.html 192.168.1.4 192.168.1.4 [webservers] [webservers] foo.example.com foo.example.com bar.example.com bar.example.com www[01:50].example.com www[01:50].example.com
  18. 35.

    Ansible Playbook --- --- # nginx.yml # nginx.yml - hosts:

    dev-box - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True # You can use variables during configuration! # You can use variables during configuration! vars: vars: - APP_PORT: 8080 - APP_PORT: 8080 tasks: tasks: - name: Ensure Nginx is installed - name: Ensure Nginx is installed apt: pkg=nginx apt: pkg=nginx
  19. 36.

    Template for Play - name: Description of the task/play -

    name: Description of the task/play module_name: arguments option1=value1 option2=value2 module_name: arguments option1=value1 option2=value2
  20. 37.

    apt

  21. 38.

    apt get install nginx? --- --- # nginx.yml # nginx.yml

    - hosts: dev-box - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True tasks: tasks: - name: Ensure Nginx is installed apt: pkg=nginx
  22. 39.

    apt-get remove nginx? --- --- # nginx.yml # nginx.yml -

    hosts: dev-box - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True tasks: tasks: - name: Ensure Nginx is removed apt: pkg=nginx state=absent
  23. 40.

    Idempotence Idempotence means for the same configuration settings, f(f(x)) =

    f(x) f(f(x)) = f(x) HTTP's GET, HEAD, PUT, DELETE, OPTIONS, TRACE satisfy idempotence.
  24. 42.

    Variable --- --- # nginx.yml # nginx.yml - hosts: dev-box

    - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True # You can use variables during configuration! # You can use variables during configuration! vars: - APP_PORT: 6543 tasks: tasks: - name: Ensure Nginx is installed - name: Ensure Nginx is installed apt: pkg=nginx apt: pkg=nginx
  25. 43.

    Variable --- --- # nginx.yml # nginx.yml - hosts: dev-box

    - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True # You can use variables during configuration! # You can use variables during configuration! vars: - NGINX_NAME: nginx tasks: tasks: - name: Ensure Nginx is installed - name: Ensure Nginx is installed apt: pkg= apt: pkg=$NGINX_NAME $NGINX_NAME
  26. 44.

    Variable --- --- # nginx.yml # nginx.yml - hosts: dev-box

    - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True # You can use variables during configuration! # You can use variables during configuration! vars: - NGINX_NAME: nginx tasks: tasks: - name: Ensure Nginx is installed - name: Ensure Nginx is installed apt: pkg= apt: pkg=$NGINX_NAME $NGINX_NAME
  27. 45.

    Template + Variable --- --- # nginx.yml # nginx.yml -

    hosts: dev-box - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True # You can use variables during configuration! # You can use variables during configuration! vars: - HELLO_WORLD: “Hi, World” tasks: tasks: - name: Configure a foo.j2 - name: Configure a foo.j2 template: src=foo.j2 dest=/home/vagrant/foo.txt # call this foo.j2 - a Jinja template file # call this foo.j2 - a Jinja template file {{ {{ HELLO_WORLD }} }} # result # result $ cat foo.txt $ cat foo.txt Hi, World Hi, World
  28. 46.

    Gather Facts Ansible can gather basic facts about the target

    machine. By default, this feature is on. If you don't need it, set gather_facts: no gather_facts: no to speed up configuration. --- --- # nginx.yml # nginx.yml - hosts: dev-box - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True gather_facts: no gather_facts: no … …. .
  29. 47.

    Gather Facts ansible -m setup 127.0.0.1 -u vagrant -k ansible

    -m setup 127.0.0.1 -u vagrant -k 127.0.0.1 | success >> { 127.0.0.1 | success >> { "ansible_facts": { "ansible_facts": { …. …. "ansible_lsb": { "ansible_lsb": { "codename": "precise", "codename": "precise", "description": "Ubuntu 12.04.1 LTS", "description": "Ubuntu 12.04.1 LTS", "id": "Ubuntu", "id": "Ubuntu", "major_release": "12", "major_release": "12", "release": "12.04" "release": "12.04" }, }, "ansible_pkg_mgr": "apt", "ansible_pkg_mgr": "apt", "ansible_processor": [ "ansible_processor": [ "Intel(R) Pentium(R) 4 CPU 3.20GHz" "Intel(R) Pentium(R) 4 CPU 3.20GHz" ], ], "ansible_processor_cores": 1, "ansible_processor_cores": 1, "ansible_processor_count": "ansible_processor_count": 1 1 "ansible_python_version": "2.7.3", "ansible_python_version": "2.7.3", …. …. } } see https://gist.github.com/yeukhon/4689592 see https://gist.github.com/yeukhon/4689592
  30. 48.

    Templating Nginx nginx.conf and sites-available/default require configuration. o worker counts

    o application port and proxy settings o SSL (although we won't do it here) o many more
  31. 49.

    nginx.conf.j2 # nginx.conf.j2 # nginx.conf.j2 user www-data; user www-data; worker_processes

    worker_processes {{ WORKER_COUNT }} {{ WORKER_COUNT }}; ; pid /var/run/nginx.pid; pid /var/run/nginx.pid; events { events { worker_connections worker_connections {{ WORKER_COUNT * 1024 }} {{ WORKER_COUNT * 1024 }}; ; # multi_accept on; # multi_accept on; } }
  32. 50.

    Configure nginx.conf.j2 --- --- # nginx.yml # nginx.yml - hosts:

    dev-box - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True vars: - WORKER_COUNT: ${ansible_processor_count} tasks: tasks: - name: Configure nginx.conf - name: Configure nginx.conf template: src=nginx.conf.j2 path=/etc/nginx/nginx.conf
  33. 51.

    Var files --- --- # nginx.yml # nginx.yml - hosts:

    dev-box - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True # vars: # - WORKER_COUNT: ${ansible_processor_count} var_files: var_files: - nginx_vars.yml - nginx_vars.yml tasks: tasks: - name: Configure nginx.conf - name: Configure nginx.conf template: src=nginx.conf.j2 path=/etc/nginx/nginx.conf # nginx_vars.yml # nginx_vars.yml WORKER_COUNT: ${ansible_processor_count}
  34. 53.

    with_items - name: Install Debian pkgs - name: Install Debian

    pkgs apt: pkg= apt: pkg=$item $item with_items: with_items: - build-essential - build-essential - python-dev - python-dev - python-pip - python-pip - python-setuptools - python-setuptools - name: Pip install Python tools - name: Pip install Python tools pip: name= pip: name=$item $item with_items: with_items: - distribute - distribute - virtualenv - virtualenv
  35. 56.

    hg vars: vars: - URL: - URL: https://bitbucket.org/ideapiteam/repo-api-master https://bitbucket.org/ideapiteam/repo-api-master tasks:

    tasks: - name: Clone Master - name: Clone Master hg: repo=$URL dest=/home/vagrant/repoapi hg: repo=$URL dest=/home/vagrant/repoapi
  36. 57.

    service tasks: tasks: - name: Ensure Nginx has started -

    name: Ensure Nginx has started service: name=nginx state=started service: name=nginx state=started # started, restarted, stopped
  37. 58.

    file tasks: tasks: - name: Ensure /home/vagrant/repoapi exists - name:

    Ensure /home/vagrant/repoapi exists file: path=/home/vagrant/repoapi state=directory file: path=/home/vagrant/repoapi state=directory # state=directory/file/link
  38. 59.

    user tasks: tasks: - name: Ensure user vagrant actually exist

    - name: Ensure user vagrant actually exist user: name=vagrant uid=1001 state=present user: name=vagrant uid=1001 state=present
  39. 60.

    postgres_* tasks: tasks: - name: Create a Postgres database -

    name: Create a Postgres database postgres_db: db=mydb owner=user123 login_password: $md5_pwd postgres_db: db=mydb owner=user123 login_password: $md5_pwd - name: Create a Postgres user - name: Create a Postgres user postgres_user: user=user123 password=$hash_pwd priv=ALL postgres_user: user=user123 password=$hash_pwd priv=ALL
  40. 62.

    Role You can have a cluster of webservice and dbserver.

    You can narrow down to a cluster of mongo instances, project1 webservice, project2 webservice, etc. Group them inside inventory.
  41. 63.

    Inventory groups (roles) [webservers] [webservers] 192.168.1.1 192.168.1.1 192.168.1.2 192.168.1.2 [mongoservers]

    192.168.1.4 192.168.1.4 192.168.1.5 192.168.1.5 192.168.1.7 192.168.1.7 [all] [all] Webservers Webservers mongoservers mongoservers
  42. 64.

    Inventory name can be role # production # production [webservers]

    [webservers] www[01:20].example.com www[01:20].example.com [mongoservers] db[01:20].example.com db[01:20].example.com [all] [all] webservers webservers mongoservers mongoservers # development # development [webservers] [webservers] 192.168.1.1 192.168.1.1 192.168.1.2 192.168.1.2 [mongoservers] 192.168.1.4 192.168.1.4 192.168.1.5 192.168.1.5 192.168.1.7 192.168.1.7 [all] [all] webservers webservers mongoservers mongoservers ansible-playbook -i ansible-playbook -i development setup.yml setup.yml ansible-playbook -i ansible-playbook -i production setup.yml setup.yml
  43. 65.

    Slow? --- --- # setup.yml # setup.yml - hosts: dev-box

    - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True tasks: tasks: - name: Ensure Nginx is installed - name: Ensure Nginx is installed apt: pkg=nginx apt: pkg=nginx - name: Update web service - name: Update web service hg: repo=$URL dest=/home/vagrant/repoapi hg: repo=$URL dest=/home/vagrant/repoapi … … more tasks x 100 ... more tasks x 100 ...
  44. 66.

    Tags? --- --- # setup.yml # setup.yml - hosts: dev-box

    - hosts: dev-box user: vagrant user: vagrant sudo: True sudo: True tags: webserver tags: webserver tasks: tasks: - name: Ensure Nginx is installed - name: Ensure Nginx is installed apt: pkg=nginx apt: pkg=nginx tags: tags: - nginx - nginx - name: Update web service - name: Update web service hg: repo=$URL dest=/home/vagrant/repoapi hg: repo=$URL dest=/home/vagrant/repoapi tags: tags: - repo - repo ansible-playbook -i development setup.yml ansible-playbook -i development setup.yml ansible-playbook -i development setup.yml ansible-playbook -i development setup.yml --tags webserver --tags webserver ansible-playbook -i development setup.yml ansible-playbook -i development setup.yml --tags nginx --tags nginx ansible-playbook -i development setup.yml ansible-playbook -i development setup.yml --tags repo --tags repo ansible-playbook -i development setup.yml ansible-playbook -i development setup.yml --tags nginx,repo --tags nginx,repo
  45. 69.

    PULL? To do full automated deploymet, you probably want to

    do pull rather than push. Ansible can do that too. http://jpmens.net/2012/07/14/ansible-pull-instead-of-push/
  46. 70.

    BUILD CUSTOM BOX Ansible is great for configuration. Vagrant is

    great for manging development vm. But building a base box to begin with? Use veewee.
  47. 71.

    ANSIBLE HAS VAGRANT MODULES I haven't tried it, but in

    theory you could configure a bunch of vagrant instances using ansible and vagrant module!