Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Vagrant: Not just for developers - LOPSA-East 2014

Vagrant: Not just for developers - LOPSA-East 2014

Vagrant’s tag line is ‘Development environments made easy’, and it is indeed a tool that vastly improves working with developer virtual machines. However, it can also make the life of a Systems Administrator much easier, providing throwaway test environments to help with troubleshooting, deploying new infrastructure, and playing with new technologies.

This class will take you through the fundamentals of vagrant, what it provides over and above virtualbox/vmware alone, getting your first virtual machine running, setting upnetworking, provisioning with configuration management tools, and creating your own boxes.

E6ba9f8a392923525d3426a7b738cf9c?s=128

Mark Harrison

May 02, 2014
Tweet

Transcript

  1. Vagrant: Not just for developers Mark Harrison Chef

  2. Who am I • Mark Harrison • @mivok on twitter

    • Senior Systems Administrator at Chef • Vagrant user
  3. Why am I here? • Tell you why vagrant is

    awesome. • Teach you how to use Vagrant. • Teach you how to do awesome things with Vagrant.
  4. Why are you here?

  5. What is Vagrant? Vagrant is a tool for building complete

    development environments. With an easy-to-use workflow and focus on automation, Vagrant lowers development environment setup time, increases development/production parity, and makes the "works on my machine" excuse a relic of the past. (From: http://www.vagrantup.com/about.html)
  6. What is Vagrant Vagrant is some developer tool that devs

    use because they don’t know how to install Linux or use Virtualbox. It helps you make dev environments, deploy your code, capistrano your ruby gems and other dev stuff that I’ll never use. (From: Me, before I actually started using vagrant)
  7. What is Vagrant? I was wrong!

  8. What is Vagrant Vagrant is tool for making working with

    virtual machines not suck.
  9. Normal VM workflow • Start virtualbox • Dismiss random error

    message • Click new • Enter a name for your VM • Pick memory, disk space, OS, favorite color • Start VM
  10. Normal VM Workflow • Realize you don’t have an ISO

    • Download ISO • Attach CD • Install OS • Wait… • Look at cat pictures
  11. http://www.flickr.com/photos/plizzba/239595759

  12. http://www.flickr.com/photos/mathias-erhart/2562106102

  13. None
  14. Normal VM Workflow • Finish looking at cat pictures •

    Reboot • Configure networking, users, hostname, software • Forget why you wanted the stupid VM in the first place
  15. Vagrant Workflow me@laptop:~$ vagrant init hashicorp/precise32 me@laptop:~$ vagrant up me@laptop:~$

    vagrant ssh vagrant@my_awesome_vm:~$
  16. What does vagrant get you? • Pre-built ‘gold master’ images

    (called boxes) • Runs virtualbox headless (no console window) • Automatically does port forwarding for SSH • vagrant user with ssh key and sudo • /vagrant shared folder with the host • Built in support for provisioning with chef, puppet, ansible, shell scripts
  17. Vagrant use cases • Throwaway test server • Testing new

    software • Testing config changes • Build system • Isolated environment for software
  18. Example use cases • ATS and OmniOS build machines •

    Installing/testing logstash configurations • Developing OmniOS zone migration procedure • Testing firewall failover procedures • Developing cookbooks/playbooks/manifests • SRE interview VM
  19. Convinced? Good! • Install Virtualbox from http://www.virtualbox.org/ • Install vagrant

    from http://www.vagrantup.com/ downloads.html • Packages are provided for common linux distributions, and installers are available for Windows and Mac
  20. Starting out Vagrant < 1.5: $ mkdir -p ~/vagrant/precise_test $

    cd ~/vagrant/precise_test $ vagrant init hashicorp/precise32 \
 http://files.vagrantup.com/precise32.box
  21. Starting out Vagrant >= 1.5: $ mkdir -p ~/vagrant/precise_test $

    cd ~/vagrant/precise_test $ vagrant init hashicorp/precise32 (No URL required - uses boxes from vagrantcloud.com)
  22. Starting out When you have been using vagrant for a

    while: $ mkdir -p ~/vagrant/precise_test $ cd ~/vagrant/precise_test $ vim Vagrantfile
  23. The Vagrantfile Vagrant.configure(2) do |config| config.vm.box = “hashicorp/precise64” #config.vm.box_url =

    “http://…” end
  24. Boxes • “Gold master” images • .box extension • Usually

    downloaded via http • List at http://www.vagrantbox.es/ (unofficial) • Also at https://vagrantcloud.com/ (newer, official) • Can create your own using Packer (http://packer.io)
  25. vagrantcloud.com

  26. Example - Starting the VM

  27. Starting the VM Steps vagrant performs when starting up: •

    Import the base box, downloading if needed • Create the VM in virtualbox • Set up port forwarding • Set up folder sharing • Boot the VM • Provision the VM (chef, puppet, ansible, shell scripts)
  28. Using the VM $ cd ~/vagrant/my_vm $ vagrant ssh •

    Vagrant automatically knows which port to connect to • Vagrant boxes automatically have a vagrant user with a common (insecure) ssh key • The insecure key isn’t normally an issue for test machines, but be aware of it
  29. Example - SSH into VM

  30. Managing your VM • vagrant status - view the current

    state of the VM • vagrant halt - shut down the VM • vagrant up - bring it back up (and create if needed) • vagrant reload - halt + up in one command • vagrant destroy - delete the VM • vagrant suspend/resume - save VM state
  31. Example - Managing your VM

  32. Managing boxes • Download/manage boxes without starting the VM •

    vagrant box add hashicorp/precise64 • vagrant box list • vagrant box remove BOXNAME • vagrant box outdated [--global] • vagrant box update
  33. Providers • Vagrant’s way of abstracting away different VM software

    • Comes with out of the box support for Virtualbox • Supports other providers too, including • Vmware fusion • ec2 • hyper-v (windows 8.1+)
  34. Vagrant configuration • Folder sharing • Port forwarding • Custom

    networking • Provisioning • Multi-VM projects
  35. Folder sharing • config.vm.synced_folder “some/folder", “/var/foo” • Not normally needed

    - /vagrant is usually sufficient • Uses Virtualbox shared folders • Somewhat slow, but not usually a problem • If performance is a concern, NFS or other options available.
  36. Example - Folder Sharing

  37. Folder sharing - nfs • To use nfs, add the

    “type” option: config.vm.synced_folder ".", "/vagrant", type: "nfs" • If you’re on Virtualbox, you also need to set up private networking to make this work: config.vm.network “private_network”, ip: “192.168.50.2”
  38. Example - Folder sharing with NFS

  39. Folder sharing - rsync • Rsync is new in 1.5

    config.vm.synced_folder ".", "/vagrant", type: "rsync" • Need to run vagrant rsync or vagrant rsync-auto to sync • Sync is one-way - host to guest • Warning: rsync also uses the --delete option
  40. Example - Folder sharing with Rsync

  41. Port forwarding config.vm.network "forwarded_port", guest: 80, host: 8080 config.vm.network "forwarded_port",

    guest: 80, host: 8080, auto_correct: true • This is done automatically for port 22 (ssh)
  42. Example - Port forwarding

  43. Custom networking • Port forwarding is usually enough, but sometimes

    you need something different • Multiple VMs that need to talk to each other (private networking) • VMs accessible to machines on the network other than the host (public networking)
  44. Private network • Used for when you want VMs to

    talk to each other • Uses ‘Host only’ networking option in virtualbox - only the host and other VMs can see the VM config.vm.network “private_network”, ip: “192.168.50.2”
  45. Public network • Used when you want the VM to

    be part of the same network as the host • Uses virtualbox ‘bridged’ networking config.vm.network “public_network” The VM is configured to use DHCP, and will get the address from your network’s DHCP server.
  46. Provisioning • One of the more powerful features of vagrant

    • Base boxes provide the bare minimum needed for vagrant to run • Provisioning lets you install and configure the rest • Vagrant supports chef, puppet, cfengine, ansible, salt, docker, shell scripts • We’ll cover chef, shell and ansible here
  47. Provisioning • Provisioning happens: • The initial run of ‘vagrant

    up’ that creates the VM • When you run ‘vagrant provision’ • Older versions of vagrant (before 1.3.0) would provision every time the VM was booted.
  48. Provisioning • Provisioners: • Run inside the VM, as root

    • Require that the relevant tools be installed (e.g. chef/ puppet) • Some base boxes provide chef/puppet already
  49. Provisioning: shell • Often used in conjunction with other provisioners,

    e.g. to install the chef package • Useful if your provisioning needs are simple • Other provisioners are more powerful and you should use those • If you do use shell, you should make the script idempotent
  50. Provisioning: shell • Simple command, directly in Vagrantfile: ! config.vm.provision

    "shell", inline: "which python2 || (pacman -Syy; pacman -S --noconfirm python2)”
  51. Provisioning: shell • Installing chef: config.vm.provision “shell”, inline: "which chef-client

    || (curl -L https://www.opscode.com/chef/install.sh | bash)"
  52. Provisioning: shell • Run a shell script from a file

    config.vm.provision "shell", path: “myscript.sh" • The path is relative to the vagrant project directory
  53. Example - Shell provisioning

  54. Provisioning: chef solo • Place your cookbooks/roles inside the vagrant

    project directory: myproject/
 Vagrantfile
 cookbooks/
 users/
 apache/
 roles/
 base.rb
  55. Provisioning: chef solo • Add the run list to your

    vagrant file: config.vm.provision "chef_solo" do |chef|
 chef.roles_path “roles” # You need to specify this
 chef.add_role "base"
 end
  56. Provisioning: chef solo • You can also include recipes directly,

    and set node configuration: config.vm.provision "chef_solo" do |chef|
 chef.add_recipe(“users”)
 chef.json = {
 “users" => [
 {“name” => “mark”, “uid” => 1000, …}
 ]
 }
 end
  57. Example - Chef provisioning

  58. Provisioning - installing chef • You can use the shell

    provisioner • Alternative - vagrant-omnibus plugin • install with ‘vagrant plugin install vagrant-omnibus’ • Then add the following to your Vagrantfile: config.omnibus.chef_version = :latest
  59. Provisioning - chef client • This requires you have a

    chef server • Provisioning is similar to chef-solo, with some changes: config.vm.provision "chef_client" do |chef|
 chef.chef_server_url = “https://...”
 chef.validation_key_path = "validation.pem"
 end
  60. Provisioning - chef client • Vagrant doesn’t delete the chef

    client for you by default • To do this, add the following to your Vagrantfile config.vm.provision "chef_client" do |chef|
 …
 chef.delete_node = true
 chef.delete_client = true
 end
  61. Provisioning: ansible • Like with chef, you put your configuration

    in the vagrant directory: myproject/
 Vagrantfile
 site.yml
 roles/
 …
  62. Provisioning: ansible • Add the path to the playbook in

    the Vagrantfile: config.vm.provision "ansible" do |ansible|
 ansible.playbook = "site.yml"
 end • You don’t need to generate an inventory file, vagrant will create one for you.
  63. Example - Ansible provisioning

  64. Multi-VM Projects • A vagrant project doesn’t need to be

    just one VM, you can have multiple. • The usual example given is allowing you to have a web server and database server on separate machines. • But, this is a really useful for SAs in that you can easily test failover with two identical machines. • This is done using ‘config.vm.define’ in your Vagrantfile
  65. Multi-VM Projects Vagrant.configure("2") do |config| config.vm.box = “omnios-151006c-r1" config.vm.provision :shell,

    :path => "provision-common.sh" config.vm.define :src do |src| src.vm.hostname = "src" src.vm.provision :shell, :path => "provision-src.sh" src.vm.network :private_network, ip: "192.168.50.3" end config.vm.define :dst do |dst| dst.vm.hostname = "dst" dst.vm.network :private_network, ip: "192.168.50.4" end end
  66. Multi-VM projects • You need to append the VM name

    to most vagrant commands now: • vagrant up vmname • vagrant ssh vmname • vagrant destroy vmname • If you leave out the VM name, it will assume you want to work on all VMs (e.g. vagrant up brings up all VMs)
  67. Recap • What makes vagrant better than virtualbox alone •

    How to create and use a simple VM • Networking and shared folder configuration • Provisioning • Multiple VMs
  68. Bonus Section: Vagrant 1.6 • New release of vagrant in

    the next few weeks • Global status - show all your vagrant VMs • Global control - no need to cd into vagrant dir • Full windows guest support • Docker guest support (in a linux VM on OSX/Windows)
  69. Bonus Section: Packer Build your own boxes

  70. Building your own Boxes • You will nearly always use

    pre-built vagrant boxes • What if you need to build your own? • OS or OS version for which a box doesn’t exist already • Custom software preinstalled • Special configuration/disk layout
  71. What’s in a box • VM image with some special

    properties • ssh server running on boot • vagrant user with ssh key and sudo access • provisioning software (optionally) installed • guest additions/vmware tools • Metadata • vagrant package command
  72. Packer - automation • This process is error prone and

    tedious • Packer was created to automate the process • Uses a json template to build VMs • I’ll be walking through a simple example to make an Arch Linux box for Virtualbox.
  73. Packer - installation • Written in go, distributed as a

    zip file containing binaries • Download from packer.io • unzip to ~/packer (or elsewhere) • Mac Homebew - binary tap • brew tap homebrew/binary • brew install packer
  74. Packer - template.json From: https://github.com/mivok/arch_scripts/tree/master/packer { "builders": [ { "type":

    "virtualbox-iso", "guest_os_type": "ArchLinux", "iso_url": "http://mirror.umd.edu/archlinux/iso/2014.04.01/archlinux-2014.04.01-dual.iso", "iso_checksum": "ae8caf8cd356dfcc24a4b645fd815185", "iso_checksum_type": "md5", "http_directory": ".", "ssh_username": "root", "ssh_password": "vagrant",
  75. Packer - template.json "boot_command": [ "<enter><wait10><wait10>", "curl -O http://{{ .HTTPIP

    }}:{{ .HTTPPort }}/install.sh<enter><wait5>", "bash ./install.sh<enter>" ], "vboxmanage": [ ["modifyvm", "{{.Name}}", "--memory", "384"]] } ], "provisioners": [{ "type": "shell", "script": "provision.sh" }], "post-processors": [{"type": “vagrant", "compression_level": 9}] }
  76. Packer - template.json • 4 main sections: • description •

    builders - create the vm image • provisioners - install and configure additional software • post-processors - generate a box from the image
  77. Packer - builders • The ‘meat’ of the VM building

    • Specifies which VM software to use (virtualbox-iso) • Specifies VM parameters • Commands needed to install the OS • Finishes with a rebooted machine and SSH available
  78. Packer - builders {
 "type": "virtualbox-iso",
 "guest_os_type": "ArchLinux",
 "iso_url": "http://mirror.umd.edu/archlinux/iso/

    2014.04.01/archlinux-2014.04.01-dual.iso",
 "iso_checksum": "ae8caf8cd356dfcc24a4b645fd815185",
 "iso_checksum_type": "md5",
 "http_directory": “.",
 "ssh_username": "root",
 "ssh_password": "vagrant",

  79. Packer - builders "shutdown_command": "shutdown -h now",
 "boot_command": [
 "<enter><wait10><wait10>",


    "curl -O http://{{ .HTTPIP }}:{{ .HTTPPort }}/ install.sh<enter><wait5>",
 "bash ./install.sh<enter>"
 ],
 "vboxmanage": [
 ["modifyvm", "{{.Name}}", "--memory", "384"]
 ]
 }
  80. Packer - install script • See https://github.com/mivok/arch_scripts/blob/master/packer/ install.sh for specifics

    • Partitions drives • Installs packages • Sets up networking (dhcp) • Sets ssh to start on boot • Sets the root password • Reboots
  81. Packer - provisioners • Identical to vagrant • Can use

    chef, puppet, ansible, salt, shell "provisioners": [
 {
 "type": "shell",
 "script": "provision.sh"
 }
 ]
  82. Packer - provisioning tasks • Add a vagrant user •

    Add ssh key for vagrant user • Set up sudo • Disable root user (optional) • Install guest additions (packer drops an iso in place for you) • Install chef/puppet (optional)
  83. Packer - post processors "post-processors": [ { "type": "vagrant", "compression_level":

    9 } ]
  84. Packer - running • packer build [template.json] • Downloads the

    iso • Creates a new VM • Installs the OS • Provisions the machine • Creates a box
  85. Example - Packer

  86. Packer - recommendations • boot-command covers much more than booting

    • do as little as possible in boot-command • install a base system • get ssh access • use provisioners for the rest • use chef/puppet if you have it already • shell if you have simple needs
  87. Questions?