Virtualize your stack with Vagrant and Puppet - Columbus Code Camp 2013 Edition

Virtualize Your Stack With Vagrant and Puppet Jacob Mather Software
Engineer at Mashery

Virtualize your environment with Vagrant and Puppet - Columbus Code
Camp - October 12th, 2013 Who am I? What I do: • Software Engineer at Mashery • Co-organizer of the San Francisco PHP User Group • Community Evangelist Where I can be found: • Blog: http://jmather.com • Twitter: @thejmather

Camp - October 12th, 2013 Why I am giving this talk

Let’s talk about Vagrant

Camp - October 12th, 2013 What is Vagrant? A tool that allows you to create and configure lightweight, reproducible, and portable development environments.

Camp - October 12th, 2013 Vagrant is built... • in Ruby • to be simple

Camp - October 12th, 2013 Vagrant terms • Box, or base box, is the ‘initial image’ of a virtual machine. • Host, or your computer, the machine running the VMs • Guest, or vm, referring to an individual VM instance • Provider, an adapter to a VMS (like VirtualBox or VMWare) • Provisioner, provides post-boot configuration of guests

Camp - October 12th, 2013 How to use Vagrant • Install Vagrant - http://vagrantup.com • Check out your repository • Run: vagrant up

Camp - October 12th, 2013 Basic Vagrant Commands • vagrant up - Turn guest on • vagrant halt - Turn guest off • vagrant status - Show guest status • vagrant destroy - Delete guest • vagrant suspend - Suspend guest • vagrant resume - Resume guest • vagrant ssh - SSH into guest

Let’s talk about code

Camp - October 12th, 2013 Basic Vagrantfile Vagrant.configure("2") do |config| config.vm.box = "base-box-name" config.vm.box_url = ”http://some/url.box" end

Camp - October 12th, 2013 Basic Vagrantfile (add file system) Vagrant.configure("2") do |config| config.vm.box = "base-box-name" config.vm.box_url = ”http://some/url.box" config.vm.synced_folder "/data", "/vagrant_data" end

Camp - October 12th, 2013 Basic Vagrantfile (add port forwarding) Vagrant.configure("2") do |config| config.vm.box = "base-box-name" config.vm.box_url = ”http://some/url.box" config.vm.synced_folder "/data", "/vagrant_data" config.vm.network :forwarded_port, guest:80, host: 8080 end

Let’s talk about clouds

Camp - October 12th, 2013 Replicating production matters If you are writing code that will be deployed to a multiple-host environment, why aren’t you writing your code in a multiple- host environment?

Camp - October 12th, 2013 Basic Vagrantfile Vagrant.configure("2") do |config| config.vm.hostname = “box” config.vm.box = "base-box-name" config.vm.box_url = ”http://some/url.box" end

Camp - October 12th, 2013 Basic Vagrantfile (Multi-Guest ready) Vagrant.configure("2") do |config| config.vm.define “box” do |node| node.vm.hostname = “box” node.vm.box = "base-box-name" node.vm.box_url = ”http://some/url.box" end end

Camp - October 12th, 2013 Basic Vagrantfile (Multi-Guest) Vagrant.configure("2") do |config| config.vm.define “box” do |node| node.vm.hostname = “box” node.vm.box = "base-box-name" node.vm.box_url = ”http://some/url.box" end config.vm.define “other-box” do |node| node.vm.hostname = “other-box” node.vm.box = "base-box-name" node.vm.box_url = ”http://some/url.box" end end

Camp - October 12th, 2013 Basic Vagrantfile (Multi-Guest, useful, part 1) nodes = { :'web-server' => { :hostname => 'server.example.com', :ipaddress => '192.168.56.60', }, :'db-server' => { :hostname => 'db.example.com', :ipaddress => '192.168.56.61', } }

Camp - October 12th, 2013 Basic Vagrantfile (Multi-Guest, useful, part 2) nodes = { … node configuration … } Vagrant.configure("2") do |config| nodes.each_pair do |name,options| config.vm.define name do |node| node.vm.box = "base-box-name" node.vm.box_url = "http://some/url.box" node.vm.hostname = options[:hostname] node.vm.network :private_network, ip: options[:ipaddress] end end end

Let’s talk about Providers

Camp - October 12th, 2013 Vagrant Providers Providers Vagrant

Camp - October 12th, 2013 Vagrant Providers VirtualBox VMWare (WS or Fusion) Parallels Desktop Providers Vagrant

Camp - October 12th, 2013 Vagrant Providers VirtualBox VMWare (WS or Fusion) Parallels Desktop Amazon AWS Digital Ocean Providers Vagrant

Camp - October 12th, 2013 Vagrant Providers (desktop) • VirtualBox is FREE! (sort of) • Some people have trouble with crashes • Some people have trouble with suspend/resume • Built in FS bridging is pretty slow • VMWare (Workstation or Fusion!) is PAID! • You have to buy both VMWare ($49) and provider ($79) • Supposed to be more stable and faster • Parallels Desktop is FREE! • Open source projects FOR THE WIN!

Camp - October 12th, 2013 Vagrant Providers (cloud) • Amazon AWS • Digital Ocean • RackSpace Cloud

Camp - October 12th, 2013 Vagrant Provider Limits Spinning up guests is cool, but what do I do with a box once it’s up? How do I control it? How do I make it useful? Computing hardware without software is nearly useless.

Let’s talk about Provisioners

Camp - October 12th, 2013 Vagrant Provisioners Provisioners Vagrant

Shell Scripts Puppet Chef Ansible Virtualize your environment with Vagrant
and Puppet - Columbus Code Camp - October 12th, 2013 Vagrant Provisioners Provisioners Vagrant

Let’s talk about Puppet

Camp - October 12th, 2013 What is Puppet? Puppet is IT automation software that helps system administrators manage infrastructure throughout its lifecycle, from provisioning and configuration to patch management and compliance.

Camp - October 12th, 2013 What?

Camp - October 12th, 2013 What is Puppet in simpler words? Puppet gives you an object oriented way to manage the configuration of your servers, either on your laptop, in the cloud, or in your favorite data center.

Camp - October 12th, 2013 Puppet is built... • to be simple • with it’s own declarative language

Camp - October 12th, 2013 Puppet terms • Manifests - This is “application” code • Modules - This is “library” code • Templates - Exactly as it sounds • Facter - Environment data • Hiera - Configuration data

Camp - October 12th, 2013 How to use Puppet? Select a base box for Vagrant from Puppet Labs http://puppet-vagrant-boxes.puppetlabs.com/ I tend to use CentOS 6.4 for VirtualBox

Let’s talk about code

Camp - October 12th, 2013 Basic Vagrantfile (add Puppet) Vagrant.configure("2") do |config| config.vm.box = "base-box-name" config.vm.box_url = "http://some/url.box" config.vm.provision :puppet do |puppet| puppet.manifests_path = "puppet/manifests/" puppet.manifest_file = "guest.pp" puppet.module_path = "puppet/modules/" end end

Camp - October 12th, 2013 Basic Puppet Manifest (guest.pp) class guest { notify { “Look! My puppet code is working!”: } }

Camp - October 12th, 2013 Basic Puppet Manifest (add Apache) class guest { notify { “Look! My puppet code is working!”: } package { “httpd”: ensure => present, } service { “httpd”: require => Package[“httpd”], ensure => running, enabled => true, } }

Camp - October 12th, 2013 Basic Puppet Manifest (refactored Apache) # manifests/guest.pp class guest { notify { “Code working!”: } require server::httpd } # modules/server/httpd.pp class server::httpd { package { “httpd”: ensure => present, } service { “httpd”: require => Package[“httpd”], ensure => running, enabled => true, } }

Camp - October 12th, 2013 Puppet Manifest (controlling flow) class guest { notify { “Say me first”: } notify { “Say me second”: require => Notify[“Say me first”], } notify { “Say me third”: require => Notify[“Say me second”], } }

Camp - October 12th, 2013 Puppet Manifest (adding parameters) # modules/server/writetestfile.pp class server::writetestfile ($content) { file { “/tmp/test”: content => $content, } } # modules/server/writetestfiledefault.pp class server::writetestfiledefault ($content = “default”) { file { “/tmp/test”: content => $content } }

Camp - October 12th, 2013 Puppet Manifest (passing parameters) # manifests/guest.pp -- using defaults class guest { include server::writetestfiledefault } # manifests/guest.pp -- passing custom value class guest { class { “server::writetestfiledefault”: content => “test content”, } }

Camp - October 12th, 2013 Puppet Manifest (including classes) class guest { # the class below must be loaded BEFORE guest require server::writetestfiledefault # the class below must be loaded WITH guest include server::writetestfiledefault # the class below must be loaded AFTER guest class { “server::writetestfiledefault”: content => “test content”, } }

Let’s talk about how to do things conditionally

Camp - October 12th, 2013 External Input in Puppet Remember there are three primary sources of where data that branches your logic will come from: • User Space Variables • $things = “you set yourself” • Facts • Specific details about your current environment, such as the name of the host, ip address, operating system, etc... • Hiera • Configuration you pass in to enable building for multiple configurations from the same shared codebase

Camp - October 12th, 2013 Puppet Manifest (variables) class guest { # local variable $host = “mybox” if ($host == “mybox”) { notify { “Box!”: } } # Box! if ($host == “mybox”) { notify { “${host}!”: } } # mybox! }

Camp - October 12th, 2013 Puppet Manifest (facts) class guest { # fact - environment configuration if ($::hostname == “box”) { notify { “Box!”: } } # output: Box! if ($hostname == “box”) { notify { “Box!”: } } # output: Box! # assign $hostname locally... $hostname = “some string” if ($::hostname == “box”) { notify { “Box!”: } } # output: Box! if ($hostname == “box”) { notify { “Box!”: } } # no output }

Camp - October 12th, 2013 Puppet Manifest (facts, debugging) class guest { # what facts are there? Let’s find out! # this should be one line, but it is just too long to fit $template = "<%= scope.to_hash.reject { |k,v| !( k.is_a? (String) && v.is_a?(String) ) }.to_yaml %>" # create a file... file { "/tmp/facts.yaml": content => inline_template($template), } }

Camp - October 12th, 2013 Puppet Manifest (if conditionals) class guest { if (“a” == “b”) { notify { “Inconceivable!”: } } elsif (“hello” =~ “help”) { notify { “Regex!”: } } else { notify { “Echo something...”: } } }

Camp - October 12th, 2013 Puppet Manifest (case conditionals) class guest { # $::operatingsystem is a fact, provided directly by Facter case $::operatingsystem { centos, redhat: { $apache = "httpd" } debian, ubuntu: { $apache = "apache2" } default: { fail("Unrecognized operating system") } } case $::operatingsystem { /linux/: { fail(“We don’t support anything with Linux in it”) } } }

Camp - October 12th, 2013 Puppet Manifest (selector conditionals) class guest { # $::operatingsystem is a fact, provided directly by Facter $package = $::operatingsystem ? { centos => “httpd”, redhat => “httpd”, /(i?)(ubuntu|debian)/ => “apache2”, default => undef } }

Let’s talk about real multiple guest configuration

Camp - October 12th, 2013 Configure our guest nodes... nodes = { :'web-server' => { :hostname => 'server.example.com', :ipaddress => '192.168.56.60', }, :'db-server' => { :hostname => 'db.example.com', :ipaddress => '192.168.56.61', } }

Camp - October 12th, 2013 Tell Vagrant about them nodes = { … node configuration … } Vagrant.configure("2") do |config| nodes.each_pair do |name,options| config.vm.define name do |node| node.vm.box = "base-box-name" node.vm.box_url = "http://some/url.box" node.vm.hostname = options[:hostname] node.vm.network :private_network, ip: options[:ipaddress] end end end

Camp - October 12th, 2013 Tell Vagrant about Puppet nodes = { … node configuration … } Vagrant.configure("2") do |config| nodes.each_pair do |name,options| config.vm.define name do |node| node.vm... # omit general config node.vm.provision :puppet do |puppet| puppet.manifests_path = "puppet/manifests/" puppet.manifest_file = "guests.pp" puppet.module_path = "puppet/modules/" end end end end

Camp - October 12th, 2013 Tell Puppet about the guest nodes class guests { if ($::hostname == “web-server”) { # only run for web server } if ($::hostname == “db-server”) { # only run for db server } }

Camp - October 12th, 2013 Tell Puppet about the guest nodes (redux) class guests { if ($::hostname =~ /^web-/) { # only run for web servers } if ($::hostname =~ /^db-/) { # only run for db servers } }

Let’s talk about how everything ties together

Shell Scripts Puppet Chef Ansible VirtualBox VMWare (WS or Fusion)
Parallels Desktop Amazon AWS Digital Ocean Virtualize your environment with Vagrant and Puppet - Columbus Code Camp - October 12th, 2013 Vagrant Architecture Providers Provisioners Vagrant

Let’s talk about how to convince your boss

Camp - October 12th, 2013 Development environments come in all shapes and sizes • Some develop in production – Yes, really. • Some develop on a non production server – Not a copy of production, not by a long shot • Some develop locally on a “non production server” – Home grown solutions that “work” until they don’t • Some develop on a development server – Actually tries to be like production – Not always like production – May have additional tuning to help with developmenty things • Some develop in a true development environment – Locally or remote, coming soon to a computer near you

Camp - October 12th, 2013 Why developing in production is really bad Assuming, of course, that you are testing your code before you make it live. Production should always move from one stable version of code to another stable version of code. When you edit code in production, you inherently introduce instability, and can no longer be assured that your code is running as tested.

Camp - October 12th, 2013 Why developing on a non production setup is also bad Debugging environmental issues is slow and painful. Developers should be solving problems with software, not debugging configuration issues. When you build your code in an environment that isn’t related to production in any tangible way, you will often discover just how different your development environment is from production.

Camp - October 12th, 2013 Where your development environment lives Remote • Easy for operations to set up • Easy to enforce a rigid structure • Limits developer’s choice of tool chain selection • Can get expensive and complicated to maintain • Developers must be online to do any development Local • Can require beefier hardware • Can take extra work to set up • Developers have the freedom to work how they work best • Developers can assist with infrastructural initiatives

Camp - October 12th, 2013 Bad development environments will sap the resources out of your team. In addition to the time spent developing and testing new code and fixing bugs, you will spend time: When development environments attack Debugging broken environments or settings Debugging broken behavior between dev and prod Debugging broken behavior between two dev systems Working in broken ways because “it works”

Camp - October 12th, 2013 Failure to specifically build an environment for development will directly impact developer productivity and product quality.

Let’s talk about what you should remember

Camp - October 12th, 2013 Takeaways • Bad (or unstable) environments (be it dev, qa, staging, or prod) hurt developers. • Puppet lets you automate all of the configuration of your systems (dev, qa, staging, and prod), ensuring consistency between them. • Vagrant is easy to use, and will let you use your production puppet configs to build development environments. • You will manage server configuration, whether you decide to or not. If you choose not to do it formally it will come back to bite you when you forget something later.

Let’s talk about where to go from here

Camp - October 12th, 2013 If you’re still not sold... h"p://PuPHPet.com/ / PuPHPet/provides/a/ simple/conﬁg/tool/for/ those/who/don’t/want/ to/=nker/with/puppet/ and/need/a/LAMP/stack/ up/and/running/NOW./ V2 was released this morning!!!

Camp - October 12th, 2013 And also look at... • Packer • http://packer.io • Take your Vagrant and Puppet usage to the next level by using Packer to pre-provision images for easy first spins • Build prebuilt systems for multiple VMS and cloud platforms • Docker • http://docker.io • A different idea of how to build multiple machine VM environments

Camp - October 12th, 2013 Thank you! More information about this talk can be found at: http://jmather.com/talks/2013-10-12

Virtualize your stack with Vagrant and Puppet -...

Virtualize your stack with Vagrant and Puppet - Columbus Code Camp 2013 Edition

More Decks by Jacob Mather

Other Decks in Programming

Featured

Transcript