This talk was given at PuppetConf 2012. It covers how I use Vagrant with Puppet and how I automate a lot of what I do with Puppet.
Advanced VagrantUsage with Puppet
View Slide
I’m Mitchell HashimotoAlso known as @mitchellh
I made Vagrant.Hopefully you use it. I think you’ll like it.http://vagrantup.com
I’m an automation freak.This talk will show this to be true.
Vagrant Usage(Ops Focused)
Benefits we wantfrom Vagrant...
“The Cloud”but on your machine.
Self service.Instant provisioning.Cost efficient.Elastic.Pay per use.Paul Strong’s Cloud
- Manifest development,both simple and not so simpleBenefits We Want
- Manifest development,both simple and not so simple- RepeatabilityBenefits We Want
- Manifest development,both simple and not so simple- Repeatability- Fast feedbackBenefits We Want
- Manifest development,both simple and not so simple- Repeatability- Fast feedback- ConfidenceBenefits We Want
Confession: I’ve beendoing Puppet full time forawhile now.
Current state ofVagrant + Puppet...
Basic manifestdevelopment and testing.
... Yep.
We can do better.We can do much better.
We can do better withwhat is available right now.
Teaser: We will do magic withwhat is coming in the future.<3 <3 <3
My state ofVagrant + Puppet...
Fully automatedPuppet Master setup.
Testing exportedresources, hiera, and nodes.
Common deploy processacross Vagrant and EC2.
Repeatable workflow ofdev to staging to prod.
Golden master boxcreation for development.
Time to sharewhat I’ve learned.
AdvancedVagrant Usage
AdvancedAutomation for Puppet Work
Fully AutomatedPuppet Master Setup
I asked: “How do peoplebring up or recover aPuppet master?”
“Most people roll theirPuppet Master by hand.”- Anonymous PuppetLabs Employee
Puppet Master is crucial totesting realistic scenarios.
Automated Puppet Master- Pushing broken Puppetcrashes the server.
Automated Puppet Master- Pushing broken Puppetcrashes the server.- Local development against aPuppet Master has benefits.
Automated Puppet Master- Pushing broken Puppetcrashes the server.- Local development against aPuppet Master has benefits.- Automation all the way down.
Multi-level bootstrap.
1. Bash script to minimallyinstall Puppet master andagent.
2. puppet apply tominimally setup PuppetMaster infrastructure
3. puppet agent tocompletely setup andharden the master.
Result: Production-qualityPuppet Master wheneveryou need it.
Vagrant::Config.run do |config|# ...config.vm.hostname = "puppet"config.vm.provision :shell, :path =>"bootstrap.sh"end
TestingExported Resources,Hiera, and nodes.
I asked: “How do peopletest more than the mostbasic Puppet module?”
“I suspect the answer isthat they just don't testtheir modules adequately.”- Anonymous PuppetLabs Employee
Solution: AutomatedPuppet Master + Multi-VM
Automated Puppet Masteris production ready:PuppetDB, Hiera, etc.
Multi-VM enables Vagrantto manage a cluster ofmachines that cancommunicate.http://vagrantup.com/v1/docs/multivm.html
Testing Exported Resources
Create two nodes.Export one.Collect other.Ruby/Shell script.Testing Exported Resources
node 'test_exporter' {@@nginx::site { "test":content => "\n",tag => "origin",}}node 'test_collector' {include role::origin}
Vagrant::Config.run do |config|config.vm.define :export do |n|n.vm.hostname = "test_exporter"n.vm.provision :puppet_server,:options => "--verbose --debug"endconfig.vm.define :collect do |n|n.vm.hostname = "test_collecter"n.vm.provision :puppet_server,:options => "--verbose --debug"endend
#!/bin/bashtest -f /etc/nginx/sites-available/test
Testing Hiera
Create full Hiera hierarchy.Launch node.Test hierarchy.
---test_region: "us-east-1"---test_role: "hiera"---test_name: "test-hiera”
Vagrant::Config.run do |config|config.vm.hostname = "test_hiera"config.vm.provision :puppet_server,:options => "--verbose --debug"endend
node 'test_hiera' {$region = hiera("test_region")$role = hiera("test_role")$name = hiera("test_name")file { "/tmp/results":content => "$region $role $name",mode => "0644",}}
Testing Nodes
Create node.Provision.Test behavior.
Vagrant::Config.run do |config|config.vm.define :master do |master|master.vm.hostname = "puppet"master.vm.provision :shell, :path =>"bootstrap.sh"endconfig.vm.define :node do |node|node.vm.hostname = "postgresql"node.vm.provision :puppet_server,:options => "--verbose --debug"endend
Pain points: Nodedestroy/up requires certclean on master plus aPuppetDB deactivate.
Common DeployProcess AcrossVagrant and EC2
Getting your Puppet codeto your masters. Solved?
Goal: Make it the same forVagrant, production, andanything in between.
My solution: Bash scriptto git pull, rsync, andrestart the master.
fab deploy:vagrantfab deploy:productionfab deploy:dev-mitchellhhttp://fabfile.org
@taskdef deploy(environment):# ...run("sudo /opt/puppet-updater/update")
- Git pull- Find env-* branches forenvironments.- RSync- Restart Puppet Masterhttp://bit.ly/Qyg3RWUpdater Script
Note: I don’t use Puppetenvironments for devbecause I like to keepproduction master just forproduction.
(Plus, the automatedPuppet Master setup isjust so easy!)
Repeatable WorkflowFrom Dev to Stagingto Production
Dev is in VirtualBox.Staging is in EC2.Production is in EC2.
Goal: Same workflow.
rake launch:vagrant,postgresqlrake destroy:production,riak-001rake provision:staging,haproxy-002
Wrapper around vagrantand AWS library.
Hides some cruft: destroywill cert clean anddeactivate fromPuppetDB, for example.
Golden Master BoxCreation forDevelopment
vagrant up a completedev environment can beslow.
Take advantage ofPuppet’s idempotenceand vagrant package
Two-pass Puppet runfor development.
Pass 1 (pre-package):Installation andconfiguration.
Pass 2 (vagrant up):Service starting andmaybe configuration.
vagrant package takescurrent Vagrant VM andproduces a distributablebox.
Build discipline aroundupdating the base box.
Example: Work on anybox you want, update tolatest base box prior tocommitting.
Bonus points:Put this in a CI.
AutomateAll the things
THANKS!@mitchellh
mitchellh
itohiro73
kubode
shutooike
speed1313
suzuki
paper2
furukawayasuto
gpsnmeajp
10xinc
zoetrope
manfredsteyer
niftycorp
searls
mojombo
phodgson
mza
maggiecrowley
dougneiner
aki_iinuma
3n
matthewcrist
danielanewman
bkeepers
62gerente