Talk from #puppetconf 2014 all about moving towards infrastructure as code and why policy driven development is a cool idea.
Continuously TestingInfrastructurePuppet Conf, San Francisco, 2014Gareth RushgroveBeyond Module Testing
View Slide
@garethr
Gareth Rushgrove
Not talking about
Finished softwareGareth Rushgrove
Testing individual modulesGareth Rushgrove
puppet-lint, puppet-syntax,rspec-puppet, beakerGareth Rushgrove
Am talking about
ExperimentsGareth Rushgrove
Testing images andcontainersGareth Rushgrove
Test driving infrastructureas a serviceGareth Rushgrove
Testing with PuppetDBGareth Rushgrove
Testingimages andcontainers1
Packer builds imagesbased on a JSONtemplateGareth Rushgrove
It has some Puppetintegration tooGareth Rushgrove
But how do we know theimage works?Gareth Rushgrove
Lets add some tests!Gareth Rushgrove
shaunduncan/packer-provisioner-host-commandGareth Rushgrove
serverspec.orgGareth Rushgrove
Serverspec also supportsport, file, ppa, selinux,user, group, lxc, iptables,cron and moreGareth Rushgrove
Only publish the image ifthe tests passGareth Rushgrove
Run tests automaticallywith a continuousintegration systemGareth Rushgrove
garethr/packer-serverspec-exampleGareth Rushgrove
Same approach workswith containers tooGareth Rushgrove
garethr/docker-spec-exampleGareth Rushgrove
Test driveyour IaaS2
Test driven developmentGareth Rushgrove
First the developer writesan automated test casethat defines a desiredimprovement or newfunctionGareth Rushgrove
Then produces theminimum amount of codeto pass that testGareth Rushgrove
And finally refactors thenew codeGareth Rushgrove
Gareth RushgroveFirst the developer writesan automated test casethat defines a desiredimprovement or newfunction
Your infrastructure should!have an APIGareth Rushgrove
What if we writeassertions against!that API?Gareth Rushgrove
Aside: Clojure2.1
Great for building DSLsGareth Rushgrove
Don’t worry, you couldwrite the examples in anylanguageGareth Rushgrove
Policy driven developmentGareth Rushgrove
I don’t want to launch toomany nodes, they’reexpensiveGareth RushgrovePolicy
I don’t want any stoppednodes, they are costingme moneyGareth RushgrovePolicy
Large nodes are reallyexpensive, so limit theirusageGareth RushgrovePolicy
We should be backing upevery nodeGareth RushgrovePolicy
I only want nodes inLondon and !San FranciscoGareth RushgrovePolicy
All our nodes should benamed environment-nameGareth RushgrovePolicy
garethr/digitalocean-expectGareth Rushgrove
Now we have the tests,we can provision someinfrastructureGareth Rushgrove
Aside: Provisioningwith Puppet2.2
puppetlabs/gce_computeGareth Rushgrove
garethr/digitaloceanGareth Rushgrove
bobtfish/aws_apiGareth Rushgrove
Testing withPuppetDB3
Aside: PuppetDB3.1
puppetlabs/puppetdbGareth Rushgrove
PuppetDB can store a lotof data about yourinfrastructureGareth Rushgrove
The most recent factsfrom every nodeGareth Rushgrove
The most recent catalogfor every nodeGareth Rushgrove
A wide range of metricsGareth Rushgrove
I want to run the sameoperating system on allhostsGareth RushgrovePolicy
Security enforcingpackages should beinstalled everywhereGareth RushgrovePolicy
I want to limit how manypuppet resources I’musingGareth RushgrovePolicy
We should avoid heavy I/O load on the database bymaintaining a high catalogduplication rateGareth RushgrovePolicy
garethr/puppetdb-expectGareth Rushgrove
Testing based onPuppetDB3.2
PuppetDB is a greatsource of context for testsGareth Rushgrove
Generate serverspec testsfrom PuppetDB dataGareth Rushgrove
Automatically detecthosts, and generatecommandsGareth Rushgrove
Match puppet resources toserverspec resourcesGareth Rushgrove
For instance on a PuppetEnterprise masterGareth Rushgrove
Run serverspec tests onall puppet managed hostsGareth Rushgrove
garethr/serverspec-puppetdbGareth Rushgrove
Conclusions
Is this monitoring?Gareth Rushgrove
We’re still moving towardsinfrastructure as codeGareth Rushgrove
Infrastructure as coderather than infrastructurefrom codeGareth Rushgrove
Taking about policy ascode might helpcommunicate intentGareth Rushgrove
Questions?And thanks for listening