Infrastructure as code; it needs testing too. Martin B. Smith martin@mbs3.org

- Configuration management has been around for a while. - Your infrastructure as an application. - Automation, the killer app. A true app. - Version environments, speed up cycles. - Turn the focus back to the business. What is infrastructure as code?

Why does this matter?

- Bugs in (your) code are certain* - Automated does not mean tested - Business value of your ‘app’ - Tests are outcome focused* - Investment in quality - CI/CD, continuous testing or monitoring - CD is the new economy; customer service Why should you test infrastructure?

- Behavior driven development - Unit testing - Integration testing - Continuous all the things - Everyone loves DSLs - Focus on ruby tools, but extensible - API driven; how long is the build broken? Types of testing; tonight’s talk

- Dynamic type system, automatic memory, multiple programming paradigms (Imp, OO, Func, Dynamic) - Conceived 1994, released 1995, English in 1998/99, more popular than Python in Japan by 2000 - Bundler, Gemfile - rbenv, rvm, chruby - DSL friendly Ruby, toolchain overview

- Behavior driven development: Cucumber What do we mean by testing?

- Unit testing (speed, isolation) - Rspec, minitest, test-unit - Rspec with extensions for chef and puppet Does the app’s ‘code’ do what I expect? Do template contents look correct? What do we mean by testing?

#Testing for our User class describe User do context 'with admin privileges' do before :each do @admin = Admin.get(1) end it 'should exist' do expect(@admin).not_to be_nil end it 'should have a name' do expect(@admin.name).not_to be_false end end #... end Unit testing with rspec

require 'chefspec' describe 'example::default' do let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) } it 'installs foo' do expect(chef_run).to install_package('foo') end end Unit testing with chefspec - Completely compatible with most chef tools - Stub other node data and environment data - Mock objects, return results

execute "woot" do command "echo woot" action :nothing end cookbook_file "/tmp/dangerfile" do owner "root" mode "00644" notifies :run, "execute[woot]" end Chefspec demo (line cookbook)

require 'spec_helper' describe 'line::tester' do let(:chef_run) { ChefSpec::Runner.new.converge 'line::tester' } it 'creates dangerfile' do expect(chef_run).to create_cookbook_file('/tmp/dangerfile').with_owner('root'). with_mode('00644') end end Chefspec demo (line cookbook)

describe 'apache', :type => 'class' do context "On a Debian OS with no package name specified" do let :facts do { :osfamily => 'Debian' } end it { should contain_package('httpd').with( { 'name' => 'apache2' } ) should contain_service('httpd').with( { 'name' => 'apache2' } ) } end end Unit testing with puppet (labs_spec_helper)

context 'with compress => foo' do let(:params) { {:compress => 'foo'} } it do expect { should contain_file('/etc/logrotate.d/nginx') }.to raise_error(Puppet::Error, /compress must be true or false/) end end it do should contain_service('apache').with( 'ensure' => 'running', 'enable' => 'true', 'hasrestart' => 'true', ) end Unit testing with puppet (rspec-puppet)

- Integration testing - Serverspec, bats, rspec again Does the system reflect my instructions? Verify file contents, service settings, etc Isolate to a single host (requires cloudy) What do we mean by testing?

describe package('httpd'), :if => os[:family] == 'redhat' do it { should be_installed } end describe package('apache2'), :if => os[:family] == 'ubuntu' do it { should be_installed } end describe service('httpd'), :if => os[:family] == 'redhat' do it { should be_enabled } it { should be_running } end describe service('apache2'), :if => os[:family] == 'ubuntu' do it { should be_enabled } it { should be_running } end describe port(80) do it { should be_listening } end Integration testing tools: serverspec

@test "PEP8 tests for interface code" { run pep8 scripts/pontoon* [ "$status" = 0 ] } @test "monit is installed and in the path" { which monit } @test "monit configuration dir exists" { [ -d "/etc/monit" ] } Integration testing tools: bats

describe 'monit::default' do it "install monit" do assert system('apt-cache policy monit | grep Installed | grep -v none') end describe "services" do # You can assert that a service must be running following the converge: it "runs as a daemon" do assert system('/etc/init.d/monit status') end # And that it will start when the server boots: it "boots on startup" do assert File.exists?(Dir.glob("/etc/rc5.d/S*monit").first) end end end Integration testing tools: minitest

- Have tests, will travel (Vagrant) - Version your environment (TK) - Run all the things (rake) Vagrant, test-kitchen, rake

- Same machine, mock & stub everything def stub_resources stub_command('/usr/sbin/httpd -t').and_return(0) stub_command('/usr/sbin/apache2 -t').and_return(0) stub_command('which php').and_return('/usr/bin/php') end def stub_nodes(platform, version, server) Dir['./test/integration/nodes/*.json'].sort.each do |f| node_data = JSON.parse(IO.read(f), symbolize_names: false) node_name = node_data['name'] server.create_node(node_name, node_data) platform.to_s # pacify rubocop version.to_s # pacify rubocop end Dir['./test/integration/environments/*.json'].sort.each do |f| env_data = JSON.parse(IO.read(f), symbolize_names: false) env_name = env_data['name'] server.create_environment(env_name, env_data) end end Rake and unit tests

- Spawn a new machine (or container or cloud server or VM) --- driver: name: vagrant driver_config: use_vagrant_berkshelf_plugin: true provisioner: name: chef_solo platforms: - name: ubuntu-12.04 - name: debian-6.0.8 - name: centos-6.4 - name: fedora-20 TK and integration tests

- Spawn a new machine (or container or cloud server or VM) suites: - name: default run_list: - recipe[redisio::default] - recipe[redisio::enable] attributes: redisio: servers: [ { port: 6379, } ] - name: sentinel run_list: - recipe[redisio::default] - recipe[redisio::enable] - recipe[redisio::sentinel] - recipe[redisio::sentinel_enable] attributes: redisio: servers: [ { port: 6379, } ] TK and integration tests

- Examples of commands in test-kitchen using redisio cookbook Go cloud instead -- driver: rackspace driver_config: rackspace_username: rackspace_region: iad rackspace_api_key: require_chef_omnibus: true no_ssh_tcp_check: true no_ssh_tcp_check_sleep: 120 public_key_path: /home/mart6985/.ssh/id_rsa.pub flavor_id: performance1-2 TK and integration tests

- Drivers for ec2, rackspace, digital ocean, docker, virtualbox, vmware, etc - Configuration is YAML files - Multiple test suites with different input options / test-scenarios - Still stub your cfg mgmt environment Drivers, configs, options, suites

- redisio with vagrant and chef-solo - jenkins with chef-zero - (Your CI could run all these commands) Examples and provisioning

Provisioners, Bussers, chef, puppet - a furnisher of provisions - bootstraps your config mgmt of choice - chef (solo, zero) - puppet (apply, agent) - shell/bash kitchen converge

Provisioners, Bussers, chef, puppet - a person who clears tables in a restaurant or cafeteria - bootstraps tests and runs them - serverspec (rspec) - minitest - bats kitchen verify

Provisioners, Bussers, chef, puppet - TK supports Berkshelf and Librarian - TK can provision using bash - Think of the other use cases for this! kitchen test

The Holy Grail bundle install bundle exec rake style* bundle exec rake spec bundle exec kitchen test -d always (gem install rubygems-bundler)

The Holy Grail bundle install rake style rake spec kitchen test -d always

- Q/A - martin@mbs3.org Testing your own code... http://devopsreactions.tumblr.com /post/88260308392/ testing-my-own-code References, questions