$30 off During Our Annual Pro Sale. View Details »

Infrastructure code needs testing too

Martin Smith
November 18, 2014
160

Infrastructure code needs testing too

Martin Smith

November 18, 2014
Tweet

Transcript

  1. Infrastructure as code;
    it needs testing too.
    Martin B. Smith
    [email protected]

    View Slide

  2. - Configuration management has been
    around for a while.
    - Your infrastructure as an application.
    - Automation, the killer app. A true app.
    - Version environments, speed up cycles.
    - Turn the focus back to the business.
    What is infrastructure as code?

    View Slide

  3. Why does this matter?

    View Slide

  4. - Bugs in (your) code are certain*
    - Automated does not mean tested
    - Business value of your ‘app’
    - Tests are outcome focused*
    - Investment in quality
    - CI/CD, continuous testing or monitoring
    - CD is the new economy; customer service
    Why should you test infrastructure?

    View Slide

  5. - Behavior driven development
    - Unit testing
    - Integration testing
    - Continuous all the things
    - Everyone loves DSLs
    - Focus on ruby tools, but extensible
    - API driven; how long is the build broken?
    Types of testing; tonight’s talk

    View Slide

  6. - Dynamic type system, automatic memory, multiple
    programming paradigms (Imp, OO, Func, Dynamic)
    - Conceived 1994, released 1995, English in 1998/99,
    more popular than Python in Japan by 2000
    - Bundler, Gemfile
    - rbenv, rvm, chruby
    - DSL friendly
    Ruby, toolchain overview

    View Slide

  7. - Behavior driven development: Cucumber
    What do we mean by testing?

    View Slide

  8. - Unit testing (speed, isolation)
    - Rspec, minitest, test-unit
    - Rspec with extensions for chef and
    puppet
    Does the app’s ‘code’ do what I expect?
    Do template contents look correct?
    What do we mean by testing?

    View Slide

  9. #Testing for our User class
    describe User do
    context 'with admin privileges' do
    before :each do
    @admin = Admin.get(1)
    end
    it 'should exist' do
    expect(@admin).not_to be_nil
    end
    it 'should have a name' do
    expect(@admin.name).not_to be_false
    end
    end
    #...
    end
    Unit testing with rspec

    View Slide

  10. require 'chefspec'
    describe 'example::default' do
    let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) }
    it 'installs foo' do
    expect(chef_run).to install_package('foo')
    end
    end
    Unit testing with chefspec
    - Completely compatible with most chef tools
    - Stub other node data and environment data
    - Mock objects, return results

    View Slide

  11. execute "woot" do
    command "echo woot"
    action :nothing
    end
    cookbook_file "/tmp/dangerfile" do
    owner "root"
    mode "00644"
    notifies :run, "execute[woot]"
    end
    Chefspec demo (line cookbook)

    View Slide

  12. require 'spec_helper'
    describe 'line::tester' do
    let(:chef_run) { ChefSpec::Runner.new.converge 'line::tester' }
    it 'creates dangerfile' do
    expect(chef_run).to create_cookbook_file('/tmp/dangerfile').with_owner('root').
    with_mode('00644')
    end
    end
    Chefspec demo (line cookbook)

    View Slide

  13. describe 'apache', :type => 'class' do
    context "On a Debian OS with no package name specified" do
    let :facts do
    {
    :osfamily => 'Debian'
    }
    end
    it {
    should contain_package('httpd').with( { 'name' => 'apache2' } )
    should contain_service('httpd').with( { 'name' => 'apache2' } )
    }
    end
    end
    Unit testing with puppet (labs_spec_helper)

    View Slide

  14. context 'with compress => foo' do
    let(:params) { {:compress => 'foo'} }
    it do
    expect {
    should contain_file('/etc/logrotate.d/nginx')
    }.to raise_error(Puppet::Error, /compress must be true or false/)
    end
    end
    it do
    should contain_service('apache').with(
    'ensure' => 'running',
    'enable' => 'true',
    'hasrestart' => 'true',
    )
    end
    Unit testing with puppet (rspec-puppet)

    View Slide

  15. - Integration testing
    - Serverspec, bats, rspec again
    Does the system reflect my instructions?
    Verify file contents, service settings, etc
    Isolate to a single host (requires cloudy)
    What do we mean by testing?

    View Slide

  16. describe package('httpd'), :if => os[:family] == 'redhat' do
    it { should be_installed }
    end
    describe package('apache2'), :if => os[:family] == 'ubuntu' do
    it { should be_installed }
    end
    describe service('httpd'), :if => os[:family] == 'redhat' do
    it { should be_enabled }
    it { should be_running }
    end
    describe service('apache2'), :if => os[:family] == 'ubuntu' do
    it { should be_enabled }
    it { should be_running }
    end
    describe port(80) do
    it { should be_listening }
    end
    Integration testing tools: serverspec

    View Slide

  17. @test "PEP8 tests for interface code" {
    run pep8 scripts/pontoon*
    [ "$status" = 0 ]
    }
    @test "monit is installed and in the path" {
    which monit
    }
    @test "monit configuration dir exists" {
    [ -d "/etc/monit" ]
    }
    Integration testing tools: bats

    View Slide

  18. describe 'monit::default' do
    it "install monit" do
    assert system('apt-cache policy monit | grep Installed | grep -v none')
    end
    describe "services" do
    # You can assert that a service must be running following the converge:
    it "runs as a daemon" do
    assert system('/etc/init.d/monit status')
    end
    # And that it will start when the server boots:
    it "boots on startup" do
    assert File.exists?(Dir.glob("/etc/rc5.d/S*monit").first)
    end
    end
    end
    Integration testing tools: minitest

    View Slide

  19. - Have tests, will travel (Vagrant)
    - Version your environment (TK)
    - Run all the things (rake)
    Vagrant, test-kitchen, rake

    View Slide

  20. - Same machine, mock & stub everything
    def stub_resources
    stub_command('/usr/sbin/httpd -t').and_return(0)
    stub_command('/usr/sbin/apache2 -t').and_return(0)
    stub_command('which php').and_return('/usr/bin/php')
    end
    def stub_nodes(platform, version, server)
    Dir['./test/integration/nodes/*.json'].sort.each do |f|
    node_data = JSON.parse(IO.read(f), symbolize_names: false)
    node_name = node_data['name']
    server.create_node(node_name, node_data)
    platform.to_s # pacify rubocop
    version.to_s # pacify rubocop
    end
    Dir['./test/integration/environments/*.json'].sort.each do |f|
    env_data = JSON.parse(IO.read(f), symbolize_names: false)
    env_name = env_data['name']
    server.create_environment(env_name, env_data)
    end
    end
    Rake and unit tests

    View Slide

  21. - Spawn a new machine (or container or cloud server or VM)
    ---
    driver:
    name: vagrant
    driver_config:
    use_vagrant_berkshelf_plugin: true
    provisioner:
    name: chef_solo
    platforms:
    - name: ubuntu-12.04
    - name: debian-6.0.8
    - name: centos-6.4
    - name: fedora-20
    TK and integration tests

    View Slide

  22. - Spawn a new machine (or container or cloud server or VM)
    suites:
    - name: default
    run_list:
    - recipe[redisio::default]
    - recipe[redisio::enable]
    attributes:
    redisio:
    servers: [ { port: 6379, } ]
    - name: sentinel
    run_list:
    - recipe[redisio::default]
    - recipe[redisio::enable]
    - recipe[redisio::sentinel]
    - recipe[redisio::sentinel_enable]
    attributes:
    redisio:
    servers: [ { port: 6379, } ]
    TK and integration tests

    View Slide

  23. - Examples of commands in test-kitchen using redisio cookbook
    Go cloud instead --
    driver: rackspace
    driver_config:
    rackspace_username:
    rackspace_region: iad
    rackspace_api_key:
    require_chef_omnibus: true
    no_ssh_tcp_check: true
    no_ssh_tcp_check_sleep: 120
    public_key_path: /home/mart6985/.ssh/id_rsa.pub
    flavor_id: performance1-2
    TK and integration tests

    View Slide

  24. - Drivers for ec2, rackspace, digital ocean,
    docker, virtualbox, vmware, etc
    - Configuration is YAML files
    - Multiple test suites with different input
    options / test-scenarios
    - Still stub your cfg mgmt environment
    Drivers, configs, options, suites

    View Slide

  25. - redisio with vagrant and chef-solo
    - jenkins with chef-zero
    - (Your CI could run all these commands)
    Examples and provisioning

    View Slide

  26. Provisioners, Bussers, chef, puppet
    - a furnisher of provisions
    - bootstraps your config mgmt of choice
    - chef (solo, zero)
    - puppet (apply, agent)
    - shell/bash
    kitchen converge

    View Slide

  27. Provisioners, Bussers, chef, puppet
    - a person who clears tables in a
    restaurant or cafeteria
    - bootstraps tests and runs them
    - serverspec (rspec)
    - minitest
    - bats
    kitchen verify

    View Slide

  28. Provisioners, Bussers, chef, puppet
    - TK supports Berkshelf and Librarian
    - TK can provision using bash
    - Think of the other use cases for this!
    kitchen test

    View Slide

  29. The Holy Grail
    bundle install
    bundle exec rake style*
    bundle exec rake spec
    bundle exec kitchen test -d always
    (gem install rubygems-bundler)

    View Slide

  30. The Holy Grail
    bundle install
    rake style
    rake spec
    kitchen test -d always

    View Slide

  31. - Q/A
    - [email protected]
    Testing your own code...
    http://devopsreactions.tumblr.com /post/88260308392/ testing-my-own-code
    References, questions

    View Slide