Infrastructure code needs testing too

Transcript

1. Infrastructure as code;
   it needs testing too.
   Martin B. Smith
   martin@mbs3.org
   

   View Slide

2. - Configuration management has been
   around for a while.
   - Your infrastructure as an application.
   - Automation, the killer app. A true app.
   - Version environments, speed up cycles.
   - Turn the focus back to the business.
   What is infrastructure as code?
   

   View Slide

3. Why does this matter?
   

   View Slide

4. - Bugs in (your) code are certain*
   - Automated does not mean tested
   - Business value of your ‘app’
   - Tests are outcome focused*
   - Investment in quality
   - CI/CD, continuous testing or monitoring
   - CD is the new economy; customer service
   Why should you test infrastructure?
   

   View Slide

5. - Behavior driven development
   - Unit testing
   - Integration testing
   - Continuous all the things
   - Everyone loves DSLs
   - Focus on ruby tools, but extensible
   - API driven; how long is the build broken?
   Types of testing; tonight’s talk
   

   View Slide

6. - Dynamic type system, automatic memory, multiple
   programming paradigms (Imp, OO, Func, Dynamic)
   - Conceived 1994, released 1995, English in 1998/99,
   more popular than Python in Japan by 2000
   - Bundler, Gemfile
   - rbenv, rvm, chruby
   - DSL friendly
   Ruby, toolchain overview
   

   View Slide

7. - Behavior driven development: Cucumber
   What do we mean by testing?
   

   View Slide

8. - Unit testing (speed, isolation)
   - Rspec, minitest, test-unit
   - Rspec with extensions for chef and
   puppet
   Does the app’s ‘code’ do what I expect?
   Do template contents look correct?
   What do we mean by testing?
   

   View Slide

9. #Testing for our User class
   describe User do
   context 'with admin privileges' do
   before :each do
   @admin = Admin.get(1)
   end
   it 'should exist' do
   expect(@admin).not_to be_nil
   end
   it 'should have a name' do
   expect(@admin.name).not_to be_false
   end
   end
   #...
   end
   Unit testing with rspec
   

   View Slide

10. require 'chefspec'
    describe 'example::default' do
    let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) }
    it 'installs foo' do
    expect(chef_run).to install_package('foo')
    end
    end
    Unit testing with chefspec
    - Completely compatible with most chef tools
    - Stub other node data and environment data
    - Mock objects, return results
    

    View Slide

11. execute "woot" do
    command "echo woot"
    action :nothing
    end
    cookbook_file "/tmp/dangerfile" do
    owner "root"
    mode "00644"
    notifies :run, "execute[woot]"
    end
    Chefspec demo (line cookbook)
    

    View Slide

12. require 'spec_helper'
    describe 'line::tester' do
    let(:chef_run) { ChefSpec::Runner.new.converge 'line::tester' }
    it 'creates dangerfile' do
    expect(chef_run).to create_cookbook_file('/tmp/dangerfile').with_owner('root').
    with_mode('00644')
    end
    end
    Chefspec demo (line cookbook)
    

    View Slide

13. describe 'apache', :type => 'class' do
    context "On a Debian OS with no package name specified" do
    let :facts do
    {
    :osfamily => 'Debian'
    }
    end
    it {
    should contain_package('httpd').with( { 'name' => 'apache2' } )
    should contain_service('httpd').with( { 'name' => 'apache2' } )
    }
    end
    end
    Unit testing with puppet (labs_spec_helper)
    

    View Slide

14. context 'with compress => foo' do
    let(:params) { {:compress => 'foo'} }
    it do
    expect {
    should contain_file('/etc/logrotate.d/nginx')
    }.to raise_error(Puppet::Error, /compress must be true or false/)
    end
    end
    it do
    should contain_service('apache').with(
    'ensure' => 'running',
    'enable' => 'true',
    'hasrestart' => 'true',
    )
    end
    Unit testing with puppet (rspec-puppet)
    

    View Slide

15. - Integration testing
    - Serverspec, bats, rspec again
    Does the system reflect my instructions?
    Verify file contents, service settings, etc
    Isolate to a single host (requires cloudy)
    What do we mean by testing?
    

    View Slide

16. describe package('httpd'), :if => os[:family] == 'redhat' do
    it { should be_installed }
    end
    describe package('apache2'), :if => os[:family] == 'ubuntu' do
    it { should be_installed }
    end
    describe service('httpd'), :if => os[:family] == 'redhat' do
    it { should be_enabled }
    it { should be_running }
    end
    describe service('apache2'), :if => os[:family] == 'ubuntu' do
    it { should be_enabled }
    it { should be_running }
    end
    describe port(80) do
    it { should be_listening }
    end
    Integration testing tools: serverspec
    

    View Slide

17. @test "PEP8 tests for interface code" {
    run pep8 scripts/pontoon*
    [ "$status" = 0 ]
    }
    @test "monit is installed and in the path" {
    which monit
    }
    @test "monit configuration dir exists" {
    [ -d "/etc/monit" ]
    }
    Integration testing tools: bats
    

    View Slide

18. describe 'monit::default' do
    it "install monit" do
    assert system('apt-cache policy monit | grep Installed | grep -v none')
    end
    describe "services" do
    # You can assert that a service must be running following the converge:
    it "runs as a daemon" do
    assert system('/etc/init.d/monit status')
    end
    # And that it will start when the server boots:
    it "boots on startup" do
    assert File.exists?(Dir.glob("/etc/rc5.d/S*monit").first)
    end
    end
    end
    Integration testing tools: minitest
    

    View Slide

19. - Have tests, will travel (Vagrant)
    - Version your environment (TK)
    - Run all the things (rake)
    Vagrant, test-kitchen, rake
    

    View Slide

20. - Same machine, mock & stub everything
    def stub_resources
    stub_command('/usr/sbin/httpd -t').and_return(0)
    stub_command('/usr/sbin/apache2 -t').and_return(0)
    stub_command('which php').and_return('/usr/bin/php')
    end
    def stub_nodes(platform, version, server)
    Dir['./test/integration/nodes/*.json'].sort.each do |f|
    node_data = JSON.parse(IO.read(f), symbolize_names: false)
    node_name = node_data['name']
    server.create_node(node_name, node_data)
    platform.to_s # pacify rubocop
    version.to_s # pacify rubocop
    end
    Dir['./test/integration/environments/*.json'].sort.each do |f|
    env_data = JSON.parse(IO.read(f), symbolize_names: false)
    env_name = env_data['name']
    server.create_environment(env_name, env_data)
    end
    end
    Rake and unit tests
    

    View Slide

21. - Spawn a new machine (or container or cloud server or VM)
    ---
    driver:
    name: vagrant
    driver_config:
    use_vagrant_berkshelf_plugin: true
    provisioner:
    name: chef_solo
    platforms:
    - name: ubuntu-12.04
    - name: debian-6.0.8
    - name: centos-6.4
    - name: fedora-20
    TK and integration tests
    

    View Slide

22. - Spawn a new machine (or container or cloud server or VM)
    suites:
    - name: default
    run_list:
    - recipe[redisio::default]
    - recipe[redisio::enable]
    attributes:
    redisio:
    servers: [ { port: 6379, } ]
    - name: sentinel
    run_list:
    - recipe[redisio::default]
    - recipe[redisio::enable]
    - recipe[redisio::sentinel]
    - recipe[redisio::sentinel_enable]
    attributes:
    redisio:
    servers: [ { port: 6379, } ]
    TK and integration tests
    

    View Slide

23. - Examples of commands in test-kitchen using redisio cookbook
    Go cloud instead --
    driver: rackspace
    driver_config:
    rackspace_username:
    rackspace_region: iad
    rackspace_api_key:
    require_chef_omnibus: true
    no_ssh_tcp_check: true
    no_ssh_tcp_check_sleep: 120
    public_key_path: /home/mart6985/.ssh/id_rsa.pub
    flavor_id: performance1-2
    TK and integration tests
    

    View Slide

24. - Drivers for ec2, rackspace, digital ocean,
    docker, virtualbox, vmware, etc
    - Configuration is YAML files
    - Multiple test suites with different input
    options / test-scenarios
    - Still stub your cfg mgmt environment
    Drivers, configs, options, suites
    

    View Slide

25. - redisio with vagrant and chef-solo
    - jenkins with chef-zero
    - (Your CI could run all these commands)
    Examples and provisioning
    

    View Slide

26. Provisioners, Bussers, chef, puppet
    - a furnisher of provisions
    - bootstraps your config mgmt of choice
    - chef (solo, zero)
    - puppet (apply, agent)
    - shell/bash
    kitchen converge
    

    View Slide

27. Provisioners, Bussers, chef, puppet
    - a person who clears tables in a
    restaurant or cafeteria
    - bootstraps tests and runs them
    - serverspec (rspec)
    - minitest
    - bats
    kitchen verify
    

    View Slide

28. Provisioners, Bussers, chef, puppet
    - TK supports Berkshelf and Librarian
    - TK can provision using bash
    - Think of the other use cases for this!
    kitchen test
    

    View Slide

29. The Holy Grail
    bundle install
    bundle exec rake style*
    bundle exec rake spec
    bundle exec kitchen test -d always
    (gem install rubygems-bundler)
    

    View Slide

30. The Holy Grail
    bundle install
    rake style
    rake spec
    kitchen test -d always
    

    View Slide

31. - Q/A
    - martin@mbs3.org
    Testing your own code...
    http://devopsreactions.tumblr.com /post/88260308392/ testing-my-own-code
    References, questions
    

    View Slide