Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Don't Shrug! Integration Testing for Infrastructure with Ruby Frameworks

Bryan Berry
February 02, 2013

Don't Shrug! Integration Testing for Infrastructure with Ruby Frameworks

code for talk is located at https://github.com/bryanwb/tk-demo

this presentation focuses heavily on test-kitchen

Bryan Berry

February 02, 2013
Tweet

More Decks by Bryan Berry

Other Decks in Programming

Transcript

  1. Don't Shrug! Using Ruby Testing Frameworks To Bring Sanity to

    Your Infrastructure FOSDEM 2013, Bryan W. Berry @bryanwb Code for this demo: https://github.com/bryanwb/tk-demo
  2. Bryan W. Berry • Sysadmin UN Food and Agr. Org

    in Rome, Italy • Co-host of FoodFightShow podcast along w/ Nathen Harvey • @bryanwb
  3. Two Types of Testing White Box Black Box code in

    isolation Integration Testing
  4. You should use Ruby for Integration Testing • Lightweight (mostly)

    • Great libraries (mostly) • Awesome Test DSLs like Rspec • Easy to mock services (Sinatra) • Vagrant, Vagrant, Vagrant Note: not the current Vagrant logo
  5. Prod,QA,Dev,Vagrant Vagrant is your Development environment QA,Dev in use by

    developers are almost production • VirtualBox • Openstack* • KVM* * support coming Note: not the current Vagrant logo
  6. Basic Vagrantfile Vagrant::Config.run do |config| config.vm.box = "lucid32" config.vm.provision :chef_solo

    do |chef| # We're going to download our cookbooks from the web chef.recipe_url = "http://files.vagrantup.com/cookbooks.tar.gz" # Tell chef what recipe to run chef.add_recipe("vagrant_main") end end
  7. Multiple Distros distros = { :lucid32 => { :url =>

    'https://example.com/ubuntu-10.04-i386.box' }, :centos6_3_32 => { :url => 'https://example.com/centos-6.3-i386.box' } } Vagrant::Config.run do |config| distros.each_pair do |name,options| config.vm.define name do |dist_config| dist_config.vm.box = name.to_s dist_config.vm.box_url = options[:url] # rest of vagrant config # . . . end end
  8. Workflow and Vagrant • Vagrantfile defines vms but not actions

    • Need a separate tool for this, Rake or shell script • Start Apache, haproxy, and back-end web service, then test communication between all 3
  9. Multi-VM Vagrantfile Vagrant::Config.run do |config| config.vm.define “apache” do # .

    . . end config.vm.define “haproxy” do # . . . end config.vm.define “backend-app” do # . . . end end
  10. Workflow with Rake task :up do puts "About to run

    vagrant-up..." $env = Vagrant::Environment.new $env.vms.each do |vm| vm.cli(“up”) end end task :test => :up do sleep 100 # wait a long time result = $env.vms[“apache”].cli('curl localhost | grep “Hello World”') exit result # result is 0 or 1 to indicate success or failure end
  11. Rspec it "should say It works!" do response = Faraday.get

    “http://localhost” response.content.should =~ /It works/ end Focus on Behavior
  12. Rspec Already used by whitebox-testing frameworks like • Chefspec •

    Rspec-Puppet Tests expectations for how a program should behave
  13. minitest-chef-handler • Runs minitest suites after Chef recipes • Has

    full access to the node object, chef-server • Can also access underlying system minitest supports more traditional TestCase syntax and Rspec style expectations
  14. Apache # apache2/files/default/minitest/default_test.rb describe_recipe “apache2::default” do it “responds with 'It

    Works!'” do response = Faraday.get “localhost” assert_match response.content, /It works/ end end
  15. More Apache Tests It “installs httpd package” do package('httpd').must_be_installed end

    It “starts httpd service” do service('httpd').must_be_running end
  16. Chef Integration It “installs httpd package” do package(node['apache2']['package']).must_be_installed end It

    “starts httpd service” do service(node['apache2']['package']).must_be_running end
  17. Custom Matchers It “installs httpd package” do package('httpd').must_be_installed end It

    “starts httpd service” do service('httpd').must_be_running end extensions to minitest
  18. Vagrant + Chef + Minitest # apache2/Vagrantfile # …. config.vm.box

    = “Centos6-i386” chef.run_list = [ “minitest-handler”, “apache2” ]
  19. Test-Kitchen • Framework to test a single cookbook across different

    operating systems • Can run tests on VirtualBox or Openstack • Supports minitest or cucumber • Configuration DSL
  20. $ kitchen test Runs each configuration in each of the

    defined platforms (4) Number of Test Runs = 4 x No. of configurations Apache cookbook 96 test runs = 4 x 24 configurations
  21. Usage To test all VMs against all configurations $ kitchen

    test # tests all configurations on # ubuntu-12.04 # ubuntu-10.10 # centos-5.8 # centos-6.3
  22. Usage To test a single VM against a single “configuration”

    $ kitchen test –platform ubuntu-12.04 –configuration default
  23. Kitchenfile cookbook “apache2” do configuration "mod_php5" configuration "mod_ssl" exclude {

    :platform => 'centos', :configuration => 'mod_auth_cas' } run_list_extras ['apache2_test::setup'] end
  24. Scaffolding # apache2_test/attributes/setup.rb default['apache']['auth_username'] = 'bork' default['apache']['auth_password'] = 'secret' default['apache']['cache_expiry_seconds']

    = 60 default['apache']['app_dir'] = '/home/apache2/env' # apache2_test/setup.rb chef_gem “faraday” # our tests will use this node.set[:apache][:default_site_enabled] = true
  25. Our Test # apache2/files/default/minitest/default_test.rb require 'faraday' describe_recipe “apache2::default” do it

    “responds with 'Hello World!'” do response = Faraday.get “localhost” assert_match response.content, /Hello World/ end end
  26. Disadvantages • Designed for single-node configurations • Doesn't expose underlying

    Vagrant config • Test runs are sequential, no support for concurrent test runs
  27. Jamie-CI Test-Kitchen 1.0 • Written by Fletcher Nichol • Only

    does two things 1. Defines machines 2. Exposes Driver interface for different virtualization backends • Drivers* exist for: • Linux containers • Vagrant • EC2 *Some still need to be renamed
  28. Test-Kitchen 1.0 • Define VMs using a single yaml file

    • Supports Librarian-Chef and Berkshelf • Tests VMs concurrently! • No Workflow! Not opinionated! • No docs either!
  29. .kitchen.yml • Why YAML? • Expressive • Easy for other

    programming languages to parse • Platforms – base VMs with a minimum config • Suite – actual configuration you want tested • Number of test runs = Platforms x Suites Note: There is debate whether there will also be code-based configuration for Test-Kitchen 1.0. Stay tuned . . .
  30. Kitchen 1.0 commands • Destroy • Create • Converge •

    Test! $ bundle exec kitchen test OR $ bundle exec kitchen test mod-proxy-centos-63 $ bundle exec kitchen test #{suite}-#{platform}
  31. Issues for “Smoke” tests • Don't need to test multiple

    OS versions • How to DRY up common driver configurations? • Where to put IP Configuration?
  32. tk-demo Demo project for test-kitchen, to demonstrate how to use

    it for mulit-vm integration tests https://github.com/bryanwb/tk-demo
  33. Chef-Workflow • Written by Erik Hollensbe • 3 libraries •

    Chef-workflow • chef-workflow-tasklib • chef-workflow-testlib • Deserves more attention than it is given here
  34. In Summary • You can do basic integration testing right

    now with vagrant + rake • Concurrent VM task execution important • Need DSL for integration tests • Test-Kitchen is still alpha but very exciting • There is gold in Chef-Workflow • We need Public CI for chef cookbooks, puppet modules, and cfengine bundles
  35. References • minitest-chef-handler • chef-workflow • RSpec • Test-Kitchen •

    Test-Kitchen 1.0 (formerly known as Jamie-CI) • Vagrant • berkshelf • Apache Mesos • Cloudera Impala Copyright Bryan W. Berry, Creative Commons 3.0, Unported
  36. References • kitchen-vagrant • Celluloid • BATS • tk-demo, the

    code used in this presentation Copyright Bryan W. Berry, Creative Commons 3.0, Unported
  37. Special Thanks To • Fletcher Nichol • Jamie Winsor •

    Nathen Harvey • Joshua Timberman • Erik Hollensbe for their assistance w/ this presentation