Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Don't Shrug! Integration Testing for Infrastruc...

Don't Shrug! Integration Testing for Infrastructure with Ruby Frameworks

code for talk is located at https://github.com/bryanwb/tk-demo

this presentation focuses heavily on test-kitchen

Bryan Berry

February 02, 2013

Transcript

80 slides
  1. Don't Shrug! Using Ruby Testing Frameworks To Bring Sanity to

    Your Infrastructure FOSDEM 2013, Bryan W. Berry @bryanwb Code for this demo: https://github.com/bryanwb/tk-demo
  2. Bryan W. Berry • Sysadmin UN Food and Agr. Org

    in Rome, Italy • Co-host of FoodFightShow podcast along w/ Nathen Harvey • @bryanwb
  3. Two Types of Testing White Box Black Box code in

    isolation Integration Testing
  4. You should use Ruby for Integration Testing • Lightweight (mostly)

    • Great libraries (mostly) • Awesome Test DSLs like Rspec • Easy to mock services (Sinatra) • Vagrant, Vagrant, Vagrant Note: not the current Vagrant logo
  5. Prod,QA,Dev,Vagrant Vagrant is your Development environment QA,Dev in use by

    developers are almost production • VirtualBox • Openstack* • KVM* * support coming Note: not the current Vagrant logo
  6. Basic Vagrantfile Vagrant::Config.run do |config| config.vm.box = "lucid32" config.vm.provision :chef_solo

    do |chef| # We're going to download our cookbooks from the web chef.recipe_url = "http://files.vagrantup.com/cookbooks.tar.gz" # Tell chef what recipe to run chef.add_recipe("vagrant_main") end end
  7. Multiple Distros distros = { :lucid32 => { :url =>

    'https://example.com/ubuntu-10.04-i386.box' }, :centos6_3_32 => { :url => 'https://example.com/centos-6.3-i386.box' } } Vagrant::Config.run do |config| distros.each_pair do |name,options| config.vm.define name do |dist_config| dist_config.vm.box = name.to_s dist_config.vm.box_url = options[:url] # rest of vagrant config # . . . end end
  8. Workflow and Vagrant • Vagrantfile defines vms but not actions

    • Need a separate tool for this, Rake or shell script • Start Apache, haproxy, and back-end web service, then test communication between all 3
  9. Multi-VM Vagrantfile Vagrant::Config.run do |config| config.vm.define “apache” do # .

    . . end config.vm.define “haproxy” do # . . . end config.vm.define “backend-app” do # . . . end end
  10. Workflow with Rake task :up do puts "About to run

    vagrant-up..." $env = Vagrant::Environment.new $env.vms.each do |vm| vm.cli(“up”) end end task :test => :up do sleep 100 # wait a long time result = $env.vms[“apache”].cli('curl localhost | grep “Hello World”') exit result # result is 0 or 1 to indicate success or failure end
  11. Rspec it "should say It works!" do response = Faraday.get

    “http://localhost” response.content.should =~ /It works/ end Focus on Behavior
  12. Rspec Already used by whitebox-testing frameworks like • Chefspec •

    Rspec-Puppet Tests expectations for how a program should behave
  13. minitest-chef-handler • Runs minitest suites after Chef recipes • Has

    full access to the node object, chef-server • Can also access underlying system minitest supports more traditional TestCase syntax and Rspec style expectations
  14. Apache # apache2/files/default/minitest/default_test.rb describe_recipe “apache2::default” do it “responds with 'It

    Works!'” do response = Faraday.get “localhost” assert_match response.content, /It works/ end end
  15. More Apache Tests It “installs httpd package” do package('httpd').must_be_installed end

    It “starts httpd service” do service('httpd').must_be_running end
  16. Chef Integration It “installs httpd package” do package(node['apache2']['package']).must_be_installed end It

    “starts httpd service” do service(node['apache2']['package']).must_be_running end
  17. Custom Matchers It “installs httpd package” do package('httpd').must_be_installed end It

    “starts httpd service” do service('httpd').must_be_running end extensions to minitest
  18. Vagrant + Chef + Minitest # apache2/Vagrantfile # …. config.vm.box

    = “Centos6-i386” chef.run_list = [ “minitest-handler”, “apache2” ]
  19. Test-Kitchen • Framework to test a single cookbook across different

    operating systems • Can run tests on VirtualBox or Openstack • Supports minitest or cucumber • Configuration DSL
  20. $ kitchen test Runs each configuration in each of the

    defined platforms (4) Number of Test Runs = 4 x No. of configurations Apache cookbook 96 test runs = 4 x 24 configurations
  21. Usage To test all VMs against all configurations $ kitchen

    test # tests all configurations on # ubuntu-12.04 # ubuntu-10.10 # centos-5.8 # centos-6.3
  22. Usage To test a single VM against a single “configuration”

    $ kitchen test –platform ubuntu-12.04 –configuration default
  23. Kitchenfile cookbook “apache2” do configuration "mod_php5" configuration "mod_ssl" exclude {

    :platform => 'centos', :configuration => 'mod_auth_cas' } run_list_extras ['apache2_test::setup'] end
  24. Scaffolding # apache2_test/attributes/setup.rb default['apache']['auth_username'] = 'bork' default['apache']['auth_password'] = 'secret' default['apache']['cache_expiry_seconds']

    = 60 default['apache']['app_dir'] = '/home/apache2/env' # apache2_test/setup.rb chef_gem “faraday” # our tests will use this node.set[:apache][:default_site_enabled] = true
  25. Our Test # apache2/files/default/minitest/default_test.rb require 'faraday' describe_recipe “apache2::default” do it

    “responds with 'Hello World!'” do response = Faraday.get “localhost” assert_match response.content, /Hello World/ end end
  26. Disadvantages • Designed for single-node configurations • Doesn't expose underlying

    Vagrant config • Test runs are sequential, no support for concurrent test runs
  27. Jamie-CI Test-Kitchen 1.0 • Written by Fletcher Nichol • Only

    does two things 1. Defines machines 2. Exposes Driver interface for different virtualization backends • Drivers* exist for: • Linux containers • Vagrant • EC2 *Some still need to be renamed
  28. Test-Kitchen 1.0 • Define VMs using a single yaml file

    • Supports Librarian-Chef and Berkshelf • Tests VMs concurrently! • No Workflow! Not opinionated! • No docs either!
  29. .kitchen.yml • Why YAML? • Expressive • Easy for other

    programming languages to parse • Platforms – base VMs with a minimum config • Suite – actual configuration you want tested • Number of test runs = Platforms x Suites Note: There is debate whether there will also be code-based configuration for Test-Kitchen 1.0. Stay tuned . . .
  30. Kitchen 1.0 commands • Destroy • Create • Converge •

    Test! $ bundle exec kitchen test OR $ bundle exec kitchen test mod-proxy-centos-63 $ bundle exec kitchen test #{suite}-#{platform}
  31. Issues for “Smoke” tests • Don't need to test multiple

    OS versions • How to DRY up common driver configurations? • Where to put IP Configuration?
  32. tk-demo Demo project for test-kitchen, to demonstrate how to use

    it for mulit-vm integration tests https://github.com/bryanwb/tk-demo
  33. Chef-Workflow • Written by Erik Hollensbe • 3 libraries •

    Chef-workflow • chef-workflow-tasklib • chef-workflow-testlib • Deserves more attention than it is given here
  34. In Summary • You can do basic integration testing right

    now with vagrant + rake • Concurrent VM task execution important • Need DSL for integration tests • Test-Kitchen is still alpha but very exciting • There is gold in Chef-Workflow • We need Public CI for chef cookbooks, puppet modules, and cfengine bundles
  35. References • minitest-chef-handler • chef-workflow • RSpec • Test-Kitchen •

    Test-Kitchen 1.0 (formerly known as Jamie-CI) • Vagrant • berkshelf • Apache Mesos • Cloudera Impala Copyright Bryan W. Berry, Creative Commons 3.0, Unported
  36. References • kitchen-vagrant • Celluloid • BATS • tk-demo, the

    code used in this presentation Copyright Bryan W. Berry, Creative Commons 3.0, Unported
  37. Special Thanks To • Fletcher Nichol • Jamie Winsor •

    Nathen Harvey • Joshua Timberman • Erik Hollensbe for their assistance w/ this presentation

More Decks by Bryan Berry

Other Decks in Programming