Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Don't Shrug! Integration Testing for Infrastructure with Ruby Frameworks

3b2917349b488b1f7511918ab0c409d4?s=47 Bryan Berry
February 02, 2013

Don't Shrug! Integration Testing for Infrastructure with Ruby Frameworks

code for talk is located at https://github.com/bryanwb/tk-demo

this presentation focuses heavily on test-kitchen

3b2917349b488b1f7511918ab0c409d4?s=128

Bryan Berry

February 02, 2013
Tweet

More Decks by Bryan Berry

Other Decks in Programming

Transcript

  1. Don't Shrug! Using Ruby Testing Frameworks To Bring Sanity to

    Your Infrastructure FOSDEM 2013, Bryan W. Berry @bryanwb Code for this demo: https://github.com/bryanwb/tk-demo
  2. Bryan W. Berry • Sysadmin UN Food and Agr. Org

    in Rome, Italy • Co-host of FoodFightShow podcast along w/ Nathen Harvey • @bryanwb
  3. None
  4. Or not . . .

  5. Already got it all figured out?

  6. Infrastructure as Infrastructure as Code? Code? No Problem!

  7. Maybe this isn't so complicated But . . .

  8. Big Data Deathstar Coming for U!

  9. Apache Mesos It's about to get a lot more complicated

  10. The Data Center needs an Operating System Cloudera Impala

  11. Two Types of Testing White Box Black Box code in

    isolation Integration Testing
  12. You should use Ruby for Integration Testing • Lightweight (mostly)

    • Great libraries (mostly) • Awesome Test DSLs like Rspec • Easy to mock services (Sinatra) • Vagrant, Vagrant, Vagrant Note: not the current Vagrant logo
  13. Prod,QA,Dev,Vagrant Vagrant is your Development environment QA,Dev in use by

    developers are almost production • VirtualBox • Openstack* • KVM* * support coming Note: not the current Vagrant logo
  14. Basic Vagrantfile Vagrant::Config.run do |config| config.vm.box = "lucid32" config.vm.provision :chef_solo

    do |chef| # We're going to download our cookbooks from the web chef.recipe_url = "http://files.vagrantup.com/cookbooks.tar.gz" # Tell chef what recipe to run chef.add_recipe("vagrant_main") end end
  15. Multiple Distros distros = { :lucid32 => { :url =>

    'https://example.com/ubuntu-10.04-i386.box' }, :centos6_3_32 => { :url => 'https://example.com/centos-6.3-i386.box' } } Vagrant::Config.run do |config| distros.each_pair do |name,options| config.vm.define name do |dist_config| dist_config.vm.box = name.to_s dist_config.vm.box_url = options[:url] # rest of vagrant config # . . . end end
  16. Workflow and Vagrant • Vagrantfile defines vms but not actions

    • Need a separate tool for this, Rake or shell script • Start Apache, haproxy, and back-end web service, then test communication between all 3
  17. Example Setup Apache Haproxy Web Service Hello World! GET /

    localhost
  18. Multi-VM Vagrantfile Vagrant::Config.run do |config| config.vm.define “apache” do # .

    . . end config.vm.define “haproxy” do # . . . end config.vm.define “backend-app” do # . . . end end
  19. Workflow with Rake task :up do puts "About to run

    vagrant-up..." $env = Vagrant::Environment.new $env.vms.each do |vm| vm.cli(“up”) end end task :test => :up do sleep 100 # wait a long time result = $env.vms[“apache”].cli('curl localhost | grep “Hello World”') exit result # result is 0 or 1 to indicate success or failure end
  20. Most important Vagrant Command $ vagrant destroy -f

  21. Frameworks Basic • Cucumber • Rspec • Minitest Advanced •

    Test-Kitchen • Chef-workflow
  22. Rspec it "should say It works!" do response = Faraday.get

    “http://localhost” response.content.should =~ /It works/ end Focus on Behavior
  23. Rspec Already used by whitebox-testing frameworks like • Chefspec •

    Rspec-Puppet Tests expectations for how a program should behave
  24. minitest-chef-handler • Runs minitest suites after Chef recipes • Has

    full access to the node object, chef-server • Can also access underlying system minitest supports more traditional TestCase syntax and Rspec style expectations
  25. Apache # apache2/files/default/minitest/default_test.rb describe_recipe “apache2::default” do it “responds with 'It

    Works!'” do response = Faraday.get “localhost” assert_match response.content, /It works/ end end
  26. More Apache Tests It “installs httpd package” do package('httpd').must_be_installed end

    It “starts httpd service” do service('httpd').must_be_running end
  27. Chef Integration It “installs httpd package” do package(node['apache2']['package']).must_be_installed end It

    “starts httpd service” do service(node['apache2']['package']).must_be_running end
  28. Custom Matchers It “installs httpd package” do package('httpd').must_be_installed end It

    “starts httpd service” do service('httpd').must_be_running end extensions to minitest
  29. Vagrant + Chef + Minitest # apache2/Vagrantfile # …. config.vm.box

    = “Centos6-i386” chef.run_list = [ “minitest-handler”, “apache2” ]
  30. Result • Screenshot of minitest results

  31. CI Integration • Minitest-chef-handler returns non-zero if any test fails

  32. Wouldn't this be awesome?

  33. Test-Kitchen • Framework to test a single cookbook across different

    operating systems • Can run tests on VirtualBox or Openstack • Supports minitest or cucumber • Configuration DSL
  34. Usage Test-kitchen defines base machines for you

  35. None
  36. $ kitchen test Runs each configuration in each of the

    defined platforms (4) Number of Test Runs = 4 x No. of configurations Apache cookbook 96 test runs = 4 x 24 configurations
  37. Usage To test all VMs against all configurations $ kitchen

    test # tests all configurations on # ubuntu-12.04 # ubuntu-10.10 # centos-5.8 # centos-6.3
  38. Test-kitchen structure

  39. Usage To test a single VM against a single “configuration”

    $ kitchen test –platform ubuntu-12.04 –configuration default
  40. Kitchenfile cookbook “apache2” do configuration "mod_php5" configuration "mod_ssl" exclude {

    :platform => 'centos', :configuration => 'mod_auth_cas' } run_list_extras ['apache2_test::setup'] end
  41. Scaffolding # apache2_test/attributes/setup.rb default['apache']['auth_username'] = 'bork' default['apache']['auth_password'] = 'secret' default['apache']['cache_expiry_seconds']

    = 60 default['apache']['app_dir'] = '/home/apache2/env' # apache2_test/setup.rb chef_gem “faraday” # our tests will use this node.set[:apache][:default_site_enabled] = true
  42. Our Test # apache2/files/default/minitest/default_test.rb require 'faraday' describe_recipe “apache2::default” do it

    “responds with 'Hello World!'” do response = Faraday.get “localhost” assert_match response.content, /Hello World/ end end
  43. Disadvantages • Designed for single-node configurations • Doesn't expose underlying

    Vagrant config • Test runs are sequential, no support for concurrent test runs
  44. Jamie-CI Test-Kitchen 1.0 • Written by Fletcher Nichol • Only

    does two things 1. Defines machines 2. Exposes Driver interface for different virtualization backends • Drivers* exist for: • Linux containers • Vagrant • EC2 *Some still need to be renamed
  45. Test-Kitchen 1.0 • Define VMs using a single yaml file

    • Supports Librarian-Chef and Berkshelf • Tests VMs concurrently! • No Workflow! Not opinionated! • No docs either!
  46. No Dependency on Chef

  47. None
  48. None
  49. None
  50. .kitchen.yml • Why YAML? • Expressive • Easy for other

    programming languages to parse • Platforms – base VMs with a minimum config • Suite – actual configuration you want tested • Number of test runs = Platforms x Suites Note: There is debate whether there will also be code-based configuration for Test-Kitchen 1.0. Stay tuned . . .
  51. .kitchen.local.ec2.yml

  52. None
  53. Kitchen 1.0 commands • Destroy • Create • Converge •

    Test! $ bundle exec kitchen test OR $ bundle exec kitchen test mod-proxy-centos-63 $ bundle exec kitchen test #{suite}-#{platform}
  54. BATS!!!!

  55. Bash Automated Testing System

  56. Bash Automated Testing System

  57. Issues for “Smoke” tests • Don't need to test multiple

    OS versions • How to DRY up common driver configurations? • Where to put IP Configuration?
  58. None
  59. None
  60. Directory Layout One log file per VM “Smoke” Test

  61. tk-demo Demo project for test-kitchen, to demonstrate how to use

    it for mulit-vm integration tests https://github.com/bryanwb/tk-demo
  62. test-kitchen + Rake

  63. None
  64. Wiring things together is hard Apache Haproxy App Server Hello

    World! GET / localhost
  65. Chef-Workflow Apache Haproxy App Server Chef-Server Chef-client Chef-client Chef-client A

    set of helpers for managing workflow
  66. Chef-Workflow • Written by Erik Hollensbe • 3 libraries •

    Chef-workflow • chef-workflow-tasklib • chef-workflow-testlib • Deserves more attention than it is given here
  67. Chef-workflow Task Example

  68. Chef-workflow Test Example

  69. We need a layered approach to systems management

  70. Image credit: http://i.technet.microsoft.com/dynimg/IC197700.gif

  71. None
  72. BOSS* Bryan's Orchestration is Super-Simple DSL *Just a mock-up

  73. even simpler

  74. Awesome Ruby Libraries for use in integration tests • Faraday

    • Sinatra • Rspec-dns • Ruby-dns
  75. None
  76. Integration Testing Heroes

  77. In Summary • You can do basic integration testing right

    now with vagrant + rake • Concurrent VM task execution important • Need DSL for integration tests • Test-Kitchen is still alpha but very exciting • There is gold in Chef-Workflow • We need Public CI for chef cookbooks, puppet modules, and cfengine bundles
  78. References • minitest-chef-handler • chef-workflow • RSpec • Test-Kitchen •

    Test-Kitchen 1.0 (formerly known as Jamie-CI) • Vagrant • berkshelf • Apache Mesos • Cloudera Impala Copyright Bryan W. Berry, Creative Commons 3.0, Unported
  79. References • kitchen-vagrant • Celluloid • BATS • tk-demo, the

    code used in this presentation Copyright Bryan W. Berry, Creative Commons 3.0, Unported
  80. Special Thanks To • Fletcher Nichol • Jamie Winsor •

    Nathen Harvey • Joshua Timberman • Erik Hollensbe for their assistance w/ this presentation