Testing Your Automation

F820550e0c386dcc73b3ca0565a8bfff?s=47 Nathen Harvey
March 01, 2013
2.3k

Testing Your Automation

TDD for Chef Cookbooks

F820550e0c386dcc73b3ca0565a8bfff?s=128

Nathen Harvey

March 01, 2013
Tweet

Transcript

  1. Testing Your Automation TDD for Chef Cookbooks Nathen Harvey @nathenharvey

    https://github.com/nathenharvey/bigruby
  2. What is Chef? Recipes and Cookbooks that describe Infrastructure as

    Code. Chef enables people to easily build & manage complex & dynamic applications at massive scale • New model for describing infrastructure that promotes reuse • Programmatically provision and configure • Reconstruct business from code repository, data backup, and bare metal resources Chef is an automation platform for developers & systems engineers to continuously define, build, and manage infrastructure. CHEF USES: “ ”
  3. None
  4. Writing Infrastructure Code https://github.com/nathenharvey/bigruby

  5. Evolving towards Configuration Management •Just build it •Keep notes in

    server.txt •Move notes to the wiki •Custom scripts (in scm?!) •Snapshot & Clone
  6. Building something with Chef •Come up with your policy •Abstract

    the resources •Write recipes •Apply recipes to nodes
  7. Trivial Example

  8. Cookbook $ knife cookbook create website ** Creating cookbook website

    ** Creating README for cookbook: website ** Creating CHANGELOG for cookbook: website ** Creating metadata for cookbook: website
  9. Resources in Recipe package "apache2" template "/var/www/index.html" do owner "www-data"

    group "www-data" mode 644 source "index.html.erb" end service "apache2" do action [:start, :enable] end
  10. templates/default/index.html.erb <html> <head> <title><%= node['conference']['name'] %></title> </head> <body> <h1>Hello, <%=

    node['conference']['name'] %></h1> </body> </html> • I told you, this is a trivial example! Stick with me. • No, you wouldn’t really manage your content in Chef but this may be easier to grok than sysctl settings.
  11. attributes/default.rb default['conference']['name'] = "Big Ruby"

  12. Apply This Recipe to a Node $ vagrant init web

    A `Vagrantfile` has been placed in this directory. You are now ready to `vagrant up` your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on `vagrantup.com` for more information on using Vagrant.
  13. Vagrantfile # -*- mode: ruby -*- # vi: set ft=ruby

    : Vagrant::Config.run do |config| config.vm.box = "web" config.vm.forward_port 80, 8080 config.vm.provision :chef_client do |chef| chef.chef_server_url = "https://api.opscode.com/organizations/bigruby" chef.validation_key_path = "bigruby-validator.pem" chef.validation_client_name = "bigruby-validator" end end
  14. Launch the VM $ vagrant up [default] Importing base box

    'web'... [default] Matching MAC address for NAT networking... [default] Forwarding ports... [default] -- 22 => 2222 (adapter 1) [default] -- 80 => 8080 (adapter 1) ... [2013-02-01T05:24:19+00:00] WARN: Node vagrant.vm has an empty run list. [2013-02-01T05:24:20+00:00] INFO: Chef Run complete in 1.962777136 seconds [2013-02-01T05:24:20+00:00] INFO: Running report handlers [2013-02-01T05:24:20+00:00] INFO: Report handlers complete
  15. Provision the Node $ knife cookbook upload website Uploading website

    [0.1.0] Uploaded 1 cookbook. $ knife node run_list add vagrant.vm "recipe[website]" vagrant.vm: run_list: recipe[website] $ vagrant provision
  16. Test

  17. Testing Infrastructure Code https://github.com/nathenharvey/bigruby

  18. Why test?

  19. Testing Tools • Vagrant • knife cookbook test

  20. knife cookbook test $ knife cookbook test website checking website

    Running syntax check on website Validating ruby files Validating templates
  21. Testing Tools • Vagrant • knife cookbook test • Foodcritic

  22. Foodcritic • A lint tool for your Opscode Chef cookbooks

    • Flag problems in your Chef cookbooks that will cause Chef to blow up when you attempt to converge • Encourage discussion within the Chef community on the more subjective stuff - what does a good cookbook look like?
  23. Foodcritic $ foodcritic cookbooks/website FC006: Mode should be quoted or

    fully specified when setting file permissions: cookbooks/website/recipes/default.rb:11 FC008: Generated cookbook metadata needs updating: cookbooks/website/metadata.rb:2 FC008: Generated cookbook metadata needs updating: cookbooks/website/metadata.rb:3
  24. Foodcritic

  25. Foodcritic

  26. Testing Tools • Vagrant • knife cookbook test • Foodcritic

    • Chefspec
  27. Chefspec require 'chefspec' describe 'website::default' do chef_run = ChefSpec::ChefRunner.new chef_run.converge

    "website::default" it "should install apache package" do chef_run.should install_package "apache2" end it "should create a home page" do chef_run.should create_file "/var/www/index.html" end
  28. Chefspec it "should create a home page with our content"

    do chef_run.should create_file_with_content( "/var/www/index.html","Big Ruby") end it "should start the apache service" do chef_run.should start_service "apache2" end it "should enable the apache service" do chef_run.should enable_service "apache2" end end
  29. Chefspec chef_run.should create_file_with_content( "/var/www/index.html","Big Ruby") <%= node['conference']['name'] %> default['conference']['name'] =

    "Big Ruby"
  30. "memory": { "swap": { "cached": "0kB", "total": "4128760kB", "free": "4128760kB"

    }, "total": "2055676kB", "free": "1646524kB", "buffers": "35032kB", "cached": "210276kB", "active": "125336kB", "inactive": "142884kB", "dirty": "8kB", "writeback": "0kB", "anon_pages": "22976kB", "mapped": "8416kB", "slab": "121512kB", "slab_reclaimable": "41148kB", "slab_unreclaim": "80364kB", "page_tables": "1784kB", "nfs_unstable": "0kB", "bounce": "0kB", "commit_limit": "5156596kB", "committed_as": "74980kB", "vmalloc_total": "34359738367kB", "vmalloc_used": "274512kB", "vmalloc_chunk": "34359449936kB" }, Ohai! "block_device": { "ram0": { "size": "32768", "removable": "0" }, "ram1": { "size": "32768", "removable": "0" }, "ram2": { "size": "32768", "removable": "0" }, "hostname": "server-1", "fqdn": "server-1.example.com", "domain": "example.com", "network": { "interfaces": { "eth0": { "type": "eth", "number": "0", "encapsulation": "Ethernet", "addresses": { "00:0C:29:43:26:C5": { "family": "lladdr" }, "192.168.177.138": { "family": "inet", "broadcast": "192.168.177.255", "netmask": "255.255.255.0" }, "fe80::20c:29ff:fe43:26c5": { "family": "inet6", "prefixlen": "64", "scope": "Link" } },
  31. Chefspec & Fauxhai

  32. Testing Tools • Vagrant • knife cookbook test • Foodcritic

    • Chefspec • Fauxhai • Minitest and the Minitest Chef Handler
  33. Handlers $ vagrant up [default] Importing base box 'web'... [default]

    Matching MAC address for NAT networking... [default] Forwarding ports... [default] -- 22 => 2222 (adapter 1) [default] -- 80 => 8080 (adapter 1) ... [2013-02-01T05:24:19+00:00] WARN: Node vagrant.vm has an empty run list. [2013-02-01T05:24:20+00:00] INFO: Chef Run complete in 1.962777136 seconds [2013-02-01T05:24:20+00:00] INFO: Running report handlers [2013-02-01T05:24:20+00:00] INFO: Report handlers complete
  34. minitest class TestWebsite < MiniTest::Chef::TestCase include MiniTest::Chef::Assertions include MiniTest::Chef::Context include

    MiniTest::Chef::Resources def test_succeed assert run_status.success? end def test_that_the_package_installed package("apache2").must_be_installed end def test_that_the_service_is_running service("apache2").must_be_running end
  35. minitest def test_that_the_service_is_enabled service("apache2").must_be_enabled end def test_home_page_file file("/var/www/index.html").must_exist end def

    test_home_page_content file("/var/www/index.html").must_include node['conference']['name'] end
  36. minitest-handler $ vagrant provision [2013-02-01T06:43:34+00:00] INFO: Running report handlers Run

    options: -v --seed 12405 # Running tests: TestWebsite#test_succeed = ... Finished tests in 0.098367s, 60.9959 tests/s, 60.9959 assertions/s. 6 tests, 6 assertions, 0 failures, 0 errors, 0 skips [2013-02-01T06:43:34+00:00] INFO: Report handlers complete
  37. Testing Tools • Vagrant • knife cookbook test • Foodcritic

    • Chefspec • Fauxhai • Minitest and the Minitest Chef Handler • Why-run
  38. Launch a Staging Node $ knife ec2 server create -I

    ami-f4fb7c9d -f m1.small -E staging Instance ID: i-1e3a4b6d Flavor: m1.small Image: ami-f4fb7c9d Region: us-east-1 Availability Zone: us-east-1d Security Groups: default Tags: {"Name"=>"i-1e3a4b6d"} SSH Key: nharvey-oc Waiting for server.................... Public DNS Name: ec2-107-22-155-182.compute-1.amazonaws.com Public IP Address: 107.22.155.182 Private DNS Name: ip-10-190-223-78.ec2.internal Private IP Address: 10.190.223.78 Waiting for sshd....done Bootstrapping Chef on ec2-107-22-155-182.compute-1.amazonaws.com
  39. Launch a Staging Node $ knife node run_list add i-1e3a4b6d

    "recipe[website]" i-1e3a4b6d: run_list: recipe[website]
  40. Why Run $ knife ssh "chef_environment:staging" "sudo chef-client --why-run"

  41. knife ssh chef_enviornment:staging

  42. knife ssh chef_environment:staging "sudo chef-client --why-run"

  43. Why Run $ knife ssh "chef_environment:staging" "sudo chef-client --why-run" Starting

    Chef Client, version 11.4.0 ... Converging 3 resources * package[apache2] action install - Would install version 2.2.22-1ubuntu1 of package apache2 * template[/var/www/index.html] action create * Parent directory /var/www does not exist. * Assuming directory /var/www would have been created - Would create template[/var/www/index.html] * service[apache2] action start - Would start service service[apache2] * service[apache2] action enable - Would enable service service[apache2] WARN: In whyrun mode, so NOT performing node save. Chef Client finished, 4 resources would have been updated
  44. Why Run • Promises, Lies, and Dry-Run Mode • http://bit.ly/guessrun

    • Why Run reported 4 resource updated • “real run” only 2 resources are updated
  45. Testing Tools • Vagrant - Local development and testing •

    knife cookbook test - Verify ruby syntax • Foodcritic - Cookbook linter • Chefspec - Unit testing recipes • Fauxhai - Mock all the things • Minitest Chef Handler - post-converge tests • Why-run - Best guess
  46. Moar Testing Tools • Test Kitchen • Cross-platform testing •

    Cucumber Chef • acceptance & integration testing
  47. Continuous Integration •Travis •Jenkins •etc.

  48. Gating Your Releases https://github.com/nathenharvey/bigruby

  49. Requirements Have Changed!

  50. Bump Cookbook Version

  51. Update Tests

  52. Watch Tests Fail $ rpsec . 5 examples, 1 failure

    Failed examples: rspec ./cookbooks/website/spec/default_spec.rb:18 # website::default should create a home page with our content $ vagrant provision FATAL: RuntimeError: MiniTest failed with 0 failure(s) and 1 error(s). Error: test_home_page_content(TestWebsite):
  53. Update Cookbook

  54. Converge Development Node $ rspec . 5 examples, 0 failures

    $ knife cookbook upload website && vagrant provision 6 tests, 7 assertions, 0 failures, 0 errors, 0 skips
  55. Verify

  56. Converge Staging Node $ knife ssh "chef_environment:staging" "sudo chef-client" Starting

    Chef Client, version 11.4.0 ... Chef Client finished, 0 resources updated
  57. Y U No Change? { "name": "staging", "description": "", "cookbook_versions":

    { "website" : "0.1.2" }, "json_class": "Chef::Environment", "chef_type": "environment", "default_attributes": { }, "override_attributes": { } }
  58. Promote the New Version

  59. Converge the Staging Node $ knife environment from file staging.json

    Updated Environment staging $ knife ssh "chef_environment:staging" "sudo chef-client" Starting Chef Client, version 11.4.0 ... Chef Client finished, 1 resources updated
  60. Environments & Continuous Integration • Tests + Environments + Continuous

    Integration • ONE Build Pipeline to Rule them All
  61. Monitoring!

  62. So, What Have We Learned? https://github.com/nathenharvey/bigruby

  63. Testing Your Infrastructure • There are a number of frameworks

    to automate your infrastructure testing • Graduate changes through your environment • Trust your monitoring to catch issues quickly
  64. Chef is Awesome! • Manage all of the infrastructure for

    your single page, static website! •
  65. Chef is Awesome!

  66. Knife Rocks! • Command line interface for • Creating Cookbooks

    • Searching the Chef Server • Provisioning Infrastructure • ...and more!
  67. The Chef Community is Awesome! • Things in this presentation

    that exist because of the Community: • Foodcritic • Chefspec • minitest-chef-handler • Fauxhai • Cucumber Chef
  68. How can you help? • You are a developer well-versed

    in Ruby test frameworks & TDD • Test frameworks & TDD are relatively new to infrastructure code
  69. How can you help? •Stop testurbation!

  70. #ChefConf 2013

  71. Thank You •What questions do you have? •Nathen Harvey •nharvey@opscode.com

    •@nathenharvey