Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing Your Automation

Nathen Harvey
March 01, 2013
2.6k

Testing Your Automation

TDD for Chef Cookbooks

Nathen Harvey

March 01, 2013
Tweet

Transcript

  1. Testing Your Automation
    TDD for Chef Cookbooks
    Nathen Harvey
    @nathenharvey
    https://github.com/nathenharvey/bigruby

    View full-size slide

  2. What is Chef?
    Recipes and Cookbooks
    that describe Infrastructure as Code.
    Chef enables people to easily build &
    manage complex & dynamic applications
    at massive scale
    • New model for describing infrastructure that
    promotes reuse
    • Programmatically provision and configure
    • Reconstruct business from code repository,
    data backup, and bare metal resources
    Chef is an automation platform for developers & systems engineers to continuously
    define, build, and manage infrastructure.
    CHEF USES:


    View full-size slide

  3. Writing Infrastructure Code
    https://github.com/nathenharvey/bigruby

    View full-size slide

  4. Evolving towards Configuration Management
    •Just build it
    •Keep notes in server.txt
    •Move notes to the wiki
    •Custom scripts (in scm?!)
    •Snapshot & Clone

    View full-size slide

  5. Building something with Chef
    •Come up with your
    policy
    •Abstract the resources
    •Write recipes
    •Apply recipes to nodes

    View full-size slide

  6. Trivial Example

    View full-size slide

  7. Cookbook
    $ knife cookbook create website
    ** Creating cookbook website
    ** Creating README for cookbook: website
    ** Creating CHANGELOG for cookbook: website
    ** Creating metadata for cookbook: website

    View full-size slide

  8. Resources in Recipe
    package "apache2"
    template "/var/www/index.html" do
    owner "www-data"
    group "www-data"
    mode 644
    source "index.html.erb"
    end
    service "apache2" do
    action [:start, :enable]
    end

    View full-size slide

  9. templates/default/index.html.erb


    <%= node['conference']['name'] %>


    Hello, <%= node['conference']['name'] %>


    • I told you, this is a trivial example! Stick with me.
    • No, you wouldn’t really manage your content in
    Chef but this may be easier to grok than sysctl
    settings.

    View full-size slide

  10. attributes/default.rb
    default['conference']['name'] = "Big Ruby"

    View full-size slide

  11. Apply This Recipe to a Node
    $ vagrant init web
    A `Vagrantfile` has been placed in this directory. You are now
    ready to `vagrant up` your first virtual environment! Please
    read the comments in the Vagrantfile as well as documentation
    on `vagrantup.com` for more information on using Vagrant.

    View full-size slide

  12. Vagrantfile
    # -*- mode: ruby -*-
    # vi: set ft=ruby :
    Vagrant::Config.run do |config|
    config.vm.box = "web"
    config.vm.forward_port 80, 8080
    config.vm.provision :chef_client do |chef|
    chef.chef_server_url = "https://api.opscode.com/organizations/bigruby"
    chef.validation_key_path = "bigruby-validator.pem"
    chef.validation_client_name = "bigruby-validator"
    end
    end

    View full-size slide

  13. Launch the VM
    $ vagrant up
    [default] Importing base box 'web'...
    [default] Matching MAC address for NAT networking...
    [default] Forwarding ports...
    [default] -- 22 => 2222 (adapter 1)
    [default] -- 80 => 8080 (adapter 1)
    ...
    [2013-02-01T05:24:19+00:00] WARN: Node vagrant.vm has an empty run list.
    [2013-02-01T05:24:20+00:00] INFO: Chef Run complete in 1.962777136 seconds
    [2013-02-01T05:24:20+00:00] INFO: Running report handlers
    [2013-02-01T05:24:20+00:00] INFO: Report handlers complete

    View full-size slide

  14. Provision the Node
    $ knife cookbook upload website
    Uploading website [0.1.0]
    Uploaded 1 cookbook.
    $ knife node run_list add vagrant.vm "recipe[website]"
    vagrant.vm:
    run_list: recipe[website]
    $ vagrant provision

    View full-size slide

  15. Testing Infrastructure Code
    https://github.com/nathenharvey/bigruby

    View full-size slide

  16. Testing Tools
    • Vagrant
    • knife cookbook test

    View full-size slide

  17. knife cookbook test
    $ knife cookbook test website
    checking website
    Running syntax check on website
    Validating ruby files
    Validating templates

    View full-size slide

  18. Testing Tools
    • Vagrant
    • knife cookbook test
    • Foodcritic

    View full-size slide

  19. Foodcritic
    • A lint tool for your Opscode Chef
    cookbooks
    • Flag problems in your Chef cookbooks
    that will cause Chef to blow up when
    you attempt to converge
    • Encourage discussion within the Chef
    community on the more subjective stuff
    - what does a good cookbook look like?

    View full-size slide

  20. Foodcritic
    $ foodcritic cookbooks/website
    FC006: Mode should be quoted or fully specified when setting file permissions:
    cookbooks/website/recipes/default.rb:11
    FC008: Generated cookbook metadata needs updating:
    cookbooks/website/metadata.rb:2
    FC008: Generated cookbook metadata needs updating:
    cookbooks/website/metadata.rb:3

    View full-size slide

  21. Testing Tools
    • Vagrant
    • knife cookbook test
    • Foodcritic
    • Chefspec

    View full-size slide

  22. Chefspec
    require 'chefspec'
    describe 'website::default' do
    chef_run = ChefSpec::ChefRunner.new
    chef_run.converge "website::default"
    it "should install apache package" do
    chef_run.should install_package "apache2"
    end
    it "should create a home page" do
    chef_run.should create_file "/var/www/index.html"
    end

    View full-size slide

  23. Chefspec
    it "should create a home page with our content" do
    chef_run.should create_file_with_content(
    "/var/www/index.html","Big Ruby")
    end
    it "should start the apache service" do
    chef_run.should start_service "apache2"
    end
    it "should enable the apache service" do
    chef_run.should enable_service "apache2"
    end
    end

    View full-size slide

  24. Chefspec
    chef_run.should create_file_with_content(
    "/var/www/index.html","Big Ruby")
    <%= node['conference']['name'] %>
    default['conference']['name'] = "Big Ruby"

    View full-size slide

  25. "memory": {
    "swap": {
    "cached": "0kB",
    "total": "4128760kB",
    "free": "4128760kB"
    },
    "total": "2055676kB",
    "free": "1646524kB",
    "buffers": "35032kB",
    "cached": "210276kB",
    "active": "125336kB",
    "inactive": "142884kB",
    "dirty": "8kB",
    "writeback": "0kB",
    "anon_pages": "22976kB",
    "mapped": "8416kB",
    "slab": "121512kB",
    "slab_reclaimable": "41148kB",
    "slab_unreclaim": "80364kB",
    "page_tables": "1784kB",
    "nfs_unstable": "0kB",
    "bounce": "0kB",
    "commit_limit": "5156596kB",
    "committed_as": "74980kB",
    "vmalloc_total": "34359738367kB",
    "vmalloc_used": "274512kB",
    "vmalloc_chunk": "34359449936kB"
    },
    Ohai!
    "block_device": {
    "ram0": {
    "size": "32768",
    "removable": "0"
    },
    "ram1": {
    "size": "32768",
    "removable": "0"
    },
    "ram2": {
    "size": "32768",
    "removable": "0"
    },
    "hostname": "server-1",
    "fqdn": "server-1.example.com",
    "domain": "example.com",
    "network": {
    "interfaces": {
    "eth0": {
    "type": "eth",
    "number": "0",
    "encapsulation": "Ethernet",
    "addresses": {
    "00:0C:29:43:26:C5": {
    "family": "lladdr"
    },
    "192.168.177.138": {
    "family": "inet",
    "broadcast": "192.168.177.255",
    "netmask": "255.255.255.0"
    },
    "fe80::20c:29ff:fe43:26c5": {
    "family": "inet6",
    "prefixlen": "64",
    "scope": "Link"
    }
    },

    View full-size slide

  26. Chefspec & Fauxhai

    View full-size slide

  27. Testing Tools
    • Vagrant
    • knife cookbook test
    • Foodcritic
    • Chefspec
    • Fauxhai
    • Minitest and the Minitest Chef Handler

    View full-size slide

  28. Handlers
    $ vagrant up
    [default] Importing base box 'web'...
    [default] Matching MAC address for NAT networking...
    [default] Forwarding ports...
    [default] -- 22 => 2222 (adapter 1)
    [default] -- 80 => 8080 (adapter 1)
    ...
    [2013-02-01T05:24:19+00:00] WARN: Node vagrant.vm has an empty run list.
    [2013-02-01T05:24:20+00:00] INFO: Chef Run complete in 1.962777136 seconds
    [2013-02-01T05:24:20+00:00] INFO: Running report handlers
    [2013-02-01T05:24:20+00:00] INFO: Report handlers complete

    View full-size slide

  29. minitest
    class TestWebsite < MiniTest::Chef::TestCase
    include MiniTest::Chef::Assertions
    include MiniTest::Chef::Context
    include MiniTest::Chef::Resources
    def test_succeed
    assert run_status.success?
    end
    def test_that_the_package_installed
    package("apache2").must_be_installed
    end
    def test_that_the_service_is_running
    service("apache2").must_be_running
    end

    View full-size slide

  30. minitest
    def test_that_the_service_is_enabled
    service("apache2").must_be_enabled
    end
    def test_home_page_file
    file("/var/www/index.html").must_exist
    end
    def test_home_page_content
    file("/var/www/index.html").must_include node['conference']['name']
    end

    View full-size slide

  31. minitest-handler
    $ vagrant provision
    [2013-02-01T06:43:34+00:00] INFO: Running report handlers
    Run options: -v --seed 12405
    # Running tests:
    TestWebsite#test_succeed =
    ...
    Finished tests in 0.098367s, 60.9959 tests/s, 60.9959 assertions/s.
    6 tests, 6 assertions, 0 failures, 0 errors, 0 skips
    [2013-02-01T06:43:34+00:00] INFO: Report handlers complete

    View full-size slide

  32. Testing Tools
    • Vagrant
    • knife cookbook test
    • Foodcritic
    • Chefspec
    • Fauxhai
    • Minitest and the Minitest Chef Handler
    • Why-run

    View full-size slide

  33. Launch a Staging Node
    $ knife ec2 server create -I ami-f4fb7c9d -f m1.small -E staging
    Instance ID: i-1e3a4b6d
    Flavor: m1.small
    Image: ami-f4fb7c9d
    Region: us-east-1
    Availability Zone: us-east-1d
    Security Groups: default
    Tags: {"Name"=>"i-1e3a4b6d"}
    SSH Key: nharvey-oc
    Waiting for server....................
    Public DNS Name: ec2-107-22-155-182.compute-1.amazonaws.com
    Public IP Address: 107.22.155.182
    Private DNS Name: ip-10-190-223-78.ec2.internal
    Private IP Address: 10.190.223.78
    Waiting for sshd....done
    Bootstrapping Chef on ec2-107-22-155-182.compute-1.amazonaws.com

    View full-size slide

  34. Launch a Staging Node
    $ knife node run_list add i-1e3a4b6d "recipe[website]"
    i-1e3a4b6d:
    run_list:
    recipe[website]

    View full-size slide

  35. Why Run
    $ knife ssh "chef_environment:staging" "sudo chef-client --why-run"

    View full-size slide

  36. knife ssh chef_enviornment:staging

    View full-size slide

  37. knife ssh chef_environment:staging "sudo chef-client --why-run"

    View full-size slide

  38. Why Run
    $ knife ssh "chef_environment:staging" "sudo chef-client --why-run"
    Starting Chef Client, version 11.4.0
    ...
    Converging 3 resources
    * package[apache2] action install
    - Would install version 2.2.22-1ubuntu1 of package apache2
    * template[/var/www/index.html] action create
    * Parent directory /var/www does not exist.
    * Assuming directory /var/www would have been created
    - Would create template[/var/www/index.html]
    * service[apache2] action start
    - Would start service service[apache2]
    * service[apache2] action enable
    - Would enable service service[apache2]
    WARN: In whyrun mode, so NOT performing node save.
    Chef Client finished, 4 resources would have been updated

    View full-size slide

  39. Why Run
    • Promises, Lies, and Dry-Run Mode
    • http://bit.ly/guessrun
    • Why Run reported 4 resource updated
    • “real run” only 2 resources are updated

    View full-size slide

  40. Testing Tools
    • Vagrant - Local development and testing
    • knife cookbook test - Verify ruby syntax
    • Foodcritic - Cookbook linter
    • Chefspec - Unit testing recipes
    • Fauxhai - Mock all the things
    • Minitest Chef Handler - post-converge tests
    • Why-run - Best guess

    View full-size slide

  41. Moar Testing Tools
    • Test Kitchen
    • Cross-platform testing
    • Cucumber Chef
    • acceptance & integration testing

    View full-size slide

  42. Continuous Integration
    •Travis
    •Jenkins
    •etc.

    View full-size slide

  43. Gating Your Releases
    https://github.com/nathenharvey/bigruby

    View full-size slide

  44. Requirements Have Changed!

    View full-size slide

  45. Bump Cookbook Version

    View full-size slide

  46. Update Tests

    View full-size slide

  47. Watch Tests Fail
    $ rpsec .
    5 examples, 1 failure
    Failed examples:
    rspec ./cookbooks/website/spec/default_spec.rb:18
    # website::default should create a home page with our content
    $ vagrant provision
    FATAL: RuntimeError: MiniTest failed with 0 failure(s) and 1 error(s).
    Error:
    test_home_page_content(TestWebsite):

    View full-size slide

  48. Update Cookbook

    View full-size slide

  49. Converge Development Node
    $ rspec .
    5 examples, 0 failures
    $ knife cookbook upload website && vagrant provision
    6 tests, 7 assertions, 0 failures, 0 errors, 0 skips

    View full-size slide

  50. Converge Staging Node
    $ knife ssh "chef_environment:staging" "sudo chef-client"
    Starting Chef Client, version 11.4.0
    ...
    Chef Client finished, 0 resources updated

    View full-size slide

  51. Y U No Change?
    {
    "name": "staging",
    "description": "",
    "cookbook_versions": {
    "website" : "0.1.2"
    },
    "json_class": "Chef::Environment",
    "chef_type": "environment",
    "default_attributes": {
    },
    "override_attributes": {
    }
    }

    View full-size slide

  52. Promote the New Version

    View full-size slide

  53. Converge the Staging Node
    $ knife environment from file staging.json
    Updated Environment staging
    $ knife ssh "chef_environment:staging" "sudo chef-client"
    Starting Chef Client, version 11.4.0
    ...
    Chef Client finished, 1 resources updated

    View full-size slide

  54. Environments & Continuous Integration
    • Tests + Environments + Continuous Integration
    • ONE Build Pipeline to Rule them All

    View full-size slide

  55. So, What Have We Learned?
    https://github.com/nathenharvey/bigruby

    View full-size slide

  56. Testing Your Infrastructure
    • There are a number of frameworks to
    automate your infrastructure testing
    • Graduate changes through your
    environment
    • Trust your monitoring to catch issues
    quickly

    View full-size slide

  57. Chef is Awesome!
    • Manage all of the infrastructure for your
    single page, static website!

    View full-size slide

  58. Chef is Awesome!

    View full-size slide

  59. Knife Rocks!
    • Command line interface for
    • Creating Cookbooks
    • Searching the Chef Server
    • Provisioning Infrastructure
    • ...and more!

    View full-size slide

  60. The Chef Community is Awesome!
    • Things in this presentation that exist
    because of the Community:
    • Foodcritic
    • Chefspec
    • minitest-chef-handler
    • Fauxhai
    • Cucumber Chef

    View full-size slide

  61. How can you help?
    • You are a developer well-versed in Ruby
    test frameworks & TDD
    • Test frameworks & TDD are relatively
    new to infrastructure code

    View full-size slide

  62. How can you help?
    •Stop testurbation!

    View full-size slide

  63. #ChefConf 2013

    View full-size slide

  64. Thank You
    •What questions do you have?
    •Nathen Harvey
    [email protected]
    •@nathenharvey

    View full-size slide