Testing Your Automation

TDD for Chef Cookbooks

Transcript

1. Testing Your Automation
   TDD for Chef Cookbooks
   Nathen Harvey
   @nathenharvey
   https://github.com/nathenharvey/bigruby
   

   View Slide

2. What is Chef?
   Recipes and Cookbooks
   that describe Infrastructure as Code.
   Chef enables people to easily build &
   manage complex & dynamic applications
   at massive scale
   • New model for describing infrastructure that
   promotes reuse
   • Programmatically provision and configure
   • Reconstruct business from code repository,
   data backup, and bare metal resources
   Chef is an automation platform for developers & systems engineers to continuously
   define, build, and manage infrastructure.
   CHEF USES:
   “
   ”
   

   View Slide

3. View Slide

4. Writing Infrastructure Code
   https://github.com/nathenharvey/bigruby
   

   View Slide

5. Evolving towards Configuration Management
   •Just build it
   •Keep notes in server.txt
   •Move notes to the wiki
   •Custom scripts (in scm?!)
   •Snapshot & Clone
   

   View Slide

6. Building something with Chef
   •Come up with your
   policy
   •Abstract the resources
   •Write recipes
   •Apply recipes to nodes
   

   View Slide

7. Trivial Example
   

   View Slide

8. Cookbook
   $ knife cookbook create website
   ** Creating cookbook website
   ** Creating README for cookbook: website
   ** Creating CHANGELOG for cookbook: website
   ** Creating metadata for cookbook: website
   

   View Slide

9. Resources in Recipe
   package "apache2"
   template "/var/www/index.html" do
   owner "www-data"
   group "www-data"
   mode 644
   source "index.html.erb"
   end
   service "apache2" do
   action [:start, :enable]
   end
   

   View Slide

10. templates/default/index.html.erb
    
    
    <%= node['conference']['name'] %>
    
    
    Hello, <%= node['conference']['name'] %>
    
    
    • I told you, this is a trivial example! Stick with me.
    • No, you wouldn’t really manage your content in
    Chef but this may be easier to grok than sysctl
    settings.
    

    View Slide

11. attributes/default.rb
    default['conference']['name'] = "Big Ruby"
    

    View Slide

12. Apply This Recipe to a Node
    $ vagrant init web
    A `Vagrantfile` has been placed in this directory. You are now
    ready to `vagrant up` your first virtual environment! Please
    read the comments in the Vagrantfile as well as documentation
    on `vagrantup.com` for more information on using Vagrant.
    

    View Slide

13. Vagrantfile
    # -*- mode: ruby -*-
    # vi: set ft=ruby :
    Vagrant::Config.run do |config|
    config.vm.box = "web"
    config.vm.forward_port 80, 8080
    config.vm.provision :chef_client do |chef|
    chef.chef_server_url = "https://api.opscode.com/organizations/bigruby"
    chef.validation_key_path = "bigruby-validator.pem"
    chef.validation_client_name = "bigruby-validator"
    end
    end
    

    View Slide

14. Launch the VM
    $ vagrant up
    [default] Importing base box 'web'...
    [default] Matching MAC address for NAT networking...
    [default] Forwarding ports...
    [default] -- 22 => 2222 (adapter 1)
    [default] -- 80 => 8080 (adapter 1)
    ...
    [2013-02-01T05:24:19+00:00] WARN: Node vagrant.vm has an empty run list.
    [2013-02-01T05:24:20+00:00] INFO: Chef Run complete in 1.962777136 seconds
    [2013-02-01T05:24:20+00:00] INFO: Running report handlers
    [2013-02-01T05:24:20+00:00] INFO: Report handlers complete
    

    View Slide

15. Provision the Node
    $ knife cookbook upload website
    Uploading website [0.1.0]
    Uploaded 1 cookbook.
    $ knife node run_list add vagrant.vm "recipe[website]"
    vagrant.vm:
    run_list: recipe[website]
    $ vagrant provision
    

    View Slide

16. Test
    

    View Slide

17. Testing Infrastructure Code
    https://github.com/nathenharvey/bigruby
    

    View Slide

18. Why test?
    

    View Slide

19. Testing Tools
    • Vagrant
    • knife cookbook test
    

    View Slide

20. knife cookbook test
    $ knife cookbook test website
    checking website
    Running syntax check on website
    Validating ruby files
    Validating templates
    

    View Slide

21. Testing Tools
    • Vagrant
    • knife cookbook test
    • Foodcritic
    

    View Slide

22. Foodcritic
    • A lint tool for your Opscode Chef
    cookbooks
    • Flag problems in your Chef cookbooks
    that will cause Chef to blow up when
    you attempt to converge
    • Encourage discussion within the Chef
    community on the more subjective stuff
    - what does a good cookbook look like?
    

    View Slide

23. Foodcritic
    $ foodcritic cookbooks/website
    FC006: Mode should be quoted or fully specified when setting file permissions:
    cookbooks/website/recipes/default.rb:11
    FC008: Generated cookbook metadata needs updating:
    cookbooks/website/metadata.rb:2
    FC008: Generated cookbook metadata needs updating:
    cookbooks/website/metadata.rb:3
    

    View Slide

24. Foodcritic
    

    View Slide

25. Foodcritic
    

    View Slide

26. Testing Tools
    • Vagrant
    • knife cookbook test
    • Foodcritic
    • Chefspec
    

    View Slide

27. Chefspec
    require 'chefspec'
    describe 'website::default' do
    chef_run = ChefSpec::ChefRunner.new
    chef_run.converge "website::default"
    it "should install apache package" do
    chef_run.should install_package "apache2"
    end
    it "should create a home page" do
    chef_run.should create_file "/var/www/index.html"
    end
    

    View Slide

28. Chefspec
    it "should create a home page with our content" do
    chef_run.should create_file_with_content(
    "/var/www/index.html","Big Ruby")
    end
    it "should start the apache service" do
    chef_run.should start_service "apache2"
    end
    it "should enable the apache service" do
    chef_run.should enable_service "apache2"
    end
    end
    

    View Slide

29. Chefspec
    chef_run.should create_file_with_content(
    "/var/www/index.html","Big Ruby")
    <%= node['conference']['name'] %>
    default['conference']['name'] = "Big Ruby"
    

    View Slide

30. "memory": {
    "swap": {
    "cached": "0kB",
    "total": "4128760kB",
    "free": "4128760kB"
    },
    "total": "2055676kB",
    "free": "1646524kB",
    "buffers": "35032kB",
    "cached": "210276kB",
    "active": "125336kB",
    "inactive": "142884kB",
    "dirty": "8kB",
    "writeback": "0kB",
    "anon_pages": "22976kB",
    "mapped": "8416kB",
    "slab": "121512kB",
    "slab_reclaimable": "41148kB",
    "slab_unreclaim": "80364kB",
    "page_tables": "1784kB",
    "nfs_unstable": "0kB",
    "bounce": "0kB",
    "commit_limit": "5156596kB",
    "committed_as": "74980kB",
    "vmalloc_total": "34359738367kB",
    "vmalloc_used": "274512kB",
    "vmalloc_chunk": "34359449936kB"
    },
    Ohai!
    "block_device": {
    "ram0": {
    "size": "32768",
    "removable": "0"
    },
    "ram1": {
    "size": "32768",
    "removable": "0"
    },
    "ram2": {
    "size": "32768",
    "removable": "0"
    },
    "hostname": "server-1",
    "fqdn": "server-1.example.com",
    "domain": "example.com",
    "network": {
    "interfaces": {
    "eth0": {
    "type": "eth",
    "number": "0",
    "encapsulation": "Ethernet",
    "addresses": {
    "00:0C:29:43:26:C5": {
    "family": "lladdr"
    },
    "192.168.177.138": {
    "family": "inet",
    "broadcast": "192.168.177.255",
    "netmask": "255.255.255.0"
    },
    "fe80::20c:29ff:fe43:26c5": {
    "family": "inet6",
    "prefixlen": "64",
    "scope": "Link"
    }
    },
    

    View Slide

31. Chefspec & Fauxhai
    

    View Slide

32. Testing Tools
    • Vagrant
    • knife cookbook test
    • Foodcritic
    • Chefspec
    • Fauxhai
    • Minitest and the Minitest Chef Handler
    

    View Slide

33. Handlers
    $ vagrant up
    [default] Importing base box 'web'...
    [default] Matching MAC address for NAT networking...
    [default] Forwarding ports...
    [default] -- 22 => 2222 (adapter 1)
    [default] -- 80 => 8080 (adapter 1)
    ...
    [2013-02-01T05:24:19+00:00] WARN: Node vagrant.vm has an empty run list.
    [2013-02-01T05:24:20+00:00] INFO: Chef Run complete in 1.962777136 seconds
    [2013-02-01T05:24:20+00:00] INFO: Running report handlers
    [2013-02-01T05:24:20+00:00] INFO: Report handlers complete
    

    View Slide

34. minitest
    class TestWebsite < MiniTest::Chef::TestCase
    include MiniTest::Chef::Assertions
    include MiniTest::Chef::Context
    include MiniTest::Chef::Resources
    def test_succeed
    assert run_status.success?
    end
    def test_that_the_package_installed
    package("apache2").must_be_installed
    end
    def test_that_the_service_is_running
    service("apache2").must_be_running
    end
    

    View Slide

35. minitest
    def test_that_the_service_is_enabled
    service("apache2").must_be_enabled
    end
    def test_home_page_file
    file("/var/www/index.html").must_exist
    end
    def test_home_page_content
    file("/var/www/index.html").must_include node['conference']['name']
    end
    

    View Slide

36. minitest-handler
    $ vagrant provision
    [2013-02-01T06:43:34+00:00] INFO: Running report handlers
    Run options: -v --seed 12405
    # Running tests:
    TestWebsite#test_succeed =
    ...
    Finished tests in 0.098367s, 60.9959 tests/s, 60.9959 assertions/s.
    6 tests, 6 assertions, 0 failures, 0 errors, 0 skips
    [2013-02-01T06:43:34+00:00] INFO: Report handlers complete
    

    View Slide

37. Testing Tools
    • Vagrant
    • knife cookbook test
    • Foodcritic
    • Chefspec
    • Fauxhai
    • Minitest and the Minitest Chef Handler
    • Why-run
    

    View Slide

38. Launch a Staging Node
    $ knife ec2 server create -I ami-f4fb7c9d -f m1.small -E staging
    Instance ID: i-1e3a4b6d
    Flavor: m1.small
    Image: ami-f4fb7c9d
    Region: us-east-1
    Availability Zone: us-east-1d
    Security Groups: default
    Tags: {"Name"=>"i-1e3a4b6d"}
    SSH Key: nharvey-oc
    Waiting for server....................
    Public DNS Name: ec2-107-22-155-182.compute-1.amazonaws.com
    Public IP Address: 107.22.155.182
    Private DNS Name: ip-10-190-223-78.ec2.internal
    Private IP Address: 10.190.223.78
    Waiting for sshd....done
    Bootstrapping Chef on ec2-107-22-155-182.compute-1.amazonaws.com
    

    View Slide

39. Launch a Staging Node
    $ knife node run_list add i-1e3a4b6d "recipe[website]"
    i-1e3a4b6d:
    run_list:
    recipe[website]
    

    View Slide

40. Why Run
    $ knife ssh "chef_environment:staging" "sudo chef-client --why-run"
    

    View Slide

41. knife ssh chef_enviornment:staging
    

    View Slide

42. knife ssh chef_environment:staging "sudo chef-client --why-run"
    

    View Slide

43. Why Run
    $ knife ssh "chef_environment:staging" "sudo chef-client --why-run"
    Starting Chef Client, version 11.4.0
    ...
    Converging 3 resources
    * package[apache2] action install
    - Would install version 2.2.22-1ubuntu1 of package apache2
    * template[/var/www/index.html] action create
    * Parent directory /var/www does not exist.
    * Assuming directory /var/www would have been created
    - Would create template[/var/www/index.html]
    * service[apache2] action start
    - Would start service service[apache2]
    * service[apache2] action enable
    - Would enable service service[apache2]
    WARN: In whyrun mode, so NOT performing node save.
    Chef Client finished, 4 resources would have been updated
    

    View Slide

44. Why Run
    • Promises, Lies, and Dry-Run Mode
    • http://bit.ly/guessrun
    • Why Run reported 4 resource updated
    • “real run” only 2 resources are updated
    

    View Slide

45. Testing Tools
    • Vagrant - Local development and testing
    • knife cookbook test - Verify ruby syntax
    • Foodcritic - Cookbook linter
    • Chefspec - Unit testing recipes
    • Fauxhai - Mock all the things
    • Minitest Chef Handler - post-converge tests
    • Why-run - Best guess
    

    View Slide

46. Moar Testing Tools
    • Test Kitchen
    • Cross-platform testing
    • Cucumber Chef
    • acceptance & integration testing
    

    View Slide

47. Continuous Integration
    •Travis
    •Jenkins
    •etc.
    

    View Slide

48. Gating Your Releases
    https://github.com/nathenharvey/bigruby
    

    View Slide

49. Requirements Have Changed!
    

    View Slide

50. Bump Cookbook Version
    

    View Slide

51. Update Tests
    

    View Slide

52. Watch Tests Fail
    $ rpsec .
    5 examples, 1 failure
    Failed examples:
    rspec ./cookbooks/website/spec/default_spec.rb:18
    # website::default should create a home page with our content
    $ vagrant provision
    FATAL: RuntimeError: MiniTest failed with 0 failure(s) and 1 error(s).
    Error:
    test_home_page_content(TestWebsite):
    

    View Slide

53. Update Cookbook
    

    View Slide

54. Converge Development Node
    $ rspec .
    5 examples, 0 failures
    $ knife cookbook upload website && vagrant provision
    6 tests, 7 assertions, 0 failures, 0 errors, 0 skips
    

    View Slide

55. Verify
    

    View Slide

56. Converge Staging Node
    $ knife ssh "chef_environment:staging" "sudo chef-client"
    Starting Chef Client, version 11.4.0
    ...
    Chef Client finished, 0 resources updated
    

    View Slide

57. Y U No Change?
    {
    "name": "staging",
    "description": "",
    "cookbook_versions": {
    "website" : "0.1.2"
    },
    "json_class": "Chef::Environment",
    "chef_type": "environment",
    "default_attributes": {
    },
    "override_attributes": {
    }
    }
    

    View Slide

58. Promote the New Version
    

    View Slide

59. Converge the Staging Node
    $ knife environment from file staging.json
    Updated Environment staging
    $ knife ssh "chef_environment:staging" "sudo chef-client"
    Starting Chef Client, version 11.4.0
    ...
    Chef Client finished, 1 resources updated
    

    View Slide

60. Environments & Continuous Integration
    • Tests + Environments + Continuous Integration
    • ONE Build Pipeline to Rule them All
    

    View Slide

61. Monitoring!
    

    View Slide

62. So, What Have We Learned?
    https://github.com/nathenharvey/bigruby
    

    View Slide

63. Testing Your Infrastructure
    • There are a number of frameworks to
    automate your infrastructure testing
    • Graduate changes through your
    environment
    • Trust your monitoring to catch issues
    quickly
    

    View Slide

64. Chef is Awesome!
    • Manage all of the infrastructure for your
    single page, static website!
    •
    

    View Slide

65. Chef is Awesome!
    

    View Slide

66. Knife Rocks!
    • Command line interface for
    • Creating Cookbooks
    • Searching the Chef Server
    • Provisioning Infrastructure
    • ...and more!
    

    View Slide

67. The Chef Community is Awesome!
    • Things in this presentation that exist
    because of the Community:
    • Foodcritic
    • Chefspec
    • minitest-chef-handler
    • Fauxhai
    • Cucumber Chef
    

    View Slide

68. How can you help?
    • You are a developer well-versed in Ruby
    test frameworks & TDD
    • Test frameworks & TDD are relatively
    new to infrastructure code
    

    View Slide

69. How can you help?
    •Stop testurbation!
    

    View Slide

70. #ChefConf 2013
    

    View Slide

71. Thank You
    •What questions do you have?
    •Nathen Harvey
    •nharvey@opscode.com
    •@nathenharvey
    

    View Slide