TDD Chef

Using Test Driven Development to write Chef cookbooks and recipes and adopting the "Infrastructure as Code" way. Including an introduction of what infrastructure is about, what is using testing, etc...

Transcript

1. TD Chef D Test Driven Infrastructure
2. Evgeny Zislis www.devops.co.il
3. Agenda ◦ History of Infrastructure Automation ◦ Infrastructure as Code ◦ OpsCode Chef ◦ Unit Tests ◦ Test Driving Chef ◦ ... Code examples ...
4. What is Infrastructure? ◦ Configuration ◦ Services ◦ OS tuning ◦ Patches ◦ Cloud Server Provision
5. Cloud Infrastructure
6. awsofa.info aws.amazon.com/solutions/case-studies/obama/ Disposable Infrastructure
7. requests # of servers CPU aggregate techblog.netflix.com/2012/01/auto-scaling-in-amazon-cloud.html
8. “... began in 1993 as a way for author Mark Burgess to get his work done by automating the management of a small group of workstations in the Department of Theoretical Physics. Scripting took too much time, the flavours of Unix were significantly different, and scripts had to be maintained for multiple platforms, drowning in exception logic.”
9. Infrastructure as Code “In today’s computer industry, we still typically install and maintain computers the way the automotive industry built cars in the early 1900s. An individual craftsman manually manipulates a machine into being, and manually maintains it afterwards. The automotive industry discovered first mass production, then mass customisation using standard tooling. The systems administration industry has a long way to go, but is getting there.” — 1997, infrastructures.org
10. “Earlier today members were disconnected from Xbox LIVE and found themselves unable to log back in. … The root cause of this outage was human error.” blogs.msdn.com/b/xblops/archive/2011/10/03/issues-with-xbox-live-earlier-today.aspx
13. zeroturnaround.com/labs/rebel-labs-release-it-ops-devops-productivity-report-2013/
14. zeroturnaround.com/labs/rebel-labs-release-it-ops-devops-productivity-report-2013/
15. zeroturnaround.com/labs/rebel-labs-release-it-ops-devops-productivity-report-2013/
16. Chef
17. ◦ open source ◦ configures servers ◦ cookbooks & recipes ◦ ruby dsl OpsCode Chef
18. redis cookbook cookbooks/redis/recipes/default.rb package "redis-server" do action :upgrade end service "redis-server" do action :nothing supports status: true, restart: true end template "/etc/redis/redis.conf" do source "redis.conf.erb" owner "root" mode "0644" variables({ bind: node[:redis][:bind] }) notifies :restart, "service[redis-server]" end 1. cookbook/redis/attributes/default.rb 2. solo.json
19. cookbooks/redis/attributes/default.rb default[:redis][:bind] = "127.0.0.1" cookbooks/redis/templates/default/redis.conf.erb # If you want you can bind a single interface, if the bind option is not # specified all the interfaces will listen for incoming connections. bind <%= @bind %> ... template "/etc/redis/redis.conf" do ... variables({ bind: node[:redis][:bind] }) end redis cookbook
20. running chef-solo $ chef-solo -c solo.rb -j solo.json ... solo.rb cookbook_path "cookbooks" solo.json { "run_list": "recipe[redis]" }
21. $ sudo chef-solo -c solo.rb -j solo.json Starting Chef Client, version 11.4.0 Compiling Cookbooks... Converging 3 resources Recipe: redis::default * package[redis-server] action upgrade - upgrade package redis-server from uninstalled to 2:2.2.12-1build1 * service[redis-server] action nothing (up to date) * template[/etc/redis/redis.conf] action create (up to date) Chef Client finished, 1 resources updated First run installs redis, creates configuration file.
22. Second time, everything is already up to date. Idempotency! ƒ[ƒ(x)] ≡ ƒ(x) $ sudo chef-solo -c solo.rb -j solo.json Starting Chef Client, version 11.4.0 Compiling Cookbooks... Converging 3 resources Recipe: redis::default * package[redis-server] action upgrade (up to date) * service[redis-server] action nothing (up to date) * template[/etc/redis/redis.conf] action create (up to date) Chef Client finished, 0 resources updated
23. making a small change to JSON parameters passed to Chef run. running chef-solo + modification solo.json { "redis": { "bind": "0.0.0.0" }, "run_list": "recipe[redis]" }
24. $ sudo chef-solo -c solo.rb -j solo.json Starting Chef Client, version 11.4.0 Compiling Cookbooks... Converging 3 resources Recipe: redis::default * package[redis-server] action upgrade (up to date) * service[redis-server] action nothing (up to date) * template[/etc/redis/redis.conf] action create - update template[/etc/redis/redis.conf] from 81b4f1 to 8a6cec --- /etc/redis/redis.conf 2011-07-27 17:26:50.000000000 +0000 +++ /tmp/chef-rendered-template20130406-2537-f8vlv6 2013-04-06 ... @@ -27,7 +27,7 @@ # If you want you can bind a single interface, if the bind option is not # specified all the interfaces will listen for incoming connections. # -bind 127.0.0.1 +bind 0.0.0.0 # Specify the path for the unix socket that will be used to listen for # incoming connections. There is no default, so Redis will not listen * service[redis-server] action restart - restart service service[redis-server] Chef Client finished, 2 resources updated install package run service configuration + restart
25. Unit Testing
26. What is a Test? #!/usr/bin/env bats @test "git binary is found in PATH" { run "which git" [ "$status" -eq 0 ] }
27. What is TDD? ◦ Test First ◦ Red - Green - Refactor ◦ Unit vs. Integration
28. codinghorror.com/blog/2006/07/i-pity-the-fool-who-doesnt-write-unit-tests.html
29. ◦ test-unit ◦ minitest ◦ rspec ruby-toolbox.com/categories/testing_frameworks Unit Test libraries in Ruby
30. sethvargo.com/unit-testing-correctly/ require 'net/http' class Scraper attr_reader :webpage def initialize(webpage = 'http://sethvargo.com') @webpage = URI.parse(webpage) write Net::HTTP.get(@webpage) end def write(contents) File.open("#{@webpage.host}.html", 'w') do |file| file.write(contents) end end end Scraper.new
31. sethvargo.com/unit-testing-correctly/ describe Scraper do it 'parses the uri' do subject.webpage.should be_a URI subject.webpage.host.should == 'sethvargo.com' end it 'writes the file' do File.read('sethvargo.com.html').should match "Unit Testing Chef Cookbooks" end end
32. • It requires an Internet connection • It's slowwwwwwww • It's not idempotent This is NOT a Unit Test! sethvargo.com/unit-testing-correctly/ describe Scraper do it 'parses the uri' do subject.webpage.should be_a URI subject.webpage.host.should == 'sethvargo.com' end it 'writes the file' do File.read('sethvargo.com.html').should match "Unit Testing Chef Cookbooks" end end
33. What is a Unit Test? ◦ individual units - asap ◦ control data ◦ fit for use In computer programming, unit testing is a method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures are tested to determine if they are fit for use. wikipedia.org/wiki/Unit_testing
34. Unit Test Benefits ◦ Find problems early ◦ Facilitate change ◦ Simplify integration ◦ Documentation ◦ Design
35. a Chef Testing
37. Chef Testing ◦ chefspec code.sethvargo.com/chefspec/ ◦ cucumber chef cucumber-chef.org ◦ minitest handler github.com/btm/minitest-handler-cookbook ◦ test kitchen / busser kitchen.ci 2011
38. require 'chefspec' describe 'mycompany-nginx::default' do let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) } it 'includes the nginx cookbook' do expect(chef_run).to include_recipe(nginx::default') end it 'installs nginx' do expect(chef_run).to install_package('nginx') end end chefspec
39. Feature: Technical team members can log into team server So that we are more efficient with our time As a technical team We can connect to a shared server to collaborate on client work Scenario: Users can connect to server via ssh key Given a newly bootstrapped server When the technical users recipe is applied Then a user should be able to ssh to the server Scenario: Default shell is bash Given a newly bootstrapped server When the technical users recipe is applied And a user connects to the server via ssh Then their login shell should be "bash" 2 scenarios (2 undefined) 12 steps (7 undefined) 0m0.013s Given /^a newly bootstrapped server$/ do ... When /^the technical users recipe is applied$/ do ... Then /^a user should be able to ssh to the server$/ do When /^a user connects to the server via ssh$/ do ... Then /^their login shell should be "([^"]*)"$/ do |arg1| Given /^with a user's default shell changed to "([^"]*)"$/ do |arg1| ... When /^the user connects to the server via ssh$/ do ... Cucumber Chef
40. # Test Cases class TestNginx < MiniTest::Chef::TestCase def test_config_file_exist assert File.exist?('/etc/nginx.conf') end def test_succeed assert run_status.success? end end chef-minitest # Spec Cases describe_recipe 'nginx:configuration' do it 'creates nginx.conf' it 'installs version 1.0.15' do node[:nginx][:version].should == '1.4.4' end end
41. $ kitchen verify default-ubuntu-1204 -----> Starting Kitchen (v1.0.0) -----> Verifying <default-ubuntu-1204>... Removing /tmp/busser/suites/bats Uploading /tmp/busser/suites/bats/git_installed.bats (mode=0644) -----> Running bats test suite ✓ git binary is found in PATH 1 test, 0 failures Finished verifying <default-ubuntu-1204> (0m0.89s). -----> Kitchen is finished. (0m1.19s) kitchen.ci | test-kitchen | busser
42. #!/usr/bin/env bats @test "addition using bc" { result="$(echo 2+2 | bc)" [ "$result" -eq 4 ] } @test "addition using dc" { result="$(echo 2 2+p | dc)" [ "$result" -eq 4 ] } @test "invoking foo without arguments prints usage" { run foo [ "$status" -eq 1 ] [ "${lines[0]}" = "usage: foo <filename>" ] } kitchen.ci - bats via busser