Test Driven Infrastructure with Chef

Test Driven Infrastructure with Chef

These are the slides from my 2013 Velocity talk on Chef Driven Infrastructure with Chef using ChefSpec, Test Kitchen, GitHub, Jenkins, and more!

502828deee7e3b38ca1e527dded8a1a9?s=128

Seth Vargo

June 18, 2013
Tweet

Transcript

  1. Test-Driven Infrastructure

  2. sethvargo@opscode.com E b yz

  3. 1WORKSTATION SETUP BBB

  4. 1#LEARNCHEF BBB

  5. 1BBB

  6. 1cl.ly/PiLZ sha: 821ecd0 BBB

  7. $ knife client list

  8. 1GITHUB  

  9. 2CREATE THE COOKBOOK fdd

  10. $ knife cookbook create {username}-myface

  11. $ knife cookbook create sethvargo-myface

  12. fmyface d metadata.rb frecipes d default.rb

  13. fmyface d metadata.rb frecipes d default.rb

  14. fmyface d metadata.rb frecipes d default.rb

  15. fmyface d metadata.rb frecipes d default.rb

  16. fmyface d metadata.rb frecipes d default.rb

  17. imyface

  18. None
  19. $ (sudo) gem install bundler

  20. u gist.github.com/5737141

  21. fmyface d Gemfile

  22. Gemfile source 'https://rubygems.org' group :test do gem 'chefspec', '~> 1.3'

    gem 'foodcritic', '~> 2.1' gem 'strainer', '~> 3.0' gem 'test-kitchen', '~> 1.0.0.alpha' gem 'kitchen-lxc', '~> 0.0.1.beta1' gem 'knife-spork', '~> 1.0.17' gem 'hipchat', '~> 0.10.0' gem 'guard', '~> 1.8' gem 'guard-foodcritic', '~> 1.0' gem 'guard-rspec', '~> 3.0' end
  23. $ (sudo) bundle

  24. fmyface d spec_helper.rb fspec d default_spec.rb

  25. fmyface d spec_helper.rb fspec d default_spec.rb

  26. fmyface d spec_helper.rb fspec d default_spec.rb

  27. spec/default_spec.rb require 'spec_helper' describe 'sethvargo-myface::default' do let(:chef_run) do run =

    ChefSpec::ChefRunner.new(platfrom: 'ubuntu', version: '12.04') run.converge('sethvargo-myface::default') end it 'installs apache2' do expect(chef_run).to install_package('apache2') end # ... end
  28. fmyface d spec_helper.rb fspec d default_spec.rb

  29. spec/spec_helper.rb require 'chefspec'

  30. fmyface d Guardfile d Strainerfile

  31. fmyface d Guardfile d Strainerfile

  32. Guardfile guard :rspec, all_on_start: false do watch(%r{^spec/.+_spec\.rb$}) watch(%r{^(recipes)/(.+)\.rb$}) { |m|

    "spec/#{m[1]}_spec.rb" } watch('spec/spec_helper.rb') { 'spec' } end guard :foodcritic, cookbook_paths: '.', all_on_start: false do watch(%r{attributes/.+\.rb$}) watch(%r{providers/.+\.rb$}) watch(%r{recipes/.+\.rb$}) watch(%r{resources/.+\.rb$}) end
  33. $ bundle exec guard

  34. None
  35. ⌘ + S

  36. None
  37. None
  38. 0.0134s

  39. fmyface d Guardfile d Strainerfile

  40. Strainerfile knife: bundle exec knife cookbook test $COOKBOOK foodcritic: bundle

    exec foodcritic $SANDBOX/$COOKBOOK -f any rspec: (cd $COOKBOOK && bundle exec rspec --color)
  41. fmyface d metadata.rb frecipes d default.rb

  42. fmyface d metadata.rb frecipes d default.rb

  43. package 'apache2' service 'apache2' do action [:enable, :start] end recipes/default.rb

  44. ⌘ + S

  45. POLICY CHANGE G

  46. RUN UNIT TESTS MASTER BEFORE MERGING TO

  47. ! RUN UNIT TESTS MASTER BEFORE MERGING TO

  48. u HUMAN (PROS) D J h a K

  49. 7 r c L M t HUMAN (CONS)

  50. None
  51. None
  52. None
  53. None
  54. None
  55. 1CREATE REPO

  56. 1GITHUB INTERFACE ?

  57. 1 myface T

  58. 1 (anything) T

  59. 1Create repository J

  60. 1

  61. 1

  62. $ git add . $ git commit -m "myface" $

    git push
  63. 1

  64. 2SETUP JOB

  65. S

  66. I, _________, solemnly swear under penalty of my GPU catching

    fire, that I will not touch anyone's build but my own.  NO ASSHOLE PLEDGE
  67. 2ci.trychef.com O

  68. 2 New Job J

  69. 2

  70. 2{username}-myface T

  71. 2Build a free-style... J

  72. 2 OK J

  73. 2

  74. 2(github URL) T

  75. 2 Git J

  76. 2git://github.com/... T

  77. 2Build when a change... J

  78. 2Color ANSI Console... J

  79. 2Add Build Step J

  80. 2Execute Shell J

  81. 2 bundle install --path ../../support/vendor bundle exec strainer test T

  82. 2Email Notification J

  83. 2(your email) T

  84. 2 Save J

  85. 2

  86. 2 Build Now J

  87. 2 (the build) J

  88. 3SETUP HOOK

  89. 3 Settings J

  90. 3

  91. 3Service Hooks J

  92. 3 Jenkins (GitHub Plugin) J

  93. 3http://ci.trychef.com/github-webhook/ T

  94. 3http://ci.trychef.com/github-webhook/ T IMPORTANT

  95. 3 Active J

  96. 3Update settings J

  97. $ git checkout -b add_site

  98. fmyface d spec_helper.rb fspec d default_spec.rb

  99. spec/default_spec.rb describe 'sethvargo-myface::default' do # pre-existing tests # ... it

    'creates the default template' do expect(chef_run).to create_file('/var/www/index.html') end it 'creates the site with the correct content' do template = chef_run.template('/var/www/index.html') expect(template.owner).to eq('root') expect(template.group).to eq('root') end end
  100. None
  101. fmyface d metadata.rb frecipes d default.rb

  102. recipes/default.rb package 'apache2' service 'apache2' do action [:enable, :start] end

    template '/var/www/index.html' do owner 'root' group 'root' mode '0755' source 'index.html.erb' end
  103. fmyface ftemplates d index.html.erb fdefault

  104. templates/default/index.html.erb <html> <head> <title>Welcome to <%= node['fqdn'] %></title> </head> <body>

    <p>Here's everything you need to know about <%= node['fqdn'] %>:</p> <pre><%= JSON.pretty_generate(node.to_hash) %></pre> </body> </html>
  105. $ git add . $ git commit -m "Write out

    default site"
  106. $ git push origin add_site

  107. Pull Request J

  108. Send pull request J

  109. c

  110.  Merge pull request J

  111. J

  112. J (the build) J

  113. JConsole Output J

  114. J

  115. J

  116. J

  117. LET'S FIX THAT Y

  118. $ git pull origin master

  119. metadata.rb name 'myface' maintainer 'YOUR_COMPANY_NAME' maintainer_email 'YOUR_EMAIL' license 'All rights

    reserved' description 'Installs/Configures myface' long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version '0.1.0'
  120. metadata.rb name 'myface' maintainer 'Seth Vargo' maintainer_email 'sethvargo@opscode.com' license 'All

    rights reserved' description 'Installs/Configures myface' long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version '0.1.0'
  121. $ git add metadata.rb $ git commit -m "Fix metadata"

  122. $ git push origin master

  123. AWESOME

  124. $ knife cookbook upload myface

  125. S

  126. WHY?

  127. 1 WE HAVE TESTS

  128. 1 WE HAVE TESTS 2 WE HAVE JENKINS

  129. 1 WE HAVE TESTS 2 WE HAVE JENKINS 3 WE

    HAVE THE TECHNOLOGY
  130. $ knife cookbook upload myface

  131. $ knife cookbook upload myface G

  132. $ knife cookbook upload myface G M

  133. J Configure J

  134. J

  135. J

  136. JAdd post-build action J

  137. JPost build task J

  138. J

  139. J Strainer marked build OK T

  140. J bundle exec knife cookbook upload {username}-myface T

  141. J Save J

  142. J

  143. J Build Now J

  144. J

  145. J (the build) J

  146. JConsole Output J

  147. J

  148. J > knife cookbook list | grep sethvargo-myface sethvargo-myface 0.1.0

  149. AWESOME

  150. R

  151. WE KNOW WHEN OUR BUILD FAILS BECAUSE WE GET AN

    EMAIL FROM JENKINS
  152. BUT DO WE KNOW WHEN OUR COOKBOOKS ARE UPLOADED TO

    OUR CHEF SERVER?
  153. BUT DO WE KNOW WHEN OUR COOKBOOKS ARE UPLOADED TO

    OUR CHEF SERVER?
  154.  jonlives/knife-spork

  155. jonlives/knife-spork

  156. jonlives/knife-spork

  157. J Configure J

  158. J

  159. J

  160. J

  161. J bundle exec knife spork upload {username}-myface T

  162. J Escalate script execution... J

  163. J Save J

  164. q hipchat.com/sign_in

  165. q Launch the web app J

  166. qTry Chef J

  167. q

  168. J Build Now J

  169. q

  170. N THERE'S MORE!

  171. N THERE'S MORE!

  172. N TEST KITCHEN!

  173. fmyface d .kitchen.yml

  174. .kitchen.yml driver_plugin: lxc driver_config: use_sudo: true platforms: - name: ubuntu-12.04

    driver_config: base_container: ubuntu_12.04 username: ubuntu password: ubuntu suites: - name: default run_list: ["recipe[myface]"] attributes: {}
  175. Strainerfile knife: bundle exec knife cookbook test $COOKBOOK foodcritic: bundle

    exec foodcritic $SANDBOX/$COOKBOOK -f any rspec: (cd $COOKBOOK && bundle exec rspec --color) kitchen: (cd $COOKBOOK && bundle exec kitchen test)
  176. metadata.rb name 'myface' maintainer 'Seth Vargo' maintainer_email 'sethvargo@opscode.com' license 'All

    rights reserved' description 'Installs/Configures myface' long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version '0.1.0'
  177. metadata.rb name 'myface' maintainer 'Seth Vargo' maintainer_email 'sethvargo@opscode.com' license 'All

    rights reserved' description 'Installs/Configures myface' long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version '0.1.1'
  178. $ git add . $ git commit -m "Add Test

    Kitchen"
  179. $ git push origin master

  180. q

  181. None
  182. qUH OH!

  183. $ git revert HEAD

  184. $ git revert HEAD

  185. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb'

    end
  186. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb'

    end IF WE REMOVE THIS WILL THE FILE BE DELETED?
  187. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb'

    end NO
  188. FAST FORWARD CHANGES ARE ABSOLUTELY ALWAYS BETTER THAN REVERTING

  189. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb'

    end
  190. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb'

    action :delete end
  191. $ git add . $ git commit -m "Revert abc123"

    $ git push origin master
  192. chef + environments safer infrastructure

  193. None
  194. None
  195. None
  196. None
  197. environments/production.json { "name" : "production", "description" : "Production cluster in

    EC2", "override_attributes" : { ... }, "default_attributes" : { ... } }
  198. environments/production.json { "name" : "production", "description" : "Production cluster in

    EC2", "override_attributes" : { ... }, "default_attributes" : { ... }, "cookbook_versions" : { "myface": "0.1.0" }, }
  199. CD

  200. CD PUSH

  201. CD PUSH

  202. sethvargo@opscode.com E b yz

  203. youtube.com/watch?v=UMnZiTL0tUc pdiffs: travis-ci.org Travis CI: