Test Driven Infrastructure with Chef

These are the slides from my 2013 Velocity talk on Chef Driven Infrastructure with Chef using ChefSpec, Test Kitchen, GitHub, Jenkins, and more!

Transcript

1. Test-Driven Infrastructure
2. sethvargo@opscode.com E b yz
3. 1WORKSTATION SETUP BBB
4. 1#LEARNCHEF BBB
5. 1BBB
6. 1cl.ly/PiLZ sha: 821ecd0 BBB
7. $ knife client list
8. 1GITHUB  
9. 2CREATE THE COOKBOOK fdd
10. $ knife cookbook create {username}-myface
11. $ knife cookbook create sethvargo-myface
12. fmyface d metadata.rb frecipes d default.rb
13. fmyface d metadata.rb frecipes d default.rb
14. fmyface d metadata.rb frecipes d default.rb
15. fmyface d metadata.rb frecipes d default.rb
16. fmyface d metadata.rb frecipes d default.rb
17. imyface
19. $ (sudo) gem install bundler
20. u gist.github.com/5737141
21. fmyface d Gemfile
22. Gemfile source 'https://rubygems.org' group :test do gem 'chefspec', '~> 1.3' gem 'foodcritic', '~> 2.1' gem 'strainer', '~> 3.0' gem 'test-kitchen', '~> 1.0.0.alpha' gem 'kitchen-lxc', '~> 0.0.1.beta1' gem 'knife-spork', '~> 1.0.17' gem 'hipchat', '~> 0.10.0' gem 'guard', '~> 1.8' gem 'guard-foodcritic', '~> 1.0' gem 'guard-rspec', '~> 3.0' end
23. $ (sudo) bundle
24. fmyface d spec_helper.rb fspec d default_spec.rb
25. fmyface d spec_helper.rb fspec d default_spec.rb
26. fmyface d spec_helper.rb fspec d default_spec.rb
27. spec/default_spec.rb require 'spec_helper' describe 'sethvargo-myface::default' do let(:chef_run) do run = ChefSpec::ChefRunner.new(platfrom: 'ubuntu', version: '12.04') run.converge('sethvargo-myface::default') end it 'installs apache2' do expect(chef_run).to install_package('apache2') end # ... end
28. fmyface d spec_helper.rb fspec d default_spec.rb
29. spec/spec_helper.rb require 'chefspec'
30. fmyface d Guardfile d Strainerfile
31. fmyface d Guardfile d Strainerfile
32. Guardfile guard :rspec, all_on_start: false do watch(%r{^spec/.+_spec\.rb$}) watch(%r{^(recipes)/(.+)\.rb$}) { |m| "spec/#{m[1]}_spec.rb" } watch('spec/spec_helper.rb') { 'spec' } end guard :foodcritic, cookbook_paths: '.', all_on_start: false do watch(%r{attributes/.+\.rb$}) watch(%r{providers/.+\.rb$}) watch(%r{recipes/.+\.rb$}) watch(%r{resources/.+\.rb$}) end
33. $ bundle exec guard
35. ⌘ + S
38. 0.0134s
39. fmyface d Guardfile d Strainerfile
40. Strainerfile knife: bundle exec knife cookbook test $COOKBOOK foodcritic: bundle exec foodcritic $SANDBOX/$COOKBOOK -f any rspec: (cd $COOKBOOK && bundle exec rspec --color)
41. fmyface d metadata.rb frecipes d default.rb
42. fmyface d metadata.rb frecipes d default.rb
43. package 'apache2' service 'apache2' do action [:enable, :start] end recipes/default.rb
44. ⌘ + S
45. POLICY CHANGE G
46. RUN UNIT TESTS MASTER BEFORE MERGING TO
47. ! RUN UNIT TESTS MASTER BEFORE MERGING TO
48. u HUMAN (PROS) D J h a K
49. 7 r c L M t HUMAN (CONS)
55. 1CREATE REPO
56. 1GITHUB INTERFACE ?
57. 1 myface T
58. 1 (anything) T
59. 1Create repository J
62. $ git add . $ git commit -m "myface" $ git push
64. 2SETUP JOB
65. S
66. I, _________, solemnly swear under penalty of my GPU catching fire, that I will not touch anyone's build but my own.  NO ASSHOLE PLEDGE
67. 2ci.trychef.com O
68. 2 New Job J
70. 2{username}-myface T
71. 2Build a free-style... J
72. 2 OK J
74. 2(github URL) T
75. 2 Git J
76. 2git://github.com/... T
77. 2Build when a change... J
78. 2Color ANSI Console... J
79. 2Add Build Step J
80. 2Execute Shell J
81. 2 bundle install --path ../../support/vendor bundle exec strainer test T
82. 2Email Notification J
83. 2(your email) T
84. 2 Save J
86. 2 Build Now J
87. 2 (the build) J
88. 3SETUP HOOK
89. 3 Settings J
91. 3Service Hooks J
92. 3 Jenkins (GitHub Plugin) J
93. 3http://ci.trychef.com/github-webhook/ T
94. 3http://ci.trychef.com/github-webhook/ T IMPORTANT
95. 3 Active J
96. 3Update settings J
97. $ git checkout -b add_site
98. fmyface d spec_helper.rb fspec d default_spec.rb
99. spec/default_spec.rb describe 'sethvargo-myface::default' do # pre-existing tests # ... it 'creates the default template' do expect(chef_run).to create_file('/var/www/index.html') end it 'creates the site with the correct content' do template = chef_run.template('/var/www/index.html') expect(template.owner).to eq('root') expect(template.group).to eq('root') end end
101. fmyface d metadata.rb frecipes d default.rb
102. recipes/default.rb package 'apache2' service 'apache2' do action [:enable, :start] end template '/var/www/index.html' do owner 'root' group 'root' mode '0755' source 'index.html.erb' end
103. fmyface ftemplates d index.html.erb fdefault
104. templates/default/index.html.erb <html> <head> <title>Welcome to <%= node['fqdn'] %></title> </head> <body> <p>Here's everything you need to know about <%= node['fqdn'] %>:</p> <pre><%= JSON.pretty_generate(node.to_hash) %></pre> </body> </html>
105. $ git add . $ git commit -m "Write out default site"
106. $ git push origin add_site
107. 
108. Pull Request J
109. 
110. Send pull request J
111. 
112. c
113.  Merge pull request J
114. J
115. J (the build) J
116. JConsole Output J
117. J
118. J
119. J
120. LET'S FIX THAT Y
121. $ git pull origin master
122. metadata.rb name 'myface' maintainer 'YOUR_COMPANY_NAME' maintainer_email 'YOUR_EMAIL' license 'All rights reserved' description 'Installs/Configures myface' long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version '0.1.0'
123. metadata.rb name 'myface' maintainer 'Seth Vargo' maintainer_email 'sethvargo@opscode.com' license 'All rights reserved' description 'Installs/Configures myface' long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version '0.1.0'
124. $ git add metadata.rb $ git commit -m "Fix metadata"
125. $ git push origin master
126. AWESOME
127. $ knife cookbook upload myface
128. S
129. WHY?
130. 1 WE HAVE TESTS
131. 1 WE HAVE TESTS 2 WE HAVE JENKINS
132. 1 WE HAVE TESTS 2 WE HAVE JENKINS 3 WE HAVE THE TECHNOLOGY
133. $ knife cookbook upload myface
134. $ knife cookbook upload myface G
135. $ knife cookbook upload myface G M
136. J Configure J
137. J
138. J
139. JAdd post-build action J
140. JPost build task J
141. J
142. J Strainer marked build OK T
143. J bundle exec knife cookbook upload {username}-myface T
144. J Save J
145. J
146. J Build Now J
147. J
148. J (the build) J
149. JConsole Output J
150. J
151. J > knife cookbook list | grep sethvargo-myface sethvargo-myface 0.1.0
152. AWESOME
153. R
154. WE KNOW WHEN OUR BUILD FAILS BECAUSE WE GET AN EMAIL FROM JENKINS
155. BUT DO WE KNOW WHEN OUR COOKBOOKS ARE UPLOADED TO OUR CHEF SERVER?
156. BUT DO WE KNOW WHEN OUR COOKBOOKS ARE UPLOADED TO OUR CHEF SERVER?
157.  jonlives/knife-spork
158. jonlives/knife-spork
159. jonlives/knife-spork
160. J Configure J
161. J
162. J
163. J
164. J bundle exec knife spork upload {username}-myface T
165. J Escalate script execution... J
166. J Save J
167. q hipchat.com/sign_in
168. q Launch the web app J
169. qTry Chef J
170. q
171. J Build Now J
172. q
173. N THERE'S MORE!
174. N THERE'S MORE!
175. N TEST KITCHEN!
176. fmyface d .kitchen.yml
177. .kitchen.yml driver_plugin: lxc driver_config: use_sudo: true platforms: - name: ubuntu-12.04 driver_config: base_container: ubuntu_12.04 username: ubuntu password: ubuntu suites: - name: default run_list: ["recipe[myface]"] attributes: {}
178. Strainerfile knife: bundle exec knife cookbook test $COOKBOOK foodcritic: bundle exec foodcritic $SANDBOX/$COOKBOOK -f any rspec: (cd $COOKBOOK && bundle exec rspec --color) kitchen: (cd $COOKBOOK && bundle exec kitchen test)
179. metadata.rb name 'myface' maintainer 'Seth Vargo' maintainer_email 'sethvargo@opscode.com' license 'All rights reserved' description 'Installs/Configures myface' long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version '0.1.0'
180. metadata.rb name 'myface' maintainer 'Seth Vargo' maintainer_email 'sethvargo@opscode.com' license 'All rights reserved' description 'Installs/Configures myface' long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version '0.1.1'
181. $ git add . $ git commit -m "Add Test Kitchen"
182. $ git push origin master
183. q
185. qUH OH!
186. $ git revert HEAD
187. $ git revert HEAD
188. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb' end
189. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb' end IF WE REMOVE THIS WILL THE FILE BE DELETED?
190. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb' end NO
191. FAST FORWARD CHANGES ARE ABSOLUTELY ALWAYS BETTER THAN REVERTING
192. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb' end
193. some_recipe.rb template '/etc/foo/bar' do owner 'root' group 'root' source 'bar.erb' action :delete end
194. $ git add . $ git commit -m "Revert abc123" $ git push origin master
195. chef + environments safer infrastructure
200. environments/production.json { "name" : "production", "description" : "Production cluster in EC2", "override_attributes" : { ... }, "default_attributes" : { ... } }
201. environments/production.json { "name" : "production", "description" : "Production cluster in EC2", "override_attributes" : { ... }, "default_attributes" : { ... }, "cookbook_versions" : { "myface": "0.1.0" }, }
202. CD
203. CD PUSH
204. CD PUSH
205. sethvargo@opscode.com E b yz
206. youtube.com/watch?v=UMnZiTL0tUc pdiffs: travis-ci.org Travis CI: