ChefConf 2014 - Test Kitchen: One Year Later and the Future

After one year of active development and real world use, Test Kitchen 1.0 was released on December 1, 2013. And this is only the beginning. In this session we will cover an introduction to Test Kitchen, its ecosystem, some common (and crazy) usage patterns in the wild, and ways to extend and bend the framework to your will. If you want a good insight into where Test Kitchen is going, then this talk is for you.

Transcript

1. Test Kitchen: One Year Later and the Future ChefConf 2014, San Francisco Fletcher Nichol @fnichol github.com/fnichol
2. Our time together
3. 1. Quick Test Kitchen introduction
4. 2. Usage patterns
5. 3. Future direction & challenges
6. A promise
7. Slides will be posted
8. But first...
9. KitchenCI logo!!!
11. Now in sticker form
13. Test Kitchen: Introduction RFE™
14. Rapid Fire Edition™
15. Test Kitchen is: A test harness tool to execute your configured code on one or more platforms in isolation
16. Goals
17. Simple workflow
18. Declarative static configuration
19. Favors speed in development
20. But optimizes for freshness of code
21. Decoupled plugin architecture
22. Concepts
23. Kitchen converges with provisioners and runs specific suites on target platforms called instances using drivers
24. Platform Operating system or target environment
25. --- platforms: - name: ubuntu-14.04 - name: centos-6.5 - name: ubuntu-14.04-chef10
26. Suite A Chef configuration, a run-list and node attributes
27. --- suites: - name: client-libs run_list: - recipe[derp::client] attributes: chef: is_fun - name: server - name: runit-server
28. Instance A platform + suite with auto- generated name
29. > kitchen list --bare client-libs-ubuntu-1404 server-ubuntu-1404 runit-server-ubuntu-1404 client-libs-centos-65 server-centos-65 runit-server-centos-65 ...
30. Driver Implementation of instance actions: create, converge, setup, verify, destroy
31. --- driver: name: docker socket: <%= ENV[‘DOCKER_HOST’] %>
32. Driver Need one? Write one!
33. Provisioner Implementation of infrastructure automation tool or framework
34. #ChefConf 2014 Edition
35. Provisioner “Runs your Chef code, buddy”
36. Chef Provisioners chef_solo chef_zero
37. --- provisioner: name: chef_zero require_chef_omnibus: 11.10.2 nodes_path: ./nodes
38. Provisioner Need one? Write one! (For now) just be careful
39. Kitchen converges with provisioners and runs specific blah blah blababibty blah and lets you optionally run tests on them to verify state
40. Busser Prepares instances for test suites and runs them
41. tl;dr http://kitchen.ci/docs/getting-started/
42. Usage Patterns (& Tricks)
43. Use Kitchen To…
44. Detangle cookbook dependencies
45. # metadata.rb name “zookeeper” version “14.4.15” depends “java”
46. Test chef-solo vs. chef-client
47. buddies = search(:users, “*:*”)
48. client :) solo :(
49. buddies = if node[“buddies”] Array(node[“buddies]) else search(:users, “*:*”) end
50. client :)) solo :)
51. Test Chef upgrades
52. When things...
54. or
55. “New major Chef client release!”
56. Upgrade & deploy?
57. “It’ll probably be fine.” - J. Paul Reed
58. “Maybe we should test this...” - Hopefully everyone
59. --- platforms: - name: ubuntu-14.04-chef-11 provisioner: require_chef_omnibus: 11 - name: ubuntu-14.04-chef-12 provisioner: require_chef_omnibus: 12
60. tl;dr Try your infrastructure code in isolation and watch the spectacular failures!
61. Fuzzy Matching
62. Almost all kitchen commands
63. kitchen converge [NAME|REGEXP|all] kitchen create [NAME|REGEXP|all] kitchen destroy [NAME|REGEXP|all] kitchen diagnose [NAME|REGEXP|all] kitchen list [NAME|REGEXP|all] kitchen login [NAME|REGEXP|all] kitchen setup [NAME|REGEXP|all] kitchen test [NAME|REGEXP|all] kitchen verify [NAME|REGPEX|all]
64. > kitchen list --bare client-ubuntu-1404 client-centos-65 server-ubuntu-1404 server-centos-65
65. > kitchen list all --bare client-ubuntu-1404 client-centos-65 server-ubuntu-1404 server-centos-65
66. > kitchen list \ default-ubuntu-1404 --bare client-ubuntu-1404
67. > kitchen list ubuntu --bare client-ubuntu-1404 server-ubuntu-1404
68. > kitchen list 4 --bare client-ubuntu-1404 server-ubuntu-1404
69. > kitchen list cl --bare client-ubuntu-1404 client-centos-65
70. > kitchen list \ ‘^cli.*-65$’ --bare client-centos-65
71. tl;dr > kitchen help
72. Diagnose
73. The.
74. Most.
75. Important.
76. Takeaway.
78. > kitchen diagnose
79. fin.
80. Originally designed for support
81. Helpful when debugging
82. Kitchen diagnose makes all implicit configuration explicit.
83. Kitchen diagnose gives you a shopping list of configuration tunables that you can mess with.
84. --- timestamp: 2014-04-15 18:59:58.460470000 Z kitchen_version: 1.2.2.dev instances: default-ubuntu-1404: # ... default-centos-65: # ...
85. --- instances: default-ubuntu-1204: state_file: hostname: 192.168.206.225 last_action: create port: '22' ssh_key: "/Users/fnichol/.vagrant.d/ insecure_private_key" username: vagrant
86. --- instances: default-ubuntu-1204: driver: box: opscode-ubuntu-12.04 box_url: https://opscode-vm- bento.s3.amazonaws.com/vagrant/vmware/ opscode_ubuntu-12.04_chef- provisionerless.box kitchen_root: "/Users/fnichol/ Projects/sandbox/newish"
87. --- instances: default-ubuntu-1204: provisioner: attributes: {} chef_omnibus_url: https:// www.getchef.com/chef/install.sh clients_path: name: chef_solo
88. --- instances: default-ubuntu-1204: busser: root_path: /tmp/busser ruby_bindir: /opt/chef/embedded/bin sudo: true
89. YAML merging hell?
90. kitchen diagnose
91. > kitchen diagnose --loader
92. --- loader: global_config: filename: "/Users/fnichol/.kitchen/ config.yml" raw_data: #... project_config: filename: "/Users/fnichol/Projects/ sandbox/newish/.kitchen.yml" raw_data: # ... local_config:
93. --- loader: combined_config: filename: raw_data: driver: name: vagrant socket: tcp://192.168.42.43:4243 provisioner: # ...
94. The full scoop
95. > kitchen diagnose --all
96. tl;dr > kitchen diag #everything
97. Chef Zero Provisioner
98. Have you thanked John Keiser?
100. Chef-Zero
101. The fakest Chef Server
102. chef_zero Provisioner
103. The fakest chef-client run
104. --- provisioner: name: chef_zero platforms: - name: ubuntu-14.04 suites: - name: default
105. chef_zero provisioner supports data bags, encrypted secrets, environments, nodes, & clients. Everything except roles.
107. chef_zero provisioner supports data bags, encrypted secrets, environments, nodes, clients & even roles.
108. > mkdir -p \ test/integration/roles > kitchen diagnose
109. And You Concurrency
110. er
111. Concurrency and You
112. Should be fast in development
113. Originally test --parallel
114. > kitchen test --parallel
115. Winning.
116. 'Concurrency Is Not Parallelism'
117. Thanks, Rob Pike me --> :’(
118. > kitchen test --concurrency
119. What if you have 24 instances?
120. Limit your concurrency!
121. > kitchen test --concurrency=3
122. > kitchen test --concurrency=1
123. kitchen converge --concurrency=2 kitchen create -c 2 kitchen destroy -c 2 kitchen setup -c 2 kitchen test -c 2 kitchen verify -c
124. Eventually deprecate -p in favor of -c
125. tl;dr Move faster, get more sleep
126. $ENV VARS
127. $KITCHEN_YAML
128. > KITCHEN_YAML=Kitchen.yml \ kitchen list
129. Have you thanked Noah Kantrowitz?
131. What about local overrides?
132. > KITCHEN_LOCAL_YAML=.kitchen.ci.yml \ kitchen list
133. What about global overrides?
134. No.
136. > KITCHEN_GLOBAL_YAML=~/tk_global.yaml \ kitchen list
137. I’m lost, too many files
138. > kitchen diagnose --loaders
139. tl;dr Real world use cases rock when driving out features
140. CI Tooling
141. Test Kitchen designs for CI
142. kitchen test Quick Review
143. kitchen test destroy, create, converge, setup, verify, destroy
144. Several options
145. 1. Rake & Thor tasks
146. > rake -T rake kitchen:all rake kitchen:default-centos-64 rake kitchen:default-ubuntu-1204
147. Sample CI Job
148. # Environment variables export KITCHEN_INSTANCE=default-centos-65 export GIT_REV=<rev> # CI job pseudo-code git reset --hard “$GIT_REV” git clean -d -x -f if [ -f “.kitchen.ci.yml” ] ; then export KITCHEN_LOCAL_YAML=.kitchen.ci.yml fi bundle install --path vendor/bundle bundle exec rake kitchen:$KITCHEN_INSTANCE
149. 2. CLI
150. > kitchen list --bare default-centos-64 default-ubuntu-1204
151. Sample CI Job
152. # Environment variables export KITCHEN_INSTANCE=default-centos-65 export GIT_REV=<rev> # CI job pseudo-code git reset --hard “$GIT_REV” git clean -d -x -f if [ -f “.kitchen.ci.yml” ] ; then export KITCHEN_LOCAL_YAML=.kitchen.ci.yml fi bundle install --path vendor/bundle bundle exec kitchen test $KITCHEN_INSTANCE
153. 3. Using Ruby
154. require “kitchen” loader = Kitchen::Loader::YAML. new(local_config: “.kitchen.ci.yml”) config = Kitchen::Config. new(loader: loader) instance = config.instances. get(ENV[“KITCHEN_INSTANCE”]) instance.test
155. Concurrency?
156. > kitchen test -c # insane pile of output # in non-deterministic ordering
157. CI: tools great for concurrency
158. Let your CI tool handle jobs
159. tl;dr In dev, use concurrency mode. In CI, use built in parallel job executions.
160. HDC™
161. Hyper Dynamic Caching™
163. Live demo? Crazy? Maybe?
164. Conference WIFI
165. http_proxy
166. Best.
167. Environment.
168. Variable.
169. Evar.
170. Have you thanked Adam Jacob?
172. --- driver: http_proxy: http://192.168.100.1:8123 https_proxy: http://192.168.100.1:8123
173. http_proxy is honored by: curl, wget, apt-get, yum, rubygems, chef-solo, chef- client, and more!
174. Next use polipo for caching
175. # brew install polipo # apt-get install polipo > polipo \ proxyAddress='0.0.0.0' \ disableIndexing='false' \ disableServersList='false' \ allowedClients='0.0.0.0/0' \ diskCacheRoot='~/.polipo-cache'
176. Now add kitchen-docker for speed
177. --- driver: name: docker socket: <%= ENV[“DOCKER_HOST”] %> http_proxy: http://192.168.100.1:8123 https_proxy: http://192.168.100.1:8123
178. Enable caching automatically
179. CRAZYTOWN
180. Result
181. Fast when you want it to be, easy to disable caching when timing Chef runs, etc.
182. Not for faint of heart
183. tl;dr https://gist.github.com/fnichol/7551540
184. Teh Futur
185. More Explicit Config
186. Configurable Busser Plugins
187. Windows
188. Transports
189. Stable Provisioner API
190. Clustering?
191. Sharing Code & Chef Metal
192. Complete Testing Backfill
193. Thank you! Fletcher Nichol @fnichol github.com/fnichol
194. Bonus Round!
195. Unified YAML Config
196. Warning: to the early adopters
197. --- # OLD!! driver_plugin: vagrant driver_config: customize: memory: 1024
198. --- # NEW!! driver: name: vagrant customize: memory: 1024
199. --- driver: name: vagrant provisioner: name: chef_solo busser: version: 0.6.0.beta.1
200. First class Provisioners
201. --- platforms: - name: centos-6.5 run_list: [“recipe[foo]”] suites: - name: default roles_path: custom/roles
202. --- platforms: - name: centos-6.5 provisioner: run_list: [“recipe[foo]”] suites: - name: default provisioner: roles_path: custom/roles
203. 1. Backwards compatible
204. 2. Future warning
205. 3. Deprecation
206. Config Blocks driver provisioner busser
207. 3 Locations for Config Blocks
208. --- # in a platform platforms: - name: freebsd-9.2 driver: name: vagrant provisioner: name: chef_zero
209. --- # in a suite suites: - name: server driver: name: vagrant provisioner: name: chef_zero
210. --- # in root or common location driver: name: vagrant provisioner: name: chef_zero
211. tl;dr https://gist.github.com/fnichol/7601773
212. Data!
213. Need a secret?
214. Need a package?
215. Need an artifact?
216. Use :data_path
217. > mkdir -p \ test/integration/data > kitchen diagnose
218. Uploads contents on converge
219. Available at /tmp/kitchen/data by default
220. Reference it in a Chef recipe
221. url = node[“privchef_pkg_url”] pkg = “/tmp/#{File.basename(url)}” remote_file pkg do source url action :create end package pkg do action :install end
222. --- provisioner: data_path: share platforms: - name: ubuntu-12.04 attributes: privchef_pkg_url: |- file:///tmp/kitchen/data/chef.deb suites: - name: default
223. tl;dr Generic data is useful