Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ChefConf 2014 - Test Kitchen: One Year Later and the Future

ChefConf 2014 - Test Kitchen: One Year Later and the Future

After one year of active development and real world use, Test Kitchen 1.0 was released on December 1, 2013. And this is only the beginning. In this session we will cover an introduction to Test Kitchen, its ecosystem, some common (and crazy) usage patterns in the wild, and ways to extend and bend the framework to your will. If you want a good insight into where Test Kitchen is going, then this talk is for you.

9891e8299426fb9b6e361b84b3155a2d?s=128

Fletcher Nichol

April 16, 2014
Tweet

More Decks by Fletcher Nichol

Other Decks in Technology

Transcript

  1. Test Kitchen: One Year Later and the Future ChefConf 2014,

    San Francisco Fletcher Nichol @fnichol github.com/fnichol
  2. Our time together

  3. 1. Quick Test Kitchen introduction

  4. 2. Usage patterns

  5. 3. Future direction & challenges

  6. A promise

  7. Slides will be posted

  8. But first...

  9. KitchenCI logo!!!

  10. None
  11. Now in sticker form

  12. None
  13. Test Kitchen: Introduction RFE™

  14. Rapid Fire Edition™

  15. Test Kitchen is: A test harness tool to execute your

    configured code on one or more platforms in isolation
  16. Goals

  17. Simple workflow

  18. Declarative static configuration

  19. Favors speed in development

  20. But optimizes for freshness of code

  21. Decoupled plugin architecture

  22. Concepts

  23. Kitchen converges with provisioners and runs specific suites on target

    platforms called instances using drivers
  24. Platform Operating system or target environment

  25. --- platforms: - name: ubuntu-14.04 - name: centos-6.5 - name:

    ubuntu-14.04-chef10
  26. Suite A Chef configuration, a run-list and node attributes

  27. --- suites: - name: client-libs run_list: - recipe[derp::client] attributes: chef:

    is_fun - name: server - name: runit-server
  28. Instance A platform + suite with auto- generated name

  29. > kitchen list --bare client-libs-ubuntu-1404 server-ubuntu-1404 runit-server-ubuntu-1404 client-libs-centos-65 server-centos-65 runit-server-centos-65

    ...
  30. Driver Implementation of instance actions: create, converge, setup, verify, destroy

  31. --- driver: name: docker socket: <%= ENV[‘DOCKER_HOST’] %>

  32. Driver Need one? Write one!

  33. Provisioner Implementation of infrastructure automation tool or framework

  34. #ChefConf 2014 Edition

  35. Provisioner “Runs your Chef code, buddy”

  36. Chef Provisioners chef_solo chef_zero

  37. --- provisioner: name: chef_zero require_chef_omnibus: 11.10.2 nodes_path: ./nodes

  38. Provisioner Need one? Write one! (For now) just be careful

  39. Kitchen converges with provisioners and runs specific blah blah blababibty

    blah and lets you optionally run tests on them to verify state
  40. Busser Prepares instances for test suites and runs them

  41. tl;dr http://kitchen.ci/docs/getting-started/

  42. Usage Patterns (& Tricks)

  43. Use Kitchen To…

  44. Detangle cookbook dependencies

  45. # metadata.rb name “zookeeper” version “14.4.15” depends “java”

  46. Test chef-solo vs. chef-client

  47. buddies = search(:users, “*:*”)

  48. client :) solo :(

  49. buddies = if node[“buddies”] Array(node[“buddies]) else search(:users, “*:*”) end

  50. client :)) solo :)

  51. Test Chef upgrades

  52. When things...

  53. None
  54. or

  55. “New major Chef client release!”

  56. Upgrade & deploy?

  57. “It’ll probably be fine.” - J. Paul Reed

  58. “Maybe we should test this...” - Hopefully everyone

  59. --- platforms: - name: ubuntu-14.04-chef-11 provisioner: require_chef_omnibus: 11 - name:

    ubuntu-14.04-chef-12 provisioner: require_chef_omnibus: 12
  60. tl;dr Try your infrastructure code in isolation and watch the

    spectacular failures!
  61. Fuzzy Matching

  62. Almost all kitchen commands

  63. kitchen converge [NAME|REGEXP|all] kitchen create [NAME|REGEXP|all] kitchen destroy [NAME|REGEXP|all] kitchen

    diagnose [NAME|REGEXP|all] kitchen list [NAME|REGEXP|all] kitchen login [NAME|REGEXP|all] kitchen setup [NAME|REGEXP|all] kitchen test [NAME|REGEXP|all] kitchen verify [NAME|REGPEX|all]
  64. > kitchen list --bare client-ubuntu-1404 client-centos-65 server-ubuntu-1404 server-centos-65

  65. > kitchen list all --bare client-ubuntu-1404 client-centos-65 server-ubuntu-1404 server-centos-65

  66. > kitchen list \ default-ubuntu-1404 --bare client-ubuntu-1404

  67. > kitchen list ubuntu --bare client-ubuntu-1404 server-ubuntu-1404

  68. > kitchen list 4 --bare client-ubuntu-1404 server-ubuntu-1404

  69. > kitchen list cl --bare client-ubuntu-1404 client-centos-65

  70. > kitchen list \ ‘^cli.*-65$’ --bare client-centos-65

  71. tl;dr > kitchen help

  72. Diagnose

  73. The.

  74. Most.

  75. Important.

  76. Takeaway.

  77. None
  78. > kitchen diagnose

  79. fin.

  80. Originally designed for support

  81. Helpful when debugging

  82. Kitchen diagnose makes all implicit configuration explicit.

  83. Kitchen diagnose gives you a shopping list of configuration tunables

    that you can mess with.
  84. --- timestamp: 2014-04-15 18:59:58.460470000 Z kitchen_version: 1.2.2.dev instances: default-ubuntu-1404: #

    ... default-centos-65: # ...
  85. --- instances: default-ubuntu-1204: state_file: hostname: 192.168.206.225 last_action: create port: '22'

    ssh_key: "/Users/fnichol/.vagrant.d/ insecure_private_key" username: vagrant
  86. --- instances: default-ubuntu-1204: driver: box: opscode-ubuntu-12.04 box_url: https://opscode-vm- bento.s3.amazonaws.com/vagrant/vmware/ opscode_ubuntu-12.04_chef-

    provisionerless.box kitchen_root: "/Users/fnichol/ Projects/sandbox/newish"
  87. --- instances: default-ubuntu-1204: provisioner: attributes: {} chef_omnibus_url: https:// www.getchef.com/chef/install.sh clients_path:

    name: chef_solo
  88. --- instances: default-ubuntu-1204: busser: root_path: /tmp/busser ruby_bindir: /opt/chef/embedded/bin sudo: true

  89. YAML merging hell?

  90. kitchen diagnose

  91. > kitchen diagnose --loader

  92. --- loader: global_config: filename: "/Users/fnichol/.kitchen/ config.yml" raw_data: #... project_config: filename:

    "/Users/fnichol/Projects/ sandbox/newish/.kitchen.yml" raw_data: # ... local_config:
  93. --- loader: combined_config: filename: raw_data: driver: name: vagrant socket: tcp://192.168.42.43:4243

    provisioner: # ...
  94. The full scoop

  95. > kitchen diagnose --all

  96. tl;dr > kitchen diag #everything

  97. Chef Zero Provisioner

  98. Have you thanked John Keiser?

  99. None
  100. Chef-Zero

  101. The fakest Chef Server

  102. chef_zero Provisioner

  103. The fakest chef-client run

  104. --- provisioner: name: chef_zero platforms: - name: ubuntu-14.04 suites: -

    name: default
  105. chef_zero provisioner supports data bags, encrypted secrets, environments, nodes, &

    clients. Everything except roles.
  106. None
  107. chef_zero provisioner supports data bags, encrypted secrets, environments, nodes, clients

    & even roles.
  108. > mkdir -p \ test/integration/roles > kitchen diagnose

  109. And You Concurrency

  110. er

  111. Concurrency and You

  112. Should be fast in development

  113. Originally test --parallel

  114. > kitchen test --parallel

  115. Winning.

  116. 'Concurrency Is Not Parallelism'

  117. Thanks, Rob Pike me --> :’(

  118. > kitchen test --concurrency

  119. What if you have 24 instances?

  120. Limit your concurrency!

  121. > kitchen test --concurrency=3

  122. > kitchen test --concurrency=1

  123. kitchen converge --concurrency=2 kitchen create -c 2 kitchen destroy -c

    2 kitchen setup -c 2 kitchen test -c 2 kitchen verify -c
  124. Eventually deprecate -p in favor of -c

  125. tl;dr Move faster, get more sleep

  126. $ENV VARS

  127. $KITCHEN_YAML

  128. > KITCHEN_YAML=Kitchen.yml \ kitchen list

  129. Have you thanked Noah Kantrowitz?

  130. None
  131. What about local overrides?

  132. > KITCHEN_LOCAL_YAML=.kitchen.ci.yml \ kitchen list

  133. What about global overrides?

  134. No.

  135. None
  136. > KITCHEN_GLOBAL_YAML=~/tk_global.yaml \ kitchen list

  137. I’m lost, too many files

  138. > kitchen diagnose --loaders

  139. tl;dr Real world use cases rock when driving out features

  140. CI Tooling

  141. Test Kitchen designs for CI

  142. kitchen test Quick Review

  143. kitchen test destroy, create, converge, setup, verify, destroy

  144. Several options

  145. 1. Rake & Thor tasks

  146. > rake -T rake kitchen:all rake kitchen:default-centos-64 rake kitchen:default-ubuntu-1204

  147. Sample CI Job

  148. # Environment variables export KITCHEN_INSTANCE=default-centos-65 export GIT_REV=<rev> # CI job

    pseudo-code git reset --hard “$GIT_REV” git clean -d -x -f if [ -f “.kitchen.ci.yml” ] ; then export KITCHEN_LOCAL_YAML=.kitchen.ci.yml fi bundle install --path vendor/bundle bundle exec rake kitchen:$KITCHEN_INSTANCE
  149. 2. CLI

  150. > kitchen list --bare default-centos-64 default-ubuntu-1204

  151. Sample CI Job

  152. # Environment variables export KITCHEN_INSTANCE=default-centos-65 export GIT_REV=<rev> # CI job

    pseudo-code git reset --hard “$GIT_REV” git clean -d -x -f if [ -f “.kitchen.ci.yml” ] ; then export KITCHEN_LOCAL_YAML=.kitchen.ci.yml fi bundle install --path vendor/bundle bundle exec kitchen test $KITCHEN_INSTANCE
  153. 3. Using Ruby

  154. require “kitchen” loader = Kitchen::Loader::YAML. new(local_config: “.kitchen.ci.yml”) config = Kitchen::Config.

    new(loader: loader) instance = config.instances. get(ENV[“KITCHEN_INSTANCE”]) instance.test
  155. Concurrency?

  156. > kitchen test -c # insane pile of output #

    in non-deterministic ordering
  157. CI: tools great for concurrency

  158. Let your CI tool handle jobs

  159. tl;dr In dev, use concurrency mode. In CI, use built

    in parallel job executions.
  160. HDC™

  161. Hyper Dynamic Caching™

  162. None
  163. Live demo? Crazy? Maybe?

  164. Conference WIFI

  165. http_proxy

  166. Best.

  167. Environment.

  168. Variable.

  169. Evar.

  170. Have you thanked Adam Jacob?

  171. None
  172. --- driver: http_proxy: http://192.168.100.1:8123 https_proxy: http://192.168.100.1:8123

  173. http_proxy is honored by: curl, wget, apt-get, yum, rubygems, chef-solo,

    chef- client, and more!
  174. Next use polipo for caching

  175. # brew install polipo # apt-get install polipo > polipo

    \ proxyAddress='0.0.0.0' \ disableIndexing='false' \ disableServersList='false' \ allowedClients='0.0.0.0/0' \ diskCacheRoot='~/.polipo-cache'
  176. Now add kitchen-docker for speed

  177. --- driver: name: docker socket: <%= ENV[“DOCKER_HOST”] %> http_proxy: http://192.168.100.1:8123

    https_proxy: http://192.168.100.1:8123
  178. Enable caching automatically

  179. CRAZYTOWN

  180. Result

  181. Fast when you want it to be, easy to disable

    caching when timing Chef runs, etc.
  182. Not for faint of heart

  183. tl;dr https://gist.github.com/fnichol/7551540

  184. Teh Futur

  185. More Explicit Config

  186. Configurable Busser Plugins

  187. Windows

  188. Transports

  189. Stable Provisioner API

  190. Clustering?

  191. Sharing Code & Chef Metal

  192. Complete Testing Backfill

  193. Thank you! Fletcher Nichol @fnichol github.com/fnichol

  194. Bonus Round!

  195. Unified YAML Config

  196. Warning: to the early adopters

  197. --- # OLD!! driver_plugin: vagrant driver_config: customize: memory: 1024

  198. --- # NEW!! driver: name: vagrant customize: memory: 1024

  199. --- driver: name: vagrant provisioner: name: chef_solo busser: version: 0.6.0.beta.1

  200. First class Provisioners

  201. --- platforms: - name: centos-6.5 run_list: [“recipe[foo]”] suites: - name:

    default roles_path: custom/roles
  202. --- platforms: - name: centos-6.5 provisioner: run_list: [“recipe[foo]”] suites: -

    name: default provisioner: roles_path: custom/roles
  203. 1. Backwards compatible

  204. 2. Future warning

  205. 3. Deprecation

  206. Config Blocks driver provisioner busser

  207. 3 Locations for Config Blocks

  208. --- # in a platform platforms: - name: freebsd-9.2 driver:

    name: vagrant provisioner: name: chef_zero
  209. --- # in a suite suites: - name: server driver:

    name: vagrant provisioner: name: chef_zero
  210. --- # in root or common location driver: name: vagrant

    provisioner: name: chef_zero
  211. tl;dr https://gist.github.com/fnichol/7601773

  212. Data!

  213. Need a secret?

  214. Need a package?

  215. Need an artifact?

  216. Use :data_path

  217. > mkdir -p \ test/integration/data > kitchen diagnose

  218. Uploads contents on converge

  219. Available at /tmp/kitchen/data by default

  220. Reference it in a Chef recipe

  221. url = node[“privchef_pkg_url”] pkg = “/tmp/#{File.basename(url)}” remote_file pkg do source

    url action :create end package pkg do action :install end
  222. --- provisioner: data_path: share platforms: - name: ubuntu-12.04 attributes: privchef_pkg_url:

    |- file:///tmp/kitchen/data/chef.deb suites: - name: default
  223. tl;dr Generic data is useful