ChefDK or how I fell in love with embedded Ruby.

My talk at DevOpsDays Austin 2015

Transcript

1. • This slide purposely left blank…
3. ChefDK or how I fell in love with embedded Ruby. Jonathan “JJ” Asghar May 2015
4. Who Am I? • Senior Partner Engineer • jj@chef.io • @jjasghar • Loves Beer, Hugs, and Obscure Television
5. What is Chef?
6. tl;dr of Chef • An Open Source IT Automation Platform • Supporting • Windows • Linux • Mac • AIX • Multiple Networking stacks • Multiple Storage stacks
7. tl;dr of Chef Chef turns infrastructure into code. With Chef, you can automate how you build, deploy, and manage your infrastructure. Your infrastructure becomes as versionable, testable, and repeatable as application code. stolen from https://www.chef.io/chef/
8. tl;dr of Chef Simply said: Chef is automation for your modern IT systems.
9. tl;dr of Chef Even more simply said: You can TDD your infrastructure and have confidence in your infrastructure changes.
10. The Historical Way to Get Chef Also known as the “dark times” 10
11. The dark times… gem install chef
12. The dark times…
13. This is wrong, don’t do this anymore. gem install chef
14. We have moved away from this and that’s what this talk is about
15. The new way to install Chef curl -L https://chef.sh | sudo bash
16. Back to the dark times… source 'https://rubygems.org' gem 'berkshelf', '> 3' gem 'chef', '>= 11.8' gem 'rake', '>= 10.2' gem 'rubocop', '= 0.23' gem 'foodcritic', '< 4.0' gem 'chefspec', '>= 3.4' gem 'serverspec', '>= 1.6' gem 'test-kitchen' gem 'kitchen-vagrant' gem 'guard', '>= 2.6' gem 'guard-rubocop', '>= 1.1' gem 'guard-foodcritic', '>= 1.0.2' gem 'fog' Thanks cz :)
17. The dark times… So with the Gemfile you’ll have to start learning rvm/ rbenv or the like…
18. Before you know it…..
19. And this is you at work….
20. Then something like this happens…. Fetching gem metadata from https:// rubygems.org/........ Fetching gem metadata from https://rubygems.org/.. Resolving dependencies... Bundler could not find compatible versions for gem "json": In Gemfile: chef (= 11.4.4) depends on json (<= 1.7.7, >= 1.4.4) berkshelf (= 2.0.4) depends on json (1.8.0)
21. So what would you do? •Break open you Gemfile •Cry a little •Try to figure out what was conflicting •bundle install •Cry a little more •GOTO 10
22. And before you know it…..
23. The Chef Development Kit Also known as the “light times”
24. So what is the ChefDK? ChefDK is everything you need to build that “workstation” all bundled up in one package. It has the generally accepted packages and plugins that the majority of users want. 24
25. Workstation? •A workstation is a computer that is configured to run knife, to synchronize with the chef-repo, and interact with a single Chef server. The workstation is the location from which most users will do most of their work. stolen from http://docs.chef.io/workstation.html 25
26. And to build a workstation you used to need 26 source 'https://rubygems.org' gem 'berkshelf', '> 3' gem 'chef', '>= 11.8' gem 'rake', '>= 10.2' gem 'rubocop', '= 0.23' gem 'foodcritic', '< 4.0' gem 'chefspec', '>= 3.4' gem 'serverspec', '>= 1.6' gem 'test-kitchen' gem 'kitchen-vagrant' gem 'guard', '>= 2.6' gem 'guard-rubocop', '>= 1.1' gem 'guard-foodcritic', '>= 1.0.2' gem 'fog'
27. ChefDK is built on embedded ruby Chef provides a package that installs ruby in a non- critical path for you along with multiple gems. 27
28. ChefDK installs embedded ruby Installs at /opt/chef or on Windows C:/opscode/chefdk/ 28
29. ChefDK How to get it? 29
30. ChefDK’s main download site https://downloads.chef.io/chef-dk/ https://packagecloud.io/chef/current 30
31. ChefDK on a *nix box curl -L https://chef.sh | sudo bash -s -- -P chefdk 31
32. ChefDK has an cask install too brew cask install chefdk 32
33. ChefDK on Windows https://downloads.chef.io/chef-dk/ https://chocolatey.org/packages/chefdk choco install chefdk (still deploying 0.3.5.1 :( ) 33
34. ChefDK’s Github page https://github.com/chef/chef-dk 34
35. How the ChefDK is built Also known as what was needed to create to get to the “light times”
36. How ChefDK is built Chef created two projects to deal with the lack of tools: appbundler omnibus 36
37. appbundler • https://github.com/chef/appbundler • Appbundler reads a Gemfile.lock and generates code with gem "some-dep", "= VERSION" statements to lock the app's dependencies to the versions selected by bundler. 37
38. appbundler • It “packages” up a ruby application via a Gem.lock file • It deals with the dependancies and other rubyish stuff • It removes the need for bundle exec 38
39. omnibus • Easily create full-stack installers for your project across a variety of platforms. 39
40. omnibus • Have you tried to upstream a “fast moving” piece of software to distros? • Omnibus was created to allow for packaging of quickly moving software 40
41. omnibus • Multiple companies have leveraged this to build all in one builds for their stacks. • Moves extremely fast • Extremely vibrant community • Just google “github omnibus” • Leverages test-kitchen to build packages for other distros and systems 41
42. omnibus • Multiple companies have leveraged this to build all in one builds for their stacks. • Dell with Enstratius or now known as Dell Cloud Manager • Heavy Water Ops with Sensu • Gitlab uses it for Gitlab installation 42
43. The Chef Development Kit Binaries
44. chef A fun little wrapper binary for some normal everyday commands
45. The chef binaries • ChefDK comes with a bunch of wrapper commands with chef as the way to call them. • Here’s just a few of them. 45
46. chef gem • The best way to inject a specific gem into the embedded ruby. •chef gem install knife-c2 •chef gem install knife-openstack 46
47. chef exec • Think of bundle exec for your embedded ruby. 47
48. chef generate • Builds a de-facto cookbook with some of the “best” practices for creating your cookbook: 48
49. chef generate 49 15:49:14 ~ > chef generate cookbook fake_cookbook Compiling Cookbooks... Recipe: code_generator::cookbook * directory[/Users/jasghar/fake_cookbook] action create - create new directory /Users/jasghar/fake_cookbook * template[/Users/jasghar/fake_cookbook/metadata.rb] action create_if_missing - create new file /Users/jasghar/fake_cookbook/metadata.rb - update content in file /Users/jasghar/fake_cookbook/metadata.rb from none to ba0ae2 (diff output suppressed by config) * template[/Users/jasghar/fake_cookbook/README.md] action create_if_missing - create new file /Users/jasghar/fake_cookbook/README.md - update content in file /Users/jasghar/fake_cookbook/README.md from none to 6b3b9c (diff output suppressed by config) * cookbook_file[/Users/jasghar/fake_cookbook/chefignore] action create - create new file /Users/jasghar/fake_cookbook/chefignore - update content in file /Users/jasghar/fake_cookbook/chefignore from none to 9727b1 (diff output suppressed by config) * cookbook_file[/Users/jasghar/fake_cookbook/Berksfile] action create_if_missing - create new file /Users/jasghar/fake_cookbook/Berksfile - update content in file /Users/jasghar/fake_cookbook/Berksfile from none to 4a0cca (diff output suppressed by config) * cookbook_file[/Users/jasghar/fake_cookbook/spec/spec_helper.rb] action create_if_missing - create new file /Users/jasghar/fake_cookbook/spec/spec_helper.rb - update content in file /Users/jasghar/fake_cookbook/spec/spec_helper.rb from none to 945e09 (diff output suppressed by config) * template[/Users/jasghar/fake_cookbook/spec/unit/recipes/default_spec.rb] action create_if_missing - create new file /Users/jasghar/fake_cookbook/spec/unit/recipes/default_spec.rb - update content in file /Users/jasghar/fake_cookbook/spec/unit/recipes/default_spec.rb from none to 09c3c9 (diff output suppressed by config) * directory[/Users/jasghar/fake_cookbook/recipes] action create - create new directory /Users/jasghar/fake_cookbook/recipes * template[/Users/jasghar/fake_cookbook/recipes/default.rb] action create_if_missing - create new file /Users/jasghar/fake_cookbook/recipes/default.rb - update content in file /Users/jasghar/fake_cookbook/recipes/default.rb from none to cd7656 (diff output suppressed by config) * execute[initialize-git] action run - execute git init . * cookbook_file[/Users/jasghar/fake_cookbook/.gitignore] action create - create new file /Users/jasghar/fake_cookbook/.gitignore - update content in file /Users/jasghar/fake_cookbook/.gitignore from none to dd37b2 (diff output suppressed by config)
50. test-kitchen The piece of software that changed my career.
51. test-kitchen • Created by Fletcher Nichol • Created because he got annoyed with so many VagrantFiles • Super-nice Canadian 51
52. test-kitchen • http://kitchen.ci • The quickly becoming the standard for integration testing machines • There are integrations for • Chef • Ansible • Puppet • Salt 52
53. test-kitchen • A simple yaml configuration • It can leverage multiple backends here’s a few of them: • Vagrant (default) • OpenStack • Amazon EC2 • Docker • DigitalOcean • Rackspace 53
54. test-kitchen 54 --- driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-14.04 - name: centos-6.6 suites: - name: default run_list: - recipe[blah::default] attributes:
55. test-kitchen • Runs a small application called “bussers” after the converge • serverspec • bats • bash • Do yourself a favor and go to: https://github.com/test-kitchen • If y'all want a demo of it running find me after this talk, I truly love this software. 55
56. foodcritic Standardizing coding styles can save a lot of headache down the line
57. foodcritic • Ran by volunteers but under Andrew Crump’s github account • Created for a way to help standardize code • http://www.foodcritic.io/ • You can write your own for your own coding style 57
58. foodcritic • Some examples: • FC030: Cookbook contains debugger breakpoints • FC008: Generated cookbook metadata needs updating • FC048: Prefer Mixlib::ShellOut 58
59. foodcritic • tl;dr • Rubyists: Think of Rubocop for Chef • Pythonists: Think of pylint for Chef 59
60. chefspec Unit tests are important, you know you don’t remember why you wrote that code that way 6 months down the line.
61. chefspec • Created by Seth Vargo • Unit testing your recipes • The main site you need: • https://github.com/ sethvargo/chefspec/tree/ master/examples 61
62. chefspec I tell people that it’s good to write chefspec when you hit your 1.0.0 release of your cookbook 62
63. chefspec Typical TDD with Chef is hard, you don’t know what you’re doing till you do it. Start small. 63
64. chefspec 64 # in your install.rb package 'explicit_action' do action :install end # in your install_spec.rb describe 'package::install' do let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) } it 'installs a package with an explicit action' do expect(chef_run).to install_package('explicit_action') end end
65. chefspec It runs a complete chef-client run in memory and checks against the compiled code to confirm it’s doing what it expects. It only takes seconds to run! 65
66. Wrap-up…. The following slides are some things I’d like you take away from this talk
67. curl -L https://chef.sh | sudo bash The easiest way to install chef in my opinion
68. curl -L https://chef.sh | sudo bash -s -- -P chefdk The easiest way to install chef-dk I’m trying to get chef-dk.sh too
69. Any Questions? Don’t hesitate to contact me: jj@chef.io or j^2 on freenode. Or track me down, I really do love talking about this stuff.