5 Things You Didn't Know About Chef

5 Things You Didn't Know About Chef

This is the slide deck I used to present at Big Ruby 2013.

http://www.bigrubyconf.com/

4eb6098fc8de5a5f37199c3668b11590?s=128

Joshua Timberman

February 28, 2013
Tweet

Transcript

  1. 5 Things You Didn't Know About Chef Joshua Timberman joshua@opscode.com

    jtimberman community.opscode.com Thursday, February 28, 13
  2. Chef is an Automation Platform for developers & systems engineers

    to build & manage Infrastructure as Code. "743 lines of #opschef to build new product infrastructure from scratch. Thank you, @opscode.” -@bdha, August 18, 2011 What is Chef? Thursday, February 28, 13
  3. Chef is a Framework • Configuration management • Reasonability •

    Idempotent, Convergent • Flexibility • Library & Primitives • TIMTOWTDI http://www.flickr.com/photos/elitatt/6980379333/ Thursday, February 28, 13
  4. package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source

    "haproxy.cfg.erb" owner "root" group "root" mode 0644 notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => true action [:enable, :start] end Chef Enables Infrastructure as Code • Resources • Recipes • Cookbooks and Roles • Source Code Thursday, February 28, 13
  5. Agenda • In-place File Editing for Greater Good • Use

    Chef's Built-in Version Matching • REPLs are fun, so Chef has one! • Take the Resource Collection for a Stroll • The Anatomy of Loading and Executing a Single Recipe http://www.flickr.com/photos/koalazymonkey/3590953001/ • Bonus, if we have time! Thursday, February 28, 13
  6. http://www.flickr.com/photos/dolarz/2333292651/ In-place File Editing (for Greater Good™) Thursday, February 28,

    13
  7. net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid

    = 1 net.ipv4.tcp_synack_retries = 2 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.send_redirects = 0 net.ipv4.conf.all.accept_source_route = 0 net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.all.secure_redirects = 0 net.ipv4.conf.all.log_martians = 1 net.ipv4.conf.default.accept_source_route = 0 net.ipv4.conf.default.accept_redirects = 0 net.ipv4.conf.default.secure_redirects = 0 net.ipv4.icmp_echo_ignore_broadcasts = 1 net.ipv4.tcp_syncookies = 1 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv6.conf.default.router_solicitations = 0 net.ipv6.conf.default.accept_ra_rtr_pref = 0 net.ipv6.conf.default.accept_ra_pinfo = 0 net.ipv6.conf.default.accept_ra_defrtr = 0 net.ipv6.conf.default.autoconf = 0 net.ipv6.conf.default.dad_transmits = 0 net.ipv6.conf.default.max_addresses = 1 kernel.exec-shield = 1 kernel.randomize_va_space = 1 I need to just update this one line... http://www.flickr.com/photos/dmswart/6507071701/ < Right here? Thursday, February 28, 13
  8. http://www.flickr.com/photos/rhysasplundh/4807749216/ Thursday, February 28, 13

  9. Danger: Anti-pattern? • This makes it hard to reason about

    where changes to files are coming from • Manage whole-file resources (template, cookbook_file, remote_file) • But sometimes you have to edit a line... Thursday, February 28, 13
  10. Introducing Chef::Util::FileEdit ruby_block "edit etc hosts" do block do rc

    = Chef::Util::FileEdit.new("/etc/hosts") rc.search_file_replace_line( /^127\.0\.0\.1 localhost$/, "127.0.0.1 #{new_fqdn} #{new_hostname} localhost" ) rc.write_file end end Thursday, February 28, 13
  11. Chef::Util::FileEdit methods... #if matched, replace the whole line with newline.

    search_file_replace_line(regex, newline) #if matched, replace the match (all occurances) with the replace parameter search_file_replace(regex, replace) #if matched, delete the line search_file_delete_line(regex) #if matched, delete the match (all occurances) from the line search_file_delete(regex) #if matched, insert newline after each matching line insert_line_after_match(regex, newline) #if not matched, insert newline at the end of the file insert_line_if_no_match(regex, newline) #write the new file content write_file Thursday, February 28, 13
  12. Sean OMeara's line cookbook • Sean OMeara wrote a "line"

    cookbook. • Resource/Provider for managing single lines. • http://ckbk.it/line http://www.flickr.com/photos/49889874@N05/6743622955/ Thursday, February 28, 13
  13. Use Chef's Built-in Version Matching http://www.flickr.com/photos/tschiae/8395137978/ Thursday, February 28, 13

  14. Places to match versions • Platforms • Cookbooks • Software/packages

    Thursday, February 28, 13
  15. Some Caveats • Match versions that are dotted notation up

    to 3 places • X.Y.Z • 5.6, 1.8.0, etc Thursday, February 28, 13
  16. Common Example if node['platform_version'].to_f >= 10.04 if node['platform_version'] == "6.3"

    Thursday, February 28, 13
  17. Chef::VersionConstraint • First class comparison operators • Allows X.Y.Z version

    strings • Not lossy, unlike (potentially) #to_i or #to_f. http://docs.opscode.com/essentials_cookbook_versions_constraints.html Thursday, February 28, 13
  18. How to use Chef::VersionConstraint • `require` the library • Initialize

    a Chef::VersionConstraint object w/ the constraint • Send the include? method with the version to match Thursday, February 28, 13
  19. Constraint Operator = Equal > Greater than < Less than

    >= Greater than / equal <= Less than / equal ~> Approximately greater than (pessimistic) Thursday, February 28, 13
  20. Chef::VersionConstraint require 'chef/version_constraint' Chef::VersionConstraint.new(">= 10.7.0").include?("10.6.0") #=> false Chef::VersionConstraint.new(">= 10.7.0").include?("10.7.3") #=>

    true Chef::VersionConstraint.new(">= 10.7.0").include?("10.8.2") #=> true Thursday, February 28, 13
  21. Version Examples # On CentOS 5.8 Chef::VersionConstraint.new("~> 6.0"). include?(node['platform_version']) #=>

    false # On CentOS 6.3 Chef::VersionConstraint.new("~> 6.0"). include?(node['platform_version']) #=> true Thursday, February 28, 13
  22. Debian 6.0.3 vs. #to_f 6.0 == "6.0.3".to_f # => true

    6.0 < "6.0.3".to_f # => false 6.0 > "6.0.3".to_f # => false Thursday, February 28, 13
  23. Debian 6.0.3 vs. #to_f 6.0 == "6.0.3".to_f # => true

    6.0 < "6.0.3".to_f # => false 6.0 > "6.0.3".to_f # => false Chef::VersionConstraint.new("> 6.0"). include?("6.0.3") # => true Thursday, February 28, 13
  24. REPLs are fun, Chef has one! http://www.flickr.com/photos/tanaka_juuyoh/4434232143/ Thursday, February 28,

    13
  25. chef-shell, the Chef REPL • Chef 11: chef-shell • Chef

    10: shef • Built on IRB • Three modes/contexts: • main, attributes, recipe http://www.flickr.com/photos/51064559@N06/5746903658/ Thursday, February 28, 13
  26. chef-shell main context • Access node attributes • Interact with

    a Chef Server • It's IRB... Thursday, February 28, 13
  27. chef-shell main % chef-shell loading configuration: none (standalone session) Session

    type: standalone Loading...done. This is the chef-shell. Chef Version: 11.2.0 http://www.opscode.com/chef http://wiki.opscode.com/display/chef/Home run `help' for help, `exit' or ^D to quit. Ohai2u jtimberman@champagne chef > Thursday, February 28, 13
  28. chef-shell main chef > help chef-shell Help ================================================== | Command

    | Description ================================================== | Lots of commands | and their descriptions chef > halp :-) Thursday, February 28, 13
  29. chef-shell main context • Many of the commands available work

    with a Chef Server's API. • So let's start up chef-shell configured for a Chef Server. Thursday, February 28, 13
  30. Interact with a Chef Server % chef-shell -z loading configuration:

    /etc/chef/client.rb Session type: client ... chef > Chef::Config[:chef_server_url] => "https://api.opscode.com/organizations/my_org" chef > node.run_list => role[mac_os_x], role[workstation] Thursday, February 28, 13
  31. chef-shell attributes_mode • Context of a cookbook's attributes file. •

    Useful to paste content from your cookbook. • Follows the same rules as Chef loading attributes files Thursday, February 28, 13
  32. chef-shell attributes_mode chef > attributes_mode chef:attributes > default['shef'] = "Fun"

    => "Fun" chef:attributes > node['shef'] = "Awesometown" Chef::Exceptions::ImmutableAttributeModification: Node attributes are read-only when you do not specify which precedence level to set. To set an attribute use code like `node.default["key"] = "value"' chef:attributes > node.default['shef'] = "Awesometown" => "Awesometown" Thursday, February 28, 13
  33. chef-shell recipe_mode • Context of a Chef Recipe. • All

    resources, recipe DSL methods are available. • Highly useful for debugging! • Run in client mode for interacting with a Chef Server. Thursday, February 28, 13
  34. chef-shell recipe_mode % sudo chef-shell -z chef > recipe_mode chef:recipe

    > package "git" => <package[git] @name: "git" SNIP> chef:recipe > package "emacs" => <package[emacs] @name: "emacs" SNIP> Thursday, February 28, 13
  35. chef-shell recipe_mode chef:recipe > resources ["package[git]", "package[emacs]"] => ["package[git]", "package[emacs]"]

    chef:recipe > run_chef Thursday, February 28, 13
  36. chef-shell recipe_mode chef:recipe > search(:node, "role:workstation") => [node[doppelbock], node[champagne], node[merlot]]

    chef:recipe > data_bag_item(:users, "jtimberman") => data_bag_item[:users, "jtimberman", {"groups"=>["jtimberman", "wheel", "sysadmin", "dialout", "cdrom", "floppy", "audio", "video", "plugdev", "sambashare", "sbuild"], "comment"=>"Joshua Timberman"} Thursday, February 28, 13
  37. Chef Shell Extension library % pry 1.9.3 (main):0 > require

    'chef' 1.9.3 (main):0 > require 'chef/shell/ext' 1.9.3 (main):0 > Shell::Extensions.extend_context_object(self) 1.9.3 (main):0 > Chef::Config.from_file("/etc/chef/client.rb") 1.9.3 (main):0 > nodes.all => [node[doppelbock], node[champagne], node[menthe], node[merlot], node[virt1test], node[cask]] Use this in: knife plugins your libraries your application Thursday, February 28, 13
  38. Take the Resource Collection for a Stroll http://www.flickr.com/photos/bradhoc/5682038880 Thursday, February

    28, 13
  39. The Resource Collection • Chef loads recipes looking for resources

    • Each resource is added to the ResourceCollection • Once done, Chef walks the resource collection, taking the appropriate action Thursday, February 28, 13
  40. Consider this recipe: package "apache2" template "/etc/apache2/apache2.conf" do mode 0644

    owner "www-data" group "www-data" notifies :restart, "service[apache2]" end service "apache2" do supports :reload => true, :status => true action [:enable, :start] end Thursday, February 28, 13
  41. Paste the recipe in chef-shell then use resources() chef:recipe >

    resources ["package[apache2]", "template[/etc/apache2/ apache2.conf]", "service[apache2]"] • resources returns an array of all the resources • Each resource has a type and a name. Thursday, February 28, 13
  42. run_context.resource_collection • The run_context loads and tracks the context of

    the Chef run. • The resource collection is part of the run context. • Access the resource collection directly with run_context.resource_collection Thursday, February 28, 13
  43. Show the resource collection chef:recipe > run_context.resource_collection => #<Chef::ResourceCollection:0x000000015cc690 ...

    snip ... @resources_by_name={"package[apache2]"=>0, "template[/etc/apache2/apache2.conf]"=>1, "service[apache2]"=>2}, @insert_after_idx=nil> • resource_by_name is a hash of all the resources. • This is what #resources() looks up. • Note the numeric index Thursday, February 28, 13
  44. Walk the resource collection chef:recipe > run_context.resource_collection.each do |r| puts

    "#{r} is a #{r.resource_name}" end package[apache2] is a package template[/etc/apache2/apache2.conf] is a template service[apache2] is a service Thursday, February 28, 13
  45. The Anatomy of Loading and Executing a Single Recipe http://www.flickr.com/photos/a_cooper/2552248132/

    Thursday, February 28, 13
  46. chef-apply • Started from a gist • Now included in

    Chef 11 • /usr/bin/chef-apply • Thanks, Bryan Berry! https://gist.github.com/danielsdeleo/2920702 Thursday, February 28, 13
  47. Load the required libraries require 'chef/client' require 'chef/providers' require 'chef/resources'

    Thursday, February 28, 13
  48. Set up the run_context Chef::Config[:solo] = true client = Chef::Client.new

    client.node = Chef::Node.build("apply-node") client.run_ohai client.build_node run_context = Chef::RunContext.new( client.node, {}, client.events ) Thursday, February 28, 13
  49. Load the recipe recipe = Chef::Recipe.new( "(chef-apply cookbook)", "(chef-apply recipe)",

    run_context ) recipe.from_file(recipe_path) Thursday, February 28, 13
  50. Load the context and converge runner = Chef::Runner.new(run_context) runner.converge Thursday,

    February 28, 13
  51. chef-apply takes STDIN... % while true do nc -l 8787

    | sudo chef-apply -s done # Elsewhere... telnet 10.1.1.127 8787 package "git" ^] ^D Never run this anywhere. Thursday, February 28, 13
  52. http://www.flickr.com/photos/michaelheiss/3090102907/ There is a lot more to Chef... Thursday, February

    28, 13
  53. More about Chef • learnchef.com • www.opscode.com/chef/install • docs.opscode.com •

    lists.opscode.com • community.opscode.com Thursday, February 28, 13
  54. chefconf.opscode.com Thursday, February 28, 13

  55. Round 6, BONUS ROUND! Thursday, February 28, 13

  56. FAQ: How do you test cookbooks? Thursday, February 28, 13

  57. FAQ: Testing • Chef 10.14+, “why-run” mode • Vagrant! •

    Test Kitchen (1.0 coming soon) • ChefSpec • Cucumber Chef Thursday, February 28, 13
  58. FAQ: Why is Chef Server so hard to install? Thursday,

    February 28, 13
  59. It isn't! (anymore) http://www.flickr.com/photos/peregrinari/3801964067 Thursday, February 28, 13

  60. Chef 11 • Chef 11 Solves this • Single package

    installation • Two commands! Thursday, February 28, 13
  61. FAQ: Installation difficulty • Configuration management systems are complex in

    their own right. • The Chef Server is a publishing system with an API. • Chef is a service-oriented web application. Thursday, February 28, 13