$30 off During Our Annual Pro Sale. View Details »

5 Things You Didn't Know About Chef

5 Things You Didn't Know About Chef

This is the slide deck I used to present at Big Ruby 2013.

http://www.bigrubyconf.com/

Joshua Timberman

February 28, 2013
Tweet

More Decks by Joshua Timberman

Other Decks in Technology

Transcript

  1. 5 Things You Didn't Know About Chef
    Joshua Timberman
    [email protected]
    jtimberman
    community.opscode.com
    Thursday, February 28, 13

    View Slide

  2. Chef is an Automation Platform for developers
    & systems engineers to build & manage
    Infrastructure as Code.
    "743 lines of #opschef to build new product
    infrastructure from scratch. Thank you,
    @opscode.”
    -@bdha, August 18, 2011
    What is Chef?
    Thursday, February 28, 13

    View Slide

  3. Chef is a Framework
    • Configuration management
    • Reasonability
    • Idempotent, Convergent
    • Flexibility
    • Library & Primitives
    • TIMTOWTDI
    http://www.flickr.com/photos/elitatt/6980379333/
    Thursday, February 28, 13

    View Slide

  4. package "haproxy" do
    action :install
    end
    template "/etc/haproxy/haproxy.cfg" do
    source "haproxy.cfg.erb"
    owner "root"
    group "root"
    mode 0644
    notifies :restart, "service[haproxy]"
    end
    service "haproxy" do
    supports :restart => true
    action [:enable, :start]
    end
    Chef Enables Infrastructure as Code
    • Resources
    • Recipes
    • Cookbooks and Roles
    • Source Code
    Thursday, February 28, 13

    View Slide

  5. Agenda
    • In-place File Editing for Greater Good
    • Use Chef's Built-in Version Matching
    • REPLs are fun, so Chef has one!
    • Take the Resource Collection for a
    Stroll
    • The Anatomy of Loading and
    Executing a Single Recipe
    http://www.flickr.com/photos/koalazymonkey/3590953001/
    • Bonus, if we have time!
    Thursday, February 28, 13

    View Slide

  6. http://www.flickr.com/photos/dolarz/2333292651/
    In-place File Editing
    (for Greater Good™)
    Thursday, February 28, 13

    View Slide

  7. net.ipv4.conf.default.rp_filter = 1
    net.ipv4.conf.default.accept_source_route = 0
    kernel.sysrq = 0
    kernel.core_uses_pid = 1
    net.ipv4.tcp_synack_retries = 2
    net.ipv4.conf.all.send_redirects = 0
    net.ipv4.conf.default.send_redirects = 0
    net.ipv4.conf.all.accept_source_route = 0
    net.ipv4.conf.all.accept_redirects = 0
    net.ipv4.conf.all.secure_redirects = 0
    net.ipv4.conf.all.log_martians = 1
    net.ipv4.conf.default.accept_source_route = 0
    net.ipv4.conf.default.accept_redirects = 0
    net.ipv4.conf.default.secure_redirects = 0
    net.ipv4.icmp_echo_ignore_broadcasts = 1
    net.ipv4.tcp_syncookies = 1
    net.ipv4.conf.all.rp_filter = 1
    net.ipv4.conf.default.rp_filter = 1
    net.ipv6.conf.default.router_solicitations = 0
    net.ipv6.conf.default.accept_ra_rtr_pref = 0
    net.ipv6.conf.default.accept_ra_pinfo = 0
    net.ipv6.conf.default.accept_ra_defrtr = 0
    net.ipv6.conf.default.autoconf = 0
    net.ipv6.conf.default.dad_transmits = 0
    net.ipv6.conf.default.max_addresses = 1
    kernel.exec-shield = 1
    kernel.randomize_va_space = 1
    I need to just update
    this one line...
    http://www.flickr.com/photos/dmswart/6507071701/
    < Right here?
    Thursday, February 28, 13

    View Slide

  8. http://www.flickr.com/photos/rhysasplundh/4807749216/
    Thursday, February 28, 13

    View Slide

  9. Danger: Anti-pattern?
    • This makes it hard to reason about
    where changes to files are coming
    from
    • Manage whole-file resources
    (template, cookbook_file,
    remote_file)
    • But sometimes you have to edit a
    line...
    Thursday, February 28, 13

    View Slide

  10. Introducing Chef::Util::FileEdit
    ruby_block "edit etc hosts" do
    block do
    rc = Chef::Util::FileEdit.new("/etc/hosts")
    rc.search_file_replace_line(
    /^127\.0\.0\.1 localhost$/,
    "127.0.0.1 #{new_fqdn} #{new_hostname} localhost"
    )
    rc.write_file
    end
    end
    Thursday, February 28, 13

    View Slide

  11. Chef::Util::FileEdit methods...
    #if matched, replace the whole line with newline.
    search_file_replace_line(regex, newline)
    #if matched, replace the match (all occurances) with the replace parameter
    search_file_replace(regex, replace)
    #if matched, delete the line
    search_file_delete_line(regex)
    #if matched, delete the match (all occurances) from the line
    search_file_delete(regex)
    #if matched, insert newline after each matching line
    insert_line_after_match(regex, newline)
    #if not matched, insert newline at the end of the file
    insert_line_if_no_match(regex, newline)
    #write the new file content
    write_file
    Thursday, February 28, 13

    View Slide

  12. Sean OMeara's line cookbook
    • Sean OMeara wrote a "line"
    cookbook.
    • Resource/Provider for managing
    single lines.
    • http://ckbk.it/line
    http://www.flickr.com/photos/49889874@N05/6743622955/
    Thursday, February 28, 13

    View Slide

  13. Use Chef's Built-in Version
    Matching
    http://www.flickr.com/photos/tschiae/8395137978/
    Thursday, February 28, 13

    View Slide

  14. Places to match versions
    • Platforms
    • Cookbooks
    • Software/packages
    Thursday, February 28, 13

    View Slide

  15. Some Caveats
    • Match versions that are dotted
    notation up to 3 places
    • X.Y.Z
    • 5.6, 1.8.0, etc
    Thursday, February 28, 13

    View Slide

  16. Common Example
    if node['platform_version'].to_f >= 10.04
    if node['platform_version'] == "6.3"
    Thursday, February 28, 13

    View Slide

  17. Chef::VersionConstraint
    • First class comparison operators
    • Allows X.Y.Z version strings
    • Not lossy, unlike (potentially) #to_i
    or #to_f.
    http://docs.opscode.com/essentials_cookbook_versions_constraints.html
    Thursday, February 28, 13

    View Slide

  18. How to use Chef::VersionConstraint
    • `require` the library
    • Initialize a Chef::VersionConstraint
    object w/ the constraint
    • Send the include? method with the
    version to match
    Thursday, February 28, 13

    View Slide

  19. Constraint Operator
    = Equal
    > Greater than
    < Less than
    >= Greater than / equal
    <= Less than / equal
    ~> Approximately greater than
    (pessimistic)
    Thursday, February 28, 13

    View Slide

  20. Chef::VersionConstraint
    require 'chef/version_constraint'
    Chef::VersionConstraint.new(">= 10.7.0").include?("10.6.0") #=> false
    Chef::VersionConstraint.new(">= 10.7.0").include?("10.7.3") #=> true
    Chef::VersionConstraint.new(">= 10.7.0").include?("10.8.2") #=> true
    Thursday, February 28, 13

    View Slide

  21. Version Examples
    # On CentOS 5.8
    Chef::VersionConstraint.new("~> 6.0").
    include?(node['platform_version']) #=> false
    # On CentOS 6.3
    Chef::VersionConstraint.new("~> 6.0").
    include?(node['platform_version']) #=> true
    Thursday, February 28, 13

    View Slide

  22. Debian 6.0.3 vs. #to_f
    6.0 == "6.0.3".to_f # => true
    6.0 < "6.0.3".to_f # => false
    6.0 > "6.0.3".to_f # => false
    Thursday, February 28, 13

    View Slide

  23. Debian 6.0.3 vs. #to_f
    6.0 == "6.0.3".to_f # => true
    6.0 < "6.0.3".to_f # => false
    6.0 > "6.0.3".to_f # => false
    Chef::VersionConstraint.new("> 6.0").
    include?("6.0.3") # => true
    Thursday, February 28, 13

    View Slide

  24. REPLs are fun,
    Chef has one!
    http://www.flickr.com/photos/tanaka_juuyoh/4434232143/
    Thursday, February 28, 13

    View Slide

  25. chef-shell, the Chef REPL
    • Chef 11: chef-shell
    • Chef 10: shef
    • Built on IRB
    • Three modes/contexts:
    • main, attributes, recipe
    http://www.flickr.com/photos/51064559@N06/5746903658/
    Thursday, February 28, 13

    View Slide

  26. chef-shell main context
    • Access node attributes
    • Interact with a Chef Server
    • It's IRB...
    Thursday, February 28, 13

    View Slide

  27. chef-shell main
    % chef-shell
    loading configuration: none (standalone session)
    Session type: standalone
    Loading...done.
    This is the chef-shell.
    Chef Version: 11.2.0
    http://www.opscode.com/chef
    http://wiki.opscode.com/display/chef/Home
    run `help' for help, `exit' or ^D to quit.
    Ohai2u jtimberman@champagne
    chef >
    Thursday, February 28, 13

    View Slide

  28. chef-shell main
    chef > help
    chef-shell Help
    ==================================================
    | Command | Description
    ==================================================
    | Lots of commands | and their descriptions
    chef > halp
    :-)
    Thursday, February 28, 13

    View Slide

  29. chef-shell main context
    • Many of the commands available
    work with a Chef Server's API.
    • So let's start up chef-shell
    configured for a Chef Server.
    Thursday, February 28, 13

    View Slide

  30. Interact with a Chef Server
    % chef-shell -z
    loading configuration: /etc/chef/client.rb
    Session type: client
    ...
    chef > Chef::Config[:chef_server_url]
    => "https://api.opscode.com/organizations/my_org"
    chef > node.run_list
    => role[mac_os_x], role[workstation]
    Thursday, February 28, 13

    View Slide

  31. chef-shell attributes_mode
    • Context of a cookbook's attributes
    file.
    • Useful to paste content from your
    cookbook.
    • Follows the same rules as Chef
    loading attributes files
    Thursday, February 28, 13

    View Slide

  32. chef-shell attributes_mode
    chef > attributes_mode
    chef:attributes > default['shef'] = "Fun"
    => "Fun"
    chef:attributes > node['shef'] = "Awesometown"
    Chef::Exceptions::ImmutableAttributeModification: Node attributes are
    read-only when you do not specify which precedence level to set. To
    set an attribute use code like `node.default["key"] = "value"'
    chef:attributes > node.default['shef'] = "Awesometown"
    => "Awesometown"
    Thursday, February 28, 13

    View Slide

  33. chef-shell recipe_mode
    • Context of a Chef Recipe.
    • All resources, recipe DSL methods
    are available.
    • Highly useful for debugging!
    • Run in client mode for interacting
    with a Chef Server.
    Thursday, February 28, 13

    View Slide

  34. chef-shell recipe_mode
    % sudo chef-shell -z
    chef > recipe_mode
    chef:recipe > package "git"
    =>
    chef:recipe > package "emacs"
    =>
    Thursday, February 28, 13

    View Slide

  35. chef-shell recipe_mode
    chef:recipe > resources
    ["package[git]", "package[emacs]"]
    => ["package[git]", "package[emacs]"]
    chef:recipe > run_chef
    Thursday, February 28, 13

    View Slide

  36. chef-shell recipe_mode
    chef:recipe > search(:node, "role:workstation")
    => [node[doppelbock], node[champagne], node[merlot]]
    chef:recipe > data_bag_item(:users, "jtimberman")
    => data_bag_item[:users, "jtimberman", {"groups"=>["jtimberman",
    "wheel", "sysadmin", "dialout", "cdrom", "floppy", "audio",
    "video", "plugdev", "sambashare", "sbuild"], "comment"=>"Joshua
    Timberman"}
    Thursday, February 28, 13

    View Slide

  37. Chef Shell Extension library
    % pry
    1.9.3 (main):0 > require 'chef'
    1.9.3 (main):0 > require 'chef/shell/ext'
    1.9.3 (main):0 > Shell::Extensions.extend_context_object(self)
    1.9.3 (main):0 > Chef::Config.from_file("/etc/chef/client.rb")
    1.9.3 (main):0 > nodes.all
    => [node[doppelbock],
    node[champagne],
    node[menthe],
    node[merlot],
    node[virt1test],
    node[cask]]
    Use this in:
    knife plugins
    your libraries
    your application
    Thursday, February 28, 13

    View Slide

  38. Take the Resource
    Collection for a Stroll
    http://www.flickr.com/photos/bradhoc/5682038880
    Thursday, February 28, 13

    View Slide

  39. The Resource Collection
    • Chef loads recipes looking for
    resources
    • Each resource is added to the
    ResourceCollection
    • Once done, Chef walks the resource
    collection, taking the appropriate
    action
    Thursday, February 28, 13

    View Slide

  40. Consider this recipe:
    package "apache2"
    template "/etc/apache2/apache2.conf" do
    mode 0644
    owner "www-data"
    group "www-data"
    notifies :restart, "service[apache2]"
    end
    service "apache2" do
    supports :reload => true, :status => true
    action [:enable, :start]
    end
    Thursday, February 28, 13

    View Slide

  41. Paste the recipe in chef-shell
    then use resources()
    chef:recipe > resources
    ["package[apache2]", "template[/etc/apache2/
    apache2.conf]", "service[apache2]"]
    • resources returns an array of all the resources
    • Each resource has a type and a name.
    Thursday, February 28, 13

    View Slide

  42. run_context.resource_collection
    • The run_context loads and tracks
    the context of the Chef run.
    • The resource collection is part of
    the run context.
    • Access the resource collection
    directly with
    run_context.resource_collection
    Thursday, February 28, 13

    View Slide

  43. Show the resource collection
    chef:recipe > run_context.resource_collection
    => #snip ... @resources_by_name={"package[apache2]"=>0,
    "template[/etc/apache2/apache2.conf]"=>1,
    "service[apache2]"=>2}, @insert_after_idx=nil>
    • resource_by_name is a hash of all the resources.
    • This is what #resources() looks up.
    • Note the numeric index
    Thursday, February 28, 13

    View Slide

  44. Walk the resource collection
    chef:recipe >
    run_context.resource_collection.each do |r|
    puts "#{r} is a #{r.resource_name}"
    end
    package[apache2] is a package
    template[/etc/apache2/apache2.conf] is a template
    service[apache2] is a service
    Thursday, February 28, 13

    View Slide

  45. The Anatomy of Loading and
    Executing a Single Recipe
    http://www.flickr.com/photos/a_cooper/2552248132/
    Thursday, February 28, 13

    View Slide

  46. chef-apply
    • Started from a gist
    • Now included in Chef 11
    • /usr/bin/chef-apply
    • Thanks, Bryan Berry!
    https://gist.github.com/danielsdeleo/2920702
    Thursday, February 28, 13

    View Slide

  47. Load the required libraries
    require 'chef/client'
    require 'chef/providers'
    require 'chef/resources'
    Thursday, February 28, 13

    View Slide

  48. Set up the run_context
    Chef::Config[:solo] = true
    client = Chef::Client.new
    client.node = Chef::Node.build("apply-node")
    client.run_ohai
    client.build_node
    run_context = Chef::RunContext.new(
    client.node, {}, client.events
    )
    Thursday, February 28, 13

    View Slide

  49. Load the recipe
    recipe = Chef::Recipe.new(
    "(chef-apply cookbook)",
    "(chef-apply recipe)",
    run_context
    )
    recipe.from_file(recipe_path)
    Thursday, February 28, 13

    View Slide

  50. Load the context and converge
    runner = Chef::Runner.new(run_context)
    runner.converge
    Thursday, February 28, 13

    View Slide

  51. chef-apply takes STDIN...
    % while true
    do
    nc -l 8787 | sudo chef-apply -s
    done
    # Elsewhere...
    telnet 10.1.1.127 8787
    package "git"
    ^]
    ^D
    Never run this anywhere.
    Thursday, February 28, 13

    View Slide

  52. http://www.flickr.com/photos/michaelheiss/3090102907/
    There is a lot more to
    Chef...
    Thursday, February 28, 13

    View Slide

  53. More about Chef
    • learnchef.com
    • www.opscode.com/chef/install
    • docs.opscode.com
    • lists.opscode.com
    • community.opscode.com
    Thursday, February 28, 13

    View Slide

  54. chefconf.opscode.com
    Thursday, February 28, 13

    View Slide

  55. Round 6, BONUS ROUND!
    Thursday, February 28, 13

    View Slide

  56. FAQ: How do you test
    cookbooks?
    Thursday, February 28, 13

    View Slide

  57. FAQ: Testing
    • Chef 10.14+, “why-run” mode
    • Vagrant!
    • Test Kitchen (1.0 coming soon)
    • ChefSpec
    • Cucumber Chef
    Thursday, February 28, 13

    View Slide

  58. FAQ: Why is Chef Server
    so hard to install?
    Thursday, February 28, 13

    View Slide

  59. It isn't! (anymore)
    http://www.flickr.com/photos/peregrinari/3801964067
    Thursday, February 28, 13

    View Slide

  60. Chef 11
    • Chef 11 Solves this
    • Single package installation
    • Two commands!
    Thursday, February 28, 13

    View Slide

  61. FAQ: Installation difficulty
    • Configuration management
    systems are complex in their
    own right.
    • The Chef Server is a
    publishing system with an API.
    • Chef is a service-oriented web
    application.
    Thursday, February 28, 13

    View Slide