Supporting Community Cookbooks - Patterns and Practices

For getting started with Chef, the collection of community-contributed cookbooks is an awesome secret weapon. But what happens when a community cookbook doesn't accomplish what you need? This session will explore your options, including: forking a private copy, wrapper cookbooks, monkey patching existing cookbooks, or contributing cookbook improvements back to the community. I’ll also cover cookbook design patterns that enable the varied use cases necessary for community cookbooks.

Transcript

1. Supporting Community Cookbooks Patterns and Practices paul.paradise@socrata.com natacado
2. About Me • Employee #2 at Socrata • Early Chef Adopter (version 0.5.x) • Former Lifeguard, EMT, Red Cross Instructor
3. Ghost  writer/editedfor  a  cookbook  (san  anselmo)   Hi,  I  am  a  nutri,onist  and  author  of  two  books,  (Diet  and  Weight  Loss  Lies),  which  are  going  on   Amazon  in  the  next  few  weeks,  if  not  sooner!  I  now  must  complete  my  recipe  book.  However,  the   demands  of  launching  my  books,  and  comple,ng  suppor,ng  DVDs  and  CDs,  have  leH  me  liIle   ,me  to  do  this:  I  have  many  of  the  recipes  prepared  and  in  need  of  edi,ng,  others  needing  to  be   formaIed  (in  which  I  will  give  you  the  ingredients  and  you  would  write  them).  As  such,  I  am   looking  for  a  ghostwriter  with  much  experience  in  wri,ng  cookbooks  (please  do  not  apply  if  you   do  not  specifically  have  this  experience,  as  wri,ng  recipes  is  very  different  to  wri,ng  a  book-­‐-­‐  as  I   have  found!)   ! You  must  have:   ! •  proven  skills  at  wri,ng  recipes.   •  a  firm  knowledge  and  passion  for  cooking,  par,cularly  healthy  cooking!   •  references  or  evidence  of  any  books  you  have  done,  so  I  can  research  your  style  to  see  if  we  are   a  good  fit.   ! Thank  you   writing recipes is very different to writing a book
4. https://flic.kr/p/sTTZ8
6. Working With Existing Community Cookbooks (Please Learn From My Mistakes)
7. commit 15cbb10b74ee00de52bd6d544a60bd2e90a8ab87 Author: Paul Paradise <paul.paradise@blist.com> Date: Sat Feb 21 15:32:38 2009 -0800 ! Creating motd cookbook ! --- _ _ _ _ | |__ | (_)___| |_ | '_ \| | / __| __| Host: <%= @node[:fqdn] %> | |_) | | \__ \ |_ <%= @host_motd_message %> |_.__/|_|_|___/\__| _____ _____ _____ _____ |_____|_____|_____|_____| This machine is managed by chef share your weblists
8. commit 337e301c8218d695a0fa68eb66e8b94878fd67fa Author: Paul Paradise <paul.paradise@socrata.com> Date: Sun Jun 14 21:26:14 2009 -0700 ! New company name, new motd ! --- _________ __ / _____/ ____ ________________ _/ |______ \_____ \ / _ \_/ ___\_ __ \__ \\ __\__ \ Host: <%= @node[:fqdn] %> / ( <_> ) \___| | \// __ \| | / __ \_ <%= @host_motd_message %> /_______ /\____/ \___ >__| (____ /__| (____ / \/ \/ \/ \/ Managed by Chef M a k i n g D a t a S o c i a l
9. You have reached a United States Government Server. Unauthorized access is prohibited by Public Law 99-474, The Computer Fraud and Abuse Act of 1986, and can result in administrative, disciplinary or criminal proceedings. This is a Department of Defense Server. The Utah National Guard is a command of the United States Army, an agency of the Federal Government of the United States of America. This Server, including all related equipment, networks and network devices (specifically including internet access), are provided only for authorized U.S. Government use. DoD Servers may be monitored for all lawful purposes, including to ensure that their use is authorized, for management of the system, to facilitate protection against unauthorized access, and to verify security procedure, survivability and operational security. Monitoring includes active attacks by authorized DoD entities to test or verify the security of this system. During monitoring, information may be examined, recorded, copied and used for authorized purposes. All information, including personal information, placed on or sent over this system may be monitored. Use of this DoD Server, authorized or unauthorized, constitutes consent to monitoring of this system. Unauthorized use may subject you to criminal prosecution. Evidence of unauthorized use collected during monitoring may be used for administrative, criminal or other adverse action. Use of this system constitutes consent to monitoring for these purposes. WARNING!
10. ___________________________________ < YOU ARE ON A PRODUCTION SERVER! > ----------------------------------- \ ^__^ \ (oo)\_______ (__))\/\ \\ ||----w | || ||
11. https://flic.kr/p/6grxgk Forking
12. https://flic.kr/p/6grxgk cookbooks motd attributes recipes templates default default.rb default.rb motd.erb
13. https://flic.kr/p/6grxgk cookbooks motd attributes recipes templates default default.rb default.rb motd.erb ___________________________________ < YOU ARE ON A PRODUCTION SERVER! > ----------------------------------- \ ^__^ \ (oo)\_______ (__))\/\ \ ||----w | || ||
14. https://flic.kr/p/6grxgk You are accessing a U.S. Government (USG) Information System (IS) that is provided for USG-authorized use only. ! By using this IS (which includes any device attached to this IS), you consent to the following conditions: ! ... cookbooks motd attributes recipes templates default default.rb default.rb motd.erb
15. commit 58dd958a13339dab754783755b5ff7a71b1be29c Author: Adam Jacob <adam@hjksolutions.com> Date: Thu Jan 15 00:34:19 2009 -0800 ! Initial commit ! Rakefile | 176 ++++++++++++++++++++++++++++++++++++++++++++++++++++ certificates/README | 1 + config/client.rb | 13 ++++ config/rake.rb | 54 ++++++++++++++++ config/server.rb | 16 +++++ config/solo.rb | 9 +++ cookbooks/README | 1 + site-cookbooks/README | 1 + 8 files changed, 271 insertions(+)
16. https://flic.kr/p/6grxgk cookbooks motd attributes recipes templates default default.rb default.rb motd.erb site-cookbooks motd templates default motd.erb
17. https://flic.kr/p/8vGpP3 Wrapper Cookbooks
18. https://flic.kr/p/8vGpP3 cookbooks rsyslog attributes recipes templates default default.rb default.rb client.rb server.rb Things to customize 1. Server process can only listen on udp or tcp, but not both simultaneously:
 
 attribute  'rsyslog/protocol',
    :display_name  =>  'Rsyslog  Protocol',
    :description  =>  'Set  which  network   protocol  to  use  for  rsyslog',
    :default  =>  'tcp'
 ! 2. Server should gzip old archived logs
19. https://flic.kr/p/8vGpP3 include_recipe  "rsyslog::server"   include_recipe  "socrata-­‐rsyslog::_server_udp"   ! template  "/etc/cron.d/rsyslog_gz"  do      source  "rsyslog_gz.erb"      owner  "root"      group  "root"      mode  0644      variables  :log_dir  =>  node[:rsyslog][:log_dir]   end cookbooks socrata-rsyslog attributes recipes templates default default.rb default.rb rsyslog_gz.erb _server_udp.rb
20. https://flic.kr/p/8vGpP3 cookbooks socrata-rsyslog attributes recipes templates default default.rb default.rb rsyslog_gz.erb _server_udp.rb template "/etc/rsyslog.d/01-server- udp.conf" do! source "01-server-udp.conf.erb"! owner node['rsyslog']['user']! group node['rsyslog']['group']! mode 0644! notifies :restart, ! ”service[#{node['rsyslog'] ['service_name']}]"! end!
21. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o Monkey Patching
22. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o ruby_block "make tea" do! block do! Chef::Log.info("Brewing some #{node[:tea][:leaves]} tea...")! Kernel.sleep 5! end! ! action :create! end
23. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o [2014-04-13T15:52:46-07:00] INFO: Processing ruby_block[make tea] action create (tea::default line 1)! [2014-04-13T15:52:46-07:00] INFO: Brewing some green tea...! [2014-04-13T15:52:51-07:00] INFO: ruby_block[make tea] called! [2014-04-13T15:52:51-07:00] INFO: Chef Run complete in 5.1282 seconds!
25. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o Reset Node Attributes Compile Resource Collection Converge the Node Update Node Object, Process Handlers http://docs.opscode.com/essentials_nodes_chef_run.html Anatomy of a Chef Run
26. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o ruby_block "make tea" do! block do! Chef::Log.info("Brewing some #{node[:tea][:leaves]} tea...")! Kernel.sleep 5! end! ! action :create! end Anatomy of a Chef Run 1. Reduce brew time to zero 2. Notify that tea is ready Goals:
27. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o cookbooks captain resources providers default.rb default.rb picard_tea attributes recipes default.rb default.rb actions :tea_is_ready action :tea_is_ready do! Chef::Log.info “Make it so!"! end!
28. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o include_recipe "tea"! ! captain ‘picard'! ! existing_resource = ↵! resources(‘ruby_block[make tea]')! ! existing_resource.block do! Chef::Log.info("Brewing some ↵! #{node[:tea][:leaves]} tea...")! Kernel.sleep node[:tea][:brew_time]! end! ! existing_resource.notifies :tea_is_ready, ↵! ’captain[picard]'! cookbooks captain resources providers default.rb default.rb picard_tea attributes recipes default.rb default.rb
29. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o ! [2014-04-13T16:20:37-07:00] INFO: Processing ruby_block[make tea] action create (tea::default line 1)! [2014-04-13T16:20:37-07:00] INFO: Brewing some earl grey tea...! [2014-04-13T16:20:37-07:00] INFO: ruby_block[make tea] called! [2014-04-13T16:20:37-07:00] INFO: ruby_block[make tea] sending tea_is_ready action to captain[picard] (delayed)! [2014-04-13T16:20:37-07:00] INFO: Processing captain[picard] action tea_is_ready (picard_tea::default line 3)! [2014-04-13T16:20:37-07:00] INFO: Make it so!! [2014-04-13T16:20:37-07:00] INFO: Chef Run complete in 0.014965 seconds! !
30. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o include_recipe "tea"! captain 'picard'! ! chef_gem "chef-rewind"! require 'chef/rewind'! ! rewind "ruby_block[make tea]" do! block do! Chef::Log.info("Brewing some ↵! #{node[:tea][:leaves]} tea...")! Kernel.sleep node[:tea][:brew_time]! end! ! notifies :tea_is_ready, 'captain[picard]'! end! cookbooks captain resources providers default.rb default.rb rewind_tea attributes recipes default.rb default.rb
31. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o [2014-04-13T16:38:58-07:00] INFO: Processing chef_gem[chef-rewind] action install (rewind_tea::default line 5)! [2014-04-13T16:38:58-07:00] INFO: Resource ruby_block[make tea] found, now rewinding it! [2014-04-13T16:38:58-07:00] INFO: Processing ruby_block[make tea] action create (tea::default line 1)! [2014-04-13T16:38:58-07:00] INFO: Brewing some earl grey tea...! [2014-04-13T16:38:58-07:00] INFO: ruby_block[make tea] called! [2014-04-13T16:38:58-07:00] INFO: Processing chef_gem[chef-rewind] action install (rewind_tea::default line 5)! [2014-04-13T16:38:58-07:00] INFO: ruby_block[make tea] sending tea_is_ready action to captain[picard] (delayed)! [2014-04-13T16:38:58-07:00] INFO: Processing captain[picard] action tea_is_ready (rewind_tea::default line 3)! [2014-04-13T16:38:58-07:00] INFO: Make it so!! [2014-04-13T16:38:58-07:00] INFO: Chef Run complete in 0.017244 seconds! !
32. https://flic.kr/p/8vGpP3 https://flic.kr/p/B2Q6o Caveats of Rewinding • Templates still use initial cookbook_name • Notifications are append-only • Likely to duplicate code
33. Contributing Back
36. .kitchen.yml | 89 ++++++------------- Berksfile | 2 +- CHANGELOG.md | 141 ------------------------------ CONTRIBUTING.md | 269 +++++++++++++++++++++++++++++++++++++++++++++++++++++++--- Gemfile | 6 +- README.md | 197 ++++++++++++++++-------------------------- TESTING.md | 48 ++++------- attributes/default.rb | 63 ++++++-------- files/default/tests/minitest/openjdk_test.rb | 20 +++++ files/default/tests/minitest/oracle_rpm_test.rb | 19 +++++ files/default/tests/minitest/oracle_test.rb | 20 +++++ files/default/tests/minitest/support/helpers.rb | 29 +++++++ libraries/helpers.rb | 23 +---- libraries/matchers.rb | 5 -- metadata.rb | 19 ++--- providers/alternatives.rb | 79 ----------------- providers/ark.rb | 106 ++++++++++++++--------- recipes/default.rb | 13 +-- recipes/default_java_symlink.rb | 19 ----- recipes/ibm.rb | 21 ----- recipes/ibm_tar.rb | 70 --------------- recipes/openjdk.rb | 34 ++------ recipes/oracle.rb | 24 ++---- recipes/oracle_i386.rb | 20 +---- recipes/oracle_rpm.rb | 4 +- recipes/purge_packages.rb | 20 ----- recipes/set_attributes_from_version.rb | 53 ------------ recipes/windows.rb | 9 +- resources/alternatives.rb | 29 ------- resources/ark.rb | 16 +--- spec/default_java_symlink_spec.rb | 12 --- spec/default_spec.rb | 75 ++++------------ spec/ibm_spec.rb | 63 +------------- spec/ibm_tar_spec.rb | 34 -------- spec/libraries/helpers_spec.rb | 2 +- spec/openjdk_spec.rb | 131 +++++++++------------------- spec/oracle_i386_spec.rb | 63 +------------- spec/oracle_spec.rb | 63 +------------- spec/set_attributes_from_version_spec.rb | 59 ------------- spec/set_java_home_spec.rb | 12 ++- spec/spec_helper.rb | 1 - spec/windows_spec.rb | 10 +-- 47 files changed, 662 insertions(+), 1400 deletions(-) 662 insertions < 1400 deletions
38. 41 Pull Requests 14 Releases 7 New Recipes
39. Pitfalls of Community Cookbook Maintainership
41. https://flic.kr/p/8vGpP3 https://flic.kr/p/sUzqA
42. Dependencies • Depends: Required • Recommends: Strongly suggested? • Suggests: Weakly suggested?
43. Cookbook Recommends elasticsearch build-essential, java, monit, xml interface ucarp jira mysql
44. Cookbook Suggests build- essential pkgutil (Solaris 2) gems ruby java aws, windows solr ruby
45. What does the community use? 0 450 900 1350 1800 Depends Recommends Suggests 50 47 1,664
48. LWRP-ifying • There’s More Than One Way To Do IT • Rule of thumb: if you’re ever going to need 2+ of the same thing on a single server, use LWRPs to do so.
49. # resources/lb.rb! actions :create! default_action :create! ! attribute :name, :kind_of => String, :name_attribute => true! attribute :type, :kind_of => String, :default => 'listen', :equal_to => ['listen', 'backend', 'frontend']! ! #Defining some attributes, but they can be all nil and defined in params! #if convenient.! attribute :servers, :kind_of => Array, :default => []! attribute :balance, :kind_of => String! attribute :bind, :kind_of => String, :default => nil! attribute :mode, :kind_of => String, :default => nil,! :equal_to => ['http', 'tcp', 'health', nil]! ! attribute :params, :kind_of => [Array, Hash], :default => []!
50. # providers/lb.rb! action :create do! #While there is no way to have an include directive for haproxy! #configuration file, this provider will only modify attributes !! listener = []! listener << "bind #{new_resource.bind}" unless new_resource.bind.nil?! listener << "balance #{new_resource.balance}" unless new_resource.balance.nil?! listener << "mode #{new_resource.mode}" unless new_resource.mode.nil?! listener += new_resource.servers.map {|server| "server #{server}" }! ! if new_resource.params.is_a? Hash! listener += new_resource.params.map { |k,v| "#{k} #{v}" }! else! listener += new_resource.params! end! ! node.default['haproxy']['listeners'][new_resource.type][new_resource.name] = ↵! listener! end!
51. ! # Set up application listeners here.! ! <% node['haproxy']['listeners'].each do |type, listeners | %>! <% listeners.each do |name, listen| %>! <%= type %> <%= name %>! <% listen.each do |option| %>! <%= option %>! <% end %>! ! <% end %>! ! <% end %>!
52. Packaging and Distribution • Don’t assume any Internet access on the node • Don’t assume every OS has the software you need pre-packaged in the version you want • Don’t assume everyone uses the OS-provided packages • Don’t assume every node has a compiler available • Don’t assume every node has the most recent version of Chef
53. # On Chef Solo, we use the node['rsyslog']['server_ip'] attribute, and on! # normal Chef, we leverage the search query.! if Chef::Config[:solo]! if node['rsyslog']['server_ip']! rsyslog_servers = Array(node['rsyslog']['server_ip'])! else! Chef::Application.fatal!("Chef Solo does not support search. You must set ↵! node['rsyslog']['server_ip']!")! end! else! results = search(:node, node['rsyslog']['server_search']).map do |server|! ipaddress = server['ipaddress']! # If both server and client are on the same cloud and local network, they may be! # instructed to communicate via the internal interface by enabling `use_local_ipv4`! if node['rsyslog']['use_local_ipv4'] && server.attribute?('cloud') && ↵! server['cloud']['local_ipv4']! ipaddress = server['cloud']['local_ipv4']! end! ipaddress! end! rsyslog_servers = Array(node['rsyslog']['server_ip']) + Array(results)! end Making Search Optional
54. Thanks jtimberman carmstrong_afk
55. Questions? paul.paradise@socrata.com natacado Office Hours: 4:15pm Marina Room