Reprogramming Chef

Transcript

1. Reprogramming Chef
2. Steven Danna steve@getchef.com
3. Tales of Chef Support
4. configuration management Chef
5. describes the configuration of a software package or service using a Domain Specific Language (DSL) Recipe:
6. contains recipes and the other supporting materials Cookbook:
7. package "nginx"! ! template "/etc/nginx.conf" do! source "nginx.conf.erb"! variables :doc_root => "/var/www"! notifies :restart, "service[nginx]"! end! ! service "nginx" do! action [:enable, :start]! end!
8. package "nginx"! ! template "/etc/nginx.conf" do! source "nginx.conf.erb"! variables :doc_root => "/var/www"! notifies :restart, "service[nginx]"! end! ! service "nginx" do! action [:enable, :start]! end!
9. package "nginx"! ! template "/etc/nginx.conf" do! source "nginx.conf.erb"! variables :doc_root => "/var/www"! notifies :restart, "service[nginx]"! end! ! service "nginx" do! action [:enable, :start]! end!
10. package "nginx"! ! template "/etc/nginx.conf" do! source "nginx.conf.erb"! variables :doc_root => "/var/www"! notifies :restart, "service[nginx]"! end! ! service "nginx" do! action [:enable, :start]! end!
11. package "nginx"! ! template "/etc/nginx.conf" do! source "nginx.conf.erb"! variables :doc_root => "/var/www"! notifies :restart, "service[nginx]"! end! ! service "nginx" do! action [:enable, :start]! end!
12. The Chef DSL is built on Ruby
13. packages = [ "emacs",! "tmux",! "irssi" ]! ! packages.each do |pkg|! package pkg do! action :install! end! end!
14. Chef uses Ohai to collect system information
15. pkg_name = if node['platform_family'] == 'debian'! 'openssl-dev'! elsif node['platform_family'] == 'rhel'! 'openssl-devel'! end! ! package pkg_name!
16. What else could I need?
17. Bug Fixes and Workarounds New Features Service Integrations More Expressive Recipes Better Abstractions
18. Happy ! Users
19. Plugins ! vs Reprogrammability
20. Plugins External Interface ! ≠ ! Internal Interface
21. Reprogrammability External Interface ! =! Internal Interface
22. Ohai Plugins Knife Plugins Start, Report, and Error Handlers Event Subscribers and Output Formatters Definitions LWRPs
23. Most “plugin” features of Chef are inspired by reprogrammability
24. Internally Ohai uses the same plugin interface it offers to users
25. Ticket #1: The LDAP Flood Ohai is trying to pull in my Corporation’s entire LDAP directory on every run. Please help!
26. To The Source! https://github.com/opscode/ohai
27. Etc.passwd do |entry|! user_passwd_entry = Mash.new(:dir => entry.dir, ! ! ! ! ! ! ! ! ! ! ! ! ! :gid => entry.gid, ! ! ! ! ! ! ! ! ! ! ! ! ! :uid => entry.uid, ! ! ! ! ! ! ! ! ! ! ! ! :shell => entry.shell, ! ! ! ! ! ! ! ! ! ! ! ! ! :gecos => entry.gecos)! user_passwd_entry.each_value {|v| fix_encoding(v)}! etc[:passwd][fix_encoding(entry.name)] = user_passwd_entry! end lib/ohai/plugins/passwd.rb
28. To The Source! https://github.com/ruby/ruby
29. static VALUE! passwd_iterate(void)! {! struct passwd *pw;! ! setpwent();! while (pw = getpwent()) {! rb_yield(setup_passwd(pw));! }! return Qnil;! }! ext/etc/etc.c
30. To The Man Page! http://linux.die.net/man/3/getpwent
31. “The getpwent() function returns a pointer to a structure containing the broken-out fields of a record from the password database (e.g., the local password file /etc/passwd, NIS, and LDAP)”
32. I’ve found the cause. ! ! Now what?
33. ! File a bug and wait for it to be fixed?
34. Ship an ! Ohai Plugin ! NOW
35. provides 'etc'! ! etc Mash.new! etc[:passwd] = Mash.new! etc[:group] = Mash.new! ! File.open("/etc/passwd", "r") do |f|! f.each_line do |line|! etc[:passwd].merge!(parse_passwd_line(line))! end! end! ! File.open("/etc/group", "r") do |f|! f.each_line do |line|! etc[:group].merge!(parse_group_line(line))! end! end! ! # parse_passwd_line and parse_group_line not shown!
36. Ohai::Config[:plugin_path] << "/etc/chef/plugins/ohai/"! Ohai::Config[:disabled_plugins] = ["passwd"]! client.rb
37. Ticket Closed! Customer Happy
38. Plugins Structured Reprogrammability
40. Unstructured Reprogrammability
41. Chef Recipes are ! Ruby
42. Ruby’s open classes make Chef reprogrammable by default
43. Open Classes, Part I class String! def leet(str=self)! str.gsub("e", "3").gsub("o", "0")! end! end! ! puts "hello, world".leet!
44. Open Classes, Part I module StringExtensions! def leet(str=self)! str.gsub("e", "3").gsub("o", "0")! end! end! ! class String! include StringExtensions! end! ! puts "hello, world".leet!
45. Open Classes, Part I module StringExtensions! def leet(str=self)! str.gsub("e", "3").gsub("o", "0")! end! end! ! String.send(:include, StringExtensions)! ! ! puts "hello, world".leet!
46. Ticket #2: The ugly search I have a recipe that configures remote rsyslog. But a search for the syslog server isn’t returning what I expect. Can you help?
47. rsyslog_server = search(:node, ! "role:rsyslog-server").sort.first! ! template "/etc/rsyslog.conf" do! variables :server => rsyslog_server['fqdn']! end!
48. What happens if the node running the recipe is the rsyslog-server, and this is the first run?
49. rsyslog_server = search(:node, ! "role:rsyslog-server").sort.first! ! if rsyslog_server.nil? ! rsyslog_server = node! else! ! template "/etc/rsyslog.conf" do! variables :server => rsyslog_server['fqdn']! end!
50. rsyslog_server = search(:node, ! "role:rsyslog-server").sort.first! ! if rsyslog_server.nil? && ! node['roles'].include?('rsyslog-server')! rsyslog_server = node! else! Chef::Application.fatal!(‘No rsyslog-server found.')! end! ! template "/etc/rsyslog.conf" do! variables :server => rsyslog_server['fqdn']! end!
51. “That is a bit better, but I the recipe is a lot uglier and I want to use a similar pattern in other cookbooks.”
52. Extend ! the ! DSL
53. module YourCorp! module DSL! def node_with_role(role)! candidates = search(:node, "roles:#{role}").sort! candidates << node if node['roles'].include?(role)! if candidates.empty?! fail "No node found with role:#{role}"! else! candidates.first! end! end! end! end! ! Chef::Recipe.send(:include, YourCorp::DSL)!
54. Any file in the libraries directory is loaded near the beginning of the Chef Client run
55. template "/etc/rsyslog.conf" do! variables :server => node_with_role(‘rsyslog-server’)! end
56. Chef Sugar
58. Beyond Syntactic Sugar
59. Open Classes, Part II class String! def upcase! # Keep your workstation locked!! "LOLOLOL"! end! end! ! puts "hello, world".upcase!
60. Open Classes, Part II class String! alias_method :old_upcase, :upcase! def upcase! old_upcase + "!!!"! end! end! ! puts "hello, world".upcase!
61. Monkey Patching, really?
62. Ticket #3: The Poor Typist Every time I typo my nginx configuration template, the service fails to restart properly. Why can’t Chef help me?!
63. package "nginx"! ! template "/etc/nginx.conf" do! source "nginx.conf.erb"! variables :doc_root => "/var/www"! notifies :restart, "service[nginx]"! end! ! service "nginx" do! action [:enable, :start]! end!
64. package "nginx"! ! template "/etc/nginx.conf.temp" do! source "nginx.conf.erb"! variables :doc_root => "/var/www"! notifies :run, "bash[check-nginx-template]", :immediately! end! ! bash "check-nginx-template" do! command "nginx -t -c /etc/nginx.conf.temp"! notifies :create, "file[/etc/nginx.conf]", :immediately! end! ! file "/etc/nginx.conf" do! content lazy { ::File.read(“/etc/nginx.conf.temp”) }! action :nothing! notifies :restart, "service[nginx]", :immediately! end! ! service "nginx" do! action [:enable, :start]! end!
65. “Ummm….I thought Chef was easy?”
66. # This should succeed! template "/tmp/foo" do! verify do |path|! true! end! end! ! # This should raise an error! template "/tmp/bar" do! verify do |path|! false! end! end!
67. package "nginx"! ! template "/etc/nginx.conf" do! source "nginx.conf.erb"! variables :doc_root => "/var/www"! verify do |path|! `nginx -t -c #{path}`! $? == 0! end! notifies :restart, "service[nginx]"! end! ! service "nginx" do! action [:enable, :start]! end!
69. class Chef::Resource::Template < Chef::Resource::File! def verify(&block)! if block_given?! @verify_block=block! else! @verify_block! end! end! end
70. No m ore syntax Error!! But, we expected the second tem plate ! to fail validation
71. To The Source! https://github.com/opscode/chef
72. template “foo” template “foo” do action :create end Chef::Provider::Template#action_create
73. LIES! There is no action_create method!
74. Maybe it’s here. lib/chef/providers/file.rb
75. This looks promising. lib/chef/providers/file.rb
76. lib/chef/providers/file.rb YIKES!
77. lib/chef/providers/file.rb
78. class Chef::Resource::Template < Chef::Resource::File! def verify(&block)! if block_given? ! @verify_block=block! else! @verify_block! end! end! end! ! class Chef::Provider::File! alias_method :old_dcc, :do_contents_changes! def do_contents_changes! do_verify if !@new_resource.verify.nil?! old_dcc! end! ! def do_verify! Chef::Log.debug "Validating rendered file"! unless @new_resource.verify.call(tempfile.path)! raise Chef::Exceptions::ValidationFailed, "Could not validate rendered file"! end! end! end!
79. It Failed!! Success!
80. Is it safe?
81. Learnability Reprogramming Chef Can Hurt Reliability Robustness Maintainability
82. Powerful Reprogrammability! is Dangerous
83. If it wasn’t dangerous it wouldn’t be powerful enough
84. Thank You! Questions?
85. On Reprogrammable System Customizing Chef Chef Sugar Passwd_min Ohai Plugin http://stackoverflow.com/a/4471202/618304 Stack Overflow Answer https://stevendanna.github.io/blog/2013/04/13/passwd-min-ohai-plugin/ https://sethvargo.github.io/chef-sugar/ http://shop.oreilly.com/product/0636920032984.do http://www.confreaks.com/videos/374-rubyconf2010-the-polite-programmer-s-guide ruby-etiquette Talk by Jim Weirich http://martinfowler.com/bliki/InternalReprogrammability.html