Cookbook Refactoring and Extracting Logic into Rubygems

Slides from my ChefConf 2013

Transcript

1. Cookbook Refactoring A
2. Cookbook Refactoring ... and extracting logic into Rubygems A
3. sethvargo@opscode.com E b yz
5. We're Hiring!
6. We're Hiring! Colorado
7. New Branding We're Hiring!
8. U DO YOU SOMETIMES FEEL LIKE THIS
11. template '/etc/hosts' do owner 'root' group 'root' source 'etc/hosts' end recipes/default.rb
12. # This file is managed by Chef for "<%= node['fqdn'] %>" # Do NOT modify this file by hand. <%= node['ipaddress'] %> <%= node['fqdn'] %> 127.0.0.1! localhost <%= node['fqdn'] %> 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost templates/default/etc/hosts.erb
14. default['etc']['hosts'] = [] unless node['etc']['hosts'] attributes/default.rb
15. # This file is managed by Chef for "<%= node['fqdn'] %>" # Do NOT modify this file by hand. <%= node['ipaddress'] %> <%= node['fqdn'] %> 127.0.0.1! localhost <%= node['fqdn'] %> 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost # Custom Entries <% node['etc']['hosts'].each do |h| -%> <%= h['ip'] %> <%= h['host'] %> <% end -%> templates/default/etc/hosts.erb
16. include_attribute 'hostsfile' default['etc']['hosts'] << { 'ip' => '1.2.3.4', 'host' => 'www.example.com' } other_cookbook/attributes/default.rb
17. node.default['etc']['hosts'] << { 'ip' => '1.2.3.4', 'host' => 'www.example.com' } other_cookbook/recipes/default.rb
18. default_attributes({ 'etc' => { 'hosts' => [ {'ip' => '1.2.3.4', 'host' => 'www.example.com'}, {'ip' => '4.5.6.7', 'host' => 'foo.example.com'} ] } }) roles/my_role.rb
19. { "default_attributes": { "etc": { "hosts": [ {"ip": "1.2.3.4", "host": "www.example.com"}, {"ip": "4.5.6.7", "host": "foo.example.com"} ] } } } environments/production.json
21. node.set['etc']['hosts'] = { ip: '7.8.9.0', host: 'bar.example.com' }) recipes/default.rb
23. arr = [1,2,3] arr << 4 => [1,2,3,4] arr = 4 => 4
24. arr = [1,2,3] arr << 4 => [1,2,3,4] arr = 4 => 4 Not an Array
25. TODO: Add infographics # This file is managed by Chef for "www.myapp.com" # Do NOT modify this file by hand. 1.2.3.4 www.myapp.com 127.0.0.1! localhost www.myapp.com 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost # Custom Entries 1.2.3.4 www.example.com 4.5.6.7 foo.example.com 7.8.9.0 bar.example.com /etc/hosts
26. TODO: Add infographics # This file is managed by Chef for "www.myapp.com" # Do NOT modify this file by hand. 1.2.3.4 www.myapp.com 127.0.0.1! localhost www.myapp.com 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost # Custom Entries 7.8.9.0 bar.example.com /etc/hosts
27. Post Mortem
30. << =
31. << = !=
32. Post Mortem Action Items 7
33. Monkey patch Chef to raise an exception when redefining that particular node attribute.
34. Monkey patch Chef to raise an exception when redefining that particular node attribute. t
35. Create a special cookbook that uses a threshold value and raises an exception if the size of the array doesn't "make sense".
36. Create a special cookbook that uses a threshold value and raises an exception if the size of the array doesn't "make sense". t
37. Move all entries to a data bag
38. Move all entries to a data bag u
39. Move all entries to a data bag 6 6 Add tests
40. Data Bags
41. [ "1.2.3.4 example.com www.example.com", "4.5.6.7 foo.example.com", "7.8.9.0 bar.example.com" ] data_bags/etc_hosts.json
42. hosts = data_bag('etc_hosts') template '/etc/hosts' do owner 'root' group 'root' source 'etc/hosts' variables( hosts: hosts ) end recipes/default.rb
43. # This file is managed by Chef for "<%= node['fqdn'] %>" # Do NOT modify this file by hand. <%= node['ipaddress'] %> <%= node['fqdn'] %> 127.0.0.1! localhost <%= node['fqdn'] %> 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost # Custom Entries <%= @hosts.join("\n") %> templates/default/etc/hosts.erb
44. Move all entries to a data bag 5 6 Add tests
45. require 'chefspec' spec/default_spec.rb
46. require 'chefspec' describe 'hostsfile::default' do end spec/default_spec.rb
47. require 'chefspec' describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7 bar.com'] } before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end end spec/default_spec.rb
48. require 'chefspec' describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7 bar.com'] } before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end let(:runner) { ChefSpec::ChefRunner.new.converge('hostsfile::default') } end spec/default_spec.rb
49. require 'chefspec' describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7 bar.com'] } before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end let(:runner) { ChefSpec::ChefRunner.new.converge('hostsfile::default') } it 'loads the data bag' do Chef::Recipe.any_instance.should_receive(:data_bag).with('etc_hosts') end end spec/default_spec.rb
50. require 'chefspec' describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7 bar.com'] } before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end let(:runner) { ChefSpec::ChefRunner.new.converge('hostsfile::default') } it 'loads the data bag' do Chef::Recipe.any_instance.should_receive(:data_bag).with('etc_hosts') end it 'creates the /etc/hosts template' do expect(runner).to create_template('/etc/hosts').with_content(hosts.join("\n")) end end spec/default_spec.rb
51. $ rspec cookbooks/hostsfile Running all specs
52. $ rspec cookbooks/hostsfile Running all specs ** Finished in 0.0003 seconds 2 examples, 0 failures
53. $ rspec cookbooks/hostsfile Running all specs ** Finished in 0.0003 seconds 2 examples, 0 failures Really Fucking Fast™
54. #winning
55. 10,000 tests
56. 28 seconds
57. #winning
58. ⏳ ⏳
61. hosts = data_bag('etc_hosts') hosts << search(:node, 'role:mongo_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}" end template '/etc/hosts' do owner 'root' group 'root' source 'etc/hosts' variables( hosts: hosts ) end recipes/default.rb
62. hosts = data_bag('etc_hosts') hosts << search(:node, 'role:mongo_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}" end hosts << search(:node, 'role:mysql_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}" end hosts << search(:node, 'role:redis_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}" end template '/etc/hosts' do owner 'root' group 'root' recipes/default.rb
63. LWRPs
64. # List of all actions supported by the provider actions :create, :create_if_missing, :update, :remove # Make create the default action default_action :create # Required attributes attribute :ip_address, kind_of: String, name_attribute: true, required: true attribute :hostname, kind_of: String # Optional attributes attribute :aliases, kind_of: Array attribute :comment, kind_of: String resources/entry.rb
65. action :create do ::Chef::Util::FileEdit.search_file_delete_line(entry) ::Chef::Util::FileEdit.insert_line_after_match(/\n/, entry) end protected def entry [new_resource.ip_address, new_resource.hostname, new_resource.aliases.join(' ')].compact.join(' ').squeeze(' ') end providers/entry.rb
66. hostsfile_entry '1.2.3.4' do hostname 'example.com' end providers/entry.rb
67. Chef::Util::FileEdit is slow
68. Re-writing the file on each run
69. Provider kept growning
70. Untested
71. Refactor A
72. Move to pure Ruby classes
73. Ditch Chef::Util::FileEdit and manage the entire file
74. Only implement Ruby classes in the Provider (logic-less Provider)
75. Test the Ruby code
76. Test that the Provider implements the proper Ruby classes
77. TODO: Add infographics class Entry attr_accessor :ip_address, :hostname, :aliases, :comment def initialize(options = {}) if options[:ip_address].nil? || options[:hostname].nil? raise ':ip_address and :hostname are both required options' end @ip_address = options[:ip_address] @hostname = options[:hostname] @aliases = [options[:aliases]].flatten @comment = options[:comment] end # ... end libraries/entry.rb
78. TODO: Add infographics class Manipulator def initialize contents = ::File.readlines(hostsfile_path) @entries = contents.collect do |line| Entry.parse(line) unless line.strip.nil? || line.strip.empty? end.compact end def add(options = {}) @entries << Entry.new( ip_address: options[:ip_address], hostname: options[:hostname], aliases: options[:aliases], comment: options[:comment] ) end end libraries/manipulator.rb
79. # Creates a new hosts file entry. If an entry already exists, it # will be overwritten by this one. action :create do hostsfile.add( ip_address: new_resource.ip_address, hostname: new_resource.hostname, aliases: new_resource.aliases, comment: new_resource.comment ) new_resource.updated_by_last_action(true) if hostsfile.save end providers/entry.rb
82. RSpec
83. TODO: Add infographics describe Entry do describe '.initialize' do subject { Entry.new(ip_address: '2.3.4.5', hostname: 'www.example.com', aliases: ['foo', 'bar'], comment: 'This is a comment!', priority: 100) } it 'raises an exception if :ip_address is missing' do expect { Entry.new(hostname: 'www.example.com') }.to raise_error(ArgumentError) end it 'sets the ip_address' do expect(subject.ip_address).to eq('2.3.4.5') end end spec/entry_spec.rb
85. Chef Spec
86. Chef Spec
87. TODO: Add infographics describe 'hostsfile lwrp' do let(:manipulator) { double('manipulator') } before do Manipulator.stub(:new).and_return(manipulator) Manipulator.should_receive(:new).with(kind_of(Chef::Node)) .and_return(manipulator) manipulator.should_receive(:save!) end let(:chef_run) { ChefSpec::ChefRunner.new( cookbook_path: $cookbook_paths, step_into: ['hostsfile_entry'] ) } spec/default_spec.rb
88. TODO: Add infographics context 'actions' do describe ':create' do it 'adds the entry' do manipulator.should_receive(:add).with({ ip_address: '2.3.4.5', hostname: 'www.example.com', aliases: nil, comment: nil, priority: nil }) chef_run.converge('fake::create') end end end end
89. Open It
95. Gem It
96. $ bundle gem hostsfile
97. $ bundle gem hostsfile create hostsfile/Gemfile create hostsfile/Rakefile create hostsfile/LICENSE.txt create hostsfile/README.md create hostsfile/.gitignore create hostsfile/hostsfile.gemspec create hostsfile/lib/hostsfile.rb create hostsfile/lib/hostsfile/version.rb Initializating git repo in ~Development/hostsfile
98. entry.rb manipulator.rb 9 9
100. 9 ?
101. chef_gem 'hostsfile' recipes/default.rb
102. require 'hostsfile' providers/entry.rb
103. In another cookbook...
104. # ... depends 'hostsfile' other_cookbook/metadata.rb
105. { "run_list": [ "recipe[hostsfile]" ] } www.myapp.com (Chef Node)
109. Thank You z