Save 37% off PRO during our Black Friday Sale! »

Cookbook Refactoring and Extracting Logic into Rubygems

Cookbook Refactoring and Extracting Logic into Rubygems

Slides from my ChefConf 2013

502828deee7e3b38ca1e527dded8a1a9?s=128

Seth Vargo

April 25, 2013
Tweet

Transcript

  1. Cookbook Refactoring A

  2. Cookbook Refactoring ... and extracting logic into Rubygems A

  3. sethvargo@opscode.com E b yz

  4. None
  5. We're Hiring!

  6. We're Hiring! Colorado

  7. New Branding We're Hiring!

  8. U DO YOU SOMETIMES FEEL LIKE THIS

  9. None
  10. None
  11. template '/etc/hosts' do owner 'root' group 'root' source 'etc/hosts' end

    recipes/default.rb
  12. # This file is managed by Chef for "<%= node['fqdn']

    %>" # Do NOT modify this file by hand. <%= node['ipaddress'] %> <%= node['fqdn'] %> 127.0.0.1! localhost <%= node['fqdn'] %> 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost templates/default/etc/hosts.erb
  13. None
  14. default['etc']['hosts'] = [] unless node['etc']['hosts'] attributes/default.rb

  15. # This file is managed by Chef for "<%= node['fqdn']

    %>" # Do NOT modify this file by hand. <%= node['ipaddress'] %> <%= node['fqdn'] %> 127.0.0.1! localhost <%= node['fqdn'] %> 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost # Custom Entries <% node['etc']['hosts'].each do |h| -%> <%= h['ip'] %> <%= h['host'] %> <% end -%> templates/default/etc/hosts.erb
  16. include_attribute 'hostsfile' default['etc']['hosts'] << { 'ip' => '1.2.3.4', 'host' =>

    'www.example.com' } other_cookbook/attributes/default.rb
  17. node.default['etc']['hosts'] << { 'ip' => '1.2.3.4', 'host' => 'www.example.com' }

    other_cookbook/recipes/default.rb
  18. default_attributes({ 'etc' => { 'hosts' => [ {'ip' => '1.2.3.4',

    'host' => 'www.example.com'}, {'ip' => '4.5.6.7', 'host' => 'foo.example.com'} ] } }) roles/my_role.rb
  19. { "default_attributes": { "etc": { "hosts": [ {"ip": "1.2.3.4", "host":

    "www.example.com"}, {"ip": "4.5.6.7", "host": "foo.example.com"} ] } } } environments/production.json
  20. None
  21. node.set['etc']['hosts'] = { ip: '7.8.9.0', host: 'bar.example.com' }) recipes/default.rb

  22. None
  23. arr = [1,2,3] arr << 4 => [1,2,3,4] arr =

    4 => 4
  24. arr = [1,2,3] arr << 4 => [1,2,3,4] arr =

    4 => 4 Not an Array
  25. TODO: Add infographics # This file is managed by Chef

    for "www.myapp.com" # Do NOT modify this file by hand. 1.2.3.4 www.myapp.com 127.0.0.1! localhost www.myapp.com 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost # Custom Entries 1.2.3.4 www.example.com 4.5.6.7 foo.example.com 7.8.9.0 bar.example.com /etc/hosts
  26. TODO: Add infographics # This file is managed by Chef

    for "www.myapp.com" # Do NOT modify this file by hand. 1.2.3.4 www.myapp.com 127.0.0.1! localhost www.myapp.com 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost # Custom Entries 7.8.9.0 bar.example.com /etc/hosts
  27. Post Mortem

  28. None
  29. None
  30. << =

  31. << = !=

  32. Post Mortem Action Items 7

  33. Monkey patch Chef to raise an exception when redefining that

    particular node attribute.
  34. Monkey patch Chef to raise an exception when redefining that

    particular node attribute. t
  35. Create a special cookbook that uses a threshold value and

    raises an exception if the size of the array doesn't "make sense".
  36. Create a special cookbook that uses a threshold value and

    raises an exception if the size of the array doesn't "make sense". t
  37. Move all entries to a data bag

  38. Move all entries to a data bag u

  39. Move all entries to a data bag 6 6 Add

    tests
  40. Data Bags

  41. [ "1.2.3.4 example.com www.example.com", "4.5.6.7 foo.example.com", "7.8.9.0 bar.example.com" ] data_bags/etc_hosts.json

  42. hosts = data_bag('etc_hosts') template '/etc/hosts' do owner 'root' group 'root'

    source 'etc/hosts' variables( hosts: hosts ) end recipes/default.rb
  43. # This file is managed by Chef for "<%= node['fqdn']

    %>" # Do NOT modify this file by hand. <%= node['ipaddress'] %> <%= node['fqdn'] %> 127.0.0.1! localhost <%= node['fqdn'] %> 255.255.255.255!broadcasthost ::1 localhost fe80::1%lo0! localhost # Custom Entries <%= @hosts.join("\n") %> templates/default/etc/hosts.erb
  44. Move all entries to a data bag 5 6 Add

    tests
  45. require 'chefspec' spec/default_spec.rb

  46. require 'chefspec' describe 'hostsfile::default' do end spec/default_spec.rb

  47. require 'chefspec' describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7

    bar.com'] } before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end end spec/default_spec.rb
  48. require 'chefspec' describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7

    bar.com'] } before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end let(:runner) { ChefSpec::ChefRunner.new.converge('hostsfile::default') } end spec/default_spec.rb
  49. require 'chefspec' describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7

    bar.com'] } before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end let(:runner) { ChefSpec::ChefRunner.new.converge('hostsfile::default') } it 'loads the data bag' do Chef::Recipe.any_instance.should_receive(:data_bag).with('etc_hosts') end end spec/default_spec.rb
  50. require 'chefspec' describe 'hostsfile::default' do let(:hosts) { ['1.2.3.4 example.com', '4.5.6.7

    bar.com'] } before do Chef::Recipe.any_instance.stub(:data_bag).with('etc_hosts').and_return(hosts) end let(:runner) { ChefSpec::ChefRunner.new.converge('hostsfile::default') } it 'loads the data bag' do Chef::Recipe.any_instance.should_receive(:data_bag).with('etc_hosts') end it 'creates the /etc/hosts template' do expect(runner).to create_template('/etc/hosts').with_content(hosts.join("\n")) end end spec/default_spec.rb
  51. $ rspec cookbooks/hostsfile Running all specs

  52. $ rspec cookbooks/hostsfile Running all specs ** Finished in 0.0003

    seconds 2 examples, 0 failures
  53. $ rspec cookbooks/hostsfile Running all specs ** Finished in 0.0003

    seconds 2 examples, 0 failures Really Fucking Fast™
  54. #winning

  55. 10,000 tests

  56. 28 seconds

  57. #winning

  58. ⏳ ⏳

  59. None
  60. None
  61. hosts = data_bag('etc_hosts') hosts << search(:node, 'role:mongo_master').first.tap do |n| "#{n['ip_address']}

    #{n['fqdn']}" end template '/etc/hosts' do owner 'root' group 'root' source 'etc/hosts' variables( hosts: hosts ) end recipes/default.rb
  62. hosts = data_bag('etc_hosts') hosts << search(:node, 'role:mongo_master').first.tap do |n| "#{n['ip_address']}

    #{n['fqdn']}" end hosts << search(:node, 'role:mysql_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}" end hosts << search(:node, 'role:redis_master').first.tap do |n| "#{n['ip_address']} #{n['fqdn']}" end template '/etc/hosts' do owner 'root' group 'root' recipes/default.rb
  63. LWRPs

  64. # List of all actions supported by the provider actions

    :create, :create_if_missing, :update, :remove # Make create the default action default_action :create # Required attributes attribute :ip_address, kind_of: String, name_attribute: true, required: true attribute :hostname, kind_of: String # Optional attributes attribute :aliases, kind_of: Array attribute :comment, kind_of: String resources/entry.rb
  65. action :create do ::Chef::Util::FileEdit.search_file_delete_line(entry) ::Chef::Util::FileEdit.insert_line_after_match(/\n/, entry) end protected def entry

    [new_resource.ip_address, new_resource.hostname, new_resource.aliases.join(' ')].compact.join(' ').squeeze(' ') end providers/entry.rb
  66. hostsfile_entry '1.2.3.4' do hostname 'example.com' end providers/entry.rb

  67. Chef::Util::FileEdit is slow

  68. Re-writing the file on each run

  69. Provider kept growning

  70. Untested

  71. Refactor A

  72. Move to pure Ruby classes

  73. Ditch Chef::Util::FileEdit and manage the entire file

  74. Only implement Ruby classes in the Provider (logic-less Provider)

  75. Test the Ruby code

  76. Test that the Provider implements the proper Ruby classes

  77. TODO: Add infographics class Entry attr_accessor :ip_address, :hostname, :aliases, :comment

    def initialize(options = {}) if options[:ip_address].nil? || options[:hostname].nil? raise ':ip_address and :hostname are both required options' end @ip_address = options[:ip_address] @hostname = options[:hostname] @aliases = [options[:aliases]].flatten @comment = options[:comment] end # ... end libraries/entry.rb
  78. TODO: Add infographics class Manipulator def initialize contents = ::File.readlines(hostsfile_path)

    @entries = contents.collect do |line| Entry.parse(line) unless line.strip.nil? || line.strip.empty? end.compact end def add(options = {}) @entries << Entry.new( ip_address: options[:ip_address], hostname: options[:hostname], aliases: options[:aliases], comment: options[:comment] ) end end libraries/manipulator.rb
  79. # Creates a new hosts file entry. If an entry

    already exists, it # will be overwritten by this one. action :create do hostsfile.add( ip_address: new_resource.ip_address, hostname: new_resource.hostname, aliases: new_resource.aliases, comment: new_resource.comment ) new_resource.updated_by_last_action(true) if hostsfile.save end providers/entry.rb
  80. None
  81. None
  82. RSpec

  83. TODO: Add infographics describe Entry do describe '.initialize' do subject

    { Entry.new(ip_address: '2.3.4.5', hostname: 'www.example.com', aliases: ['foo', 'bar'], comment: 'This is a comment!', priority: 100) } it 'raises an exception if :ip_address is missing' do expect { Entry.new(hostname: 'www.example.com') }.to raise_error(ArgumentError) end it 'sets the ip_address' do expect(subject.ip_address).to eq('2.3.4.5') end end spec/entry_spec.rb
  84. None
  85. Chef Spec

  86. Chef Spec

  87. TODO: Add infographics describe 'hostsfile lwrp' do let(:manipulator) { double('manipulator')

    } before do Manipulator.stub(:new).and_return(manipulator) Manipulator.should_receive(:new).with(kind_of(Chef::Node)) .and_return(manipulator) manipulator.should_receive(:save!) end let(:chef_run) { ChefSpec::ChefRunner.new( cookbook_path: $cookbook_paths, step_into: ['hostsfile_entry'] ) } spec/default_spec.rb
  88. TODO: Add infographics context 'actions' do describe ':create' do it

    'adds the entry' do manipulator.should_receive(:add).with({ ip_address: '2.3.4.5', hostname: 'www.example.com', aliases: nil, comment: nil, priority: nil }) chef_run.converge('fake::create') end end end end
  89. Open It

  90. None
  91. None
  92. None
  93. None
  94. None
  95. Gem It

  96. $ bundle gem hostsfile

  97. $ bundle gem hostsfile create hostsfile/Gemfile create hostsfile/Rakefile create hostsfile/LICENSE.txt

    create hostsfile/README.md create hostsfile/.gitignore create hostsfile/hostsfile.gemspec create hostsfile/lib/hostsfile.rb create hostsfile/lib/hostsfile/version.rb Initializating git repo in ~Development/hostsfile
  98. entry.rb manipulator.rb 9 9

  99. 9

  100. 9 ?

  101. chef_gem 'hostsfile' recipes/default.rb

  102. require 'hostsfile' providers/entry.rb

  103. In another cookbook...

  104. # ... depends 'hostsfile' other_cookbook/metadata.rb

  105. { "run_list": [ "recipe[hostsfile]" ] } www.myapp.com (Chef Node)

  106. None
  107. None
  108. None
  109. Thank You z