Refactoring Puppet

How do you prevent Bit-rot within Puppet code as complexity grows? This presentation talks about some common code-smells within Puppet and how to apply common programming principles to Infrastructure as Code

View this presentation recorded at https://www.youtube.com/watch?feature=player_embedded&v=foiF3wd9Kn4

Transcript

1. Refactoring Puppet 1
2. HAS THIS HAPPENED TO YOU? 2
4. Rockin’ It! 3
7. WIDGET 3000! 5
8. WIDGET 3000! Amazing! 5
9. WIDGET 3000! Features! Amazing! 5
10. WIDGET 3000! Features! Social! Amazing! 5
11. WIDGET 3000! Features! Social! Scale! Amazing! 5
16. err: Could not apply complete catalog: Found 1 dependency cycle: (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 9
17. err: Could not apply complete catalog: Found 1 dependency cycle: (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 9
19. err: Could not apply complete catalog: Found 1 dependency cycle: (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 11
20. err: Could not apply complete catalog: Found 1 dependency cycle: (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 11
23. James Fryman @jfryman 13
25. Why Refactor? 15
26. Why Fix your car? 16
27. Why Go To The Doctor? 17
28. Why Drink Beer!? 18
29. Why Refactor? 19
30. Why Refactor? 19
31. Why Refactor? Improve Readability 19
32. Why Refactor? Improve Readability Reduce Bugs 19
33. Why Refactor? Improve Readability Reduce Bugs Improve Velocity 19
34. Why Refactor? Improve Readability Reduce Bugs Improve Velocity 19
35. JUST DO IT 20
37. Code Reuse 21
38. Code Reuse Isolate Change 21
39. Iteration 22
41. SHIP IT 24
44. Abstract 26
45. Abstract Refactor as you go 26
46. Abstract Refactor as you go Refactor Refactors 26
48. Level 1: Hand Edited 28
49. Level 2: Central Location 29
50. Level 3: Configuration Management 30
51. Level 4: Templates 31
52. Level 5: Data Driven 32
53. Tools 33
54. rspec- puppet 34
55. @rodjek 35
56. I TESTS HATE LOVE I 36
57. $ puppet module \ generate jfryman-octokittens Generating module at .. jfryman-octokittens/spec/spec_helper.rb jfryman-octokittens/tests/init.pp 37
58. $ puppet module \ generate jfryman-octokittens Generating module at .. jfryman-octokittens/spec/spec_helper.rb jfryman-octokittens/tests/init.pp Tests Here 37
59. describe 'logrotate::rule' do let(:title) { 'nginx' } it { should include_class('logrotate::setup') } it do should contain_file('/etc/logrotate.d/ nginx').with({ 'ensure' => 'present', 'owner' => 'root', 'group' => 'root', 'mode' => '0444', }) end end 38
60. Write Negative Tests 39
61. describe 'logrotate::rule' do let(:title) { 'nginx' } let(:parameters) {{ :enabled => false }} it { should_not include_class('logrotate::setup') it do should contain_file('/etc/logrotate.d/ nginx').with({ 'ensure' => 'absent', }) end end 40
62. Test Flow Control 41
63. if $::datacenter =~ /ec2/ { $apt_repository = ‘https://hi:mom@foo.com/’ } else { $apt_repository = ‘http://foo.local/’ } file { ‘/etc/apt/sources.list.d/100basho.list’: content => “deb $apt_repository $::lsbdistcodename main”, } 42
64. internal_package_server = ‘https://hi:mom@foo.com/’ external_package_server = ‘http://foo.internal/’ context 'on internal nodes' do it 'should contain basho package repo' do should contain_file('/etc/apt/sources.list.d/ 100basho.list').with_content(internal_package_server) end end context 'on external nodes' do it 'should contain basho package repo' do should contain_file('/etc/apt/sources.list.d/ 100basho.list').with_content(external_package_server) end end 43
65. vagrant 44
66. @mitchellh 45
69. Code Smells 47
70. DRY 48
71. Extract Move 49
72. java -jar simian-2.3.33.jar \ -threshold=2 \ ~/puppet/**/*.erb \ ~/puppet/**/*.rb \ ~/puppet/**/*.pp Found 28542 duplicate lines in 7696 blocks in 1340 files 50
73. Common 51
74. Common 51
75. Common 51
76. Common Duplicated Logic 51
77. Common Duplicated Logic Duplicated Config 51
78. Common Duplicated Logic Duplicated Config Duplicated Tests 51
79. $monitor = $rails_env ? { 'production' => true, 'staging' => true, default => false } # module/lib/facter/should_monitor.rb ... case Facter.value(‘rails_env’) when ‘production’, ‘staging’ true else false end ... Extract 52
80. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content => template(‘foo/foo.conf.erb’), monitor => should_monitor(), } } if should_monitor() { ... } Extract 53
81. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content => template(‘foo/foo.conf.erb’), monitor => should_monitor(), } } if should_monitor() { ... } THAT’S READABLE WAT?! Extract 53
82. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content => template(‘foo/foo.conf.erb’), monitor => should_monitor(), } } if $rnx_prb_alpha() { ... } Extract 54
83. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content => template(‘foo/foo.conf.erb’), monitor => should_monitor(), } } if $rnx_prb_alpha() { ... } Extract 54
84. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content => template(‘foo/foo.conf.erb’), monitor => $::should_monitor, } } if $::should_monitor { ... } 55
85. DRY Partials with Templates 56
86. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content => template(‘foo/foo.conf.erb’), } } # foo/templates/foo.conf.erb <% if @ssl -%> <%= scope.function_template(‘foo/ssl.conf.erb’) %> <% end -%> 57
87. MVC Move 58
88. Move 59
89. class ntp ( $options = $ntp::params::defaults, ) inherits ntp::params { class { 'ntp::package': options => $options, } -> class { 'ntp::config': options => $options, } ~> class { 'ntp::service': options => $options, } -> Class[‘ntp’] } Move 60
90. class ntp::params { $defaults = { package => { version => ‘latest’, }, config => { servers => [‘pool.ntp.org’], }, } } Move 61
91. class ntp::params { $defaults = { package => { version => hiera(‘ntp_package_version’), }, config => { servers => hiera(‘ntp_servers’), }, } } Move 62
92. Modularize 63
93. DRY Long Classes 64
94. require => Class[‘ntp’] require => File[‘/etc/ntp.conf’] Move Extract 65
95. err: Could not apply complete catalog: Found 1 dependency cycle: (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 66
96. err: Could not apply complete catalog: Found 1 dependency cycle: (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 66
97. Manifest Task Purpose 67
98. Organize By Function Move Extract 68
99. Organize By Function Move Extract 68
100. Organize By Function Move Extract 68
101. Share 69
102. Move Move Extract exec { ‘wget awesome file’: command => ‘wget -O /tmp/file.txt http://git.io/ YHs6eg’, creates => ‘/tmp/file.txt’, } Move 70
103. Move Move Extract # modules/wget/manifests/download.pp define wget::download ( $source = undef, $dest = $name, ) include wget exec { “wget download ${name}”: command => “wget -O ${dest} ${source}” creates => $dest, } } Move 71
104. Move Move Extract Move Move Move Extract wget::download { ‘/tmp/file.txt’: source => ‘http://git.io/YHs6eg’, } Move 72
105. Modularize Move 73
108. Move frymanet.com mysql nginx rails ruby common admin package repos 76
109. Wrap Up 77
110. Why Refactor? 78
112. Improve Readability 79
113. Improve Readability Reduce Bugs 79
114. Improve Readability Reduce Bugs Improve Velocity 79
115. Improve Readability Reduce Bugs Improve Velocity 79
116. Level Up 80
117. @jfryman jamesfryman/freenode fryman@github.com 81
118. fin 82