Refactoring Puppet

Refactoring Puppet

How do you prevent Bit-rot within Puppet code as complexity grows? This presentation talks about some common code-smells within Puppet and how to apply common programming principles to Infrastructure as Code

View this presentation recorded at https://www.youtube.com/watch?feature=player_embedded&v=foiF3wd9Kn4

E1e1b442f49570070c40b735cf0688f4?s=128

James Fryman

April 05, 2013
Tweet

Transcript

  1. Refactoring Puppet 1

  2. HAS THIS HAPPENED TO YOU? 2

  3. 3

  4. Rockin’ It! 3

  5. 4

  6. 5

  7. WIDGET 3000! 5

  8. WIDGET 3000! Amazing! 5

  9. WIDGET 3000! Features! Amazing! 5

  10. WIDGET 3000! Features! Social! Amazing! 5

  11. WIDGET 3000! Features! Social! Scale! Amazing! 5

  12. 6

  13. 7

  14. 7

  15. 8

  16. err: Could not apply complete catalog: Found 1 dependency cycle:

    (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 9
  17. err: Could not apply complete catalog: Found 1 dependency cycle:

    (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 9
  18. 10

  19. err: Could not apply complete catalog: Found 1 dependency cycle:

    (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 11
  20. err: Could not apply complete catalog: Found 1 dependency cycle:

    (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 11
  21. 12

  22. 12

  23. James Fryman @jfryman 13

  24. 14

  25. Why Refactor? 15

  26. Why Fix your car? 16

  27. Why Go To The Doctor? 17

  28. Why Drink Beer!? 18

  29. Why Refactor? 19

  30. Why Refactor? 19

  31. Why Refactor? Improve Readability 19

  32. Why Refactor? Improve Readability Reduce Bugs 19

  33. Why Refactor? Improve Readability Reduce Bugs Improve Velocity 19

  34. Why Refactor? Improve Readability Reduce Bugs Improve Velocity 19

  35. JUST DO IT 20

  36. 21

  37. Code Reuse 21

  38. Code Reuse Isolate Change 21

  39. Iteration 22

  40. 23

  41. SHIP IT 24

  42. 25

  43. 26

  44. Abstract 26

  45. Abstract Refactor as you go 26

  46. Abstract Refactor as you go Refactor Refactors 26

  47. 27

  48. Level 1: Hand Edited 28

  49. Level 2: Central Location 29

  50. Level 3: Configuration Management 30

  51. Level 4: Templates 31

  52. Level 5: Data Driven 32

  53. Tools 33

  54. rspec- puppet 34

  55. @rodjek 35

  56. I TESTS HATE LOVE I 36

  57. $ puppet module \ generate jfryman-octokittens Generating module at ..

    jfryman-octokittens/spec/spec_helper.rb jfryman-octokittens/tests/init.pp 37
  58. $ puppet module \ generate jfryman-octokittens Generating module at ..

    jfryman-octokittens/spec/spec_helper.rb jfryman-octokittens/tests/init.pp Tests Here 37
  59. describe 'logrotate::rule' do let(:title) { 'nginx' } it { should

    include_class('logrotate::setup') } it do should contain_file('/etc/logrotate.d/ nginx').with({ 'ensure' => 'present', 'owner' => 'root', 'group' => 'root', 'mode' => '0444', }) end end 38
  60. Write Negative Tests 39

  61. describe 'logrotate::rule' do let(:title) { 'nginx' } let(:parameters) {{ :enabled

    => false }} it { should_not include_class('logrotate::setup') it do should contain_file('/etc/logrotate.d/ nginx').with({ 'ensure' => 'absent', }) end end 40
  62. Test Flow Control 41

  63. if $::datacenter =~ /ec2/ { $apt_repository = ‘https://hi:mom@foo.com/’ } else

    { $apt_repository = ‘http://foo.local/’ } file { ‘/etc/apt/sources.list.d/100basho.list’: content => “deb $apt_repository $::lsbdistcodename main”, } 42
  64. internal_package_server = ‘https://hi:mom@foo.com/’ external_package_server = ‘http://foo.internal/’ context 'on internal nodes'

    do it 'should contain basho package repo' do should contain_file('/etc/apt/sources.list.d/ 100basho.list').with_content(internal_package_server) end end context 'on external nodes' do it 'should contain basho package repo' do should contain_file('/etc/apt/sources.list.d/ 100basho.list').with_content(external_package_server) end end 43
  65. vagrant 44

  66. @mitchellh 45

  67. 46

  68. 46

  69. Code Smells 47

  70. DRY 48

  71. Extract Move 49

  72. java -jar simian-2.3.33.jar \ -threshold=2 \ ~/puppet/**/*.erb \ ~/puppet/**/*.rb \

    ~/puppet/**/*.pp Found 28542 duplicate lines in 7696 blocks in 1340 files 50
  73. Common 51

  74. Common 51

  75. Common 51

  76. Common Duplicated Logic 51

  77. Common Duplicated Logic Duplicated Config 51

  78. Common Duplicated Logic Duplicated Config Duplicated Tests 51

  79. $monitor = $rails_env ? { 'production' => true, 'staging' =>

    true, default => false } # module/lib/facter/should_monitor.rb ... case Facter.value(‘rails_env’) when ‘production’, ‘staging’ true else false end ... Extract 52
  80. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content =>

    template(‘foo/foo.conf.erb’), monitor => should_monitor(), } } if should_monitor() { ... } Extract 53
  81. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content =>

    template(‘foo/foo.conf.erb’), monitor => should_monitor(), } } if should_monitor() { ... } THAT’S READABLE WAT?! Extract 53
  82. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content =>

    template(‘foo/foo.conf.erb’), monitor => should_monitor(), } } if $rnx_prb_alpha() { ... } Extract 54
  83. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content =>

    template(‘foo/foo.conf.erb’), monitor => should_monitor(), } } if $rnx_prb_alpha() { ... } Extract 54
  84. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content =>

    template(‘foo/foo.conf.erb’), monitor => $::should_monitor, } } if $::should_monitor { ... } 55
  85. DRY Partials with Templates 56

  86. # foo/manifests/init.pp class foo($ssl=true) { file { “/etc/foo.conf”: content =>

    template(‘foo/foo.conf.erb’), } } # foo/templates/foo.conf.erb <% if @ssl -%> <%= scope.function_template(‘foo/ssl.conf.erb’) %> <% end -%> 57
  87. MVC Move 58

  88. Move 59

  89. class ntp ( $options = $ntp::params::defaults, ) inherits ntp::params {

    class { 'ntp::package': options => $options, } -> class { 'ntp::config': options => $options, } ~> class { 'ntp::service': options => $options, } -> Class[‘ntp’] } Move 60
  90. class ntp::params { $defaults = { package => { version

    => ‘latest’, }, config => { servers => [‘pool.ntp.org’], }, } } Move 61
  91. class ntp::params { $defaults = { package => { version

    => hiera(‘ntp_package_version’), }, config => { servers => hiera(‘ntp_servers’), }, } } Move 62
  92. Modularize 63

  93. DRY Long Classes 64

  94. require => Class[‘ntp’] require => File[‘/etc/ntp.conf’] Move Extract 65

  95. err: Could not apply complete catalog: Found 1 dependency cycle:

    (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 66
  96. err: Could not apply complete catalog: Found 1 dependency cycle:

    (Exec[apt-update] => Package[lvm2] => Class[Lvm::Setup] => Stage[stage3] => Stage[main] => Class[Main] => Exec[apt-update]) 66
  97. Manifest Task Purpose 67

  98. Organize By Function Move Extract 68

  99. Organize By Function Move Extract 68

  100. Organize By Function Move Extract 68

  101. Share 69

  102. Move Move Extract exec { ‘wget awesome file’: command =>

    ‘wget -O /tmp/file.txt http://git.io/ YHs6eg’, creates => ‘/tmp/file.txt’, } Move 70
  103. Move Move Extract # modules/wget/manifests/download.pp define wget::download ( $source =

    undef, $dest = $name, ) include wget exec { “wget download ${name}”: command => “wget -O ${dest} ${source}” creates => $dest, } } Move 71
  104. Move Move Extract Move Move Move Extract wget::download { ‘/tmp/file.txt’:

    source => ‘http://git.io/YHs6eg’, } Move 72
  105. Modularize Move 73

  106. 74

  107. 75

  108. Move frymanet.com mysql nginx rails ruby common admin package repos

    76
  109. Wrap Up 77

  110. Why Refactor? 78

  111. 79

  112. Improve Readability 79

  113. Improve Readability Reduce Bugs 79

  114. Improve Readability Reduce Bugs Improve Velocity 79

  115. Improve Readability Reduce Bugs Improve Velocity 79

  116. Level Up 80

  117. @jfryman jamesfryman/freenode fryman@github.com 81

  118. fin 82