Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Refactoring Puppet

Refactoring Puppet

How do you prevent Bit-rot within Puppet code as complexity grows? This presentation talks about some common code-smells within Puppet and how to apply common programming principles to Infrastructure as Code

View this presentation recorded at https://www.youtube.com/watch?feature=player_embedded&v=foiF3wd9Kn4

James Fryman

April 05, 2013
Tweet

More Decks by James Fryman

Other Decks in Programming

Transcript

  1. Refactoring
    Puppet
    1

    View full-size slide

  2. HAS THIS
    HAPPENED
    TO YOU?
    2

    View full-size slide

  3. Rockin’
    It!
    3

    View full-size slide

  4. WIDGET
    3000!
    5

    View full-size slide

  5. WIDGET
    3000!
    Amazing!
    5

    View full-size slide

  6. WIDGET
    3000!
    Features!
    Amazing!
    5

    View full-size slide

  7. WIDGET
    3000!
    Features!
    Social!
    Amazing!
    5

    View full-size slide

  8. WIDGET
    3000!
    Features!
    Social!
    Scale!
    Amazing!
    5

    View full-size slide

  9. err: Could not apply complete catalog:
    Found 1 dependency cycle:
    (Exec[apt-update] =>
    Package[lvm2] =>
    Class[Lvm::Setup] =>
    Stage[stage3] => Stage[main] =>
    Class[Main] => Exec[apt-update])
    9

    View full-size slide

  10. err: Could not apply complete catalog:
    Found 1 dependency cycle:
    (Exec[apt-update] =>
    Package[lvm2] =>
    Class[Lvm::Setup] =>
    Stage[stage3] => Stage[main] =>
    Class[Main] => Exec[apt-update])
    9

    View full-size slide

  11. err: Could not apply complete catalog:
    Found 1 dependency cycle:
    (Exec[apt-update] =>
    Package[lvm2] =>
    Class[Lvm::Setup] =>
    Stage[stage3] => Stage[main] =>
    Class[Main] => Exec[apt-update])
    11

    View full-size slide

  12. err: Could not apply complete catalog:
    Found 1 dependency cycle:
    (Exec[apt-update] =>
    Package[lvm2] =>
    Class[Lvm::Setup] =>
    Stage[stage3] => Stage[main] =>
    Class[Main] => Exec[apt-update])
    11

    View full-size slide

  13. James Fryman
    @jfryman
    13

    View full-size slide

  14. Why
    Refactor?
    15

    View full-size slide

  15. Why
    Fix your car?
    16

    View full-size slide

  16. Why
    Go To The Doctor?
    17

    View full-size slide

  17. Why
    Drink Beer!?
    18

    View full-size slide

  18. Why
    Refactor?
    19

    View full-size slide

  19. Why
    Refactor?
    19

    View full-size slide

  20. Why
    Refactor?
    Improve Readability
    19

    View full-size slide

  21. Why
    Refactor?
    Improve Readability
    Reduce Bugs
    19

    View full-size slide

  22. Why
    Refactor?
    Improve Readability
    Reduce Bugs
    Improve Velocity
    19

    View full-size slide

  23. Why
    Refactor?
    Improve Readability
    Reduce Bugs
    Improve Velocity
    19

    View full-size slide

  24. Code Reuse
    21

    View full-size slide

  25. Code Reuse
    Isolate Change
    21

    View full-size slide

  26. Abstract
    Refactor as you go
    26

    View full-size slide

  27. Abstract
    Refactor as you go
    Refactor Refactors
    26

    View full-size slide

  28. Level 1:
    Hand Edited
    28

    View full-size slide

  29. Level 2:
    Central Location
    29

    View full-size slide

  30. Level 3:
    Configuration
    Management
    30

    View full-size slide

  31. Level 4:
    Templates
    31

    View full-size slide

  32. Level 5:
    Data Driven
    32

    View full-size slide

  33. rspec-
    puppet
    34

    View full-size slide

  34. I
    TESTS
    HATE LOVE
    I
    36

    View full-size slide

  35. $ puppet module \
    generate jfryman-octokittens
    Generating module at ..
    jfryman-octokittens/spec/spec_helper.rb
    jfryman-octokittens/tests/init.pp
    37

    View full-size slide

  36. $ puppet module \
    generate jfryman-octokittens
    Generating module at ..
    jfryman-octokittens/spec/spec_helper.rb
    jfryman-octokittens/tests/init.pp
    Tests Here
    37

    View full-size slide

  37. describe 'logrotate::rule' do
    let(:title) { 'nginx' }
    it { should include_class('logrotate::setup') }
    it do
    should contain_file('/etc/logrotate.d/
    nginx').with({
    'ensure' => 'present',
    'owner' => 'root',
    'group' => 'root',
    'mode' => '0444',
    })
    end
    end
    38

    View full-size slide

  38. Write
    Negative
    Tests
    39

    View full-size slide

  39. describe 'logrotate::rule' do
    let(:title) { 'nginx' }
    let(:parameters) {{ :enabled => false }}
    it { should_not include_class('logrotate::setup')
    it do
    should contain_file('/etc/logrotate.d/
    nginx').with({
    'ensure' => 'absent',
    })
    end
    end
    40

    View full-size slide

  40. Test
    Flow
    Control
    41

    View full-size slide

  41. if $::datacenter =~ /ec2/ {
    $apt_repository = ‘https://hi:[email protected]/’
    } else {
    $apt_repository = ‘http://foo.local/’
    }
    file { ‘/etc/apt/sources.list.d/100basho.list’:
    content => “deb $apt_repository $::lsbdistcodename
    main”,
    }
    42

    View full-size slide

  42. internal_package_server = ‘https://hi:[email protected]/’
    external_package_server = ‘http://foo.internal/’
    context 'on internal nodes' do
    it 'should contain basho package repo' do
    should contain_file('/etc/apt/sources.list.d/
    100basho.list').with_content(internal_package_server)
    end
    end
    context 'on external nodes' do
    it 'should contain basho package repo' do
    should contain_file('/etc/apt/sources.list.d/
    100basho.list').with_content(external_package_server)
    end
    end
    43

    View full-size slide

  43. @mitchellh
    45

    View full-size slide

  44. Code
    Smells
    47

    View full-size slide

  45. Extract
    Move
    49

    View full-size slide

  46. java -jar simian-2.3.33.jar \
    -threshold=2 \
    ~/puppet/**/*.erb \
    ~/puppet/**/*.rb \
    ~/puppet/**/*.pp
    Found 28542 duplicate lines
    in 7696 blocks in 1340 files
    50

    View full-size slide

  47. Common
    Duplicated Logic
    51

    View full-size slide

  48. Common
    Duplicated Logic
    Duplicated Config
    51

    View full-size slide

  49. Common
    Duplicated Logic
    Duplicated Config
    Duplicated Tests
    51

    View full-size slide

  50. $monitor = $rails_env ? {
    'production' => true,
    'staging' => true,
    default => false
    }
    # module/lib/facter/should_monitor.rb
    ...
    case Facter.value(‘rails_env’)
    when ‘production’, ‘staging’
    true
    else
    false
    end
    ...
    Extract
    52

    View full-size slide

  51. # foo/manifests/init.pp
    class foo($ssl=true) {
    file { “/etc/foo.conf”:
    content => template(‘foo/foo.conf.erb’),
    monitor => should_monitor(),
    }
    }
    if should_monitor() {
    ...
    }
    Extract
    53

    View full-size slide

  52. # foo/manifests/init.pp
    class foo($ssl=true) {
    file { “/etc/foo.conf”:
    content => template(‘foo/foo.conf.erb’),
    monitor => should_monitor(),
    }
    }
    if should_monitor() {
    ...
    }
    THAT’S
    READABLE
    WAT?!
    Extract
    53

    View full-size slide

  53. # foo/manifests/init.pp
    class foo($ssl=true) {
    file { “/etc/foo.conf”:
    content => template(‘foo/foo.conf.erb’),
    monitor => should_monitor(),
    }
    }
    if $rnx_prb_alpha() {
    ...
    }
    Extract
    54

    View full-size slide

  54. # foo/manifests/init.pp
    class foo($ssl=true) {
    file { “/etc/foo.conf”:
    content => template(‘foo/foo.conf.erb’),
    monitor => should_monitor(),
    }
    }
    if $rnx_prb_alpha() {
    ...
    }
    Extract
    54

    View full-size slide

  55. # foo/manifests/init.pp
    class foo($ssl=true) {
    file { “/etc/foo.conf”:
    content => template(‘foo/foo.conf.erb’),
    monitor => $::should_monitor,
    }
    }
    if $::should_monitor {
    ...
    }
    55

    View full-size slide

  56. DRY
    Partials with
    Templates
    56

    View full-size slide

  57. # foo/manifests/init.pp
    class foo($ssl=true) {
    file { “/etc/foo.conf”:
    content => template(‘foo/foo.conf.erb’),
    }
    }
    # foo/templates/foo.conf.erb
    <% if @ssl -%>
    <%= scope.function_template(‘foo/ssl.conf.erb’) %>
    <% end -%>
    57

    View full-size slide

  58. class ntp (
    $options = $ntp::params::defaults,
    ) inherits ntp::params {
    class { 'ntp::package':
    options => $options,
    } ->
    class { 'ntp::config':
    options => $options,
    } ~>
    class { 'ntp::service':
    options => $options,
    } ->
    Class[‘ntp’]
    }
    Move
    60

    View full-size slide

  59. class ntp::params {
    $defaults = {
    package => {
    version => ‘latest’,
    },
    config => {
    servers => [‘pool.ntp.org’],
    },
    }
    }
    Move
    61

    View full-size slide

  60. class ntp::params {
    $defaults = {
    package => {
    version => hiera(‘ntp_package_version’),
    },
    config => {
    servers => hiera(‘ntp_servers’),
    },
    }
    }
    Move
    62

    View full-size slide

  61. Modularize
    63

    View full-size slide

  62. DRY
    Long Classes
    64

    View full-size slide

  63. require => Class[‘ntp’]
    require => File[‘/etc/ntp.conf’]
    Move
    Extract
    65

    View full-size slide

  64. err: Could not apply complete catalog:
    Found 1 dependency cycle:
    (Exec[apt-update] =>
    Package[lvm2] =>
    Class[Lvm::Setup] =>
    Stage[stage3] => Stage[main] =>
    Class[Main] => Exec[apt-update])
    66

    View full-size slide

  65. err: Could not apply complete catalog:
    Found 1 dependency cycle:
    (Exec[apt-update] =>
    Package[lvm2] =>
    Class[Lvm::Setup] =>
    Stage[stage3] => Stage[main] =>
    Class[Main] => Exec[apt-update])
    66

    View full-size slide

  66. Manifest
    Task
    Purpose
    67

    View full-size slide

  67. Organize By
    Function
    Move
    Extract
    68

    View full-size slide

  68. Organize By
    Function
    Move
    Extract
    68

    View full-size slide

  69. Organize By
    Function
    Move
    Extract
    68

    View full-size slide

  70. Move
    Move
    Extract
    exec { ‘wget awesome file’:
    command => ‘wget -O /tmp/file.txt http://git.io/
    YHs6eg’,
    creates => ‘/tmp/file.txt’,
    }
    Move
    70

    View full-size slide

  71. Move
    Move
    Extract
    # modules/wget/manifests/download.pp
    define wget::download (
    $source = undef,
    $dest = $name,
    )
    include wget
    exec { “wget download ${name}”:
    command => “wget -O ${dest} ${source}”
    creates => $dest,
    }
    }
    Move
    71

    View full-size slide

  72. Move
    Move
    Extract
    Move
    Move
    Move
    Extract
    wget::download { ‘/tmp/file.txt’:
    source => ‘http://git.io/YHs6eg’,
    }
    Move
    72

    View full-size slide

  73. Modularize
    Move
    73

    View full-size slide

  74. Move
    frymanet.com
    mysql
    nginx rails
    ruby
    common admin
    package repos
    76

    View full-size slide

  75. Why
    Refactor?
    78

    View full-size slide

  76. Improve Readability
    79

    View full-size slide

  77. Improve Readability
    Reduce Bugs
    79

    View full-size slide

  78. Improve Readability
    Reduce Bugs
    Improve Velocity
    79

    View full-size slide

  79. Improve Readability
    Reduce Bugs
    Improve Velocity
    79

    View full-size slide

  80. @jfryman jamesfryman/freenode
    [email protected]
    81

    View full-size slide