Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DevOps for the Rubyist Soul at RailsConf 2013

DevOps for the Rubyist Soul at RailsConf 2013

John Downey

May 01, 2013
Tweet

More Decks by John Downey

Other Decks in Programming

Transcript

  1. DEVOPS FOR THE
    RUBYIST SOUL
    John Downey | @jtdowney

    View Slide

  2. The authenticity of host 'apollo.local
    (10.0.2.56)' can't be established.
    RSA key fingerprint is
    04:63:c1:ba:c7:31:04:12:14:ff:b6:c4:32:cf:44:ec.
    Are you sure you want to continue connecting
    (yes/no)?

    View Slide

  3. John Downey | @jtdowney

    View Slide

  4. View Slide

  5. http://www.flickr.com/photos/jackol/3291891191/
    Highly
    Available

    View Slide

  6. View Slide

  7. Automated
    Repeatable
    Verifiable

    View Slide

  8. View Slide

  9. BRAINTREE PATH TO DEVOPS

    View Slide

  10. View Slide

  11. http://www.flickr.com/photos/orcmid/3880056930/
    Agile
    Process

    View Slide

  12. Pairing
    http://www.flickr.com/photos/damienpollet/5048830734/

    View Slide

  13. View Slide

  14. http://www.flickr.com/photos/us-pacific-command/7072151539/
    Guerrilla
    DevOps

    View Slide

  15. MY PATH TO DEVOPS

    View Slide

  16. I <3 LINUX

    View Slide

  17. View Slide

  18. Blue team!

    View Slide

  19. You’re a DevOps engineer, Harry

    View Slide

  20. cap qa puppet:noop

    View Slide

  21. http://www.flickr.com/photos/sp3ccylad/462995220/
    Puppet

    View Slide

  22. console
    $ gem install puppet

    View Slide

  23. http://www.flickr.com/photos/fortwainwright/7217500612/
    Puppet
    Master

    View Slide

  24. manifests/site.pp
    node "app01.qa" {
    package { "ruby":
    ensure => installed
    }
    $blah = "test"
    file { "/etc/foo/bar.conf":
    ensure => file,
    content => template("foo/bar.conf.erb"),
    owner => "root",
    group => "root",
    mode => "0644"
    }
    }

    View Slide

  25. manifests/site.pp
    node "app01.qa" {
    package { "ruby":
    ensure => installed
    }
    $blah = "test"
    file { "/etc/foo/bar.conf":
    ensure => file,
    content => template("foo/bar.conf.erb"),
    owner => "root",
    group => "root",
    mode => "0644"
    }
    }

    View Slide

  26. manifests/site.pp
    node "app01.qa" {
    package { "ruby":
    ensure => installed
    }
    $blah = "test"
    file { "/etc/foo/bar.conf":
    ensure => file,
    content => template("foo/bar.conf.erb"),
    owner => "root",
    group => "root",
    mode => "0644"
    }
    }

    View Slide

  27. manifests/site.pp
    node "app01.qa" {
    package { "ruby":
    ensure => installed
    }
    $blah = "test"
    file { "/etc/foo/bar.conf":
    ensure => file,
    content => template("foo/bar.conf.erb"),
    owner => "root",
    group => "root",
    mode => "0644"
    }
    }

    View Slide

  28. manifests/site.pp
    node "app01.qa" {
    package { "ruby":
    ensure => installed
    }
    $blah = "test"
    file { "/etc/foo/bar.conf":
    ensure => file,
    content => template("foo/bar.conf.erb"),
    owner => "root",
    group => "root",
    mode => "0644"
    }
    }

    View Slide

  29. manifests/site.pp
    node "app01.qa" {
    package { "ruby":
    ensure => installed
    }
    $blah = "test"
    file { "/etc/foo/bar.conf":
    ensure => file,
    content => template("foo/bar.conf.erb"),
    owner => "root",
    group => "root",
    mode => "0644"
    }
    }

    View Slide

  30. templates/foo/bar.conf.erb
    [test]
    blah = <%= blah %>
    <%- 1.upto(6) do |i| -%>
    foo<%= i %> = <%= i %>
    <%- end -%>

    View Slide

  31. templates/foo/bar.conf.erb
    [test]
    blah = <%= blah %>
    <%- 1.upto(6) do |i| -%>
    foo<%= i %> = <%= i %>
    <%- end -%>

    View Slide

  32. manifests/ruby.pp
    class ruby {
    package { "ruby":
    ensure => installed
    }
    }

    View Slide

  33. manifests/ruby.pp
    class ruby {
    package { "ruby":
    ensure => installed
    }
    }

    View Slide

  34. manifests/ruby.pp
    class ruby {
    package { "ruby":
    ensure => installed
    }
    }

    View Slide

  35. manifests/site.pp
    node "app1.qa" {
    include ruby
    $blah = "test"
    file { "/etc/foo/bar.conf":
    ensure => file,
    content => template("foo/bar.conf.erb"),
    owner => "root",
    group => "root",
    mode => "0644"
    }
    }

    View Slide

  36. http://www.flickr.com/photos/wwarby/4782904694/
    Abstraction

    View Slide

  37. http://www.flickr.com/photos/chicagobart/3364948220/
    Facter

    View Slide

  38. console
    $ gem install facter

    View Slide

  39. console
    $ facter
    architecture => x86_64
    domain => local
    facterversion => 1.6.17
    fqdn => apollo.local
    hardwareisa => i386
    hardwaremodel => x86_64
    hostname => apollo
    id => jtdowney
    is_virtual => false
    kernel => Darwin
    ...

    View Slide

  40. console
    $ facter
    architecture => x86_64
    domain => local
    facterversion => 1.6.17
    fqdn => apollo.local
    hardwareisa => i386
    hardwaremodel => x86_64
    hostname => apollo
    id => jtdowney
    is_virtual => false
    kernel => Darwin
    ...

    View Slide

  41. manifests/blah.pp
    class blah {
    case $operatingsystem {
    "Ubuntu": {
    ...
    }
    "CentOS" {
    ...
    }
    }
    }

    View Slide

  42. http://www.flickr.com/photos/evelynishere/2680853104/
    Modules

    View Slide

  43. my_module
    ├── files
    │ └── somefile
    ├── manifests
    │ ├── files.pp
    │ ├── init.pp
    │ ├── package.pp
    │ └── service.pp
    └── templates
    └── sometemplate.erb

    View Slide

  44. my_module
    ├── files
    │ └── somefile
    ├── manifests
    │ ├── files.pp
    │ ├── init.pp
    │ ├── package.pp
    │ └── service.pp
    └── templates
    └── sometemplate.erb

    View Slide

  45. my_module
    ├── files
    │ └── somefile
    ├── manifests
    │ ├── files.pp
    │ ├── init.pp
    │ ├── package.pp
    │ └── service.pp
    └── templates
    └── sometemplate.erb

    View Slide

  46. my_module
    ├── files
    │ └── somefile
    ├── manifests
    │ ├── files.pp
    │ ├── init.pp
    │ ├── package.pp
    │ └── service.pp
    └── templates
    └── sometemplate.erb

    View Slide

  47. my_module
    ├── files
    │ └── somefile
    ├── manifests
    │ ├── files.pp
    │ ├── init.pp
    │ ├── package.pp
    │ └── service.pp
    └── templates
    └── sometemplate.erb

    View Slide

  48. my_module
    ├── files
    │ └── somefile
    ├── manifests
    │ ├── files.pp
    │ ├── init.pp
    │ ├── package.pp
    │ └── service.pp
    └── templates
    └── sometemplate.erb

    View Slide

  49. manifests/site.pp
    node "app1.qa" {
    include my_module
    include ruby
    }

    View Slide

  50. Puppetforge

    View Slide

  51. http://www.flickr.com/photos/baggis/3576064503/
    Hiera

    View Slide

  52. console
    $ gem install hiera

    View Slide

  53. data
    ├── common.yaml
    ├── domains
    │ ├── chi.braintreepayments.com
    │ │ ├── db.yaml
    │ │ ├── db01.yaml
    │ │ ├── jenkins.yaml
    │ │ ├── xen01.yaml
    │ │ ├── xen02.yaml
    │ │ ├── xen03.yaml

    View Slide

  54. data
    ├── common.yaml
    ├── domains
    │ ├── chi.braintreepayments.com
    │ │ ├── db.yaml
    │ │ ├── db01.yaml
    │ │ ├── jenkins.yaml
    │ │ ├── xen01.yaml
    │ │ ├── xen02.yaml
    │ │ ├── xen03.yaml

    View Slide

  55. data
    ├── common.yaml
    ├── domains
    │ ├── chi.braintreepayments.com
    │ │ ├── db.yaml
    │ │ ├── db01.yaml
    │ │ ├── jenkins.yaml
    │ │ ├── xen01.yaml
    │ │ ├── xen02.yaml
    │ │ ├── xen03.yaml

    View Slide

  56. data
    ├── common.yaml
    ├── domains
    │ ├── chi.braintreepayments.com
    │ │ ├── db.yaml
    │ │ ├── db01.yaml
    │ │ ├── jenkins.yaml
    │ │ ├── xen01.yaml
    │ │ ├── xen02.yaml
    │ │ ├── xen03.yaml

    View Slide

  57. data/domains/chi.braintreepayments.com/xen02.yaml
    xen::guests:
    "app02.chi":
    cpus: 12
    memory_mb: 30720
    management_vlan_mac: "FF:FF:FF:FF:FF:FF"
    management_vlan_ip: "%{app02_mgmt_ip}"
    application_networks:
    - { vlan: "%{gateway_vlan}", mac: ...
    distribution: "debian-squeeze-amd64-10g"
    "mailman01.chi":
    cpus: 2

    View Slide

  58. data/domains/chi.braintreegateway.com/xen02.yaml
    xen::guests:
    "app02.chi":
    cpus: 12
    memory_mb: 30720
    management_vlan_mac: "FF:FF:FF:FF:FF:FF"
    management_vlan_ip: "%{app02_mgmt_ip}"
    application_networks:
    - { vlan: "%{gateway_vlan}", mac: ...
    distribution: "debian-squeeze-amd64-10g"
    "mailman01.chi":
    cpus: 2

    View Slide

  59. Puppet does not manage truth,
    puppet maps truth to
    complexity
    http://www.flickr.com/photos/brionybarr/4310798716/

    View Slide

  60. View Slide

  61. http://www.flickr.com/photos/baggis/3576064503/
    Puppet
    Model

    View Slide

  62. http://www.flickr.com/photos/usairforce/5977871446/
    Supply drop

    View Slide

  63. console
    $ gem install supply_drop

    View Slide

  64. pitluga/supply_drop

    View Slide

  65. console
    $ mkdir infrastructure
    $ cd infrastructure
    $ gem install supply_drop
    $ capify .
    $ vim Capfile

    View Slide

  66. console
    $ mkdir infrastructure
    $ cd infrastructure
    $ gem install supply_drop
    $ capify .
    $ vim Capfile

    View Slide

  67. console
    $ mkdir infrastructure
    $ cd infrastructure
    $ gem install supply_drop
    $ capify .
    $ vim Capfile

    View Slide

  68. Capfile
    require "supply_drop"
    def datacenter_tasks(datacenter, servers)
    task datacenter do
    role :server, *servers
    end
    servers.each do |server|
    task server do
    role :server, server
    end
    end
    end

    View Slide

  69. Capfile
    require "supply_drop"
    def datacenter_tasks(datacenter, servers)
    task datacenter do
    role :server, *servers
    end
    servers.each do |server|
    task server do
    role :server, server
    end
    end
    end

    View Slide

  70. Capfile
    require "supply_drop"
    def datacenter_tasks(datacenter, servers)
    task datacenter do
    role :server, *servers
    end
    servers.each do |server|
    task server do
    role :server, server
    end
    end
    end

    View Slide

  71. Capfile
    require "supply_drop"
    def datacenter_tasks(datacenter, servers)
    task datacenter do
    role :server, *servers
    end
    servers.each do |server|
    task server do
    role :server, server
    end
    end
    end

    View Slide

  72. Capfile
    ...
    datacenter_tasks :qa, %w[
    app01.qa
    db.qa
    ]
    datacenter_tasks :production, %w[
    app01.prod
    db.prod
    ]

    View Slide

  73. puppet.pp
    node "app01.qa" {
    package { "ruby":
    ensure => installed
    }
    }

    View Slide

  74. puppet.pp
    node "app01.qa" {
    package { "ruby":
    ensure => installed
    }
    }

    View Slide

  75. console
    $ cap app01.qa puppet:noop
    notice: /Stage[main]/Package[ruby]/ensure:
    current_value absent, should be present
    (noop)
    $ cap app01.qa puppet:apply
    $ cap qa puppet:noop
    $ git add -Av
    $ git commit -m "install ruby"
    $ git push

    View Slide

  76. console
    $ cap app01.qa puppet:noop
    notice: /Stage[main]/Package[ruby]/ensure:
    current_value absent, should be present
    (noop)
    $ cap app01.qa puppet:apply
    $ cap qa puppet:noop
    $ git add -Av
    $ git commit -m "install ruby"
    $ git push
    Always read the diff!!

    View Slide

  77. console
    $ cap app01.qa puppet:noop
    notice: /Stage[main]/Package[ruby]/ensure:
    current_value absent, should be present
    (noop)
    $ cap app01.qa puppet:apply
    $ cap qa puppet:noop
    $ git add -Av
    $ git commit -m "install ruby"
    $ git push

    View Slide

  78. console
    $ cap app01.qa puppet:noop
    notice: /Stage[main]/Package[ruby]/ensure:
    current_value absent, should be present
    (noop)
    $ cap app01.qa puppet:apply
    $ cap qa puppet:noop
    $ git add -Av
    $ git commit -m "install ruby"
    $ git push

    View Slide

  79. console
    $ cap app01.qa puppet:noop
    notice: /Stage[main]/Package[ruby]/ensure:
    current_value absent, should be present
    (noop)
    $ cap app01.qa puppet:apply
    $ cap qa puppet:noop
    $ git add -Av
    $ git commit -m "install ruby"
    $ git push

    View Slide

  80. console
    $ cap db01.qa puppet:noop
    $ cap app{01..12}.qa puppet:noop
    $ cap qa puppet:noop
    $ cap qa invoke COMMAND="ls"

    View Slide

  81. console
    $ git branch
    * master # maps to qa
    production # maps to production
    staging # maps to staging

    View Slide

  82. console
    $ git checkout staging
    $ git merge master
    $ cap staging puppet:noop
    $ cap staging puppet:apply
    $ git push

    View Slide

  83. console
    $ git checkout staging
    $ git merge master
    $ cap staging puppet:noop
    $ cap staging puppet:apply
    $ git push

    View Slide

  84. console
    $ git checkout staging
    $ git merge master
    $ cap staging puppet:noop
    $ cap staging puppet:apply
    $ git push

    View Slide

  85. console
    $ git checkout staging
    $ git merge master
    $ cap staging puppet:noop
    $ cap staging puppet:apply
    $ git push

    View Slide

  86. View Slide

  87. http://www.flickr.com/photos/michelleishere/4226094476/

    View Slide

  88. Production
    Staging
    QA

    View Slide

  89. Production
    Staging
    QA

    View Slide

  90. View Slide

  91. LESSONS LEARNED
    • Stick to the puppet style guide
    • Use modules
    • Turn on puppet backup file bucket
    • Merge regularly

    View Slide

  92. PUPPET STYLE GUIDE
    • http://docs.puppetlabs.com/guides/style_guide.html
    • Idiomatic puppet
    • Cleaner and more maintainable

    View Slide

  93. USE MODULES
    • Composeable
    • Generic components
    • apache, elasticsearch, postgresql, etc
    • Check puppetforge first
    • Unique services
    • What makes your business run

    View Slide

  94. PUPPET BACKUP
    • Keep every file version
    • Great if you screw up
    • Sucks for large files
    • Turn off per-file

    View Slide

  95. MERGE REGULARLY
    • Same reason we deploy often
    • Don’t let changes linger

    View Slide

  96. View Slide

  97. console
    $ gem install vagrant

    View Slide

  98. console
    $ gem install vagrant

    View Slide

  99. http://www.vagrantup.com/

    View Slide

  100. console
    $ vagrant init precise64
    $ vagrant up
    $ vagrant ssh

    View Slide

  101. Vagrantfile
    Vagrant::Config.run do |config|
    config.vm.box = "precise64"
    config.vm.define :web do |web_config|
    web_config.vm.box = "web"
    web_config.vm.forward_port 80, 8080
    end
    config.vm.define :db do |db_config|
    db_config.vm.box = "db"
    db_config.vm.forward_port 5432, 5432
    end
    end

    View Slide

  102. Vagrantfile
    Vagrant::Config.run do |config|
    config.vm.box = "precise64"
    config.vm.define :web do |web_config|
    web_config.vm.box = "web"
    web_config.vm.forward_port 80, 8080
    end
    config.vm.define :db do |db_config|
    db_config.vm.box = "db"
    db_config.vm.forward_port 5432, 5432
    end
    end

    View Slide

  103. Vagrantfile
    Vagrant::Config.run do |config|
    config.vm.box = "precise64"
    config.vm.define :web do |web_config|
    web_config.vm.box = "web"
    web_config.vm.forward_port 80, 8080
    end
    config.vm.define :db do |db_config|
    db_config.vm.box = "db"
    db_config.vm.forward_port 5432, 5432
    end
    end

    View Slide

  104. Vagrantfile
    config.vm.provision :puppet do |puppet|
    puppet.manifests_path = "manifests"
    puppet.manifest_file = "puppet.pp"
    end

    View Slide

  105. http://www.flickr.com/photos/nickames/5307403412/
    Rapid
    prototyping

    View Slide

  106. http://www.flickr.com/photos/jrhode/4632887921/
    Dev box

    View Slide

  107. DEVOPS HUMOR

    View Slide

  108. DEVOPS REACTIONS
    Visiting the guy that wrote the build scripts
    http://devopsreactions.tumblr.com

    View Slide

  109. @DEVOPS_BORAT

    View Slide

  110. @NeckbeardHacker

    View Slide

  111. John Downey | @jtdowney

    View Slide