DevOps for the Rubyist Soul at RubyNation 2013

DevOps for the Rubyist Soul at RubyNation 2013

58376779023f009fc13d160bb3e82515?s=128

John Downey

June 14, 2013
Tweet

Transcript

  1. DEVOPS FOR THE RUBYIST SOUL John Downey | @jtdowney

  2. The authenticity of host 'apollo.local (10.0.2.56)' can't be established. RSA

    key fingerprint is 04:63:c1:ba:c7:31:04:12:14:ff:b6:c4:32:cf:44:ec. Are you sure you want to continue connecting (yes/no)?
  3. John Downey | @jtdowney

  4. None
  5. http://www.flickr.com/photos/jackol/3291891191/ Highly Available

  6. None
  7. Automated Repeatable Verifiable

  8. None
  9. BRAINTREE PATH TO DEVOPS

  10. None
  11. http://www.flickr.com/photos/orcmid/3880056930/ Agile Process

  12. Pairing http://www.flickr.com/photos/damienpollet/5048830734/

  13. None
  14. http://www.flickr.com/photos/us-pacific-command/7072151539/ Guerrilla DevOps

  15. MY PATH TO DEVOPS

  16. I <3 LINUX

  17. None
  18. Blue team!

  19. You’re a DevOps engineer, Harry

  20. cap qa puppet:noop

  21. http://www.flickr.com/photos/sp3ccylad/462995220/ Puppet

  22. console $ gem install puppet

  23. http://www.flickr.com/photos/fortwainwright/7217500612/ Puppet Master

  24. manifests/site.pp node "app01.qa" { package { "ruby": ensure => installed

    } $blah = "test" file { "/etc/foo/bar.conf": ensure => file, content => template("foo/bar.conf.erb"), owner => "root", group => "root", mode => "0644" } }
  25. manifests/site.pp node "app01.qa" { package { "ruby": ensure => installed

    } $blah = "test" file { "/etc/foo/bar.conf": ensure => file, content => template("foo/bar.conf.erb"), owner => "root", group => "root", mode => "0644" } }
  26. manifests/site.pp node "app01.qa" { package { "ruby": ensure => installed

    } $blah = "test" file { "/etc/foo/bar.conf": ensure => file, content => template("foo/bar.conf.erb"), owner => "root", group => "root", mode => "0644" } }
  27. manifests/site.pp node "app01.qa" { package { "ruby": ensure => installed

    } $blah = "test" file { "/etc/foo/bar.conf": ensure => file, content => template("foo/bar.conf.erb"), owner => "root", group => "root", mode => "0644" } }
  28. manifests/site.pp node "app01.qa" { package { "ruby": ensure => installed

    } $blah = "test" file { "/etc/foo/bar.conf": ensure => file, content => template("foo/bar.conf.erb"), owner => "root", group => "root", mode => "0644" } }
  29. manifests/site.pp node "app01.qa" { package { "ruby": ensure => installed

    } $blah = "test" file { "/etc/foo/bar.conf": ensure => file, content => template("foo/bar.conf.erb"), owner => "root", group => "root", mode => "0644" } }
  30. templates/foo/bar.conf.erb [test] blah = <%= blah %> <%- 1.upto(6) do

    |i| -%> foo<%= i %> = <%= i %> <%- end -%>
  31. templates/foo/bar.conf.erb [test] blah = <%= blah %> <%- 1.upto(6) do

    |i| -%> foo<%= i %> = <%= i %> <%- end -%>
  32. manifests/ruby.pp class ruby { package { "ruby": ensure => installed

    } }
  33. manifests/ruby.pp class ruby { package { "ruby": ensure => installed

    } }
  34. manifests/ruby.pp class ruby { package { "ruby": ensure => installed

    } }
  35. manifests/site.pp node "app1.qa" { include ruby $blah = "test" file

    { "/etc/foo/bar.conf": ensure => file, content => template("foo/bar.conf.erb"), owner => "root", group => "root", mode => "0644" } }
  36. http://www.flickr.com/photos/wwarby/4782904694/ Abstraction

  37. http://www.flickr.com/photos/chicagobart/3364948220/ Facter

  38. console $ gem install facter

  39. console $ facter architecture => x86_64 domain => local facterversion

    => 1.6.17 fqdn => apollo.local hardwareisa => i386 hardwaremodel => x86_64 hostname => apollo id => jtdowney is_virtual => false kernel => Darwin ...
  40. console $ facter architecture => x86_64 domain => local facterversion

    => 1.6.17 fqdn => apollo.local hardwareisa => i386 hardwaremodel => x86_64 hostname => apollo id => jtdowney is_virtual => false kernel => Darwin ...
  41. manifests/blah.pp class blah { case $operatingsystem { "Ubuntu": { ...

    } "CentOS" { ... } } }
  42. http://www.flickr.com/photos/evelynishere/2680853104/ Modules

  43. my_module ├── files │ └── somefile ├── manifests │ ├──

    files.pp │ ├── init.pp │ ├── package.pp │ └── service.pp └── templates └── sometemplate.erb
  44. my_module ├── files │ └── somefile ├── manifests │ ├──

    files.pp │ ├── init.pp │ ├── package.pp │ └── service.pp └── templates └── sometemplate.erb
  45. my_module ├── files │ └── somefile ├── manifests │ ├──

    files.pp │ ├── init.pp │ ├── package.pp │ └── service.pp └── templates └── sometemplate.erb
  46. my_module ├── files │ └── somefile ├── manifests │ ├──

    files.pp │ ├── init.pp │ ├── package.pp │ └── service.pp └── templates └── sometemplate.erb
  47. my_module ├── files │ └── somefile ├── manifests │ ├──

    files.pp │ ├── init.pp │ ├── package.pp │ └── service.pp └── templates └── sometemplate.erb
  48. my_module ├── files │ └── somefile ├── manifests │ ├──

    files.pp │ ├── init.pp │ ├── package.pp │ └── service.pp └── templates └── sometemplate.erb
  49. manifests/site.pp node "app1.qa" { include my_module include ruby }

  50. Puppetforge

  51. http://www.flickr.com/photos/baggis/3576064503/ Hiera

  52. console $ gem install hiera

  53. data ├── common.yaml ├── domains │ ├── chi.braintreepayments.com │ │

    ├── db.yaml │ │ ├── db01.yaml │ │ ├── jenkins.yaml │ │ ├── xen01.yaml │ │ ├── xen02.yaml │ │ ├── xen03.yaml
  54. data ├── common.yaml ├── domains │ ├── chi.braintreepayments.com │ │

    ├── db.yaml │ │ ├── db01.yaml │ │ ├── jenkins.yaml │ │ ├── xen01.yaml │ │ ├── xen02.yaml │ │ ├── xen03.yaml
  55. data ├── common.yaml ├── domains │ ├── chi.braintreepayments.com │ │

    ├── db.yaml │ │ ├── db01.yaml │ │ ├── jenkins.yaml │ │ ├── xen01.yaml │ │ ├── xen02.yaml │ │ ├── xen03.yaml
  56. data ├── common.yaml ├── domains │ ├── chi.braintreepayments.com │ │

    ├── db.yaml │ │ ├── db01.yaml │ │ ├── jenkins.yaml │ │ ├── xen01.yaml │ │ ├── xen02.yaml │ │ ├── xen03.yaml
  57. data/domains/chi.braintreepayments.com/xen02.yaml xen::guests: "app02.chi": cpus: 12 memory_mb: 30720 management_vlan_mac: "FF:FF:FF:FF:FF:FF" management_vlan_ip:

    "%{app02_mgmt_ip}" application_networks: - { vlan: "%{gateway_vlan}", mac: ... distribution: "debian-squeeze-amd64-10g" "mailman01.chi": cpus: 2
  58. data/domains/chi.braintreegateway.com/xen02.yaml xen::guests: "app02.chi": cpus: 12 memory_mb: 30720 management_vlan_mac: "FF:FF:FF:FF:FF:FF" management_vlan_ip:

    "%{app02_mgmt_ip}" application_networks: - { vlan: "%{gateway_vlan}", mac: ... distribution: "debian-squeeze-amd64-10g" "mailman01.chi": cpus: 2
  59. Puppet does not manage truth, puppet maps truth to complexity

    http://www.flickr.com/photos/brionybarr/4310798716/
  60. modules/nagios/manifests/init.pp class nagios::init { $servers = ["proxy1.qa", "proxy2.qa"] file {

    "/etc/nagios3/host-proxy.conf": ensure => file, content => template("nagios/proxy.erb"), owner => "root", group => "root", mode => "0644" } }
  61. modules/nagios/manifests/init.pp class nagios::init { $servers = ["proxy1.qa", "proxy2.qa"] file {

    "/etc/nagios3/host-proxy.conf": ensure => file, content => template("nagios/proxy.erb"), owner => "root", group => "root", mode => "0644" } }
  62. None
  63. http://www.flickr.com/photos/baggis/3576064503/ Puppet Model

  64. http://www.flickr.com/photos/usairforce/5977871446/ Supply drop

  65. console $ gem install supply_drop

  66. pitluga/supply_drop

  67. console $ mkdir infrastructure $ cd infrastructure $ gem install

    supply_drop $ capify . $ vim Capfile
  68. console $ mkdir infrastructure $ cd infrastructure $ gem install

    supply_drop $ capify . $ vim Capfile
  69. console $ mkdir infrastructure $ cd infrastructure $ gem install

    supply_drop $ capify . $ vim Capfile
  70. Capfile require "supply_drop" def datacenter_tasks(datacenter, servers) task datacenter do role

    :server, *servers end servers.each do |server| task server do role :server, server end end end
  71. Capfile require "supply_drop" def datacenter_tasks(datacenter, servers) task datacenter do role

    :server, *servers end servers.each do |server| task server do role :server, server end end end
  72. Capfile require "supply_drop" def datacenter_tasks(datacenter, servers) task datacenter do role

    :server, *servers end servers.each do |server| task server do role :server, server end end end
  73. Capfile require "supply_drop" def datacenter_tasks(datacenter, servers) task datacenter do role

    :server, *servers end servers.each do |server| task server do role :server, server end end end
  74. Capfile ... datacenter_tasks :qa, %w[ app01.qa db.qa ] datacenter_tasks :production,

    %w[ app01.prod db.prod ]
  75. puppet.pp node "app01.qa" { package { "ruby": ensure => installed

    } }
  76. puppet.pp node "app01.qa" { package { "ruby": ensure => installed

    } }
  77. console $ cap app01.qa puppet:noop notice: /Stage[main]/Package[ruby]/ensure: current_value absent, should

    be present (noop) $ cap app01.qa puppet:apply $ cap qa puppet:noop $ git add -Av $ git commit -m "install ruby" $ git push
  78. console $ cap app01.qa puppet:noop notice: /Stage[main]/Package[ruby]/ensure: current_value absent, should

    be present (noop) $ cap app01.qa puppet:apply $ cap qa puppet:noop $ git add -Av $ git commit -m "install ruby" $ git push Always read the diff!!
  79. console $ cap app01.qa puppet:noop notice: /Stage[main]/Package[ruby]/ensure: current_value absent, should

    be present (noop) $ cap app01.qa puppet:apply $ cap qa puppet:noop $ git add -Av $ git commit -m "install ruby" $ git push
  80. console $ cap app01.qa puppet:noop notice: /Stage[main]/Package[ruby]/ensure: current_value absent, should

    be present (noop) $ cap app01.qa puppet:apply $ cap qa puppet:noop $ git add -Av $ git commit -m "install ruby" $ git push
  81. console $ cap app01.qa puppet:noop notice: /Stage[main]/Package[ruby]/ensure: current_value absent, should

    be present (noop) $ cap app01.qa puppet:apply $ cap qa puppet:noop $ git add -Av $ git commit -m "install ruby" $ git push
  82. console $ cap db01.qa puppet:noop $ cap app{01..12}.qa puppet:noop $

    cap qa puppet:noop $ cap qa invoke COMMAND="ls"
  83. console $ git branch * master # maps to qa

    production # maps to production staging # maps to staging
  84. console $ git checkout staging $ git merge master $

    cap staging puppet:noop $ cap staging puppet:apply $ git push
  85. console $ git checkout staging $ git merge master $

    cap staging puppet:noop $ cap staging puppet:apply $ git push
  86. console $ git checkout staging $ git merge master $

    cap staging puppet:noop $ cap staging puppet:apply $ git push
  87. console $ git checkout staging $ git merge master $

    cap staging puppet:noop $ cap staging puppet:apply $ git push
  88. None
  89. http://www.flickr.com/photos/michelleishere/4226094476/

  90. Production Staging QA

  91. Production Staging QA

  92. None
  93. LESSONS LEARNED • Stick to the puppet style guide •

    Use modules • Know where the truth is • Turn on puppet backup file bucket • Merge regularly
  94. PUPPET STYLE GUIDE • http://docs.puppetlabs.com/guides/style_guide.html • Idiomatic puppet • Cleaner

    and more maintainable
  95. USE MODULES • Composeable • Generic components • apache, elasticsearch,

    postgresql, etc • Check puppetforge first • Unique services • What makes your business run
  96. TRUTH • Use Hiera to manage truth • Don’t let

    it get scattered • Refactor it when you spot it
  97. PUPPET BACKUP • Keep every file version • Great if

    you screw up • Sucks for large files • Turn off per-file
  98. MERGE REGULARLY • Same reason we deploy often • Don’t

    let changes linger
  99. None
  100. console $ gem install vagrant

  101. console $ gem install vagrant

  102. http://www.vagrantup.com/

  103. console $ vagrant init precise64 $ vagrant up $ vagrant

    ssh
  104. Vagrantfile Vagrant::Config.run do |config| config.vm.box = "precise64" config.vm.define :web do

    |web_config| web_config.vm.box = "web" web_config.vm.forward_port 80, 8080 end config.vm.define :db do |db_config| db_config.vm.box = "db" db_config.vm.forward_port 5432, 5432 end end
  105. Vagrantfile Vagrant::Config.run do |config| config.vm.box = "precise64" config.vm.define :web do

    |web_config| web_config.vm.box = "web" web_config.vm.forward_port 80, 8080 end config.vm.define :db do |db_config| db_config.vm.box = "db" db_config.vm.forward_port 5432, 5432 end end
  106. Vagrantfile Vagrant::Config.run do |config| config.vm.box = "precise64" config.vm.define :web do

    |web_config| web_config.vm.box = "web" web_config.vm.forward_port 80, 8080 end config.vm.define :db do |db_config| db_config.vm.box = "db" db_config.vm.forward_port 5432, 5432 end end
  107. Vagrantfile config.vm.provision :puppet do |puppet| puppet.manifests_path = "manifests" puppet.manifest_file =

    "puppet.pp" end
  108. http://www.flickr.com/photos/nickames/5307403412/ Rapid prototyping

  109. http://www.flickr.com/photos/jrhode/4632887921/ Dev box

  110. DEVOPS HUMOR

  111. DEVOPS REACTIONS Visiting the guy that wrote the build scripts

    http://devopsreactions.tumblr.com
  112. @DEVOPS_BORAT

  113. @NeckbeardHacker

  114. John Downey | @jtdowney