Pro Yearly is on sale from $80 to $50! »

Puppet at GitHub: PuppetConf 2013

Puppet at GitHub: PuppetConf 2013

Cd839cc361ffa996be0cc8259f3d7555?s=128

Will Farrington

August 22, 2013
Tweet

Transcript

  1. Puppet at GitHub

  2. @wfarr GitHub Operations Known Aliases: King of Kebabs The Chairman

    Mr. Caremad !
  3. The State of Puppet at GitHub "

  4. " The State of Puppet at GitHub github/boxen

  5. " The State of Puppet at GitHub ~1.5 years old

  6. " The State of Puppet at GitHub open-sourced ~7 months

    ago
  7. " The State of Puppet at GitHub ~240 open-source puppet

    modules
  8. " The State of Puppet at GitHub puppet 3.latest

  9. " The State of Puppet at GitHub supports ruby 1.8.7,

    1.9.3, 2.0.0
  10. " The State of Puppet at GitHub hiera

  11. " The State of Puppet at GitHub OS X

  12. " The State of Puppet at GitHub Linux support in-progress

  13. " The State of Puppet at GitHub # tomorrow @

    2:20pm
  14. " The State of Puppet at GitHub github/puppet

  15. " The State of Puppet at GitHub $

  16. " The State of Puppet at GitHub ~5 years old

  17. " The State of Puppet at GitHub 0.24.x — 2.7.x

  18. " The State of Puppet at GitHub 121 modules

  19. " The State of Puppet at GitHub ~280k lines of

    code
  20. " The State of Puppet at GitHub every single employee

    has commit access
  21. " The State of Puppet at GitHub 15.5k commits to

    master past 12 months
  22. " The State of Puppet at GitHub by 86 contributors

  23. " The State of Puppet at GitHub % % %

    % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %
  24. " The State of Puppet at GitHub % % %

    % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 42.5%
  25. " The State of Puppet at GitHub % % %

    % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % now with ops taken out
  26. " The State of Puppet at GitHub 0 100 200

    300 400 2012-08-25 2012-10-20 2012-12-15 2013-02-09 2013-04-06 2013-06-01 2013-07-27 commits on master per week, last 12 months
  27. " The State of Puppet at GitHub 0 1000 2000

    3000 4000 total commits by author past year
  28. " The State of Puppet at GitHub 0 175 350

    525 700 total commits by author past year, except ops
  29. " The State of Puppet at GitHub 0 175 350

    525 700 total commits by authors with >10 commits, past year, except ops
  30. " The State of Puppet at GitHub single puppetmaster

  31. " The State of Puppet at GitHub rubygems

  32. " The State of Puppet at GitHub ruby 1.8.7

  33. " The State of Puppet at GitHub unicorn

  34. " The State of Puppet at GitHub puppet 2.7.latest

  35. " The State of Puppet at GitHub ~600 nodes

  36. " The State of Puppet at GitHub run hourly via

    crond
  37. " The State of Puppet at GitHub puppetdb

  38. " The State of Puppet at GitHub nagiosdb

  39. " The State of Puppet at GitHub "yo puppetdb, gimme

    all your nagios::object::* resources so I can update this nagios config"
  40. " The State of Puppet at GitHub filtergendb

  41. " The State of Puppet at GitHub "yo puppetdb, gimme

    all the filtergen::rule resources I would realize so I can update this filtergen config"
  42. " The State of Puppet at GitHub ⚡ puppetdb ⚡

  43. " The State of Puppet at GitHub as it turns

    out, an api call is faster than running puppet on a host
  44. " The State of Puppet at GitHub gpanel

  45. " The State of Puppet at GitHub "Imagine Puppet Dashboard

    meets Razor and went on a weekend trip to the beach with the Heroku API and drank epic amounts of blue drink"
  46. " The State of Puppet at GitHub aka we reinvented

    our own, smaller version of Foreman
  47. " The State of Puppet at GitHub inventory

  48. " The State of Puppet at GitHub

  49. " The State of Puppet at GitHub

  50. " The State of Puppet at GitHub app configuration

  51. " The State of Puppet at GitHub

  52. " The State of Puppet at GitHub versioning of configuration

    values
  53. " The State of Puppet at GitHub Create nil =>

    1 Update 1 => 2 Delete 2 => nil
  54. " The State of Puppet at GitHub let's make credential

    rolling less awful
  55. " The State of Puppet at GitHub provisioning

  56. " The State of Puppet at GitHub provisioning is typically

    awful
  57. " The State of Puppet at GitHub we sprinkled in

    some ChatOps
  58. " The State of Puppet at GitHub

  59. " The State of Puppet at GitHub a little bit

    later...
  60. " The State of Puppet at GitHub <MACHINE>

  61. " The State of Puppet at GitHub enc

  62. " The State of Puppet at GitHub we have the

    dumbest ENC out there
  63. " The State of Puppet at GitHub ⋆

  64. " The State of Puppet at GitHub # /usr/local/sbin/fetch_gpanel_enc \

    fe1.rs.github.com --- parameters: gpanel_cabinet: D20-13 gpanel_enabled: true gpanel_monitored: false
  65. " The State of Puppet at GitHub we never specify

    classes via the ENC
  66. " The State of Puppet at GitHub any variables we

    pass through are prefixed with gpanel_
  67. How GitHub writes Puppet (

  68. ( How GitHub Writes Puppet rodjek/puppet-lint

  69. ( How GitHub Writes Puppet

  70. ( How GitHub Writes Puppet if you aren't using puppet-lint

    to audit your puppet codebase, you are doing it wrong
  71. ( How GitHub Writes Puppet puppet-lint enforces the Puppet Labs

    style guide
  72. ( How GitHub Writes Puppet puppet-lint can even fix a

    ton of linter errors for you
  73. ( How GitHub Writes Puppet put it in a pre-commit

    hook
  74. ( How GitHub Writes Puppet $ git commit -am "can't

    lint this" modules/github/manifests/role/redis.pp: syntax ok modules/github/manifests/role/redis.pp - WARNING: => is not properly aligned on line 118 1 errors found, aborting commit.
  75. ( How GitHub Writes Puppet and then buy rodjek a

    beer
  76. ( How GitHub Writes Puppet rodjek/rspec-puppet

  77. ( How GitHub Writes Puppet

  78. ( How GitHub Writes Puppet if you aren't writing tests

    for your puppet code before running it on a server, you are doing it wrong
  79. ( How GitHub Writes Puppet use whatever framework/library

  80. ( How GitHub Writes Puppet rspec-puppet just happens to be

    a pretty good one
  81. ( How GitHub Writes Puppet put it in a pre-commit

    hook
  82. ( How GitHub Writes Puppet $ git commit -am "tests

    dont pass but whatever lol" 1) Expected redis::server would include class "more_than_a_single_c_thread" 1 failures encountered, aborting commit.
  83. ( How GitHub Writes Puppet and then buy rodjek another

    beer
  84. ( How GitHub Writes Puppet node definitions

  85. ( How GitHub Writes Puppet we don't use an ENC

    to describe node classes
  86. ( How GitHub Writes Puppet node /^github-redis\d+/ { class {

    'github::role::polling_redis': enabled => $::gpanel_enabled, environment => $::gpanel_environment, private_ipv4 => $::ipaddress, } }
  87. ( How GitHub Writes Puppet abstractions all the way down

  88. ( How GitHub Writes Puppet treat your site classes as

    cascades down to your dist classes
  89. ( How GitHub Writes Puppet class redis::server( # params )

    { class { 'redis::config': ... } -> class { 'redis::package': ... } ~> class { 'redis::service': ... } }
  90. ( How GitHub Writes Puppet class github::redis( # params )

    inherits github::defaults { $memory = $environment ? { 'stg' => '2G', default => $half_memory_gb } class { 'redis::server': ... } }
  91. ( How GitHub Writes Puppet class github::role::polling_redis( # params )

    { class { 'github::redis': # overrides based on specific node } }
  92. ( How GitHub Writes Puppet augeas

  93. ( How GitHub Writes Puppet you know what's not awesome?

  94. ( How GitHub Writes Puppet an erb template that requires

    your class to take 52876423 parameters so you can configure every possible value in my.cnf
  95. ( How GitHub Writes Puppet augeas { 'my.cnf/performance': context =>

    '/files/etc/mysql/my.cnf/mysqld', changes => [ # automatic dump/restore 'set innodb_auto_lru_dump 18000', # innodb "set innodb_buffer_pool_size ${innodb_buffer_pool_size}", 'set innodb_log_file_size 256M', 'set innodb_log_buffer_size 8M', 'set innodb_lazy_drop_table 1', ], require => Percona::Server[$::fqdn] }
  96. ( How GitHub Writes Puppet it can seem complex and

    scary
  97. ( How GitHub Writes Puppet it is an amazing tool

  98. How GitHub ships Puppet )

  99. ) How GitHub Ships Puppet continuous integration

  100. ) How GitHub Ships Puppet commit gets pushed

  101. ) How GitHub Ships Puppet jenkins runs the test suite

  102. ) How GitHub Ships Puppet status gets posted back to

    GitHub.com
  103. ) How GitHub Ships Puppet tmm1/test-queue

  104. ) How GitHub Ships Puppet

  105. ) How GitHub Ships Puppet remembers how to better parallelize

    tests
  106. ) How GitHub Ships Puppet continuous deployment

  107. ) How GitHub Ships Puppet tests passed for commit, Hubot

    auto-deploys
  108. ) How GitHub Ships Puppet branch deploy everything

  109. ) How GitHub Ships Puppet a lot of shops have

    a few environments
  110. ) How GitHub Ships Puppet testing staging production

  111. ) How GitHub Ships Puppet currently we have 181 environments

  112. ) How GitHub Ships Puppet that's not the same as

    181 nodes not running production
  113. ) How GitHub Ships Puppet we only have 5 of

    those
  114. ) How GitHub Ships Puppet Hubot automatically merges the master

    branch before allowing any branch to deploy
  115. ) How GitHub Ships Puppet ChatOps

  116. ) How GitHub Ships Puppet 202 employees 100% access to

    Puppet 100% trust
  117. ) How GitHub Ships Puppet the list of roles you'd

    never think would touch production puppet webkit developer core git developers core ruby developers svn developers search developer frontend designers windows developers mac developers support enterprise sales
  118. ) How GitHub Ships Puppet every puppet run happens in

    chat
  119. ) How GitHub Ships Puppet all puppet output goes to

    chat
  120. ) How GitHub Ships Puppet everyone can see everything

  121. ) How GitHub Ships Puppet everyone can do anything

  122. ) How GitHub Ships Puppet hands-on learning, by accident

  123. ) How GitHub Ships Puppet eliminate disruptive questions

  124. The Future of Puppet at GitHub *

  125. * The Future of Puppet at GitHub puppet 3.x

  126. * The Future of Puppet at GitHub upgrading a large,

    old codebase from 2.7 to 3 is really painful
  127. * The Future of Puppet at GitHub we've been talking

    about it for a year
  128. * The Future of Puppet at GitHub we're still trying

    to push forward, but...
  129. * The Future of Puppet at GitHub helping maintain security

    fixes for 2.7
  130. * The Future of Puppet at GitHub goal is to

    be on 3.2 by end of year
  131. * The Future of Puppet at GitHub ruby 2.0

  132. * The Future of Puppet at GitHub moar faster rubby

  133. * The Future of Puppet at GitHub we get this

    for free when we move to 3
  134. * The Future of Puppet at GitHub mcollective

  135. * The Future of Puppet at GitHub aka the story

    of github/shell and the wonders of ssh in a for-loop
  136. * The Future of Puppet at GitHub there are a

    lot of cool things about mcollective
  137. * The Future of Puppet at GitHub I think it

    will be a part of our stack in the future
  138. * The Future of Puppet at GitHub but you can

    still go pretty damn far with bash and ssh
  139. * The Future of Puppet at GitHub even more puppetdb

    tooling
  140. * The Future of Puppet at GitHub waiting for multiple

    runs to converge exported resources is painful
  141. * The Future of Puppet at GitHub use the puppetdb

    API to skip all that
  142. * The Future of Puppet at GitHub even more gpanel

    integration
  143. * The Future of Puppet at GitHub "databags"

  144. * The Future of Puppet at GitHub "node search"

  145. * The Future of Puppet at GitHub steal good ideas

    from other tools
  146. closing thoughts

  147. "all software is terrible" — anyone who's worked with software

    long enough
  148. "no software is better than no software" — rtomayko

  149. the plumbing doesn't matter when all you care about is

    the porcelain
  150. write some damned good porcelain

  151. None
  152. THIS IS RODJEK

  153. HE'S NORMALLY NOT THIS BLURRY

  154. HE IS ALSO NOT MY BROTHER, BUT YOU REALLY SHOULD

    BUY HIM A BEER
  155. YOU SHOULD BUY THIS MAN A LOT OF BEER

  156. THANKS https://speakerdeck.com/wfarr/ puppet-at-github-puppetconf-2013 !