Puppet at GitHub: PuppetConf 2013

Transcript

1. Puppet at GitHub
2. @wfarr GitHub Operations Known Aliases: King of Kebabs The Chairman Mr. Caremad !
3. The State of Puppet at GitHub "
4. " The State of Puppet at GitHub github/boxen
5. " The State of Puppet at GitHub ~1.5 years old
6. " The State of Puppet at GitHub open-sourced ~7 months ago
7. " The State of Puppet at GitHub ~240 open-source puppet modules
8. " The State of Puppet at GitHub puppet 3.latest
9. " The State of Puppet at GitHub supports ruby 1.8.7, 1.9.3, 2.0.0
10. " The State of Puppet at GitHub hiera
11. " The State of Puppet at GitHub OS X
12. " The State of Puppet at GitHub Linux support in-progress
13. " The State of Puppet at GitHub # tomorrow @ 2:20pm
14. " The State of Puppet at GitHub github/puppet
15. " The State of Puppet at GitHub $
16. " The State of Puppet at GitHub ~5 years old
17. " The State of Puppet at GitHub 0.24.x — 2.7.x
18. " The State of Puppet at GitHub 121 modules
19. " The State of Puppet at GitHub ~280k lines of code
20. " The State of Puppet at GitHub every single employee has commit access
21. " The State of Puppet at GitHub 15.5k commits to master past 12 months
22. " The State of Puppet at GitHub by 86 contributors
23. " The State of Puppet at GitHub % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %
24. " The State of Puppet at GitHub % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 42.5%
25. " The State of Puppet at GitHub % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % now with ops taken out
26. " The State of Puppet at GitHub 0 100 200 300 400 2012-08-25 2012-10-20 2012-12-15 2013-02-09 2013-04-06 2013-06-01 2013-07-27 commits on master per week, last 12 months
27. " The State of Puppet at GitHub 0 1000 2000 3000 4000 total commits by author past year
28. " The State of Puppet at GitHub 0 175 350 525 700 total commits by author past year, except ops
29. " The State of Puppet at GitHub 0 175 350 525 700 total commits by authors with >10 commits, past year, except ops
30. " The State of Puppet at GitHub single puppetmaster
31. " The State of Puppet at GitHub rubygems
32. " The State of Puppet at GitHub ruby 1.8.7
33. " The State of Puppet at GitHub unicorn
34. " The State of Puppet at GitHub puppet 2.7.latest
35. " The State of Puppet at GitHub ~600 nodes
36. " The State of Puppet at GitHub run hourly via crond
37. " The State of Puppet at GitHub puppetdb
38. " The State of Puppet at GitHub nagiosdb
39. " The State of Puppet at GitHub "yo puppetdb, gimme all your nagios::object::* resources so I can update this nagios config"
40. " The State of Puppet at GitHub filtergendb
41. " The State of Puppet at GitHub "yo puppetdb, gimme all the filtergen::rule resources I would realize so I can update this filtergen config"
42. " The State of Puppet at GitHub ⚡ puppetdb ⚡
43. " The State of Puppet at GitHub as it turns out, an api call is faster than running puppet on a host
44. " The State of Puppet at GitHub gpanel
45. " The State of Puppet at GitHub "Imagine Puppet Dashboard meets Razor and went on a weekend trip to the beach with the Heroku API and drank epic amounts of blue drink"
46. " The State of Puppet at GitHub aka we reinvented our own, smaller version of Foreman
47. " The State of Puppet at GitHub inventory
48. " The State of Puppet at GitHub
49. " The State of Puppet at GitHub
50. " The State of Puppet at GitHub app configuration
51. " The State of Puppet at GitHub
52. " The State of Puppet at GitHub versioning of configuration values
53. " The State of Puppet at GitHub Create nil => 1 Update 1 => 2 Delete 2 => nil
54. " The State of Puppet at GitHub let's make credential rolling less awful
55. " The State of Puppet at GitHub provisioning
56. " The State of Puppet at GitHub provisioning is typically awful
57. " The State of Puppet at GitHub we sprinkled in some ChatOps
58. " The State of Puppet at GitHub
59. " The State of Puppet at GitHub a little bit later...
60. " The State of Puppet at GitHub <MACHINE>
61. " The State of Puppet at GitHub enc
62. " The State of Puppet at GitHub we have the dumbest ENC out there
63. " The State of Puppet at GitHub ⋆
64. " The State of Puppet at GitHub # /usr/local/sbin/fetch_gpanel_enc \ fe1.rs.github.com --- parameters: gpanel_cabinet: D20-13 gpanel_enabled: true gpanel_monitored: false
65. " The State of Puppet at GitHub we never specify classes via the ENC
66. " The State of Puppet at GitHub any variables we pass through are prefixed with gpanel_
67. How GitHub writes Puppet (
68. ( How GitHub Writes Puppet rodjek/puppet-lint
69. ( How GitHub Writes Puppet
70. ( How GitHub Writes Puppet if you aren't using puppet-lint to audit your puppet codebase, you are doing it wrong
71. ( How GitHub Writes Puppet puppet-lint enforces the Puppet Labs style guide
72. ( How GitHub Writes Puppet puppet-lint can even fix a ton of linter errors for you
73. ( How GitHub Writes Puppet put it in a pre-commit hook
74. ( How GitHub Writes Puppet $ git commit -am "can't lint this" modules/github/manifests/role/redis.pp: syntax ok modules/github/manifests/role/redis.pp - WARNING: => is not properly aligned on line 118 1 errors found, aborting commit.
75. ( How GitHub Writes Puppet and then buy rodjek a beer
76. ( How GitHub Writes Puppet rodjek/rspec-puppet
77. ( How GitHub Writes Puppet
78. ( How GitHub Writes Puppet if you aren't writing tests for your puppet code before running it on a server, you are doing it wrong
79. ( How GitHub Writes Puppet use whatever framework/library
80. ( How GitHub Writes Puppet rspec-puppet just happens to be a pretty good one
81. ( How GitHub Writes Puppet put it in a pre-commit hook
82. ( How GitHub Writes Puppet $ git commit -am "tests dont pass but whatever lol" 1) Expected redis::server would include class "more_than_a_single_c_thread" 1 failures encountered, aborting commit.
83. ( How GitHub Writes Puppet and then buy rodjek another beer
84. ( How GitHub Writes Puppet node definitions
85. ( How GitHub Writes Puppet we don't use an ENC to describe node classes
86. ( How GitHub Writes Puppet node /^github-redis\d+/ { class { 'github::role::polling_redis': enabled => $::gpanel_enabled, environment => $::gpanel_environment, private_ipv4 => $::ipaddress, } }
87. ( How GitHub Writes Puppet abstractions all the way down
88. ( How GitHub Writes Puppet treat your site classes as cascades down to your dist classes
89. ( How GitHub Writes Puppet class redis::server( # params ) { class { 'redis::config': ... } -> class { 'redis::package': ... } ~> class { 'redis::service': ... } }
90. ( How GitHub Writes Puppet class github::redis( # params ) inherits github::defaults { $memory = $environment ? { 'stg' => '2G', default => $half_memory_gb } class { 'redis::server': ... } }
91. ( How GitHub Writes Puppet class github::role::polling_redis( # params ) { class { 'github::redis': # overrides based on specific node } }
92. ( How GitHub Writes Puppet augeas
93. ( How GitHub Writes Puppet you know what's not awesome?
94. ( How GitHub Writes Puppet an erb template that requires your class to take 52876423 parameters so you can configure every possible value in my.cnf
95. ( How GitHub Writes Puppet augeas { 'my.cnf/performance': context => '/files/etc/mysql/my.cnf/mysqld', changes => [ # automatic dump/restore 'set innodb_auto_lru_dump 18000', # innodb "set innodb_buffer_pool_size ${innodb_buffer_pool_size}", 'set innodb_log_file_size 256M', 'set innodb_log_buffer_size 8M', 'set innodb_lazy_drop_table 1', ], require => Percona::Server[$::fqdn] }
96. ( How GitHub Writes Puppet it can seem complex and scary
97. ( How GitHub Writes Puppet it is an amazing tool
98. How GitHub ships Puppet )
99. ) How GitHub Ships Puppet continuous integration
100. ) How GitHub Ships Puppet commit gets pushed
101. ) How GitHub Ships Puppet jenkins runs the test suite
102. ) How GitHub Ships Puppet status gets posted back to GitHub.com
103. ) How GitHub Ships Puppet tmm1/test-queue
104. ) How GitHub Ships Puppet
105. ) How GitHub Ships Puppet remembers how to better parallelize tests
106. ) How GitHub Ships Puppet continuous deployment
107. ) How GitHub Ships Puppet tests passed for commit, Hubot auto-deploys
108. ) How GitHub Ships Puppet branch deploy everything
109. ) How GitHub Ships Puppet a lot of shops have a few environments
110. ) How GitHub Ships Puppet testing staging production
111. ) How GitHub Ships Puppet currently we have 181 environments
112. ) How GitHub Ships Puppet that's not the same as 181 nodes not running production
113. ) How GitHub Ships Puppet we only have 5 of those
114. ) How GitHub Ships Puppet Hubot automatically merges the master branch before allowing any branch to deploy
115. ) How GitHub Ships Puppet ChatOps
116. ) How GitHub Ships Puppet 202 employees 100% access to Puppet 100% trust
117. ) How GitHub Ships Puppet the list of roles you'd never think would touch production puppet webkit developer core git developers core ruby developers svn developers search developer frontend designers windows developers mac developers support enterprise sales
118. ) How GitHub Ships Puppet every puppet run happens in chat
119. ) How GitHub Ships Puppet all puppet output goes to chat
120. ) How GitHub Ships Puppet everyone can see everything
121. ) How GitHub Ships Puppet everyone can do anything
122. ) How GitHub Ships Puppet hands-on learning, by accident
123. ) How GitHub Ships Puppet eliminate disruptive questions
124. The Future of Puppet at GitHub *
125. * The Future of Puppet at GitHub puppet 3.x
126. * The Future of Puppet at GitHub upgrading a large, old codebase from 2.7 to 3 is really painful
127. * The Future of Puppet at GitHub we've been talking about it for a year
128. * The Future of Puppet at GitHub we're still trying to push forward, but...
129. * The Future of Puppet at GitHub helping maintain security fixes for 2.7
130. * The Future of Puppet at GitHub goal is to be on 3.2 by end of year
131. * The Future of Puppet at GitHub ruby 2.0
132. * The Future of Puppet at GitHub moar faster rubby
133. * The Future of Puppet at GitHub we get this for free when we move to 3
134. * The Future of Puppet at GitHub mcollective
135. * The Future of Puppet at GitHub aka the story of github/shell and the wonders of ssh in a for-loop
136. * The Future of Puppet at GitHub there are a lot of cool things about mcollective
137. * The Future of Puppet at GitHub I think it will be a part of our stack in the future
138. * The Future of Puppet at GitHub but you can still go pretty damn far with bash and ssh
139. * The Future of Puppet at GitHub even more puppetdb tooling
140. * The Future of Puppet at GitHub waiting for multiple runs to converge exported resources is painful
141. * The Future of Puppet at GitHub use the puppetdb API to skip all that
142. * The Future of Puppet at GitHub even more gpanel integration
143. * The Future of Puppet at GitHub "databags"
144. * The Future of Puppet at GitHub "node search"
145. * The Future of Puppet at GitHub steal good ideas from other tools
146. closing thoughts
147. "all software is terrible" — anyone who's worked with software long enough
148. "no software is better than no software" — rtomayko
149. the plumbing doesn't matter when all you care about is the porcelain
150. write some damned good porcelain
152. THIS IS RODJEK
153. HE'S NORMALLY NOT THIS BLURRY
154. HE IS ALSO NOT MY BROTHER, BUT YOU REALLY SHOULD BUY HIM A BEER
155. YOU SHOULD BUY THIS MAN A LOT OF BEER
156. THANKS https://speakerdeck.com/wfarr/ puppet-at-github-puppetconf-2013 !