The Setup (Ruby Hoedown)

As a rapidly growing company, GitHub's faced some challenges in how to make sure that everyone can get up and running on projects or continue to work on one of the dozens of existing projects if they've never set it up before. What a nightmare! That's what prompted @jbarnette and @wfarr to develop The Setup. The Setup aims to solve all the problems that used to plague on-boarding GitHubbers onto projects with a lot of automation and a case of the "It Just Works". This presentation talks about the history of The Setup, how it works, and some of the lessons learned along the way.

Transcript

1. Setup the
2. managing an army of laptops ...with puppet
3. hi, my name is will farrington
4. but you might know me as @wfarr  
6. i work at a “little-known startup” called github
7. I work on System operations System operations internal apps the setup
8. why
9. once upon a time
10. there was a developer developer
11. there was a designer designer
12. oh man, this new macbook pro is sweet! let me run a bunch of stuff by hand so I can get some work done in about 6 to 8 hours, maybe. - some folks
13. once upon a time
14. there was a crazy person crazy person
15. oh man, this new macbook pro is sweet! let me run a bunch of stuff by hand so I can get some work done in about 6 to 8 hours, maybe. - a bunch of crazy folks
17. we made a huge mistake everybody
18. i thought we cared about stuff
19. testing
20. repeatability
21. Automation
22. they matter for our software
23. they matter for our servers
24. but not for our laptops
25. !!!
28. what
29. a living, breathing software project that tests and automates every little bit of your machine with love
30. @jbarnette
31. about 6000 about 3600 1415 38 lines of Ruby lines of Puppet commits contributors some numbers
32. chrome, colloquy, dropbox, elasticsearch, emacs, erlang, git, homebrew, hub, iterm2, java, macvim, memcache, mongodb, mysql, nginx, nodejs, nvm, 1passwd, personalization, postgresql, projects, python, qt, rbenv, rdio, redis, ruby, solr, sparrow, textmate, virtualbox, viscosity, wget, xcode, xquartz, zeromq, zsh, and more daily.
33. less than 6 months
34. in
36. hours of frustration getting github.com and other projects even working locally
38. reality
41. drink coffee for 20 minutes
42. Text
44. The Command Line Interface
45. » gh-setup --projects * github
46. github » script/bootstrap github » script/server github » open github.dev # Ready to ship things!
47. » gh-setup octostatus # sets it all up. boom.
48. » gh-setup --stealth # development mode
50. The Puppet DSL
51. is pretty easy
52. Resources
53. basically everything in puppet
54. package { ‘redis-server’: }
55. file { ‘/etc/motd’: content => ‘Hello, world!’ }
56. service { ‘riak’: ensure => running, subscribe => File[‘/etc/riak/vm.args’] } resource type
57. service { ‘riak’: ensure => running, subscribe => File[‘/etc/riak/vm.args’] } resource name
58. service { ‘riak’: ensure => running, subscribe => File[‘/etc/riak/vm.args’] } resource parameters
59. service { ‘riak’: ensure => running, subscribe => File[‘/etc/riak/vm.args’] } metaparameter
60. metaparameters are important
61. metaparameters are everywhere
62. before require subscribe notify this resource must run before this resource must run after this resource runs after and watches this resource runs before and tells
63. custom types
64. define git::config::local($key, $value) { $safekey = shellquote($key) $safevalue = shellquote($value) $command = "git config --local ${safekey} $ {safevalue}" exec { "${command} in ${name}": command => $command, cwd => $name, onlyif => 'test -d .git', unless => "git config --local ${safekey}", require => Package['git'] } }
65. git::config::local { ‘~/github/hubot’: key => ‘heroku.account’, value => ‘work’ }
66. Classes
67. class redis { package { ‘redis-server’: } service { ‘redis’: require => Package[‘redis-server’] } }
68. include/require
69. class myapp { require redis include amazingness::incarnate }
70. this matters a whole lot everybody
71. Puppet has its own dsl for reasons
72. puppet was designed for sysadmins
73. puppet was designed for sysadmins okay,
74. 2. puppet has semantics about execution that would be confusing and unintuitive in ruby or python or most other languages a
75. puppet uses a DAG to apply resources
76. a DAG is a directed, acyclic graph
77. a DAG means relationships matter
78. code order does not imply execution order
80. The Setup is a stdlib for puppet
81.  The Setup is a stdlib for puppet
82. class github::projects::octostatus { require redis ... }
83. class github::projects::octostatus { ... $dir = "${github::config::srcdir}/ octostatus" ... }
84. class github::projects::octostatus { ... git::repo { $dir: source => 'github/octostatus' } ... }
85. class github::projects::octostatus { ... ruby::local { $dir: version => '1.9.3-p194', require => Git::Repo[$dir] } }
87. I like doing things my way
88. I like doing things my way
89. class github::people::hubot { include github::projects::all }
90. class github::people::keavy { require gitx require onepassword require sizeup require sparrow require textmate }
91. class github::people::yossef { package { 'Alfred': provider => 'appdmg', source => $alfred_dmg_url; } }
92. class github::people::yossef { setup::tarballed_app { 'Mailplane': source => $mailplane_url; } }
93. class github::people::wfarr { git::config::global { 'alias.st': value => 'status'; 'alias.ci': value => 'commit'; } }
94. class github::people::wfarr { file { '/Users/wfarr/.gitignore': ensure => present, content => '.DS_STORE' } }
95. class github::people::wfarr { package { 'emacs': } git::repo { '/Users/wfarr/.emacs.d': source => 'wfarr/.emacs.d', submodules => true } }
96. class github::people::wfarr { setup::osx_defaults { 'require password always': ensure => present, domain => 'com.apple.screensaver', key => 'askForPassword', value => 1, user => 'wfarr' } }
97. lessons
98. puppeteer supportocat
99. people will do crazy shit to it
100. if it breaks it is your job to fix it
101. if it breaks it is your job to fix it no matter what
102. developers and designers are really good at breaking things
103. people will not report “minor” problems
104. people will barely report “major” problems
106. not having to report bugs makes people happy
107. getting regular bug reports makes us happy
108. users getting github notifications that the bug they hit and never even reported is already fixed
110. reduce potential for bugs by vendoring
111. EVERYTHING
114. » du -sh /opt/github 3.8G /opt/github
115. there’s
117. =(
118. » env | grep GH_ | wc -l 24
119. » env | grep \ GH_REDIS_PORT GH_REDIS_PORT=16379
120. » env | grep \ GH_MYSQL_PORT GH_MYSQL_PORT=13306
122. people will never remember to update regularly
123. tell people when they should update
125. sneak peek
126. => Installing qt...
129. => Fetching snapshot... => Installing qt...
130. => Fetching snapshot... => Installing ruby-1.9.3...
131. => Fetching snapshot... => Installing node-0.6.8...
132. we don't have the ability to mandate environments
133. we don't even want to mandate environments
134. we have to provide something dramatically better than just doing it yourself
135. prim donn
136. the setup is not a tool for it administrators
137. the setup is a tool for people who want to get stuff done, sanely
138. the setup has goals
139. user-extensible
140. framework
141. stable
142. easy to use
143. not a maintenance nightmare
144. zomg want
145. you don’t =(
146. ... yet
147. we are working hard to open source it
148. Soon!
149. Setup the
150. questions
151. love ops? come work with us puppet, erlang, ruby, shell, c and all the graphs you can shake a stick at
153. thanks