Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Setup (Ruby Hoedown)

Cd839cc361ffa996be0cc8259f3d7555?s=47 Will Farrington
September 22, 2012

The Setup (Ruby Hoedown)

As a rapidly growing company, GitHub's faced some challenges in how to make sure that everyone can get up and running on projects or continue to work on one of the dozens of existing projects if they've never set it up before. What a nightmare! That's what prompted @jbarnette and @wfarr to develop The Setup. The Setup aims to solve all the problems that used to plague on-boarding GitHubbers onto projects with a lot of automation and a case of the "It Just Works". This presentation talks about the history of The Setup, how it works, and some of the lessons learned along the way.

Cd839cc361ffa996be0cc8259f3d7555?s=128

Will Farrington

September 22, 2012
Tweet

Transcript

  1. Setup the

  2. managing an army of laptops ...with puppet

  3. hi, my name is will farrington

  4. but you might know me as @wfarr  

  5. None
  6. i work at a “little-known startup” called github

  7. I work on System operations System operations internal apps the

    setup
  8. why

  9. once upon a time

  10. there was a developer developer

  11. there was a designer designer

  12. oh man, this new macbook pro is sweet! let me

    run a bunch of stuff by hand so I can get some work done in about 6 to 8 hours, maybe. - some folks
  13. once upon a time

  14. there was a crazy person crazy person

  15. oh man, this new macbook pro is sweet! let me

    run a bunch of stuff by hand so I can get some work done in about 6 to 8 hours, maybe. - a bunch of crazy folks
  16. None
  17. we made a huge mistake everybody

  18. i thought we cared about stuff

  19. testing

  20. repeatability

  21. Automation

  22. they matter for our software

  23. they matter for our servers

  24. but not for our laptops

  25. !!!

  26. None
  27. None
  28. what

  29. a living, breathing software project that tests and automates every

    little bit of your machine with love
  30. @jbarnette

  31. about 6000 about 3600 1415 38 lines of Ruby lines

    of Puppet commits contributors some numbers
  32. chrome, colloquy, dropbox, elasticsearch, emacs, erlang, git, homebrew, hub, iterm2,

    java, macvim, memcache, mongodb, mysql, nginx, nodejs, nvm, 1passwd, personalization, postgresql, projects, python, qt, rbenv, rdio, redis, ruby, solr, sparrow, textmate, virtualbox, viscosity, wget, xcode, xquartz, zeromq, zsh, and more daily.
  33. less than 6 months

  34. in

  35. None
  36. hours of frustration getting github.com and other projects even working

    locally
  37. None
  38. reality

  39. None
  40. None
  41. drink coffee for 20 minutes

  42. Text

  43. None
  44. The Command Line Interface

  45. » gh-setup --projects * github

  46. github » script/bootstrap github » script/server github » open github.dev

    # Ready to ship things!
  47. » gh-setup octostatus # sets it all up. boom.

  48. » gh-setup --stealth # development mode

  49. None
  50. The Puppet DSL

  51. is pretty easy

  52. Resources

  53. basically everything in puppet

  54. package { ‘redis-server’: }

  55. file { ‘/etc/motd’: content => ‘Hello, world!’ }

  56. service { ‘riak’: ensure => running, subscribe => File[‘/etc/riak/vm.args’] }

    resource type
  57. service { ‘riak’: ensure => running, subscribe => File[‘/etc/riak/vm.args’] }

    resource name
  58. service { ‘riak’: ensure => running, subscribe => File[‘/etc/riak/vm.args’] }

    resource parameters
  59. service { ‘riak’: ensure => running, subscribe => File[‘/etc/riak/vm.args’] }

    metaparameter
  60. metaparameters are important

  61. metaparameters are everywhere

  62. before require subscribe notify this resource must run before this

    resource must run after this resource runs after and watches this resource runs before and tells
  63. custom types

  64. define git::config::local($key, $value) { $safekey = shellquote($key) $safevalue = shellquote($value)

    $command = "git config --local ${safekey} $ {safevalue}" exec { "${command} in ${name}": command => $command, cwd => $name, onlyif => 'test -d .git', unless => "git config --local ${safekey}", require => Package['git'] } }
  65. git::config::local { ‘~/github/hubot’: key => ‘heroku.account’, value => ‘work’ }

  66. Classes

  67. class redis { package { ‘redis-server’: } service { ‘redis’:

    require => Package[‘redis-server’] } }
  68. include/require

  69. class myapp { require redis include amazingness::incarnate }

  70. this matters a whole lot everybody

  71. Puppet has its own dsl for reasons

  72. puppet was designed for sysadmins

  73. puppet was designed for sysadmins okay,

  74. 2. puppet has semantics about execution that would be confusing

    and unintuitive in ruby or python or most other languages a
  75. puppet uses a DAG to apply resources

  76. a DAG is a directed, acyclic graph

  77. a DAG means relationships matter

  78. code order does not imply execution order

  79. None
  80. The Setup is a stdlib for puppet

  81.  The Setup is a stdlib for puppet

  82. class github::projects::octostatus { require redis ... }

  83. class github::projects::octostatus { ... $dir = "${github::config::srcdir}/ octostatus" ... }

  84. class github::projects::octostatus { ... git::repo { $dir: source => 'github/octostatus'

    } ... }
  85. class github::projects::octostatus { ... ruby::local { $dir: version => '1.9.3-p194',

    require => Git::Repo[$dir] } }
  86. None
  87. I like doing things my way

  88. I like doing things my way

  89. class github::people::hubot { include github::projects::all }

  90. class github::people::keavy { require gitx require onepassword require sizeup require

    sparrow require textmate }
  91. class github::people::yossef { package { 'Alfred': provider => 'appdmg', source

    => $alfred_dmg_url; } }
  92. class github::people::yossef { setup::tarballed_app { 'Mailplane': source => $mailplane_url; }

    }
  93. class github::people::wfarr { git::config::global { 'alias.st': value => 'status'; 'alias.ci':

    value => 'commit'; } }
  94. class github::people::wfarr { file { '/Users/wfarr/.gitignore': ensure => present, content

    => '.DS_STORE' } }
  95. class github::people::wfarr { package { 'emacs': } git::repo { '/Users/wfarr/.emacs.d':

    source => 'wfarr/.emacs.d', submodules => true } }
  96. class github::people::wfarr { setup::osx_defaults { 'require password always': ensure =>

    present, domain => 'com.apple.screensaver', key => 'askForPassword', value => 1, user => 'wfarr' } }
  97. lessons

  98. puppeteer supportocat

  99. people will do crazy shit to it

  100. if it breaks it is your job to fix it

  101. if it breaks it is your job to fix it

    no matter what
  102. developers and designers are really good at breaking things

  103. people will not report “minor” problems

  104. people will barely report “major” problems

  105. None
  106. not having to report bugs makes people happy

  107. getting regular bug reports makes us happy

  108. users getting github notifications that the bug they hit and

    never even reported is already fixed
  109. None
  110. reduce potential for bugs by vendoring

  111. EVERYTHING

  112. None
  113. None
  114. » du -sh /opt/github 3.8G /opt/github

  115. there’s

  116. None
  117. =(

  118. » env | grep GH_ | wc -l 24

  119. » env | grep \ GH_REDIS_PORT GH_REDIS_PORT=16379

  120. » env | grep \ GH_MYSQL_PORT GH_MYSQL_PORT=13306

  121. None
  122. people will never remember to update regularly

  123. tell people when they should update

  124. None
  125. sneak peek

  126. => Installing qt...

  127. None
  128. None
  129. => Fetching snapshot... => Installing qt...

  130. => Fetching snapshot... => Installing ruby-1.9.3...

  131. => Fetching snapshot... => Installing node-0.6.8...

  132. we don't have the ability to mandate environments

  133. we don't even want to mandate environments

  134. we have to provide something dramatically better than just doing

    it yourself
  135. prim donn

  136. the setup is not a tool for it administrators

  137. the setup is a tool for people who want to

    get stuff done, sanely
  138. the setup has goals

  139. user-extensible

  140. framework

  141. stable

  142. easy to use

  143. not a maintenance nightmare

  144. zomg want

  145. you don’t =(

  146. ... yet

  147. we are working hard to open source it

  148. Soon!

  149. Setup the

  150. questions

  151. love ops? come work with us puppet, erlang, ruby, shell,

    c and all the graphs you can shake a stick at
  152. None
  153. thanks