$30 off During Our Annual Pro Sale. View Details »

The Setup: Managing an Army of Laptops with Puppet

The Setup: Managing an Army of Laptops with Puppet

As a rapidly growing company, GitHub's faced some challenges in how to make sure that everyone can get up and running on projects or continue to work on one of the dozens of existing projects if they've never set it up before. What a nightmare! That's what prompted @jbarnette and @wfarr to develop The Setup. The Setup aims to solve all the problems that used to plague on-boarding GitHubbers onto projects with a lot of automation and a case of the "It Just Works". This presentation talks about the history of The Setup, how it works, and some of the lessons learned along the way.

Will Farrington

September 12, 2012
Tweet

More Decks by Will Farrington

Other Decks in Technology

Transcript

  1. Setup
    the

    View Slide

  2. managing an
    army of laptops
    ...with puppet

    View Slide

  3. hi, my name is will farrington
    but @wfarr is way shorter

    View Slide

  4. i work at a “little-known startup” called
    github

    View Slide

  5. I work on System operations
    System operations
    internal apps
    the setup

    View Slide

  6. why

    View Slide

  7. once
    upon a
    time

    View Slide

  8. there
    was a
    developer

    View Slide

  9. there
    was a
    designer

    View Slide

  10. oh man, this new macbook pro is sweet! let me run
    a bunch of stuff by hand so I can get some work
    done in about 6 to 8 hours, maybe.
    - some perfectly normal people

    View Slide

  11. once
    upon a
    time

    View Slide

  12. there
    was a
    crazy
    person

    View Slide

  13. oh man, this new macbook pro is sweet! let me run
    a bunch of stuff by hand so I can get some work
    done in about 6 to 8 hours, maybe.
    ~ crazy people

    View Slide

  14. View Slide

  15. we made a
    HUGE
    mistake

    View Slide

  16. yep, it’s
    pretty
    bad

    View Slide

  17. i thought we cared about testing

    View Slide

  18. i thought we cared about repeatability

    View Slide

  19. i thought we cared about automation

    View Slide

  20. we say those things matter for software
    and they totally do

    View Slide

  21. we do not treat our laptops like they matter

    View Slide

  22. wtf is wrong with us

    View Slide

  23. wtf is wrong with us
    i mean srsly guise

    View Slide

  24. spock
    approves
    this
    message

    View Slide

  25. what

    View Slide

  26. a living, breathing software project
    that tests and automates every little
    bit of your machine with love

    View Slide

  27. @jbarnette

    View Slide

  28. about 6000
    about 3600
    1186
    26
    lines of Ruby (not incl. tests)
    lines of Puppet (not incl. tests)
    commits
    contributors
    some numbers

    View Slide

  29. chrome, colloquy, dropbox, elasticsearch, emacs,
    erlang, git, homebrew, hub, iterm2, java, macvim,
    memcache, mongodb, mysql, nginx, nodejs, nvm,
    1passwd, personalization, postgresql, projects,
    python, qt, rbenv, rdio, redis, ruby, solr, sparrow,
    textmate, virtualbox, viscosity, wget, xcode,
    xquartz, zeromq, zsh, and more daily.

    View Slide

  30. less than 6 months

    View Slide

  31. a githubber’s first day
    in

    View Slide

  32. View Slide

  33. hours of frustration getting
    github.com and other projects
    even working locally

    View Slide

  34. View Slide

  35. a githubber’s first day
    now

    View Slide

  36. View Slide

  37. View Slide

  38. drink coffee for 20 minutes

    View Slide

  39. Text

    View Slide

  40. View Slide

  41. The Command Line Interface

    View Slide

  42. » gh-setup --projects
    * github

    View Slide

  43. github » script/bootstrap
    github » script/server
    github » open github.dev
    # Ready to ship things!

    View Slide

  44. » gh-setup octostatus
    # sets it all up. boom.

    View Slide

  45. » gh-setup --stealth
    # development mode

    View Slide

  46. View Slide

  47. The Puppet DSL

    View Slide

  48. The Puppet DSL is pretty easy actually

    View Slide

  49. Resources

    View Slide

  50. Resources are the basic unit of puppet

    View Slide

  51. package { ‘redis-server’: }

    View Slide

  52. file { ‘/etc/motd’:
    content => ‘Hello, world!’
    }

    View Slide

  53. service { ‘riak’:
    ensure => running,
    subscribe => File[‘/etc/riak/vm.args’]
    }
    resource type

    View Slide

  54. service { ‘riak’:
    ensure => running,
    subscribe => File[‘/etc/riak/vm.args’]
    }
    resource name

    View Slide

  55. service { ‘riak’:
    ensure => running,
    subscribe => File[‘/etc/riak/vm.args’]
    }
    resource parameters

    View Slide

  56. service { ‘riak’:
    ensure => running,
    subscribe => File[‘/etc/riak/vm.args’]
    }
    metaparameter

    View Slide

  57. metaparameters work on every resource

    View Slide

  58. metaparameters describe relationships

    View Slide

  59. before
    require
    subscribe
    notify
    this resource must run before
    this resource must run after
    this resource runs after and watches
    this resource runs before and tells

    View Slide

  60. resources can have custom types

    View Slide

  61. define git::config::local($key, $value) {
    $safekey = shellquote($key)
    $safevalue = shellquote($value)
    $command = "git config --local ${safekey} ${safevalue}"
    exec { "${command} in ${name}":
    command => $command,
    cwd => $name,
    onlyif => 'test -d .git',
    unless => "git config --local ${safekey}",
    require => Package['git']
    }
    }

    View Slide

  62. git::config::local { ‘~/github/hubot’:
    key => ‘heroku.account’,
    value => ‘work’
    }

    View Slide

  63. Resources can be grouped with classes

    View Slide

  64. class redis {
    package { ‘redis-server’: }
    service { ‘redis’:
    require => Package[‘redis-server’]
    }
    }

    View Slide

  65. classes can mixin other classes

    View Slide

  66. classes can require other classes

    View Slide

  67. this matters a whole lot everybody

    View Slide

  68. Puppet has its own dsl for a couple reasons

    View Slide

  69. 1. puppet was designed for sysadmin-types

    View Slide

  70. 2. puppet has semantics about execution that
    would be confusing and unintuitive in ruby or
    python or most other languages

    View Slide

  71. puppet uses a DAG to apply resources

    View Slide

  72. a DAG is a directed, acyclic graph

    View Slide

  73. a DAG means relationships matter

    View Slide

  74. a DAG also means order of execution does not matter

    View Slide

  75. View Slide

  76. The Setup as a standard library for puppet
    but designed for personal laptops

    View Slide

  77. class github::projects::octostatus {
    require redis
    ...
    }

    View Slide

  78. class github::projects::octostatus {
    ...
    $dir = "${github::config::srcdir}/octostatus"
    ...
    }

    View Slide

  79. class github::projects::octostatus {
    ...
    git::repo { $dir:
    source => 'github/octostatus'
    }
    ...
    }

    View Slide

  80. class github::projects::octostatus {
    ...
    ruby::local { $dir:
    version => '1.9.3-p194',
    require => Git::Repo[$dir]
    }
    }

    View Slide

  81. I like doing things my way

    View Slide

  82. class github::people::hubot {
    include github::projects::all
    }

    View Slide

  83. class github::people::keavy {
    require gitx
    require onepassword
    require sizeup
    require sparrow
    require textmate
    }

    View Slide

  84. class github::people::yossef {
    package {
    'Alfred':
    provider => 'appdmg',
    source => $alfred_dmg_url;
    }
    }

    View Slide

  85. class github::people::yossef {
    setup::tarballed_app {
    'Mailplane':
    source => $mailplane_url;
    }
    }

    View Slide

  86. class github::people::wfarr {
    git::config::global {
    'alias.st':
    value => 'status';
    'alias.ci':
    value => 'commit';
    }
    }

    View Slide

  87. class github::people::wfarr {
    file { '/Users/wfarr/.gitignore':
    ensure => present,
    content => '.DS_STORE'
    }
    }

    View Slide

  88. class github::people::wfarr {
    package { 'emacs': }
    git::repo { '/Users/wfarr/.emacs.d':
    source => 'wfarr/.emacs.d',
    submodules => true
    }
    }

    View Slide

  89. class github::people::wfarr {
    setup::osx_defaults { 'require password always':
    ensure => present,
    domain => 'com.apple.screensaver',
    key => 'askForPassword',
    value => 1,
    user => 'wfarr'
    }
    }

    View Slide

  90. lessons

    View Slide

  91. puppeteer
    supportocat

    View Slide

  92. people will do crazy shit to it

    View Slide

  93. if it breaks it is your job to fix it

    View Slide

  94. if it breaks it is your job to fix it
    no matter what

    View Slide

  95. devs are really good at breaking things

    View Slide

  96. people report about 1% of issues they have

    View Slide

  97. people report about 1% of issues they have
    on a really good day

    View Slide

  98. View Slide

  99. rapid iteration on bugs makes people happy

    View Slide

  100. avoiding more bugs makes us happy

    View Slide

  101. reduce potential for bugs by vendoring

    View Slide

  102. EVERYTHING

    View Slide

  103. View Slide

  104. » du -sh /opt/github
    3.8G /opt/github

    View Slide

  105. there’s

    View Slide

  106. View Slide

  107. =(

    View Slide

  108. » env | grep GH_ | wc -l
    24

    View Slide

  109. » env | grep GH_REDIS_PORT
    GH_REDIS_PORT=16379

    View Slide

  110. » env | grep GH_MYSQL_PORT
    GH_MYSQL_PORT=13306

    View Slide

  111. View Slide

  112. people just will never remember to update

    View Slide

  113. tell people when they should update

    View Slide

  114. View Slide

  115. omg sneak
    peek time

    View Slide

  116. => Installing qt...

    View Slide

  117. View Slide

  118. View Slide

  119. => Fetching snapshot...
    => Installing qt...

    View Slide

  120. => Fetching snapshot...
    => Installing ruby-1.9.3...

    View Slide

  121. => Fetching snapshot...
    => Installing node-0.6.8...

    View Slide

  122. we don't have the ability to
    mandate environments

    View Slide

  123. we have to provide something
    dramatically better than just
    doing it yourself

    View Slide

  124. prim
    donn

    View Slide

  125. the setup is not a tool for it administrators

    View Slide

  126. the setup is a tool for people to get stuff done

    View Slide

  127. the setup is extensible by users

    View Slide

  128. the setup is a stable framework for config

    View Slide

  129. the setup is about enabling good patterns

    View Slide

  130. the setup is not about dictating machine policy

    View Slide

  131. how do i get it

    View Slide

  132. you don’t =(

    View Slide

  133. ... yet

    View Slide

  134. we are working
    hard to open
    source it

    View Slide

  135. Soon!

    View Slide

  136. Setup
    the

    View Slide

  137. questions

    View Slide

  138. love ops? come work with us
    puppet, erlang, ruby, shell, c
    and all the graphs you can shake a stick at

    View Slide

  139. View Slide

  140. thanks

    View Slide