Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Setup: Managing an Army of Laptops with Puppet

The Setup: Managing an Army of Laptops with Puppet

As a rapidly growing company, GitHub's faced some challenges in how to make sure that everyone can get up and running on projects or continue to work on one of the dozens of existing projects if they've never set it up before. What a nightmare! That's what prompted @jbarnette and @wfarr to develop The Setup. The Setup aims to solve all the problems that used to plague on-boarding GitHubbers onto projects with a lot of automation and a case of the "It Just Works". This presentation talks about the history of The Setup, how it works, and some of the lessons learned along the way.

Will Farrington

September 12, 2012
Tweet

More Decks by Will Farrington

Other Decks in Technology

Transcript

  1. managing an
    army of laptops
    ...with puppet

    View full-size slide

  2. hi, my name is will farrington
    but @wfarr is way shorter

    View full-size slide

  3. i work at a “little-known startup” called
    github

    View full-size slide

  4. I work on System operations
    System operations
    internal apps
    the setup

    View full-size slide

  5. once
    upon a
    time

    View full-size slide

  6. there
    was a
    developer

    View full-size slide

  7. there
    was a
    designer

    View full-size slide

  8. oh man, this new macbook pro is sweet! let me run
    a bunch of stuff by hand so I can get some work
    done in about 6 to 8 hours, maybe.
    - some perfectly normal people

    View full-size slide

  9. once
    upon a
    time

    View full-size slide

  10. there
    was a
    crazy
    person

    View full-size slide

  11. oh man, this new macbook pro is sweet! let me run
    a bunch of stuff by hand so I can get some work
    done in about 6 to 8 hours, maybe.
    ~ crazy people

    View full-size slide

  12. we made a
    HUGE
    mistake

    View full-size slide

  13. yep, it’s
    pretty
    bad

    View full-size slide

  14. i thought we cared about testing

    View full-size slide

  15. i thought we cared about repeatability

    View full-size slide

  16. i thought we cared about automation

    View full-size slide

  17. we say those things matter for software
    and they totally do

    View full-size slide

  18. we do not treat our laptops like they matter

    View full-size slide

  19. wtf is wrong with us

    View full-size slide

  20. wtf is wrong with us
    i mean srsly guise

    View full-size slide

  21. spock
    approves
    this
    message

    View full-size slide

  22. a living, breathing software project
    that tests and automates every little
    bit of your machine with love

    View full-size slide

  23. about 6000
    about 3600
    1186
    26
    lines of Ruby (not incl. tests)
    lines of Puppet (not incl. tests)
    commits
    contributors
    some numbers

    View full-size slide

  24. chrome, colloquy, dropbox, elasticsearch, emacs,
    erlang, git, homebrew, hub, iterm2, java, macvim,
    memcache, mongodb, mysql, nginx, nodejs, nvm,
    1passwd, personalization, postgresql, projects,
    python, qt, rbenv, rdio, redis, ruby, solr, sparrow,
    textmate, virtualbox, viscosity, wget, xcode,
    xquartz, zeromq, zsh, and more daily.

    View full-size slide

  25. less than 6 months

    View full-size slide

  26. a githubber’s first day
    in

    View full-size slide

  27. hours of frustration getting
    github.com and other projects
    even working locally

    View full-size slide

  28. a githubber’s first day
    now

    View full-size slide

  29. drink coffee for 20 minutes

    View full-size slide

  30. The Command Line Interface

    View full-size slide

  31. » gh-setup --projects
    * github

    View full-size slide

  32. github » script/bootstrap
    github » script/server
    github » open github.dev
    # Ready to ship things!

    View full-size slide

  33. » gh-setup octostatus
    # sets it all up. boom.

    View full-size slide

  34. » gh-setup --stealth
    # development mode

    View full-size slide

  35. The Puppet DSL

    View full-size slide

  36. The Puppet DSL is pretty easy actually

    View full-size slide

  37. Resources are the basic unit of puppet

    View full-size slide

  38. package { ‘redis-server’: }

    View full-size slide

  39. file { ‘/etc/motd’:
    content => ‘Hello, world!’
    }

    View full-size slide

  40. service { ‘riak’:
    ensure => running,
    subscribe => File[‘/etc/riak/vm.args’]
    }
    resource type

    View full-size slide

  41. service { ‘riak’:
    ensure => running,
    subscribe => File[‘/etc/riak/vm.args’]
    }
    resource name

    View full-size slide

  42. service { ‘riak’:
    ensure => running,
    subscribe => File[‘/etc/riak/vm.args’]
    }
    resource parameters

    View full-size slide

  43. service { ‘riak’:
    ensure => running,
    subscribe => File[‘/etc/riak/vm.args’]
    }
    metaparameter

    View full-size slide

  44. metaparameters work on every resource

    View full-size slide

  45. metaparameters describe relationships

    View full-size slide

  46. before
    require
    subscribe
    notify
    this resource must run before
    this resource must run after
    this resource runs after and watches
    this resource runs before and tells

    View full-size slide

  47. resources can have custom types

    View full-size slide

  48. define git::config::local($key, $value) {
    $safekey = shellquote($key)
    $safevalue = shellquote($value)
    $command = "git config --local ${safekey} ${safevalue}"
    exec { "${command} in ${name}":
    command => $command,
    cwd => $name,
    onlyif => 'test -d .git',
    unless => "git config --local ${safekey}",
    require => Package['git']
    }
    }

    View full-size slide

  49. git::config::local { ‘~/github/hubot’:
    key => ‘heroku.account’,
    value => ‘work’
    }

    View full-size slide

  50. Resources can be grouped with classes

    View full-size slide

  51. class redis {
    package { ‘redis-server’: }
    service { ‘redis’:
    require => Package[‘redis-server’]
    }
    }

    View full-size slide

  52. classes can mixin other classes

    View full-size slide

  53. classes can require other classes

    View full-size slide

  54. this matters a whole lot everybody

    View full-size slide

  55. Puppet has its own dsl for a couple reasons

    View full-size slide

  56. 1. puppet was designed for sysadmin-types

    View full-size slide

  57. 2. puppet has semantics about execution that
    would be confusing and unintuitive in ruby or
    python or most other languages

    View full-size slide

  58. puppet uses a DAG to apply resources

    View full-size slide

  59. a DAG is a directed, acyclic graph

    View full-size slide

  60. a DAG means relationships matter

    View full-size slide

  61. a DAG also means order of execution does not matter

    View full-size slide

  62. The Setup as a standard library for puppet
    but designed for personal laptops

    View full-size slide

  63. class github::projects::octostatus {
    require redis
    ...
    }

    View full-size slide

  64. class github::projects::octostatus {
    ...
    $dir = "${github::config::srcdir}/octostatus"
    ...
    }

    View full-size slide

  65. class github::projects::octostatus {
    ...
    git::repo { $dir:
    source => 'github/octostatus'
    }
    ...
    }

    View full-size slide

  66. class github::projects::octostatus {
    ...
    ruby::local { $dir:
    version => '1.9.3-p194',
    require => Git::Repo[$dir]
    }
    }

    View full-size slide

  67. I like doing things my way

    View full-size slide

  68. class github::people::hubot {
    include github::projects::all
    }

    View full-size slide

  69. class github::people::keavy {
    require gitx
    require onepassword
    require sizeup
    require sparrow
    require textmate
    }

    View full-size slide

  70. class github::people::yossef {
    package {
    'Alfred':
    provider => 'appdmg',
    source => $alfred_dmg_url;
    }
    }

    View full-size slide

  71. class github::people::yossef {
    setup::tarballed_app {
    'Mailplane':
    source => $mailplane_url;
    }
    }

    View full-size slide

  72. class github::people::wfarr {
    git::config::global {
    'alias.st':
    value => 'status';
    'alias.ci':
    value => 'commit';
    }
    }

    View full-size slide

  73. class github::people::wfarr {
    file { '/Users/wfarr/.gitignore':
    ensure => present,
    content => '.DS_STORE'
    }
    }

    View full-size slide

  74. class github::people::wfarr {
    package { 'emacs': }
    git::repo { '/Users/wfarr/.emacs.d':
    source => 'wfarr/.emacs.d',
    submodules => true
    }
    }

    View full-size slide

  75. class github::people::wfarr {
    setup::osx_defaults { 'require password always':
    ensure => present,
    domain => 'com.apple.screensaver',
    key => 'askForPassword',
    value => 1,
    user => 'wfarr'
    }
    }

    View full-size slide

  76. puppeteer
    supportocat

    View full-size slide

  77. people will do crazy shit to it

    View full-size slide

  78. if it breaks it is your job to fix it

    View full-size slide

  79. if it breaks it is your job to fix it
    no matter what

    View full-size slide

  80. devs are really good at breaking things

    View full-size slide

  81. people report about 1% of issues they have

    View full-size slide

  82. people report about 1% of issues they have
    on a really good day

    View full-size slide

  83. rapid iteration on bugs makes people happy

    View full-size slide

  84. avoiding more bugs makes us happy

    View full-size slide

  85. reduce potential for bugs by vendoring

    View full-size slide

  86. » du -sh /opt/github
    3.8G /opt/github

    View full-size slide

  87. » env | grep GH_ | wc -l
    24

    View full-size slide

  88. » env | grep GH_REDIS_PORT
    GH_REDIS_PORT=16379

    View full-size slide

  89. » env | grep GH_MYSQL_PORT
    GH_MYSQL_PORT=13306

    View full-size slide

  90. people just will never remember to update

    View full-size slide

  91. tell people when they should update

    View full-size slide

  92. omg sneak
    peek time

    View full-size slide

  93. => Installing qt...

    View full-size slide

  94. => Fetching snapshot...
    => Installing qt...

    View full-size slide

  95. => Fetching snapshot...
    => Installing ruby-1.9.3...

    View full-size slide

  96. => Fetching snapshot...
    => Installing node-0.6.8...

    View full-size slide

  97. we don't have the ability to
    mandate environments

    View full-size slide

  98. we have to provide something
    dramatically better than just
    doing it yourself

    View full-size slide

  99. the setup is not a tool for it administrators

    View full-size slide

  100. the setup is a tool for people to get stuff done

    View full-size slide

  101. the setup is extensible by users

    View full-size slide

  102. the setup is a stable framework for config

    View full-size slide

  103. the setup is about enabling good patterns

    View full-size slide

  104. the setup is not about dictating machine policy

    View full-size slide

  105. how do i get it

    View full-size slide

  106. you don’t =(

    View full-size slide

  107. we are working
    hard to open
    source it

    View full-size slide

  108. love ops? come work with us
    puppet, erlang, ruby, shell, c
    and all the graphs you can shake a stick at

    View full-size slide