Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Unsucking Your Team's Development Environment

Zach Holman
September 14, 2012

Unsucking Your Team's Development Environment

Success can bring many glamorous changes to your company: hiring more employees, getting free coffee, and giving everyone a private jet filled with cash and endangered African predatory cats.

Success can lead to less-glamorous problems, though. As you grow, your team's development environment becomes really important. How long does it take to clone, set up, and boot your apps? Can your employees still be productive on an aging codebase? How can you automate CI, hooks, and other setup for new projects? Is any of this fun anymore?

GitHub ran into these problems as we expanded our team tremendously over the last two years. Let's look at some of the ways we've improved our employees' development environments.

Zach Holman

September 14, 2012
Tweet

More Decks by Zach Holman

Other Decks in Programming

Transcript

  1. unsucking
    YOUR TEAM’S
    ENVIRONMENT
    DEVELOPMENT

    View Slide

  2. NO
    FORKING
    HOLMAN
    @
    LOST
    YO QUIT READING THIS SHIT

    View Slide

  3. githu

    View Slide

  4. WE STARTED WITH
    one QUEUE

    View Slide

  5. THIS QUEUE WORKED SORTA
    OBJECTS IN,
    TASK OCCURS

    View Slide

  6. OH SHIT
    BLOCKING
    OPERATIONS

    View Slide

  7. THEN SPLIT INTO 2 QUEUES
    SEGREGATE BY
    CATEGORY

    View Slide

  8. 2 QUEUES, 3x THROUGHPUT
    EXPONENTIAL
    GROWTH

    View Slide

  9. SATURATED AVAILABILITY
    EXPERIENCING
    BOTTLENECKS

    View Slide

  10. THIS IS GITHUB’S GODDAMN
    TOILET SITUATION

    View Slide

  11. TWO GENDERED BATHROOMS
    ZOMG WHY
    AM I WAITING

    View Slide

  12. It’s because of this:

    View Slide

  13. GITHUB EMPLOYEES
    5 8
    2009
    2008

    View Slide

  14. GITHUB EMPLOYEES
    2009 2010
    5 8 14
    2008

    View Slide

  15. GITHUB EMPLOYEES
    2009 2010 2011
    5 8
    14 54
    2008

    View Slide

  16. GITHUB EMPLOYEES
    2008 2009 2010 2011 2012
    5 8
    14 54
    114

    View Slide

  17. WE’VE THOUGHT A LOT
    ABOUT ONBOARDING

    View Slide

  18. IT’S NOT JUST
    TOILET LATENCY
    ABOUT

    View Slide

  19. 2.5 YEARS AGO:
    1 WEEK
    TODAY:
    20 MINUTES
    DURATION FROM HIRE TO BOOT
    (a )
    (every a )

    View Slide

  20. THINK THROUGH MAKING
    YOUR ENVIRONMENT
    SUCK LESS

    View Slide

  21. PAIN IS MULTIPLIED
    ACROSS YOUR
    WHOLE COMPANY

    View Slide

  22. THE APP
    THE STACK
    THE DEPLOY
    T H E C O M P A N Y

    View Slide

  23. THE APP
    THE STACK
    THE DEPLOY
    T H E C O M P A N Y

    View Slide

  24. I S
    Y O U R
    F O U N D A T I O N
    THE APP

    View Slide

  25. remembe :
    EVERY APP SHOULD HAVE A
    ONE-LINER INSTALL
    REALLY.

    View Slide

  26. EVEN LARGE PROJECTS CAN HAVE A
    TUTORIAL-STYLE SCRIPT
    remembe :

    View Slide

  27. THINGS
    SUCK
    LESS HOMEBREW
    FOREMAN
    RBENV
    BUNDLER

    View Slide

  28. AUTOMATE
    AUTOMATE
    AUTOMATE
    AUTOMATE
    AUTOMATE
    AUTOMATE

    View Slide

  29. APP BOOTSTRAP

    View Slide

  30. BOOTSTRAP
    REPLICATE-REPO
    CIBUILD
    SCRIPT/

    View Slide

  31. BOOTSTRAP
    SCRIPT/
    ALL-IN-ONE SCRIPT
    THAT SETS UP
    EVERYTHING

    View Slide

  32. BOOTSTRAP
    SCRIPT/
    DEPENDENCY CHECKS + MIGRATIONS
    +
    BUNDLER ASSET COMPILATION

    View Slide

  33. BOOTSTRAP
    SCRIPT/
    S P E E D
    MD5 CHECKSUM:
    GEMFILE + ASSETS

    View Slide

  34. BOOTSTRAP
    SCRIPT/
    everywhere SCRIPT/SERVER
    SCRIPT/CONSOLE
    CI

    View Slide

  35. BOOTSTRAP
    SCRIPT/
    DESIGNERS ❤ THIS

    View Slide

  36. BOOTSTRAP
    SCRIPT/
    NON-TECHNICALS ❤ THIS

    View Slide

  37. BOOTSTRAP
    SCRIPT/
    DEVELOPERS ❤ THIS

    View Slide

  38. REPLICATE-REPO
    SCRIPT/
    A SUPPORT SCRIPT
    THAT LOADS
    PRODUCTION DATA
    LOCALLY

    View Slide

  39. REPLICATE-REPO
    SCRIPT/
    FLY YOUR
    PRODUCTION AR
    ASSOCIATIONS TO
    YOU IN DEVELOPMENT
    an rail
    dhh

    View Slide

  40. REPLICATE-REPO
    SCRIPT/
    an rail

    View Slide

  41. REPLICATE-REPO
    SCRIPT/
    FOR US, IT’S REPLICATE-REPO
    REPLICATE-#{OBJECT}
    BUT IT COULD BE YOUR

    View Slide

  42. REPLICATE-REPO
    SCRIPT/
    /RTOMAYKO/REPLICATE
    HTTP://

    View Slide

  43. REPLICATE-REPO
    SCRIPT/
    DUMPS BELONGS_TO &
    HAS_ONE ASSOCIATIONS

    View Slide

  44. REPLICATE-REPO
    SCRIPT/
    $ script/replicate-repo holman/spark
    CLONES ISSUES, PULL REQUESTS,
    USERS, GIT DATA, ETC.

    View Slide

  45. REPLICATE-REPO
    SCRIPT/
    AUTOMATE
    AUTOMATE
    AUTOMATE
    AUTOMATE
    AUTOMATE
    AUTOMATE
    BORING
    SUPPORT TASKS

    View Slide

  46. CIBUILD
    SCRIPT/
    OUR
    CONTINUOUS INTEGRATION
    ENVIRONMENT
    PREPARATION SCRIPT

    View Slide

  47. CIBUILD
    SCRIPT/
    SET ENV VARS + LOAD GC CONFIG
    +
    CLEAN WORK DIR BOOTSTRAP

    View Slide

  48. CIBUILD
    SCRIPT/
    VERSION CONTROL
    you TEST ENVIRONMENT

    View Slide

  49. USE
    COMPUTERS
    TO DO
    THINGS

    View Slide

  50. USE
    HUMANS
    TO DRINK
    WITH

    View Slide

  51. USE
    COMPUTERS
    TO DO
    THINGS

    View Slide

  52. THE STACK
    THE APP
    THE DEPLOY
    T H E C O M P A N Y

    View Slide

  53. { GROUP PARTICIPATION TIME }

    View Slide

  54. EVERYONE LIFT YOUR

    View Slide

  55. OKAY NOW
    SMASH THEM

    View Slide

  56. NO?

    View Slide

  57. OKAY, MAYBE THEY’RE
    EXPENSIVE, BUT THEY’RE
    ALSO A PAIN TO SET UP

    View Slide

  58. MACHINES SHOULD BE
    expendable

    View Slide

  59. WE’VE DONE THAT

    View Slide

  60. I COULD
    SMASH
    THIS
    LAPTOP

    View Slide

  61. JUST KIDDING
    OMG I’D GET MURDERED

    View Slide

  62. AUTOMATED
    MACHINE
    SETUP

    20 min

    View Slide

  63. setu
    THE

    View Slide

  64. FIRST DAY
    GET A LAPTOP
    RUN A ONE-LINER
    HAVE A BEER
    HACK

    View Slide

  65. $ gh-setup project
    < >

    View Slide

  66. DEPENDENCIES PROCESSES
    +
    fo 50 SERVICES
    & APPS

    View Slide

  67. APPS
    DROPBOX HOMEBREW
    + +
    CHROME + 1PASSWORD + ETC.

    View Slide

  68. DEPENDENCIES
    MYSQL NODE
    + + REDIS +
    POSTGRES
    RIAK
    + + RBENV + ETC.

    View Slide

  69. +
    CONFIG
    FULL-DISK ENCRYPTION
    SCREEN LOCKS
    VPN
    + ETC.

    View Slide

  70. PUPPET

    View Slide

  71. AUTO-REPORTING
    OF ANY FAILURE TO GITHUB ISSUES

    View Slide

  72. USE
    COMPUTERS
    TO DO
    THINGS

    View Slide

  73. THE DEPLOY
    THE APP
    THE STACK
    T H E C O M P A N Y

    View Slide

  74. remembe :
    DON’T HANDLE
    YOUR OWN DEPLOYS

    View Slide

  75. HUBOT
    HEAVEN
    +

    View Slide

  76. HUBOT
    NODE.JS CHAT BOT
    HUNDREDS OF PLUGINS

    View Slide

  77. HUBOT DEPLOY X TO Y

    View Slide

  78. HUBOT DEPLOY X TO Y
    DEPLOY GITHUB TO PRODUCTION
    DEPLOY GITHUB TO PRODUCTION/FE
    DEPLOY SMOKE TO FS12,FE1

    View Slide

  79. HUBOT DEPLOY X TO Y
    DEPLOY GITHUB TO PRODUCTION
    DEPLOY GITHUB TO PRODUCTION/FE
    DEPLOY SMOKE/HOTFIX TO FS12,FE1

    View Slide

  80. HUBOT
    HEAVEN
    +

    View Slide

  81. HEAVEN
    CAPISTRANO OR HEROKU
    SINATRA APP
    API TO HUBOT
    RUNS DEPLOYS ON OUR BOXES

    View Slide

  82. HEAVEN
    DEPLOY LOCKING
    1. YOU PUSH A BRANCH
    2. CI STARTS A BUILD
    3. BUILD PASSES
    4. YOU DEPLOY BRANCH
    5. HEAVEN PREVENTS DEPLOY CONFLICTS
    6.YOU MERGE BRANCH
    7. HEAVEN UNLOCKS DEPLOYS

    View Slide

  83. HUBOT
    HEAVEN
    +
    FLEXIBLE
    SAFE
    EASY

    View Slide

  84. USE
    COMPUTERS
    TO DO
    THINGS

    View Slide

  85. T H E C O M P A N Y
    THE APP
    THE STACK
    THE DEPLOY

    View Slide

  86. remembe :
    TAKE AWAY
    THE BULLSHIT

    View Slide

  87. JUST BEING AN EMPLOYEE
    ADDS A LOT OF OVERHEAD
    LOGINS
    SITES
    TERMS
    REPORTS PAYROLL
    NAMES COMMUNICATION

    View Slide

  88. REDUCE IT
    OAUTH EVERYTHING
    50+ INTERNAL APPS UNDER OAUTH

    View Slide

  89. REDUCE IT
    BUILD HELPERS
    TEAM APP HELPS US COMMUNICATE

    View Slide

  90. CLARIFY IT
    WRITE A GUIDE
    FIRST-DAY GUIDE TO THE COMPANY

    View Slide

  91. CLARIFY IT
    MAKE IT ACCESSIBLE
    MONITORS APP AND OFFICE TVS

    View Slide

  92. USE
    COMPUTERS
    TO BUILD
    THINGS THAT CAN HELP YOUR
    ORGANIZATION SCALE TO THE EXTENT
    THAT NEW HIRES ACCLIMATE
    WITHOUT A LOT OF STRESS

    View Slide

  93. THE APP
    THE STACK
    THE DEPLOY
    T H E C O M P A N Y

    View Slide

  94. AUTOMATE EVERYTHING

    View Slide

  95. WE HAVE ZERO MANAGERS
    BECAUSE WE AUTOMATE

    View Slide

  96. LONG-TERM BENEFIT
    WILL ALWAYS OUTWEIGH
    SHORT-TERM COST

    View Slide

  97. THANKS

    View Slide

  98. NO
    FORKING
    HOLMAN
    @
    LOST
    YO QUIT READING THIS SHIT
    ZACHHOLMAN.COM/TALKS

    View Slide