Upgrade to Pro — share decks privately, control downloads, hide ads and more …

GitHub: Behind the Feature

GitHub: Behind the Feature

In the beginning of 2013, we shipped a new version of Search to GitHub.com. This talk takes a peek behind the scenes to illustrate how we build features at GitHub. It's a good beginner-level look at how startups work in teams together and deploy a product to the general public.


Zach Holman

March 28, 2013


  1. github Behind the Feature

  2. None
  3. Search

  4. January 2013 Search

  5. 4-9 months Search

  6. elasticsearch Search

  7. design Search

  8. @jonrohan @TwP

  9. @holman 

  10. Planning

  11. We knew search sucked

  12. Start by talking about it

  13. None
  14. From: tim.pease@github.com Subject: Search Results If you have a spare

    moment, go to github.com/search. These search pages need some lovin’ in a bad, bad way.
  15. Backend Two-person teams

  16. Two-person teams Frontend

  17. Building

  18. Okay, let’s build something


  20. git checkout -b search-next BRANCHES

  21. Really simple branching BRANCHES

  22. Cut a branch from master BRANCHES (rarely from non-master branch)

  23. Push whenever; let others help BRANCHES

  24. No need for complicated forks BRANCHES

  25. COMMITS

  26. git commit -m “Design Search” COMMITS

  27. git commit -m “:lipstick:” COMMITS

  28. COMMITS

  29. We don’t rebase or do any fancy merge strategies COMMITS

  30. Small, incremental commits COMMITS


  32. Open a Pull Request PULL REQUESTS

  33. None
  34. Sometimes we open pulls before code PULL REQUESTS

  35. PULL REQUESTS Sometimes we open pulls after code

  36. PULL REQUESTS Sometimes pulls aren’t for merging

  37. 130 commits 10 contributors 2 months

  38. PULL REQUESTS Let your pulls evolve


  40. None
  41. None
  42. MENTIONS Low-overhead way to ask for help

  43. MENTIONS Non-intrusive way to ask for help

  44. MENTIONS Feel good about a ship by mentioning the relevant

  45. TESTING

  46. TESTING Tests are kinda critical

  47. TESTING Every commit we push triggers three builds: ruby 1.8

    ruby 1.9 github enterprise
  48. Build notifications

  49. TESTING Make it hard to ignore failures

  50. Shipping

  51. STAFF-SHIPPING Limit your exposure

  52. STAFF-SHIPPING We have ~35 features staff-shipped

  53. STAFF-SHIPPING My GitHub looks different from yours:


  55. STAFF-SHIPPING Ranges from four-line config changes to entire feature redesigns

  56. STAFF-SHIPPING Coworkers are the only real beta testers

  57. STAFF-SHIPPING def search_enabled? current_user.staff? end master

  58. STAFF-SHIPPING def search_enabled? true end ship-search

  59. STAFF-SHIPPING No risky deploys; it’s already production-tested

  60. STAFF-SHIPPING Search’s design was staff-shipped for two months

  61. STAFF-SHIPPING Search’s backend was staff-shipped for four months

  62. STAFF-SHIPPING Gist 2.0 was staff-shipped for a year

  63. STAFF-SHIPPING More real-world usage means fewer surprises

  64. STAFF-SHIPPING Shipping is scary; reduce surprises


  66. DEPLOYMENT Deploying is scary; reduce surprises

  67. DEPLOYMENT Deploying to production involves 80 servers and hundreds of

    Unicorn processes and magic and it’s a wonder this stuff works
  68. DEPLOYMENT We want it easy enough for a designer to

  69. github/hubot

  70. DEPLOYMENT hubot deploy github

  71. DEPLOYMENT Hubot talks to an API that knows how to

    deploy every app
  72. DEPLOYMENT hubot deploy github/ship-search

  73. DEPLOYMENT Deploy a branch; rollback by deploying master

  74. DEPLOYMENT hubot deploy github/master

  75.  CHATOPS

  76.  CHATOPS Chat’s important to us

  77. None
  78.  CHATOPS Search didn’t need standup meetings

  79.  CHATOPS Chat is asynchronous and non-blocking

  80.  CHATOPS “Chatops”

  81.  CHATOPS Transparent way of handling ops

  82.  CHATOPS Search deployed, check perf

  83.  CHATOPS /graph me -1h substr(es-*_github.load)

  84.  CHATOPS

  85.  CHATOPS load on es-storage15 is higher than the other

    storage nodes
  86.  CHATOPS let's let it go a bit- could just

    be due to relocating shards causing iowait
  87.  CHATOPS Working over SSH is non-collaborative

  88.  CHATOPS Makes it obvious what needs automation and tooling

  89. Maintaining

  90.  POST-SHIP

  91.  POST-SHIP Keep shipping

  92. None
  93.  BUGS

  94.  BUGS Keep fixing

  95. None
  96.  BUGS Don’t assume someone else will fix ‘em

  97. Thanks.