Addicted to Stable

Addicted to Stable

Closing keynote for RubyConf Brazil.

E13c31390e0369fcd5972292ce0e7b92?s=128

John Nunemaker
PRO

September 01, 2012
Tweet

Transcript

  1. Addicted to Stable a.k.a how to sleep like an octocat

  2. None
  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. Hundreds of Sites

  10. Hundreds of Sites Thousands of Decks

  11. Hundreds of Sites Thousands of Decks Millions of People

  12. Hundreds of Sites Thousands of Decks Millions of People Billions

    of Views
  13. “ People want features, but need stability.

  14. Sta·ble Not likely to change or fail

  15. App Code and Servers

  16. Testing Automate

  17. ?

  18. ? test/unit

  19. ? rspec test/unit

  20. ? rspec test/unit minitest

  21. YES

  22. design YES

  23. design YES change

  24. design YES change regression

  25. None
  26. None
  27. Work by me

  28. Work by me Pushed code fix

  29. None
  30. Work by robots

  31. Work by robots Closed issue

  32. Work by robots Closed issue Ran test suite

  33. Work by robots Closed issue Ran test suite Deployed code

  34. Backup and Restore Automate

  35. A Single Server Never trust

  36. None
  37. Replicate

  38. Replicate Promote

  39. Replicate Promote Practice

  40. “ Me, at 8pm on a Saturday I’m confirming the

    steps and order of commands.
  41. None
  42. Deleted Everything I accidentally Sincerely, Your Employee

  43. Deleted Everything I accidentally Sincerely, Your Customer

  44. users users.archived gauges gauges.archived

  45. hubot restore <repo>

  46. Deleted Everything I accidentally Sincerely, You

  47. None
  48. Deployment Automate

  49. None
  50. hubot deploy github

  51. hubot deploy speakerdeck

  52. hubot deploy gauges

  53. hubot deploy gauges/my-branch

  54. puppet https://github.com/puppetlabs/puppet

  55. moonshine https://github.com/railsmachine/moonshine

  56. HOSTFILTER=‘app7.foo.com’ \ cap production deploy:setup

  57. HOSTFILTER=‘app7.foo.com’ \ cap production deploy

  58. Exceptions Collect and Fix

  59. None
  60. None
  61. < 1 exception per million page views (gauges)

  62. Metrics Collect and Graph

  63. None
  64. None
  65. None
  66. None
  67. None
  68. None
  69. Alerts Email, Push and SMS

  70. None
  71. None
  72. None
  73. Nagios

  74. Pingdom Nagios

  75. Pager Duty Pingdom Nagios

  76. Failover Automate

  77. web1 web2 haproxy/heartbeat

  78. Tracking app3 app4 app5 app6 web1 web2

  79. Tracking app3 app4 app5 app6 web1 web2 - option httpchk

    GET /pulse/tracking
  80. Tracking app3 app4 app5 app6 web1 web2 class PulseApp <

    Sinatra::Base get '/pulse/tracking' do pulse :kestrel end end
  81. Tracking app3 app4 app5 app6 web1 web2 app1 app2 Reporting

  82. Tracking app3 app4 app5 app6 web1 web2 app1 app2 Reporting

    - option httpchk GET /pulse/reporting
  83. Tracking app3 app4 app5 app6 web1 web2 app1 app2 Reporting

    class PulseApp < Sinatra::Base get '/pulse/reporting' do pulse :mongo end end
  84. Tracking app3 app4 app5 app6 web1 web2 q1 q2 app1

    app2 Reporting
  85. Tracking app3 app4 app5 app6 web1 web2 q1 q2 Workers

    db1 db2 db3 app1 app2 Reporting
  86. Team Collaboration and Freedom

  87. 46 hubbers When I started in December

  88. 115 hubbers As of August 31, 2012 at 10:43am EST

  89. How is that stable???

  90. Collaborate Always

  91. Pull Requests Discussion + ❤

  92. None
  93. None
  94. Chat Campfire to your phone

  95. None
  96. Video Face to face and Pairing

  97. None
  98. Internal Tools Ideas, Hiring, Music and more!

  99. None
  100. None
  101. 300+ ideas

  102. 300+ ideas 3000+ comments

  103. 300+ ideas 3000+ comments 100+ people

  104. Communicate Always

  105. None
  106. None
  107. 4300+ statuses

  108. 4300+ statuses 4200+ comments

  109. 4300+ statuses 4200+ comments 100+ people

  110. None
  111. None
  112. 170 talks

  113. 170 talks 112 people

  114. 170 talks 112 people 2 summits/year

  115. None
  116. Document Always

  117. None
  118. None
  119. Empower Always

  120. None
  121. Anyone can view

  122. Anyone can view Anyone can commit

  123. Anyone can view Anyone can commit Anyone can deploy

  124. None
  125. 296 branches

  126. 296 branches 2980 commits

  127. 296 branches 2980 commits 307 issues

  128. IN ONE MONTH

  129. ONLY THE WEB APP

  130. Brand Surprise and Delight

  131. Feedback Stupid Easy

  132. None
  133. None
  134. support@github.com speakerdeck@github.com gauges@github.com

  135. Support Fast and Friendly

  136. None
  137. None
  138. None
  139. None
  140. Documentation Useful and Thorough

  141. None
  142. None
  143. None
  144. None
  145. Communication Honest and Open

  146. None
  147. None
  148. None
  149. If you present, GitHub covers the bill for you.

  150. If you present, GitHub covers the bill for you and

    a buddy.
  151. None
  152. “ On Saturday, June 2nd, we had a perfect storm

    of cascading failures resulting in around four hours of lost tracking data.
  153. “ We have commissioned a new database cluster with automatic

    failover that we hope to be using sometime next week.
  154. “ We hate that we lost your data. Know that

    we have learned from this and will rise from the ashes of your data loss like a GLORIOUS PHOENIX.
  155. Celebration Public and Genuine

  156. None
  157. None
  158. None
  159. None
  160. automate testing, backing up, restoring, deploying, collect exceptions, metrics, send

    alerts, fail gracefully, collaborate, communicate, document all the things, empower, get feedback, provide awesome support, communicate, celebrate
  161. The End.