ChatOps at GitHub

ChatOps at GitHub

Hubot, GitHub's open source chat bot, is completely revolutionizing how we do ops at GitHub. Automating deployment, graphing, monitoring, provisioning, tweeting, and many other things with Hubot has enabled and enhanced our culture of remote work. We're standardizing how we interact with the applications and servers that run GitHub by continuously expanding our library of Hubot commands that any GitHubber can run in multiple Campfire chat rooms. Interacting with Hubot in the middle other of conversations has increased our awareness of what other GitHubbers are working on and the speed at which new team members learn common practices. In this talk, I'll give examples of how we deploy and interact with github.com via Hubot, describe benefits we've seen from this approach, and describe how you can bring ChatOps into your daily workflow.

56348b545d905e840ef32db4a1c85eed?s=128

Jesse Newland

February 07, 2013
Tweet

Transcript

  1. at ChatOps GitHub

  2. Jesse Newland jnewland

  3. at ChatOps GitHub

  4. *

  5. None
  6. None
  7. git checkout -b git-gh13 # bump version number git commit

    -am ‘bump git version’ git push origin git-gh13 # create pull request github/puppet
  8. Build #108816 (5fe75932f26ea62cb5fc5e3d0cb302cc2461d11e) of puppet/git-gh13 was successful(421s) github/ puppet@567ea48...5fe7593

  9. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  10. ** [out :: REDACTED ] Bootstrapping... ** [out :: REDACTED

    ] Gem environment up-to-date. ** [out :: REDACTED ] Running librarian-puppet... ** [out :: REDACTED ] Generating puppet environments... ** [out :: REDACTED ] Cleaning up deleted branches... ** [out :: REDACTED ] Done! ** [out :: REDACTED ] Sending 'restart' command ** [out :: REDACTED ] The following watches were affected: ** [out :: REDACTED ] puppetmaster_unicorn ...
  11. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  12. ... ** [out :: fs1a.rs.github.com] info: Applying configuration version '8fb1a2716d5f950b836e511471a2bdac3ed27090'

    ** [out :: fs1a.rs.github.com] notice: /Stage[main]/ Github::Common_packages/Package[git]/ensure: would have changed from '1:1.7.10-1+github12' to '1:1.7.10-1+github13' ...
  13. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  14. ** [out :: fs1a.rs.github.com] info: Applying configuration version ** [out

    :: fs1a.rs.github.com] notice: /Stage[main]/ Github::Common_packages/Package[git]/ensure: ensure changed '1:1.7.10-1+github12' to '1:1.7.10-1+github13' ...
  15. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  16. None
  17. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  18. [puppet/master] Merge pull request #8406 from github/git-gh13 - Jesse Newland

    ( github/puppet@567ea48 )
  19. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  20. Build #108817 (5fe75932f26ea62cb5fc5e3d0cb302cc2461d11e) of puppet/master was successful(399s)

  21. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  22. jnewland is deploying puppet/master (863f7ff..567ea48) to production [#37789]

  23. ** [out :: REDACTED ] Bootstrapping... ** [out :: REDACTED

    ] Gem environment up-to-date. ** [out :: REDACTED ] Running librarian-puppet... ** [out :: REDACTED ] Generating puppet environments... ** [out :: REDACTED ] Cleaning up deleted branches... ** [out :: REDACTED ] Done! ** [out :: REDACTED ] Sending 'restart' command ** [out :: REDACTED ] The following watches were affected: ** [out :: REDACTED ] puppetmaster_unicorn ...
  24. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs*) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  25. ** [out :: fs2a.rs.github.com] info: Applying configuration version '8fb1a2716d5f950b836e511471a2bdac3ed27090' **

    [out :: fs7b.rs.github.com] info: Applying configuration version '8fb1a2716d5f950b836e511471a2bdac3ed27090' ** [out :: fs2a.rs.github.com] notice: /Stage[main]/ Github::Common_packages/Package[git]/ensure: ensure changed '1:1.7.10-1+github12' to '1:1.7.10-1+github13' ** [out :: fs7b.rs.github.com] notice: /Stage[main]/ Github::Common_packages/Package[git]/ensure: ensure changed '1:1.7.10-1+github12' to '1:1.7.10-1+github13' ...
  26. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  27. None
  28. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  29. None
  30. Boom

  31. ChatOps

  32.  Hubot

  33. heaven janky shell graphme Hubot

  34. hubot heaven janky shell graphme ALL OF THE APIS

  35. Why is this stupid chat bot so important to Ops?

  36. hubot deploy puppet to production puppet noop git-gh13 fs1 puppet

    force git-gh13 fs1 graph me -1h @collectd.load(fs1) # merge pull request # ci runs # auto-deploy hubot puppet force production fs graph me -1h @collectd.load(fs*) log me hooks github/github
  37. Everyone sees all of that happen

  38. Everyone sees all of that happen on their first day

  39. None
  40. hubot ci status github/smoke-perf

  41. hubot deploy github/smoke-perf to prod/fe1

  42. hubot graph me -10min @app-perf

  43. hubot procs unicorn

  44. hubot resque critical

  45. hubot graph me -10min @collectd.load(fe*)

  46. hubot conns fe1

  47. hubot log me smoke fe1

  48. hubot lbctl disable fe1

  49. hubot status yellow Bad deploy. Reverting now.

  50. hubot who’s on call

  51. hubot pingdom checks

  52. hubot upset me

  53. hubot deploy github to prod/fe1

  54. hubot log me smoke fe1

  55. hubot air drum me

  56. hubot lbctl enable fe1

  57. hubot status green All systems go.

  58. hubot whois 4.9.23.22

  59. hubot khanify spammers

  60. hubot play in the air tonight

  61. ChatOps

  62. By placing tools directly in the middle of the conversation

  63. Everyone is pairing all of the time

  64. Teaching by doing

  65. This was always my main motivation with hubot - teaching

    by doing by making things visible. It's an extremely powerful teaching technique - @rtomayko
  66. None
  67. Communicate by doing

  68. THINGS I HAVEN’T ASKED RECENTLY

  69. THINGS I HAVEN’T ASKED RECENTLY how’s that deploy going?

  70. THINGS I HAVEN’T ASKED RECENTLY how’s that deploy going? are

    you deploying that or should i?
  71. THINGS I HAVEN’T ASKED RECENTLY how’s that deploy going? are

    you deploying that or should i? is anyone responding to that nagios alert?
  72. THINGS I HAVEN’T ASKED RECENTLY how’s that deploy going? are

    you deploying that or should i? is anyone responding to that nagios alert? is that branch green?
  73. THINGS I HAVEN’T ASKED RECENTLY is that branch green? how’s

    that deploy going? are you deploying that or should i? is anyone responding to that nagios alert? how does load look?
  74. THINGS I HAVEN’T ASKED RECENTLY is that branch green? how’s

    that deploy going? are you deploying that or should i? is anyone responding to that nagios alert? how does load look? did anyone update the status page?
  75. THINGS I HAVEN’T ASKED RECENTLY how’s that deploy going? are

    you deploying that or should i? is anyone responding to that nagios alert? is that branch green? how does load look? did that deploy finish? did anyone update the status page?
  76. None
  77. None
  78. http://www.flickr.com/photos/7997249@N06/6061305639/

  79. Hide the ugly

  80. None
  81. [nines] hubot opened issue #4263: Nagios (229906) - fs3b/syslog -

    Tue Sept 25 23:40:18 PDT 2012. github/nines#4263
  82. hubot nagios ack fs3b/syslog # fix stuff nagios check fs3b/syslog

    nagios status fs3b/syslog hubot nagios downtime fs3b/syslog 90 nagios mute fs3b/syslog nagios unmute fs3b/syslog
  83. None
  84. None
  85. Mobile FTW

  86. None
  87. ChatOps

  88. None
  89. None
  90. Work at GitHub jesse@github.com