ChatOps: Infrastructure As Conversation

ChatOps: Infrastructure As Conversation

Those tasked with building and maintaining resilient services and systems at scale are aware of the concepts behind “Infrastructure as Code”. Configuration management tools and services such as Puppet, Ansible, Chef, and others have become a staple in the provisioning, management, and more of critical components of systems worldwide.

With the growing popularity of ChatOps (the concept of moving actions and context related to Operations into a persistent group-chat tool) more and more teams are beginning to find even more efficiencies and benefits. What does it look like when teams have begun to manage much of their infrastructure from the interface of group-chat? What concerns and pitfalls must be considered? Where should teams begin their efforts towards a ChatOps approach? What’s on the horizon and beyond as more and more shift their actions away from the command-line-interface and web browsers and begin to leverage API’s, bots, and group-chat tools in a secure and consistent way?

516fcd20ab7b946f50090ce1d557638c?s=128

j.hand

May 17, 2016
Tweet

Transcript

  1. CHATOPS: INFRASTRUCTURE AS CONVERSATION

  2. JASON HAND jasonhand #ChatOps

  3. @jasonhand ~ #ChatOps https://www.cheatography.com/uploads/davidpol_1439395467_Agile%20Manifesto.png

  4. Delivery Pipeline Plan Code Build Test Release Deploy Operate Agile

    Infrastructure (DevOps) Continuous Delivery Continuous Integration Agile Development @jasonhand ~ #ChatOps
  5. @jasonhand ~ #ChatOps http://2.bp.blogspot.com/-3Vv68jBEYdo/VfGlfe3aB0I/AAAAAAAAAMo/vrJl7PBd8m0/s1600/contact-center-acd-bottleneck.jpg

  6. USER & CUSTOMER EXPECTATIONS Users expect working software, high availability,

    innovation, and a sense of empathy
  7. AVAILABILITY & RELIABILITY High availability is the result of high

    resiliency & reliability
  8. AVAILABILITY & RELIABILITY High resiliency & reliability is the result

    of improved MTTR Fail Fast == Respond & Recover Fast
  9. Continuous Delivery @jasonhand ~ #ChatOps

  10. Continuous Improvement @jasonhand ~ #ChatOps

  11. A CONTINUOUS IMPROVEMENT It is not enough to do your

    best; you must know what to do, and then do your best. - W. Edwards Deming Act Check Do @jasonhand ~ #ChatOps Plan
  12. @jasonhand ~ #ChatOps

  13. DevOps Buzzword Bingo Continuous Integration Continuous Testing Continuous Delivery Continuous

    Monitoring Continuous Deployment ChatOps Containerization Microservices High Availability Devs On-call Delivery Pipeline Configuration Management Infrastructure As Code Virtualization Immutable Infrastructure Feedback Loops Experimentation Culture Release Automation Lean Kanban @jasonhand ~ #ChatOps Measurement Sharing
  14. DEVOPS PRINCIPLES CULTURE AUTOMATION MEASUREMENT SHARING LEAN @jasonhand ~ #ChatOps

  15. What is ChatOps? ChatOps is an approach to communication that

    allows teams to collaborate and manage many aspects of their infrastructure, code, and data from the comfort and safety of a chat room. Through the use of chatbots and scripts, teams can execute commands, query information, and distribute knowledge across the entire organization and all from the space where conversations are already taking place, a chat room. “We're taking the things we're already doing and putting them in line with the conversations we're already having” - James Fryman (StackStorm) @jasonhand ~ #ChatOps
  16. 1 1 1 2 1 3 1 4 Origins IRC

    XMPP/ Jabber Campfire HipChat/ Slack/Flowdock api @jasonhand ~ #ChatOps “Conversation-driven development”
  17. A HUBOT: THE EARLY YEARS Placing tools right in the

    middle of the conversation - Jesse Newland (GitHub) - 2012 @jasonhand ~ #ChatOps
  18. Problems Devs & Ops Teams Have Problems Our Bots Solve

    Maturation of ChatOps : Phase 1 adopted from @davidjbland @jasonhand ~ #ChatOps
  19. Problems Devs & Ops Teams Have Problems Our Bots Solve

    Maturation of ChatOps : Phase 2 adopted from @davidjbland @jasonhand ~ #ChatOps
  20. GitHub Pull Request Notifier Merge code Show current status Display

    “commit” link Commit Notifier Display “issue” link Display repo commuters Show Open Pull Requests Show Repo Activity Show open issues Search for string in code @jasonhand ~ #ChatOps
  21. 1 1 1 2 1 3 1 4 THE NEW

    INTERFACE Notifications/ Information Alerting/ Paging Command Execution Collaboration/ Communication @jasonhand ~ #ChatOps Feedback Loops Lower MTTR Higher availability & happier users
  22. A THE NEW INTERFACE A tool to help the way

    our people systems work that in turn helps the way our technical systems work. - Mark Imbriaco (operable.io) https://youtu.be/pCVvYCjvoZI @jasonhand ~ #ChatOps
  23. SHIELDS UP @jasonhand ~ #ChatOps laserfirecreations.com/Large%20Images/ShldsUpNew.jpg https://stardate.org/sites/default/files/images/gallery/10%2012%2018.jpg DDOS Attack Shields

    Up!
  24. Problems Devs & Ops Teams Have Problems Our Bots Solve

    Maturation of ChatOps : Phase 3 adopted from @davidjbland @jasonhand ~ #ChatOps
  25. Timelines @jasonhand ~ #ChatOps

  26. SITUATIONAL AWARENESS @jasonhand ~ #ChatOps blog.vmware.com

  27. A SITUATIONAL AWARENESS What makes a great team — everyone

    on the field is aware of everything, all the time. The team with the most situational awareness will win. - Mary Poppendieck (Lean Software Development) DevOpsDays Minneapolis - 2015 @jasonhand ~ #ChatOps
  28. Examples @jasonhand ~ #ChatOps

  29. Deploy Code git push origin master Counting objects: 7, done.

    Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 407 bytes, done. Total 4 (delta 2), reused 0 (delta 0) To git@github.com:qrush/gitready.git 361303d..f2cd831 master -> master jhand$ git commit -am "Adding ChatOps link" Created commit f2cd831: Adding ChatOps link 1 files changed, 1 insertions(+), 0 deletions(-) @jasonhand ~ #ChatOps jhand$
  30. Deploy Code Created commit f2cd831: Adding ChatOps link 1 files

    changed, 1 insertions(+), 0 deletions(-) foobot git commit -am "Adding ChatOps link" @jasonhand ~ #ChatOps Counting objects: 7, done. Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 407 bytes, done. Total 4 (delta 2), reused 0 (delta 0) To git@github.com:qrush/gitready.git 361303d..f2cd831 master -> master foobot git push origin master
  31. Build Infrastructure Platform install commit-1248994 (#789). Started by user: jhand

    (09.04.2015.11:03:24) Platform install commit-1248994 complete jhand$ build commit-1248994 @jasonhand ~ #ChatOps
  32. Build Infrastructure building commit-1248994 Platform install commit-1248994 (#789). Started by

    user: jhand (09.04.2015.11:03:24) Platform install commit-1248994 complete foobot build commit-1248994 @jasonhand ~ #ChatOps
  33. Manage Incidents Incident 778467 has been acknowledged by (jhand) Stopped

    paging jhand - (Backend Engineering Team) @ 2:51 p.m. MT jhand$ ack incident 778467 @jasonhand ~ #ChatOps
  34. Manage Incidents Incident 778467 has been acknowledged by (jhand) Stopped

    paging jhand - (Backend Engineering Team) @ 2:51 p.m. MT foobot ack me incident 778467 @jasonhand ~ #ChatOps
  35. Manage Incidents Runbook says to look at the graph first

    Please review runbook for prod2.cass24.victorops.com @jasonhand ~ #ChatOps
  36. Pull Graphs foobot graph me prod2.cass24.victorops.com - cpu -15 @jasonhand

    ~ #ChatOps
  37. Kick Services foobot restart apache 192.168.0.345 @jasonhand ~ #ChatOps apache

    on host 192.168.0.345 has been restarted one moment .. restarting apache on requested host
  38. Moving terminal and API to Chat @jasonhand ~ #ChatOps

  39. 1 1 1 2 1 3 1 4 Benefits of

    ChatOps Sharing Speed Learning Security @jasonhand ~ #ChatOps
  40. Security continued @jasonhand ~ #ChatOps Security, compliance, and regulatory requirements

    https://youtu.be/18PNEwt18P0
  41. Bots github.com/litaio/lita @jasonhand ~ #ChatOps github.com/github/hubot github.com/errbotio/errbot Lita

  42. More Bots github.com/devth/yetibot @jasonhand ~ #ChatOps Hal-9001 github.com/Netflix/hal-9001 github.com/elastic/jarvis (Previously

    Scorebot) github.com/micro/micro/tree/master/bot
  43. Cog : ChatOps Platform operable.io @jasonhand ~ #ChatOps github.com/operable/cog w/

    Access Control Mechanism
  44. Stackstorm @jasonhand ~ #ChatOps stackstorm.com Event-driven automation

  45. http://eng.localytics.com/serverless-slackbots-powered-by-aws/

  46. OSS (Persistent) Group Chat @jasonhand ~ #ChatOps Zulip Jappix Mattermost

    RocketChat
  47. It’s getting even easier @jasonhand ~ #ChatOps

  48. @jasonhand ~ #ChatOps 3rd-party App Directories

  49. Problems Devs & Ops Teams Have Problems Our Bots Solve

    Maturation of ChatOps : Phase 4 & Beyond adopted from @davidjbland @jasonhand ~ #ChatOps
  50. CONCERNS ALERT FATIGUE CONTEXT SWITCHING LARGE TEAMS AUTHENTICATION @jasonhand ~

    #ChatOps
  51. ADDITIONAL RESOURCES reddit.com/r/chatops @jasonhand talk-chatops.slack.com twitter.com/hashtag/chatops medium.com/search?q=chatops jasonhand.com/tag/chatops

  52. ADDITIONAL RESOURCES https://youtu.be/9C2aZIhVk3U https://youtu.be/mBbyO4UsWiE

  53. ChatOps For Dummies jhand.co/ChatOps4Dummies

  54. ChatOps on O’Reilly jhand.co/ChatOpsSurvey

  55. THANK YOU! Slides: jhand.co/1siEWHd

  56. !56 More https://medium.com/@cloudoapp/we-asked-hipchat-what-their-next-big-move-is-will-hipchat-join-the-bot-hype-heres-their-answer-90bae12e0cce#.pvzw4tzf7 https://www.usenix.org/conference/srecon16/program/presentation/tobey https://getcarina.com/blog/moving-chatops-into-carina/ http://dev.haufe.com/irc-chatops/ https://blog.micro.mu/2016/04/25/the-micro-bot.html http://www.infoworld.com/article/3062703/devops/how-chatops-can-revolutionize-your-business.html https://aws.amazon.com/blogs/aws/new-slack-integration-blueprints-for-aws-lambda/ https://victorops.com/chatops/

    http://www.jasonhand.com/tag/chatops/
  57. !57 Abstract Those tasked with building and maintaining resilient services

    and systems at scale are aware of the concepts behind “Infrastructure as Code”. Configuration management tools and services such as Puppet, Ansible, Chef, and others have become a staple in the provisioning, management, and more of critical components of systems worldwide. With the growing popularity of ChatOps (the concept of moving actions and context related to Operations into a persistent group-chat tool) more and more teams are beginning to find even more efficiencies and benefits. What does it look like when teams have begun to manage much of their infrastructure from the interface of group- chat? What concerns and pitfalls must be considered? Where should teams begin their efforts towards a ChatOps approach? What’s on the horizon and beyond as more and more shift their actions away from the command-line-interface and web browsers and begin to leverage API’s, bots, and group-chat tools in a secure and consistent way?