ChatOps: Infrastructure As Conversation

Slide 1

Slide 1 text

CHATOPS: INFRASTRUCTURE AS CONVERSATION

Slide 2

Slide 2 text

JASON HAND jasonhand #ChatOps

Slide 3

Slide 3 text

@jasonhand ~ #ChatOps https://www.cheatography.com/uploads/davidpol_1439395467_Agile%20Manifesto.png

Slide 4

Slide 4 text

Delivery Pipeline Plan Code Build Test Release Deploy Operate Agile Infrastructure (DevOps) Continuous Delivery Continuous Integration Agile Development @jasonhand ~ #ChatOps

Slide 5

Slide 5 text

@jasonhand ~ #ChatOps http://2.bp.blogspot.com/-3Vv68jBEYdo/VfGlfe3aB0I/AAAAAAAAAMo/vrJl7PBd8m0/s1600/contact-center-acd-bottleneck.jpg

Slide 6

Slide 6 text

USER & CUSTOMER EXPECTATIONS Users expect working software, high availability, innovation, and a sense of empathy

Slide 7

Slide 7 text

AVAILABILITY & RELIABILITY High availability is the result of high resiliency & reliability

Slide 8

Slide 8 text

AVAILABILITY & RELIABILITY High resiliency & reliability is the result of improved MTTR Fail Fast == Respond & Recover Fast

Slide 9

Slide 9 text

Continuous Delivery @jasonhand ~ #ChatOps

Slide 10

Slide 10 text

Continuous Improvement @jasonhand ~ #ChatOps

Slide 11

Slide 11 text

A CONTINUOUS IMPROVEMENT It is not enough to do your best; you must know what to do, and then do your best. - W. Edwards Deming Act Check Do @jasonhand ~ #ChatOps Plan

Slide 12

Slide 12 text

@jasonhand ~ #ChatOps

Slide 13

Slide 13 text

DevOps Buzzword Bingo Continuous Integration Continuous Testing Continuous Delivery Continuous Monitoring Continuous Deployment ChatOps Containerization Microservices High Availability Devs On-call Delivery Pipeline Configuration Management Infrastructure As Code Virtualization Immutable Infrastructure Feedback Loops Experimentation Culture Release Automation Lean Kanban @jasonhand ~ #ChatOps Measurement Sharing

Slide 14

Slide 14 text

DEVOPS PRINCIPLES CULTURE AUTOMATION MEASUREMENT SHARING LEAN @jasonhand ~ #ChatOps

Slide 15

Slide 15 text

What is ChatOps? ChatOps is an approach to communication that allows teams to collaborate and manage many aspects of their infrastructure, code, and data from the comfort and safety of a chat room. Through the use of chatbots and scripts, teams can execute commands, query information, and distribute knowledge across the entire organization and all from the space where conversations are already taking place, a chat room. “We're taking the things we're already doing and putting them in line with the conversations we're already having” - James Fryman (StackStorm) @jasonhand ~ #ChatOps

Slide 16

Slide 16 text

1 1 1 2 1 3 1 4 Origins IRC XMPP/ Jabber Campfire HipChat/ Slack/Flowdock api @jasonhand ~ #ChatOps “Conversation-driven development”

Slide 17

Slide 17 text

A HUBOT: THE EARLY YEARS Placing tools right in the middle of the conversation - Jesse Newland (GitHub) - 2012 @jasonhand ~ #ChatOps

Slide 18

Slide 18 text

Problems Devs & Ops Teams Have Problems Our Bots Solve Maturation of ChatOps : Phase 1 adopted from @davidjbland @jasonhand ~ #ChatOps

Slide 19

Slide 19 text

Problems Devs & Ops Teams Have Problems Our Bots Solve Maturation of ChatOps : Phase 2 adopted from @davidjbland @jasonhand ~ #ChatOps

Slide 20

Slide 20 text

GitHub Pull Request Notifier Merge code Show current status Display “commit” link Commit Notifier Display “issue” link Display repo commuters Show Open Pull Requests Show Repo Activity Show open issues Search for string in code @jasonhand ~ #ChatOps

Slide 21

Slide 21 text

1 1 1 2 1 3 1 4 THE NEW INTERFACE Notifications/ Information Alerting/ Paging Command Execution Collaboration/ Communication @jasonhand ~ #ChatOps Feedback Loops Lower MTTR Higher availability & happier users

Slide 22

Slide 22 text

A THE NEW INTERFACE A tool to help the way our people systems work that in turn helps the way our technical systems work. - Mark Imbriaco (operable.io) https://youtu.be/pCVvYCjvoZI @jasonhand ~ #ChatOps

Slide 23

Slide 23 text

SHIELDS UP @jasonhand ~ #ChatOps laserfirecreations.com/Large%20Images/ShldsUpNew.jpg https://stardate.org/sites/default/files/images/gallery/10%2012%2018.jpg DDOS Attack Shields Up!

Slide 24

Slide 24 text

Problems Devs & Ops Teams Have Problems Our Bots Solve Maturation of ChatOps : Phase 3 adopted from @davidjbland @jasonhand ~ #ChatOps

Slide 25

Slide 25 text

Timelines @jasonhand ~ #ChatOps

Slide 26

Slide 26 text

SITUATIONAL AWARENESS @jasonhand ~ #ChatOps blog.vmware.com

Slide 27

Slide 27 text

A SITUATIONAL AWARENESS What makes a great team — everyone on the field is aware of everything, all the time. The team with the most situational awareness will win. - Mary Poppendieck (Lean Software Development) DevOpsDays Minneapolis - 2015 @jasonhand ~ #ChatOps

Slide 28

Slide 28 text

Examples @jasonhand ~ #ChatOps

Slide 29

Slide 29 text

Deploy Code git push origin master Counting objects: 7, done. Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 407 bytes, done. Total 4 (delta 2), reused 0 (delta 0) To [email protected]:qrush/gitready.git 361303d..f2cd831 master -> master jhand$ git commit -am "Adding ChatOps link" Created commit f2cd831: Adding ChatOps link 1 files changed, 1 insertions(+), 0 deletions(-) @jasonhand ~ #ChatOps jhand$

Slide 30

Slide 30 text

Deploy Code Created commit f2cd831: Adding ChatOps link 1 files changed, 1 insertions(+), 0 deletions(-) foobot git commit -am "Adding ChatOps link" @jasonhand ~ #ChatOps Counting objects: 7, done. Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 407 bytes, done. Total 4 (delta 2), reused 0 (delta 0) To [email protected]:qrush/gitready.git 361303d..f2cd831 master -> master foobot git push origin master

Slide 31

Slide 31 text

Build Infrastructure Platform install commit-1248994 (#789). Started by user: jhand (09.04.2015.11:03:24) Platform install commit-1248994 complete jhand$ build commit-1248994 @jasonhand ~ #ChatOps

Slide 32

Slide 32 text

Build Infrastructure building commit-1248994 Platform install commit-1248994 (#789). Started by user: jhand (09.04.2015.11:03:24) Platform install commit-1248994 complete foobot build commit-1248994 @jasonhand ~ #ChatOps

Slide 33

Slide 33 text

Manage Incidents Incident 778467 has been acknowledged by (jhand) Stopped paging jhand - (Backend Engineering Team) @ 2:51 p.m. MT jhand$ ack incident 778467 @jasonhand ~ #ChatOps

Slide 34

Slide 34 text

Manage Incidents Incident 778467 has been acknowledged by (jhand) Stopped paging jhand - (Backend Engineering Team) @ 2:51 p.m. MT foobot ack me incident 778467 @jasonhand ~ #ChatOps

Slide 35

Slide 35 text

Manage Incidents Runbook says to look at the graph first Please review runbook for prod2.cass24.victorops.com @jasonhand ~ #ChatOps

Slide 36

Slide 36 text

Pull Graphs foobot graph me prod2.cass24.victorops.com - cpu -15 @jasonhand ~ #ChatOps

Slide 37

Slide 37 text

Kick Services foobot restart apache 192.168.0.345 @jasonhand ~ #ChatOps apache on host 192.168.0.345 has been restarted one moment .. restarting apache on requested host

Slide 38

Slide 38 text

Moving terminal and API to Chat @jasonhand ~ #ChatOps

Slide 39

Slide 39 text

1 1 1 2 1 3 1 4 Benefits of ChatOps Sharing Speed Learning Security @jasonhand ~ #ChatOps

Slide 40

Slide 40 text

Security continued @jasonhand ~ #ChatOps Security, compliance, and regulatory requirements https://youtu.be/18PNEwt18P0

Slide 41

Slide 41 text

Bots github.com/litaio/lita @jasonhand ~ #ChatOps github.com/github/hubot github.com/errbotio/errbot Lita

Slide 42

Slide 42 text

More Bots github.com/devth/yetibot @jasonhand ~ #ChatOps Hal-9001 github.com/Netflix/hal-9001 github.com/elastic/jarvis (Previously Scorebot) github.com/micro/micro/tree/master/bot

Slide 43

Slide 43 text

Cog : ChatOps Platform operable.io @jasonhand ~ #ChatOps github.com/operable/cog w/ Access Control Mechanism

Slide 44

Slide 44 text

Stackstorm @jasonhand ~ #ChatOps stackstorm.com Event-driven automation

Slide 45

Slide 45 text

http://eng.localytics.com/serverless-slackbots-powered-by-aws/

Slide 46

Slide 46 text

OSS (Persistent) Group Chat @jasonhand ~ #ChatOps Zulip Jappix Mattermost RocketChat

Slide 47

Slide 47 text

It’s getting even easier @jasonhand ~ #ChatOps

Slide 48

Slide 48 text

@jasonhand ~ #ChatOps 3rd-party App Directories

Slide 49

Slide 49 text

Problems Devs & Ops Teams Have Problems Our Bots Solve Maturation of ChatOps : Phase 4 & Beyond adopted from @davidjbland @jasonhand ~ #ChatOps

Slide 50

Slide 50 text

CONCERNS ALERT FATIGUE CONTEXT SWITCHING LARGE TEAMS AUTHENTICATION @jasonhand ~ #ChatOps

Slide 51

Slide 51 text

ADDITIONAL RESOURCES reddit.com/r/chatops @jasonhand talk-chatops.slack.com twitter.com/hashtag/chatops medium.com/search?q=chatops jasonhand.com/tag/chatops

Slide 52

Slide 52 text

ADDITIONAL RESOURCES https://youtu.be/9C2aZIhVk3U https://youtu.be/mBbyO4UsWiE

Slide 53

Slide 53 text

ChatOps For Dummies jhand.co/ChatOps4Dummies

Slide 54

Slide 54 text

ChatOps on O’Reilly jhand.co/ChatOpsSurvey

Slide 55

Slide 55 text

THANK YOU! Slides: jhand.co/1siEWHd

Slide 56

Slide 56 text

!56 More https://medium.com/@cloudoapp/we-asked-hipchat-what-their-next-big-move-is-will-hipchat-join-the-bot-hype-heres-their-answer-90bae12e0cce#.pvzw4tzf7 https://www.usenix.org/conference/srecon16/program/presentation/tobey https://getcarina.com/blog/moving-chatops-into-carina/ http://dev.haufe.com/irc-chatops/ https://blog.micro.mu/2016/04/25/the-micro-bot.html http://www.infoworld.com/article/3062703/devops/how-chatops-can-revolutionize-your-business.html https://aws.amazon.com/blogs/aws/new-slack-integration-blueprints-for-aws-lambda/ https://victorops.com/chatops/ http://www.jasonhand.com/tag/chatops/

Slide 57

Slide 57 text

!57 Abstract Those tasked with building and maintaining resilient services and systems at scale are aware of the concepts behind “Infrastructure as Code”. Configuration management tools and services such as Puppet, Ansible, Chef, and others have become a staple in the provisioning, management, and more of critical components of systems worldwide. With the growing popularity of ChatOps (the concept of moving actions and context related to Operations into a persistent group-chat tool) more and more teams are beginning to find even more efficiencies and benefits. What does it look like when teams have begun to manage much of their infrastructure from the interface of group- chat? What concerns and pitfalls must be considered? Where should teams begin their efforts towards a ChatOps approach? What’s on the horizon and beyond as more and more shift their actions away from the command-line-interface and web browsers and begin to leverage API’s, bots, and group-chat tools in a secure and consistent way?