Dealing with Incidents
using Monzo Response ⚡
Chris Evans
Platform / On-call Lead
@evnsio
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
If any of that sounds interesting, we’re hiring…
monzo.com/careers
Slide 8
Slide 8 text
Let’s talk about incidents...
Slide 9
Slide 9 text
“When you assign an incident lead,
let customer operations know who it is”
Slide 10
Slide 10 text
“...and remember to set a severity
within the first 5 mins of the incident”
Slide 11
Slide 11 text
“...and if it’s a major severity, follow this procedure”
Slide 12
Slide 12 text
“...and if it’s a critical severity,
here’s this other stuff you need to do”
Slide 13
Slide 13 text
“...and if you’re going to put the status page up, contact this team
so they can decide whether we need to inform the regulator”
Slide 14
Slide 14 text
“...and check this spreadsheet for risk ratings to know
whether you need to notify a senior exec”
Slide 15
Slide 15 text
“...oh, and the incident needs to be dealt with too”
Slide 16
Slide 16 text
No content
Slide 17
Slide 17 text
Response ⚡ is the result of
applying software engineering to
incident response
Slide 18
Slide 18 text
Limit context switching
Context switching during an incident is often unavoidable. We
aim to limit this, by enabling actions to be carried out without
leaving the conversation.
Slide 19
Slide 19 text
Make the easiest thing the right thing
If something needs doing, bring it to the attention of the
responder when it makes sense, or better still automate it away.
@slack_event
@notification
@action_handler
@keyword_handler
@incident_command
@dialog_handler
Building blocks
Slide 31
Slide 31 text
@slack_event
@notification
@action_handler
@keyword_handler
@incident_command
@dialog_handler
Building blocks
Slide 32
Slide 32 text
Functions that are called when a user
wants the tool to do something...
Slide 33
Slide 33 text
@incident
Slide 34
Slide 34 text
No content
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
No content
Slide 39
Slide 39 text
Some other commands we use at Monzo
action log an action to be followed up later
escalate escalate to another set of on-callers
statuspage update our customer facing status page
rename rename a comms channel to something descriptive
Slide 40
Slide 40 text
@slack_event
@notification
@action_handler
@keyword_handler
@incident_command
@dialog_handler
Building blocks
Slide 41
Slide 41 text
Functions that are called when a word or
phrase is mentioned...
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
@slack_event
@notification
@action_handler
@keyword_handler
@incident_command
@dialog_handler
Building blocks
Slide 44
Slide 44 text
Functions that are called on a particular schedule...
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
No content
Slide 47
Slide 47 text
@slack_event
@notification
@action_handler
@keyword_handler
@incident_command
@dialog_handler
Building blocks
Slide 48
Slide 48 text
Functions that are called when a particular
Slack event happens...
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
Some other events that might be interesting...
Users posting messages
Users pasting links
Reactions
Slide 52
Slide 52 text
@slack_event
@notification
@action_handler
@keyword_handler
@incident_command
@dialog_handler
Building blocks
Slide 53
Slide 53 text
Functions that are called when a Slack button is clicked...
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
@slack_event
@notification
@action_handler
@keyword_handler
@incident_command
@dialog_handler
Building blocks
Slide 56
Slide 56 text
Functions that are called when a Slack dialog is submitted...
Slide 57
Slide 57 text
No content
Slide 58
Slide 58 text
“I want to be able to Page engineers
without leaving Slack”
Slide 59
Slide 59 text
No content
Slide 60
Slide 60 text
Page Engineer
One line of code
One function
Slide 61
Slide 61 text
One incident command
to post the message
One function to handle
buttons and raise dialog
One function to
handle dialog
submission
Slide 62
Slide 62 text
Takeaways
Map out your
incident handling
process
Understand
what you’re
being asked to
do and when
Use the
building blocks
of Response ⚡
to codify it
Let your
on-callers do
what they do
best
Slide 63
Slide 63 text
Thanks for listening!
@evnsio
github.com/monzo/response