Upgrade to Pro — share decks privately, control downloads, hide ads and more …

It's 3AM, Do You Know Why You Got Paged?

RyanFrantz
September 16, 2014

It's 3AM, Do You Know Why You Got Paged?

My presentation at Velocity NYC 2014. I discuss ideas around what good context can be, how computers should augment operators' abilities, and how nagios-herald can be used to add valuable information to Nagios alerts.

RyanFrantz

September 16, 2014
Tweet

More Decks by RyanFrantz

Other Decks in Technology

Transcript

  1. @Ryan_Frantz 3AM Is the Worst • Takeaways • What makes

    for good context? • Computers should augment our work
  2. @Ryan_Frantz Alert Fatigue • High volume, low value alerts •

    Heightened anxiety, distraction • Sleep loss • Lack of context
  3. @Ryan_Frantz Easy to Implement • Current value vs threshold •

    Impact on site availability/application function • Downstream effects • Author’s intentions
  4. @Ryan_Frantz Augmentation “...increasing the capability of a [person] to approach

    a complex problem situation, to gain comprehension to suit [one’s] particular needs, and to derive solutions to problems.” - Doug Englebart “Augmenting Human Intellect: A Conceptual Framework”
  5. @Ryan_Frantz OODA Patrick Edwin Moran - Own work. Licensed under

    Creative Commons Attribution 3.0 via Wikimedia Commons
  6. @Ryan_Frantz Orientation “The second O, orientation – as the repository

    of our genetic heritage, cultural tradition, and previous experiences – is the most important part of the O-O-D- A loop since it shapes the way we observe, the way we decide, the way we act.” Col. Boyd “Organic Design for Command and Control”
  7. @Ryan_Frantz Adding Context • Within Nagios checks • Always do

    this! • Before alert notification • nagios-herald
  8. @Ryan_Frantz nagios-herald is Calling • Write formatters • Need tests

    • Need more docs • https://github.com/etsy/nagios-herald
  9. @Ryan_Frantz Resources and Reads • Project: github.com/etsy/nagios-herald • Blog: bit.ly/alert-design

    • Color contrast ratio: bit.ly/contrastratio • Englebart: bit.ly/augmenthumans • OODA Loop: bit.ly/oodaloops