Fallible Humans: Dealing With Failure In The Absence Of Scapegoats

Functional siloing - "dev", "ops", etc. - make it easier to pass the buck and blame others when things go wrong. In an environment where dev, ops, and other business functions co-operate and work together, it's not so easy to fall back on traditional scapegoats. DevOps approaches encourage us to move away from the harmful (and counterproductive) apportioning of blame for failure, but it takes effort and commitment from all involved.

I'll discuss the philosophy and motivations of "blamelessness", along with ways to understand the vital role humans play in making complex socio-technical systems safe, even when things go awry, along with advice on how to promote and foster a blameless culture in your workplace, and techniques for analyzing and learning from failures without resorting to finger-pointing and scapegoating.

This talk was originally given at DevOps Days Minneapolis on 18 July, 2014: http://devopsdays.org/events/2014-minneapolis/

A transcript is available at http://ind.ec/fallible

Transcript

1. Fallible Humans Dealing With Failure In The Absence Of Scapegoats Ian Malpass, Etsy @indec
2. "Jerusalem Ugglan 1". Licensed under Public domain via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:Jerusalem_Ugglan_1.jpg
3. By Rodhullandemu (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons GET IN THE CAR GET IN THE CAR NO TIME TO EXPLAIN NO TIME TO EXPLAIN
4. “Fire of Troy" by Kerstiaen de Keuninck (Coninck) - http://fotki.yandex.ru/users/fchstudents/view/133880?page=69. Licensed under Public domain via Wikimedia Commons
5. It’s not me, it’s you
6. Dev Ops Legal Marketing Support Finance Security
7. Dev Ops Legal Marketing Support Finance Security
8. With the unknown, one is confronted with danger, discomfort, and care; the first instinct is to abolish these painful states. First principle: any explanation is better than none. ! Friedrich Nietzsche, The Twilight Of The Idols
9. A person who has been punished is not thereby simply less inclined to behave in a given way; at best, he learns how to avoid punishment. ! B. F. Skinner, Beyond Freedom and Dignity
10. Failure is an emergent property of a complex system
11. Chalk Farm Banksy by grahamc99 https://flic.kr/p/39M4pM
12. We should learn from our mistakes
13. You don’t come to work to do a bad job
14. Get the feature out on time Don’t have any bugs Don’t use too many resources
15. Monitor everything Alert appropriately Avoid alert fatigue
16. Efficiency ! ! ! Thoroughness
17. If I'd observed all the rules, I'd never have got anywhere. ! Marilyn Monroe
18. Every day I’m ETTOing
19. Why?
20. Trust and understanding
21. Don’t blame yourself
22. Blamelessness is more fun with friends
23. Practice makes perfect as close to perfect as we can hope for in a complex system based on reasonable expectations
24. Looking back, we should have eliminated hindsight
25. Could have, would have, should have
26. There is no root cause
27. Seek good remediation items
28. Process is an embedded reaction to prior stupidity ! Clay Shirky, Wikis, Grafitti, and Process
29. Communicate and advocate
30. etsy.me/justculture
31. Good luck