The devops field guide to cognitive biases

As devops practitioners we focus on improving the culture of collaboration so that others play nicely with us & we play nicely with others - but what if the biggest thing holding us back from change is our own brains?

Cognitive biases can deeply affect our behaviours towards others by herding us towards mental shortcuts that are optimised for timeliness over accuracy, at the expense of rationalising irrational behaviour.

You are probably pushing these biases onto other people every day but don't even know it. Does that idea make you feel uncomfortable? You are probably experiencing the Semmelweis reflex kicking your confirmation bias right now.

Knowing is half the battle. This talk delves into some of the well-known and less well-known biases that may be affecting your ability to work with your peers, and your team's ability to work constructively with other teams.

Viewers of the talk will come away with an overview of biases they run into every day, how to hack their brains to use these biases to their advantage, and some tips on how to mitigate the effects of the limitations baked into their wetware.

We have met the enemy and he is us.

--

These are the resources used in the presentation:

http://www.flickr.com/photos/williamhook/4742869256
http://www.flickr.com/photos/10268891@N07/856565056
http://www.flickr.com/photos/brenderous/3671106784
http://www.flickr.com/photos/frostnova/2268471558
http://www.flickr.com/photos/buttim/1297081125
http://www.flickr.com/photos/dhdesign/1096464615
http://www.flickr.com/photos/princess_l_88/4948893120
http://www.flickr.com/photos/cphotos/4780625450
http://www.flickr.com/photos/amesis/3777130839
http://www.flickr.com/photos/artotemsco/5465529987
http://www.flickr.com/photos/fish2000/4648423198
http://www.comedycentral.com/press/images/colbertreport/stephencolbert2.jpg
http://althouse.blogspot.com.au/2006/05/who-is-clueless-enough-to-think.html
http://www.alternet.org/story/137918/why_do_conservatives_like_stephen_colbert
http://crooksandliars.com/john-amato/ohio-study-finds-conservatives-believe
http://www.youtube.com/watch?v=Ahg6qcgoay4
http://news.ninemsn.com.au/world/2013/07/10/12/43/asiana-pilot-couldn-t-see-runway-before-crash
http://au.news.yahoo.com/thewest/a/-/world/17940112/asiana-flight-crew-saw-trouble-at-500-feet-u-s-safety-board-says/
http://www.theaustralian.com.au/news/world/crash-pilots-wanted-to-abort-landing-as-boeing-777-approached-airport-too-low-investigators/story-e6frg6so-1226675762698
http://www.theguardian.com.au/story/1628513/inexperienced-pilots-may-be-cause-of-asiana-jet-crash/?cs=5
http://www.news.com.au/world-news/fears-pilots-manual-flying-skills-eroding-after-asiana-airlines-crash-in-san-francisco/story-fndir2ev-1226676769126
http://en.wikipedia.org/wiki/Scientific_method
http://en.wikipedia.org/wiki/Publication_bias
http://en.wikipedia.org/wiki/Hindsight_bias#Judicial_system
http://www.flickr.com/photos/studiogabe/5535916719
http://www.flickr.com/photos/sjliew/1311490235
http://www.flickr.com/photos/pcb85/4258475681
http://www.flickr.com/photos/jjackowski/3812946635
http://library.mpib-berlin.mpg.de/ft/rh/RH_Fluency_2008.pdf
http://www.wrayherbert.com/blog/memory-learning-and-the-fluency-heuristic
http://www.youtube.com/watch?v=IGQmdoK_ZfY
http://www.amazon.com/You-Are-Not-So-Smart/dp/1592407366/
http://www.amazon.com/Sway-Irresistible-Pull-Irrational-Behavior/dp/0385530609/
http://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555/
http://www.amazon.com/The-Checklist-Manifesto-Things-Right/dp/0312430000/
http://www.amazon.com/Field-Guide-Understanding-Human-Error/dp/0754648265/

Transcript

1. devops field guide to understanding cognitive biases (2nd edition)
3. Your current iPhone crashes
4. Your current iPhone is slower
5. Time to upgrade!
6. Why?
7. Apple has made my device slower!
9. Clustering illusion
10. See patterns in randomness
11. See correlations where there are none
12. Your device has always been that slow
13. Clustering illusion is a Cognitive Bias
14. What is a cognitive bias?
15. “Mental shortcut”
17. Timeliness Accuracy
19. Problem solving
20. Problem solving Heuristic
21. Problem solving Heuristic Correct result
22. Problem solving Heuristic Correct result Rational choice
24. Problem solving
25. Problem solving Heuristic
26. Problem solving Heuristic Incorrect result
27. Problem solving Heuristic Incorrect result Cognitive bias!
28. Heuristic?
29. Pattern matching
30. What helped your ancestors survive!
32. March 27, 1977
34. Control tower
35. Control tower ✈
36. Control tower ✈ ✈
37. Control tower ✈ ✈
38. Control tower ✈ ✈
39. Control tower ✈ ✈
40. Control tower ✈ ✈
43. KLM:
    234 passengers
    16 crew
44. Pan Am:
    326 passengers
    9 crew
45. Frozen in place
46. Control tower ✈ ✈
47. Control tower ✈ ✈
48. Control tower ✈ ✈
49. Normalcy bias
50. Before a disaster:
52. • Underestimate:
53. • Underestimate: • risk
54. • Underestimate: • risk • effects
55. • Underestimate: • risk • effects • preparation
56. “Because something bad has never happened, it never will happen”
57. During a disaster:
59. • Response:
60. • Response: • slow reaction
61. • Response: • slow reaction • seek validation
62. • Response: • slow reaction • seek validation • optimistic interpretation
63. Reaction steps
65. • Cognition
66. • Cognition • Perception
67. • Cognition • Perception • Comprehension
68. • Cognition • Perception • Comprehension • Decision
69. • Cognition • Perception • Comprehension • Decision • Implementation
70. • Cognition • Perception • Comprehension • Decision • Implementation • Movement
71. These are
    complex tasks
72. You cannot skip
    these tasks
73. You can practice to make them more automatic
74. People who don't practice deliberate during the disaster
77. • 70% freeze
78. • 70% freeze • 15% freak out
79. • 70% freeze • 15% freak out • 15% react to situation
80. No practice == higher MTTR
81. Counter this behaviour
82. drill drill drill drill drill drill drill drill drill drill drill drill drill drill drill drill drill drill drill drill
83. Normalcy bias has your head in the sand. Drill to react when disaster is at hand.
92. see it go over here
95. was actually here
96. Confirmation bias
97. Seek information that reinforces position
98. Ignore alternative explanations
99. Interpret ambiguous information in favour of existing positions!
102. • Who is clueless enough to think Colbert is a conservative? • Why do conservatives like Stephen Colbert? • Ohio study finds conservatives believe Colbert dislikes liberalism
103. http://www.youtube.com/watch?v=Ahg6qcgoay4
104. Debugging code
105. My code is perfect!
106. I know the problem is in this other library
107. I just can't find it!
108. *30 minutes pass*
109. The bug was in
     my code
111. Counter this behaviour
112. Take opposing viewpoints
113. Is your conclusion negative?
     It could be biased.
114. Confirmation reinforces, You ignore other sources
115. Provisioning automation
     @
     
117. 18 months
119. • Provisioning: • We're being hurt badly by VMware provs!
120. • Provisioning: • We're being hurt badly by VMware provs! • R&D: • We'll automate that now!
121. 2 weeks later
123. • Provisioning: • We're being hurt badly by load balancer provs!
124. • Provisioning: • We're being hurt badly by load balancer provs! • R&D: • We'll automate that now!
125. 1 week later
127. • Provisioning: • We're being hurt badly by VMware provs!
128. • Provisioning: • We're being hurt badly by VMware provs! • R&D: • We'll automate that now!
129. 4 weeks later
131. • Provisioning: • We're being hurt badly by EC2 provs!
132. • Provisioning: • We're being hurt badly by EC2 provs! • R&D: • We'll automate that now!
134. 18 months
136. What is
     happening?
137. Working on the thing closest to us that hurts us most.
138. But it's not the biggest problem overall!
139. Would you work
     like this?
140. 18 months
141. Outcome bias
142. Hindsight bias
143. "knew it all along" effect
144. memory distortion distortion
145. Judgement
146. “You were capable
     of preventing
     a bad outcome”
147. Failure!
148. Talk about events
     in the context
     of the outcome Failure!
149. Bad outcomes
     come from
     bad decisions
151. • MH17 tragedy reveals need for flight path overhaul • Flight MH17 took abandoned flight path • MH17 flightpath questioned as Malaysia mourns second airline tragedy • Flight MH17 ‘could have avoided Ukrainian airspace for $66 per passenger’ • Why was MH17’s flight path over a conflict zone?
152. • news.com.au • Sky News • ABC • Daily Mail • PBS
153. Taints all interactions
154. Hindsight bias
     is your culture killer
156. Monitoring thresholds
157. Backend HTTP Port 81
158. “I don’t want that thing
     that woke me last night
     to wake me again”
159. “I’m going to make that problem go away”
160. But...
161. How often does
     that alert fire?
162. What percentage
     of all alerts is that?
163. Are there other alerts that are more frequent?
164. What makes this alert the most important to fix?
165. +----------------------------+---------------+-----------------+---------------------+
     | 2013-07-01T00:00:00+09:30 - 2013-07-31T23:59:59+09:30 | +----------------------------+---------------+-----------------+---------------------+ | Check | # of entities | Critical | Total | +----------------------------+---------------+-----------------+---------------------+ | Memory Utilisation | 85 | 9h 44m 31s | 2mo 5d 8h 2m 5s | | Load | 44 | 22d 2h 38m 50s | 1mo 22d 15h 51m 42s | | ** DISK Utilisation (Win) | 41 | 5m 0s | 1mo 19d 16h 2m 7s | | ** Disk Utilisation (Unix) | 43 | 3h 34m 2s | 1mo 1d 18h 24m 28s | | PING | 35 | 1mo 16h 16m 43s | 1mo 16h 16m 43s | | HOST | 37 | 1mo 2h 25m 52s | 1mo 2h 25m 52s | | SSH | 14 | 26d 21h 8m 33s | 26d 21h 8m 33s | | Flapper | 1 | 15d 11h 42m 16s | 15d 11h 42m 16s | | CPU Load | 5 | 8d 12h 21m 31s | 8d 13h 36m 22s | | CPU Utilisation | 30 | 1h 35m 18s | 4d 3h 17m 37s | | ** check_tcp Port | 56 | 2d 19h 25m 47s | 2d 19h 25m 47s | | RDP | 6 | 6h 34m 39s | 6h 34m 39s | | Swap Utilisation | 18 | 5h 43m 44s | 5h 43m 44s | | Frontend HTTP Port 80 | 4 | 4h 22m 14s | 4h 22m 14s | | Backend HTTPS Port 444 | 4 | 4h 20m 19s | 4h 20m 19s | | Frontend HTTPS Port 443 | 4 | 4h 20m 9s | 4h 20m 9s | | Backend HTTP Port 81 | 4 | 4h 17m 41s | 4h 17m 41s | | Disk Utilisation | 2 | 1h 45m 0s | 3h 9m 0s | | ** HTTP Regex | 5 | 11m 53s | 11m 53s | +----------------------------+---------------+-----------------+---------------------+
166. +----------------------------+---------------+-----------------+---------------------+
     | 2013-07-01T00:00:00+09:30 - 2013-07-31T23:59:59+09:30 | +----------------------------+---------------+-----------------+---------------------+ | Check | # of entities | Critical | Total | +----------------------------+---------------+-----------------+---------------------+ | Memory Utilisation | 85 | 9h 44m 31s | 2mo 5d 8h 2m 5s | | Load | 44 | 22d 2h 38m 50s | 1mo 22d 15h 51m 42s | | ** DISK Utilisation (Win) | 41 | 5m 0s | 1mo 19d 16h 2m 7s | | ** Disk Utilisation (Unix) | 43 | 3h 34m 2s | 1mo 1d 18h 24m 28s | | PING | 35 | 1mo 16h 16m 43s | 1mo 16h 16m 43s | | HOST | 37 | 1mo 2h 25m 52s | 1mo 2h 25m 52s | | SSH | 14 | 26d 21h 8m 33s | 26d 21h 8m 33s | | Flapper | 1 | 15d 11h 42m 16s | 15d 11h 42m 16s | | CPU Load | 5 | 8d 12h 21m 31s | 8d 13h 36m 22s | | CPU Utilisation | 30 | 1h 35m 18s | 4d 3h 17m 37s | | ** check_tcp Port | 56 | 2d 19h 25m 47s | 2d 19h 25m 47s | | RDP | 6 | 6h 34m 39s | 6h 34m 39s | | Swap Utilisation | 18 | 5h 43m 44s | 5h 43m 44s | | Frontend HTTP Port 80 | 4 | 4h 22m 14s | 4h 22m 14s | | Backend HTTPS Port 444 | 4 | 4h 20m 19s | 4h 20m 19s | | Frontend HTTPS Port 443 | 4 | 4h 20m 9s | 4h 20m 9s | | Backend HTTP Port 81 | 4 | 4h 17m 41s | 4h 17m 41s | | Disk Utilisation | 2 | 1h 45m 0s | 3h 9m 0s | | ** HTTP Regex | 5 | 11m 53s | 11m 53s | +----------------------------+---------------+-----------------+---------------------+ | Backend HTTP Port 81 | 4 | 4h 17m 41s | 4h 17m 41s |
167. Harness this behaviour
168. Counterfactuals
169. *we’re about to launch* “What do you think will break?”
170. versus
171. *imagine 5 minutes post-launch* “What just broke?”
172. Counter this behaviour
173. During
175. • 1. collect data
176. • 1. collect data • 2. analyse that data
177. • 1. collect data • 2. analyse that data • 3. make decisions based on that data
178. • 1. collect data • 2. analyse that data • 3. make decisions based on that data • 4. act on data, not feelings
179. • 1. collect data • 2. analyse that data • 3. make decisions based on that data • 4. act on data, and feelings
180. After:
181. Explain in terms of foresight
182. Local rationality
183. “People make what they think are best decisions based on the information at hand”
185. Hindsight affords us
     global rationality
187. Hindsight blinds,
     Foresight finds
189. Normalcy bias
190. Confirmation bias
191. Hindsight bias
193. How vulnerable to these biases are you?
194. – you, right now “I’m aware of my biases.”
195. – you, right now “These don’t affect me.”
196. – you, right now “I am better than average.”
197. Better-than-average effect
     &
     Illusory superiority
198. Bias blind spot
200. • Positive attributes
201. • Positive attributes • Above average
203. • Negative attributes
204. • Negative attributes • Below average
205. Why?
206. Self-esteem
207. Control
209. Normalcy bias has your head in the sand. Drill to react when disaster is at hand.
210. Confirmation reinforces,
     You ignore other sources
211. Hindsight blinds,
     Foresight finds
212. Rhyme-as-reason
     effect
213. Aesthetics affect truth
215. • Fluent, Fast, Smooth
216. • Fluent, Fast, Smooth • Higher value
217. This is a test
219. 1. Read the statement once
220. 1. Read the statement once 2. Count the letter F
222. FINAL FOLIOS SEEM TO RESULT FROM YEARS OF DUTIFUL STUDY OF TEXTS ALONG WITH YEARS OF SCIENTIFIC EXPERIENCE.
224. How many
     did you see?
     
225. How many
     did you see?
     The answer is 8
226. Fluency heuristic
227. FINAL FOLIOS SEEM TO RESULT FROM YEARS OF DUTIFUL STUDY OF TEXTS ALONG WITH YEARS OF SCIENTIFIC EXPERIENCE.
228. Brain expects pattern to continue
229. Brain skips
     other information
230. Harness this behaviour
231. Rhyme your mantras
233. Have you learnt anything?
234. https://www.youtube.com/watch?v=IGQmdoK_ZfY
235. How many passes?
236. What did the
     gorilla do?
237. Did you see the curtains change colour?
238. Or one of the
     players leave?
239. https://www.youtube.com/watch?v=IGQmdoK_ZfY
240. – Mark Twain “It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.”
241. Culture is made
     from people.
242. Everyone has biases.
243. Understand how
     you think.
244. Become more rational.
245. Become less rationalising.
246. Be aware of bias.
247. Improve ourselves.
248. Culture will flow.
249. I’m Lindsay @auxesis
250. Thank you!
251. Thank you! Liked the talk? Let @auxesis know.
253. • Further reading • You Are Not So Smart – David McRaney • The field guide to understanding 
 human error – Sidney Dekker • Sway: The irresistible pull of irrational behavior – Ori Brafman • Thinking, Fast and Slow – Daniel Kahneman