Safety science - a primer (ConFoo 2023)

Incidents are a fact of life, an inevitable cost of doing business. But they can teach us a lot about our organizations and our work.

How we look at & learn from incidents has been evolving over the past decades, both in- and outside our industry.

Join me to learn about some of the most important current themes and theories in incident investigation and safety science. Then I will discuss some pointers that you can apply in your own organization. And don’t forget to bring your notebook, I’ll be sharing plenty of literature tips!

Transcript

1. SAFETY SCIENCE
   
   
   A PRIMER
   Michiel Rook
   

   @michieltcs
   

   View Slide

2. LEARNING FROM
   INCIDENTS
   
   
   A PRIMER
   Michiel Rook
   

   @michieltcs
   

   View Slide

3. View Slide

4. View Slide

5. @michieltcs
   #StandWithUkraine
   (JUST) AN
   INTRODUCTION TO
   

   View Slide

6. @michieltcs
   #StandWithUkraine
   SAFETY SCIENCE
   

   View Slide

7. @michieltcs
   #StandWithUkraine
   INCIDENTS &
   ACCIDENTS
   

   View Slide

8. View Slide

9. View Slide

10. @michieltcs
    #StandWithUkraine
    LEARNING,
    UNDERSTANDING,
    IMPROVING
    

    View Slide

11. @michieltcs
    #StandWithUkraine
    LEARNING,
    UNDERSTANDING,
    IMPROVING?
    

    View Slide

12. @michieltcs
    #StandWithUkraine
    IMPORTANT
    
    
    THEMES AND
    THEORIES
    

    View Slide

13. @michieltcs
    #StandWithUkraine
    NEWTONIAN /
    CARTESIAN VIEWS
    

    View Slide

14. View Slide

15. @michieltcs
    #StandWithUkraine
    "Cartesians view the
    mind as being wholly
    separate from the
    corporeal body"
    

    View Slide

16. @michieltcs
    #StandWithUkraine
    REDUCTIONIST
    
    
    VIEW
    

    View Slide

17. View Slide

18. @michieltcs
    #StandWithUkraine
    2ND & 3RD LAWS
    

    View Slide

19. @michieltcs
    #StandWithUkraine
    "The change of motion of
    an object is proportional
    to the force impressed;
    and is made in the
    direction... in which the
    force is impressed."
    

    View Slide

20. @michieltcs
    #StandWithUkraine
    "To every action, there is
    always opposed an
    equal reaction"
    

    View Slide

21. @michieltcs
    #StandWithUkraine
    CAUSALITY
    

    View Slide

22. @michieltcs
    #StandWithUkraine
    SYMMETRY
    BETWEEN
    

    CAUSE & EFFECT
    

    View Slide

23. @michieltcs
    #StandWithUkraine
    “What is the cause of the
    accident? This question is
    just as bizarre as asking
    what the cause is of not
    having an accident.”
    S I D N E Y D E K K E R
    

    View Slide

24. @michieltcs
    #StandWithUkraine
    "SWISS CHEESE
    MODEL" AND SAFETY
    BOUNDARIES
    

    View Slide

25. @michieltcs
    #StandWithUkraine
    

    View Slide

26. @michieltcs
    #StandWithUkraine
    "UNCONTAINED AND
    HARMFUL ENERGY"
    

    View Slide

27. @michieltcs
    @michieltcs
    #StandWithUkraine
    

    View Slide

28. @michieltcs
    @michieltcs
    #StandWithUkraine
    

    View Slide

29. @michieltcs
    #StandWithUkraine
    NORMAL ACCIDENTS
    

    View Slide

30. @michieltcs
    #StandWithUkraine
    

    View Slide

31. @michieltcs
    #StandWithUkraine
    POTENTIAL FOR
    CATASTROPHE =
    INHERENT PROPERTY
    OF SOME SYSTEMS
    

    View Slide

32. @michieltcs
    #StandWithUkraine
    COMPLEXITY &
    COUPLING
    

    View Slide

33. @michieltcs
    #StandWithUkraine
    TIGHT
    COUPLING
    
    
    NO DELAYS & SLACK
    
    
    RESPONSIBILITIES
    CLEARLY STATED
    
    
    ONE METHOD TO
    REACH GOAL
    COMPLEX
    SYSTEM
    
    
    UNFAMILIAR
    FEEDBACK LOOPS
    
    
    NON-LINEAR
    
    
    LIMITED
    UNDERSTANDING
    
    
    FAILURES NOT
    ISOLATED
    

    View Slide

34. @michieltcs
    #StandWithUkraine
    REDUNDANCY &
    AUTOMATION
    BEGETS
    COMPLEXITY?
    

    View Slide

35. @michieltcs
    #StandWithUkraine
    COMPLEX SYSTEMS
    CAN BE HIGHLY
    RELIABLE!
    

    View Slide

36. View Slide

37. @michieltcs
    #StandWithUkraine
    NORMALIZATION OF
    DEVIANCE &
    PRACTICAL DRIFT
    

    View Slide

38. @michieltcs
    #StandWithUkraine
    

    View Slide

39. @michieltcs
    #StandWithUkraine
    ACCIDENTS HAVE
    HISTORY
    

    View Slide

40. @michieltcs
    #StandWithUkraine
    ACCIDENTS HAPPEN
    EVEN WHEN EVERY
    ACTOR BEHAVES AS
    EXPECTED!
    

    View Slide

41. @michieltcs
    #StandWithUkraine
    

    View Slide

42. @michieltcs
    #StandWithUkraine
    WHAT CAN GO
    WRONG USUALLY
    GOES RIGHT
    

    View Slide

43. @michieltcs
    #StandWithUkraine
    HUMAN FACTORS &
    HUMAN ERROR
    

    View Slide

44. @michieltcs
    #StandWithUkraine
    “A preliminary FAA review of last week’s
    outage of the Notice to Air Missions
    (NOTAM) system determined that
    contract personnel unintentionally
    deleted
    fi
    les while working to correct
    synchronization between the live
    primary database and a backup
    database”
    "According to the RBQ, a mechanic
    carried out a simple visual inspection of
    the lifts after a malfunction message —
    contrary to the manufacturer's
    guidelines — and authorized a restart."
    

    View Slide

45. @michieltcs
    #StandWithUkraine
    "LOSS OF
    SITUATIONAL
    AWARENESS"
    

    View Slide

46. @michieltcs
    #StandWithUkraine
    "INEFFECTIVE
    COMMUNICATION"
    

    View Slide

47. @michieltcs
    #StandWithUkraine
    "COMPLACENT"
    

    View Slide

48. @michieltcs
    #StandWithUkraine
    "NOT FOLLOWING
    THE ESTABLISHED
    RULES"
    

    View Slide

49. @michieltcs
    #StandWithUkraine
    “There is almost no
    human action or decision
    that cannot be made to
    look
    fl
    awed and less
    sensible in the
    misleading light of
    hindsight.”
    S I D N E Y D E K K E R
    

    View Slide

50. @michieltcs
    #StandWithUkraine
    SHARP END
    
    
    &
    
    
    BLUNT END
    

    View Slide

51. @michieltcs
    #StandWithUkraine
    HINDSIGHT BIAS
    

    View Slide

52. @michieltcs
    #StandWithUkraine
    OUTCOME BIAS
    

    View Slide

53. @michieltcs
    #StandWithUkraine
    CONFIRMATION
    
    
    BIAS
    

    View Slide

54. @michieltcs
    #StandWithUkraine
    WORK AS IMAGINED
    
    
    VS.
    
    
    WORK AS DONE
    

    View Slide

55. @michieltcs
    #StandWithUkraine
    LOCAL RATIONALITY
    

    View Slide

56. @michieltcs
    #StandWithUkraine
    “People do things that make
    sense to them, given their
    goals, understanding of the
    situation and focus of
    attention at that time.”
    E U R O C O N T R O L
    

    View Slide

57. @michieltcs
    #StandWithUkraine
    HUMANS AS A
    PROBLEM TO
    CONTROL
    

    View Slide

58. @michieltcs
    #StandWithUkraine
    VS
    

    View Slide

59. @michieltcs
    #StandWithUkraine
    HUMANS AS AN
    ASSET TO USE
    

    View Slide

60. @michieltcs
    #StandWithUkraine
    “Human error is both
    universal & inevitable.”
    JA M ES R E AS O N
    

    View Slide

61. @michieltcs
    #StandWithUkraine
    CONSERVATIVE &
    PROGRESSIVE
    
    
    VIEWS
    

    View Slide

62. @michieltcs
    #StandWithUkraine
    FINDING
    MALFUNCTIONS IN
    SYSTEMS
    

    View Slide

63. @michieltcs
    #StandWithUkraine
    VS
    

    View Slide

64. @michieltcs
    #StandWithUkraine
    LEARNING HOW A
    SYSTEM WORKS IN
    PRACTICE
    

    View Slide

65. @michieltcs
    #StandWithUkraine
    HUMANS AS
    
    
    A CAUSE
    

    View Slide

66. @michieltcs
    #StandWithUkraine
    VS
    

    View Slide

67. @michieltcs
    #StandWithUkraine
    HUMANS ARE PART
    OF THE SYSTEM &
    SUBJECT TO ITS
    COMPLEXITY
    

    View Slide

68. @michieltcs
    #StandWithUkraine
    ASKING WHY PEOPLE
    DO THINGS
    

    View Slide

69. @michieltcs
    #StandWithUkraine
    VS
    

    View Slide

70. @michieltcs
    #StandWithUkraine
    UNDERSTANDING
    HOW PEOPLE ADAPT
    

    View Slide

71. BACK TO HAWAII
    https:/
    /
    fl
    ic.kr/p/9cJyEY
    

    View Slide

72. @michieltcs
    #StandWithUkraine
    TWO CLICKS OF A
    MOUSE?
    

    View Slide

73. @michieltcs
    #StandWithUkraine
    Events Leading Up to the False Alert
    3
    Time Events
    0805
    • HI-EMA’s midnight shift supervisor begins a no-notice ballistic missile defense drill at
    a shift change by placing a call, pretending to be U.S. Pacific Command, to the day
    shift warning officers.
    • The midnight shift supervisor plays a recording over the phone that properly includes
    the drill language “EXERCISE, EXERCISE, EXERCISE,” but also erroneously
    contains the text of an EAS message for a live ballistic missile alert, including the
    language, “THIS IS NOT A DRILL.” The recording does not follow the script
    contained in HI-EMA’s standard operating procedure for this drill.
    • The day shift warning officers receive this recorded message on speakerphone.
    • While other warning officers understand that this is a drill, the warning officer at the
    alert origination terminal claimed to believe, in a written statement provided to HI-
    EMA, that this was a real emergency, not a drill.
    0807
    • This day shift warning officer responds, as trained for a real event, by transmitting a
    live incoming ballistic missile alert to the State of Hawaii.
    • In doing so, the day shift warning officer selects the template for a live alert from a
    drop-down menu, and clicks “yes” in response to a prompt that reads, “Are you sure
    that you want to send this Alert?”
    Preliminary Report: Hawaii Emergency Management Agency’s January 13, 2018 False Ballistic Missile Alert.
    

    View Slide

74. @michieltcs
    #StandWithUkraine
    Preliminary Report: Hawaii Emergency Management Agency’s January 13, 2018 False Ballistic Missile Alert.
    Events After the False Alert
    4
    Time Events
    0808 • Day shift warning officer receives false WEA on mobile device
    0809 • HI-EMA notifies Hawaii Governor of false alert
    0810 • HI-EMA to U.S. Pacific Command and Honolulu PD: no missile launch
    0812 • HI-EMA issues a cancellation, ceasing retransmission over EAS, WEA
    0813 • HI-EMA begins outreach, but its phone lines become congested
    0820 • HI-EMA posts on Facebook, Twitter – “NO missile threat to Hawaii”
    0824 • Hawaii Governor retweets notice that there is no missile threat
    0827 • HI-EMA determines that an EAS, WEA Civil Emergency Message (CEM) is the
    best vehicle for correction
    0830 • FEMA confirms HI-EMA’s view on CEM; Hawaii Governor posts correction on
    Facebook
    0831 • HI-EMA supervisor logs into alert system, begins to create false alert correction
    0845 • HI-EMA issues correction through EAS and WEA that there is no missile threat
    

    View Slide

75. @michieltcs
    #StandWithUkraine
    HUMAN ERROR?
    

    View Slide

76. @michieltcs
    #StandWithUkraine
    False Ballistic Missile Alert Investigation for January 13, 2018, Hawaii Department of Defense
    

    View Slide

77. @michieltcs
    #StandWithUkraine
    Stress or stressors ‣ Stress and anxiety created by recent North Korea nuclear attack concerns.
    
    
    ‣ Drills were a recently added training exercise.
    
    
    ‣ Drills were administered spontaneously.
    
    
    ‣
    Experience and training ‣ Employee was a 10-year veteran of the agency and a supervisor.
    
    
    ‣ Standard practice test conducted during each shift change three times a day were not yet routine enough to be predictable, but
    they were not entirely new.
    
    
    ‣ Employees had not been trained on the new FEMA software currently being used.
    
    
    ‣ No records maintained for employee training and to what degree.
    Ergonomics or HMI ‣ Interfaces had not been updated.
    
    
    ‣ File names were very similar with little distinction between the test missile alert and the genuine missile alert.
    Procedures ‣ The recording does not follow the script contained in HI-EMA’s standard operating procedure for this drill.
    
    
    ‣ Recorded message was incorrectly interpreted as part of an unscheduled drill.
    
    
    ‣ “The day-shift manager was not prepared to supervise the morning test,” the FCC said.
    
    
    ‣ Following standard procedures, the night-shift supervisor posing as Paci
    fi
    c Command played a recorded message to the
    emergency workers warning them of the fake threat. The message included the phrase “Exercise, exercise, exercise.” But the
    message inaccurately included the phrase “This is not a drill.”
    Fitness for duty ‣ Reports stated that Hawaii emergency management o
    ff i
    cials knew for years that the employee had problems performing his
    job.
    
    
    ‣ In the past, the employee in question had mistakenly believed drills for tsunami and
    fi
    re warnings were actual events.
    
    
    ‣ Colleagues reported not feeling comfortable working with the individual. Supervisors had counselled the employee repeatedly,
    yet retained him for a decade in a position that had to be renewed each year.
    
    
    ‣ During the drill, the employee reported that he did not hear the word “exercise” repeated six times. All of the fellow o
    ff
    i
    cers
    participating in the drill con
    fi
    rmed that they had clearly heard the word “exercise” during the drill.
    Work processes ‣ FEMA and HI-EMA did not have clear policies for issuing alerts. HI-EMA waited for permission to issue a second alert stating that
    the
    fi
    rst alert was a false alarm. According to FEMA, this was not a requirement.
    
    
    ‣ The agency had a vague checklist for missile alerts, allowing workers to interpret the steps they should follow di
    ff
    erently.
    
    
    ‣ HI-EMA Managers didn’t require a second person to sign o
    ff
    on alerts before they were sent.
    
    
    ‣ The agency lacked any preparation on how to correct a false warning. New computer software programs had been added to
    the agency’s computers but no training had been conducted to teach o
    ff i
    cers how to apply it.
    Human Reliability Considerations from the Hawaii Ballistic Missile Alert Event
    

    View Slide

78. @michieltcs
    #StandWithUkraine
    “another clear example of
    how failures occur in
    complex technological
    systems”
    H E AT H E R M E D E M A , H A R O L D B L AC K M A N ,
    K AT E RY N A SAVC H E N KO, R O N A L D B O R I N G
    

    View Slide

79. View Slide

80. @michieltcs
    #StandWithUkraine
    "Incidents are a fact of life.
    

    

    How well you respond is your
    choice."
    J I M S E V E R I N O
    

    View Slide

81. @michieltcs
    #StandWithUkraine
    POST-INCIDENT
    REVIEWS
    

    View Slide

82. @michieltcs
    #StandWithUkraine
    (HUMAN)
    LIMITATIONS AND
    TENDENCIES
    

    View Slide

83. @michieltcs
    #StandWithUkraine
    STRESS
    
    
    &
    
    
    EMOTIONS
    

    View Slide

84. @michieltcs
    #StandWithUkraine
    BLAMELESS
    

    VS
    

    BLAME-AWARE
    

    View Slide

85. @michieltcs
    #StandWithUkraine
    BIASES
    

    View Slide

86. @michieltcs
    #StandWithUkraine
    COUNTERFACTUALS
    

    View Slide

87. @michieltcs
    #StandWithUkraine
    COGNITIVE EFFECTS
    OF CHANGES
    

    View Slide

88. @michieltcs
    #StandWithUkraine
    ROOT CAUSE
    ANALYSIS?
    

    View Slide

89. @michieltcs
    #StandWithUkraine
    "What you call 'root
    cause' is simply the
    place where you stop
    looking any further."
    S I D N E Y D E K K E R
    

    View Slide

90. @michieltcs
    #StandWithUkraine
    ROOT CAUSES?
    

    View Slide

91. @michieltcs
    #StandWithUkraine
    5 WHYS
    

    View Slide

92. @michieltcs
    #StandWithUkraine
    Figure 1 A causal event tree (adapted from ref. 21). ID, identification; IT, information technology.
    The problem with…
    Alan J. Card - The problem with '5 whys'
    

    BMJ quality & safety · September 2016 DOI: 10.1136/bmjqs-2016-005849
    

    View Slide

93. @michieltcs
    #StandWithUkraine
    “For every complex
    problem, there is an
    answer that is clear,
    simple, and wrong.”
    H . L . M E N C K E N
    

    View Slide

94. @michieltcs
    #StandWithUkraine
    DIAGNOSING AND
    FIXING PROBLEMS
    

    View Slide

95. @michieltcs
    #StandWithUkraine
    PREVENTING
    INCIDENTS?
    

    View Slide

96. @michieltcs
    #StandWithUkraine
    CREATING NEW
    INCIDENTS?
    

    View Slide

97. @michieltcs
    #StandWithUkraine
    WHAT THEN?
    

    View Slide

98. @michieltcs
    #StandWithUkraine
    WHAT INFORMATION
    WAS AVAILABLE?
    

    View Slide

99. @michieltcs
    #StandWithUkraine
    WHAT INFORMATION
    MADE SENSE?
    

    View Slide

100. @michieltcs
     #StandWithUkraine
     WHAT FACTORS
     CONTRIBUTED TO
     THE EVENT?
     

     View Slide

101. @michieltcs
     #StandWithUkraine
     HOW DID THIS
     DIFFER FROM
     "NORMAL" OPS?
     

     View Slide

102. @michieltcs
     #StandWithUkraine
     WEAK VS STRONG
     
     
     SIGNALS
     

     View Slide

103. @michieltcs
     #StandWithUkraine
     HOW DID PEOPLE
     ADAPT?
     

     View Slide

104. @michieltcs
     #StandWithUkraine
     OPERATOR
     EXPERIENCE
     

     View Slide

105. @michieltcs
     #StandWithUkraine
     JUST CULTURE &
     PSYCHOLOGICAL
     SAFETY
     

     View Slide

106. @michieltcs
     #StandWithUkraine
     INCIDENTS &
     METRICS
     

     View Slide

107. @michieltcs
     #StandWithUkraine
     DAYS SINCE LAST
     INCIDENT?
     

     View Slide

108. @michieltcs
     #StandWithUkraine
     

     View Slide

109. @michieltcs
     #StandWithUkraine
     NUMBER OF
     INCIDENTS?
     

     View Slide

110. @michieltcs
     #StandWithUkraine
     

     View Slide

111. @michieltcs
     #StandWithUkraine
     TIME TO DETECT?
     

     View Slide

112. @michieltcs
     #StandWithUkraine
     TIME TO RECOVER?
     

     View Slide

113. @michieltcs
     #StandWithUkraine
     READERS OF
     INCIDENT REPORTS?
     

     View Slide

114. @michieltcs
     #StandWithUkraine
     NEAR MISSES?
     

     View Slide

115. @michieltcs
     #StandWithUkraine
     HUMANS AS AN
     

     (ADAPTABLE)
     

     ASSET TO USE
     

     View Slide

116. @michieltcs
     #StandWithUkraine
     H T T P S : // V I M EO.C O M /370 0 0 8 1 57
     

     View Slide

117. @michieltcs
     #StandWithUkraine
     WHERE TO GO
     

     FROM HERE?
     

     View Slide

118. @michieltcs
     #StandWithUkraine
     

     View Slide

119. @michieltcs
     #StandWithUkraine
     https:/
     /www.humanfactors.lth.se/
     
     
     https:/
     /www.learningfromincidents.io/
     
     
     https:/
     /www.thevoid.community/
     
     
     http:/
     /resiliencepapers.club/
     

     View Slide

120. @michieltcs
     #StandWithUkraine
     “Everything is
     unprecedented until it
     happens for the
     fi
     rst
     time.”
     C H ES L E Y B . S U L L E N B E R G E R
     

     View Slide

121. @michieltcs
     @michieltcs
     #StandWithUkraine
     THANK YOU FOR
     LISTENING!
     
     
     twitter: @michieltcs
     

     mastodon: @michielrook@mastodon.nl
     

     email: michiel@michielrook.nl
     
     
     www.michielrook.nl
     

     View Slide