Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Safety science - a primer (ConFoo 2023)

Michiel Rook
February 22, 2023

Safety science - a primer (ConFoo 2023)

Incidents are a fact of life, an inevitable cost of doing business. But they can teach us a lot about our organizations and our work.

How we look at & learn from incidents has been evolving over the past decades, both in- and outside our industry.

Join me to learn about some of the most important current themes and theories in incident investigation and safety science. Then I will discuss some pointers that you can apply in your own organization. And don’t forget to bring your notebook, I’ll be sharing plenty of literature tips!

Michiel Rook

February 22, 2023
Tweet

More Decks by Michiel Rook

Other Decks in Science

Transcript

  1. SAFETY SCIENCE


    A PRIMER
    Michiel Rook

    @michieltcs

    View Slide

  2. LEARNING FROM
    INCIDENTS


    A PRIMER
    Michiel Rook

    @michieltcs

    View Slide

  3. View Slide

  4. View Slide

  5. @michieltcs
    #StandWithUkraine
    (JUST) AN
    INTRODUCTION TO

    View Slide

  6. @michieltcs
    #StandWithUkraine
    SAFETY SCIENCE

    View Slide

  7. @michieltcs
    #StandWithUkraine
    INCIDENTS &
    ACCIDENTS

    View Slide

  8. View Slide

  9. View Slide

  10. @michieltcs
    #StandWithUkraine
    LEARNING,
    UNDERSTANDING,
    IMPROVING

    View Slide

  11. @michieltcs
    #StandWithUkraine
    LEARNING,
    UNDERSTANDING,
    IMPROVING?

    View Slide

  12. @michieltcs
    #StandWithUkraine
    IMPORTANT


    THEMES AND
    THEORIES

    View Slide

  13. @michieltcs
    #StandWithUkraine
    NEWTONIAN /
    CARTESIAN VIEWS

    View Slide

  14. View Slide

  15. @michieltcs
    #StandWithUkraine
    "Cartesians view the
    mind as being wholly
    separate from the
    corporeal body"

    View Slide

  16. @michieltcs
    #StandWithUkraine
    REDUCTIONIST


    VIEW

    View Slide

  17. View Slide

  18. @michieltcs
    #StandWithUkraine
    2ND & 3RD LAWS

    View Slide

  19. @michieltcs
    #StandWithUkraine
    "The change of motion of
    an object is proportional
    to the force impressed;
    and is made in the
    direction... in which the
    force is impressed."

    View Slide

  20. @michieltcs
    #StandWithUkraine
    "To every action, there is
    always opposed an
    equal reaction"

    View Slide

  21. @michieltcs
    #StandWithUkraine
    CAUSALITY

    View Slide

  22. @michieltcs
    #StandWithUkraine
    SYMMETRY
    BETWEEN

    CAUSE & EFFECT

    View Slide

  23. @michieltcs
    #StandWithUkraine
    “What is the cause of the
    accident? This question is
    just as bizarre as asking
    what the cause is of not
    having an accident.”
    S I D N E Y D E K K E R

    View Slide

  24. @michieltcs
    #StandWithUkraine
    "SWISS CHEESE
    MODEL" AND SAFETY
    BOUNDARIES

    View Slide

  25. @michieltcs
    #StandWithUkraine

    View Slide

  26. @michieltcs
    #StandWithUkraine
    "UNCONTAINED AND
    HARMFUL ENERGY"

    View Slide

  27. @michieltcs
    @michieltcs
    #StandWithUkraine

    View Slide

  28. @michieltcs
    @michieltcs
    #StandWithUkraine

    View Slide

  29. @michieltcs
    #StandWithUkraine
    NORMAL ACCIDENTS

    View Slide

  30. @michieltcs
    #StandWithUkraine

    View Slide

  31. @michieltcs
    #StandWithUkraine
    POTENTIAL FOR
    CATASTROPHE =
    INHERENT PROPERTY
    OF SOME SYSTEMS

    View Slide

  32. @michieltcs
    #StandWithUkraine
    COMPLEXITY &
    COUPLING

    View Slide

  33. @michieltcs
    #StandWithUkraine
    TIGHT
    COUPLING


    NO DELAYS & SLACK


    RESPONSIBILITIES
    CLEARLY STATED


    ONE METHOD TO
    REACH GOAL
    COMPLEX
    SYSTEM


    UNFAMILIAR
    FEEDBACK LOOPS


    NON-LINEAR


    LIMITED
    UNDERSTANDING


    FAILURES NOT
    ISOLATED

    View Slide

  34. @michieltcs
    #StandWithUkraine
    REDUNDANCY &
    AUTOMATION
    BEGETS
    COMPLEXITY?

    View Slide

  35. @michieltcs
    #StandWithUkraine
    COMPLEX SYSTEMS
    CAN BE HIGHLY
    RELIABLE!

    View Slide

  36. View Slide

  37. @michieltcs
    #StandWithUkraine
    NORMALIZATION OF
    DEVIANCE &
    PRACTICAL DRIFT

    View Slide

  38. @michieltcs
    #StandWithUkraine

    View Slide

  39. @michieltcs
    #StandWithUkraine
    ACCIDENTS HAVE
    HISTORY

    View Slide

  40. @michieltcs
    #StandWithUkraine
    ACCIDENTS HAPPEN
    EVEN WHEN EVERY
    ACTOR BEHAVES AS
    EXPECTED!

    View Slide

  41. @michieltcs
    #StandWithUkraine

    View Slide

  42. @michieltcs
    #StandWithUkraine
    WHAT CAN GO
    WRONG USUALLY
    GOES RIGHT

    View Slide

  43. @michieltcs
    #StandWithUkraine
    HUMAN FACTORS &
    HUMAN ERROR

    View Slide

  44. @michieltcs
    #StandWithUkraine
    “A preliminary FAA review of last week’s
    outage of the Notice to Air Missions
    (NOTAM) system determined that
    contract personnel unintentionally
    deleted
    fi
    les while working to correct
    synchronization between the live
    primary database and a backup
    database”
    "According to the RBQ, a mechanic
    carried out a simple visual inspection of
    the lifts after a malfunction message —
    contrary to the manufacturer's
    guidelines — and authorized a restart."

    View Slide

  45. @michieltcs
    #StandWithUkraine
    "LOSS OF
    SITUATIONAL
    AWARENESS"

    View Slide

  46. @michieltcs
    #StandWithUkraine
    "INEFFECTIVE
    COMMUNICATION"

    View Slide

  47. @michieltcs
    #StandWithUkraine
    "COMPLACENT"

    View Slide

  48. @michieltcs
    #StandWithUkraine
    "NOT FOLLOWING
    THE ESTABLISHED
    RULES"

    View Slide

  49. @michieltcs
    #StandWithUkraine
    “There is almost no
    human action or decision
    that cannot be made to
    look
    fl
    awed and less
    sensible in the
    misleading light of
    hindsight.”
    S I D N E Y D E K K E R

    View Slide

  50. @michieltcs
    #StandWithUkraine
    SHARP END


    &


    BLUNT END

    View Slide

  51. @michieltcs
    #StandWithUkraine
    HINDSIGHT BIAS

    View Slide

  52. @michieltcs
    #StandWithUkraine
    OUTCOME BIAS

    View Slide

  53. @michieltcs
    #StandWithUkraine
    CONFIRMATION


    BIAS

    View Slide

  54. @michieltcs
    #StandWithUkraine
    WORK AS IMAGINED


    VS.


    WORK AS DONE

    View Slide

  55. @michieltcs
    #StandWithUkraine
    LOCAL RATIONALITY

    View Slide

  56. @michieltcs
    #StandWithUkraine
    “People do things that make
    sense to them, given their
    goals, understanding of the
    situation and focus of
    attention at that time.”
    E U R O C O N T R O L

    View Slide

  57. @michieltcs
    #StandWithUkraine
    HUMANS AS A
    PROBLEM TO
    CONTROL

    View Slide

  58. @michieltcs
    #StandWithUkraine
    VS

    View Slide

  59. @michieltcs
    #StandWithUkraine
    HUMANS AS AN
    ASSET TO USE

    View Slide

  60. @michieltcs
    #StandWithUkraine
    “Human error is both
    universal & inevitable.”
    JA M ES R E AS O N

    View Slide

  61. @michieltcs
    #StandWithUkraine
    CONSERVATIVE &
    PROGRESSIVE


    VIEWS

    View Slide

  62. @michieltcs
    #StandWithUkraine
    FINDING
    MALFUNCTIONS IN
    SYSTEMS

    View Slide

  63. @michieltcs
    #StandWithUkraine
    VS

    View Slide

  64. @michieltcs
    #StandWithUkraine
    LEARNING HOW A
    SYSTEM WORKS IN
    PRACTICE

    View Slide

  65. @michieltcs
    #StandWithUkraine
    HUMANS AS


    A CAUSE

    View Slide

  66. @michieltcs
    #StandWithUkraine
    VS

    View Slide

  67. @michieltcs
    #StandWithUkraine
    HUMANS ARE PART
    OF THE SYSTEM &
    SUBJECT TO ITS
    COMPLEXITY

    View Slide

  68. @michieltcs
    #StandWithUkraine
    ASKING WHY PEOPLE
    DO THINGS

    View Slide

  69. @michieltcs
    #StandWithUkraine
    VS

    View Slide

  70. @michieltcs
    #StandWithUkraine
    UNDERSTANDING
    HOW PEOPLE ADAPT

    View Slide

  71. BACK TO HAWAII
    https:/
    /
    fl
    ic.kr/p/9cJyEY

    View Slide

  72. @michieltcs
    #StandWithUkraine
    TWO CLICKS OF A
    MOUSE?

    View Slide

  73. @michieltcs
    #StandWithUkraine
    Events Leading Up to the False Alert
    3
    Time Events
    0805
    • HI-EMA’s midnight shift supervisor begins a no-notice ballistic missile defense drill at
    a shift change by placing a call, pretending to be U.S. Pacific Command, to the day
    shift warning officers.
    • The midnight shift supervisor plays a recording over the phone that properly includes
    the drill language “EXERCISE, EXERCISE, EXERCISE,” but also erroneously
    contains the text of an EAS message for a live ballistic missile alert, including the
    language, “THIS IS NOT A DRILL.” The recording does not follow the script
    contained in HI-EMA’s standard operating procedure for this drill.
    • The day shift warning officers receive this recorded message on speakerphone.
    • While other warning officers understand that this is a drill, the warning officer at the
    alert origination terminal claimed to believe, in a written statement provided to HI-
    EMA, that this was a real emergency, not a drill.
    0807
    • This day shift warning officer responds, as trained for a real event, by transmitting a
    live incoming ballistic missile alert to the State of Hawaii.
    • In doing so, the day shift warning officer selects the template for a live alert from a
    drop-down menu, and clicks “yes” in response to a prompt that reads, “Are you sure
    that you want to send this Alert?”
    Preliminary Report: Hawaii Emergency Management Agency’s January 13, 2018 False Ballistic Missile Alert.

    View Slide

  74. @michieltcs
    #StandWithUkraine
    Preliminary Report: Hawaii Emergency Management Agency’s January 13, 2018 False Ballistic Missile Alert.
    Events After the False Alert
    4
    Time Events
    0808 • Day shift warning officer receives false WEA on mobile device
    0809 • HI-EMA notifies Hawaii Governor of false alert
    0810 • HI-EMA to U.S. Pacific Command and Honolulu PD: no missile launch
    0812 • HI-EMA issues a cancellation, ceasing retransmission over EAS, WEA
    0813 • HI-EMA begins outreach, but its phone lines become congested
    0820 • HI-EMA posts on Facebook, Twitter – “NO missile threat to Hawaii”
    0824 • Hawaii Governor retweets notice that there is no missile threat
    0827 • HI-EMA determines that an EAS, WEA Civil Emergency Message (CEM) is the
    best vehicle for correction
    0830 • FEMA confirms HI-EMA’s view on CEM; Hawaii Governor posts correction on
    Facebook
    0831 • HI-EMA supervisor logs into alert system, begins to create false alert correction
    0845 • HI-EMA issues correction through EAS and WEA that there is no missile threat

    View Slide

  75. @michieltcs
    #StandWithUkraine
    HUMAN ERROR?

    View Slide

  76. @michieltcs
    #StandWithUkraine
    False Ballistic Missile Alert Investigation for January 13, 2018, Hawaii Department of Defense

    View Slide

  77. @michieltcs
    #StandWithUkraine
    Stress or stressors ‣ Stress and anxiety created by recent North Korea nuclear attack concerns.


    ‣ Drills were a recently added training exercise.


    ‣ Drills were administered spontaneously.



    Experience and training ‣ Employee was a 10-year veteran of the agency and a supervisor.


    ‣ Standard practice test conducted during each shift change three times a day were not yet routine enough to be predictable, but
    they were not entirely new.


    ‣ Employees had not been trained on the new FEMA software currently being used.


    ‣ No records maintained for employee training and to what degree.
    Ergonomics or HMI ‣ Interfaces had not been updated.


    ‣ File names were very similar with little distinction between the test missile alert and the genuine missile alert.
    Procedures ‣ The recording does not follow the script contained in HI-EMA’s standard operating procedure for this drill.


    ‣ Recorded message was incorrectly interpreted as part of an unscheduled drill.


    ‣ “The day-shift manager was not prepared to supervise the morning test,” the FCC said.


    ‣ Following standard procedures, the night-shift supervisor posing as Paci
    fi
    c Command played a recorded message to the
    emergency workers warning them of the fake threat. The message included the phrase “Exercise, exercise, exercise.” But the
    message inaccurately included the phrase “This is not a drill.”
    Fitness for duty ‣ Reports stated that Hawaii emergency management o
    ff i
    cials knew for years that the employee had problems performing his
    job.


    ‣ In the past, the employee in question had mistakenly believed drills for tsunami and
    fi
    re warnings were actual events.


    ‣ Colleagues reported not feeling comfortable working with the individual. Supervisors had counselled the employee repeatedly,
    yet retained him for a decade in a position that had to be renewed each year.


    ‣ During the drill, the employee reported that he did not hear the word “exercise” repeated six times. All of the fellow o
    ff
    i
    cers
    participating in the drill con
    fi
    rmed that they had clearly heard the word “exercise” during the drill.
    Work processes ‣ FEMA and HI-EMA did not have clear policies for issuing alerts. HI-EMA waited for permission to issue a second alert stating that
    the
    fi
    rst alert was a false alarm. According to FEMA, this was not a requirement.


    ‣ The agency had a vague checklist for missile alerts, allowing workers to interpret the steps they should follow di
    ff
    erently.


    ‣ HI-EMA Managers didn’t require a second person to sign o
    ff
    on alerts before they were sent.


    ‣ The agency lacked any preparation on how to correct a false warning. New computer software programs had been added to
    the agency’s computers but no training had been conducted to teach o
    ff i
    cers how to apply it.
    Human Reliability Considerations from the Hawaii Ballistic Missile Alert Event

    View Slide

  78. @michieltcs
    #StandWithUkraine
    “another clear example of
    how failures occur in
    complex technological
    systems”
    H E AT H E R M E D E M A , H A R O L D B L AC K M A N ,
    K AT E RY N A SAVC H E N KO, R O N A L D B O R I N G

    View Slide

  79. View Slide

  80. @michieltcs
    #StandWithUkraine
    "Incidents are a fact of life.


    How well you respond is your
    choice."
    J I M S E V E R I N O

    View Slide

  81. @michieltcs
    #StandWithUkraine
    POST-INCIDENT
    REVIEWS

    View Slide

  82. @michieltcs
    #StandWithUkraine
    (HUMAN)
    LIMITATIONS AND
    TENDENCIES

    View Slide

  83. @michieltcs
    #StandWithUkraine
    STRESS


    &


    EMOTIONS

    View Slide

  84. @michieltcs
    #StandWithUkraine
    BLAMELESS

    VS

    BLAME-AWARE

    View Slide

  85. @michieltcs
    #StandWithUkraine
    BIASES

    View Slide

  86. @michieltcs
    #StandWithUkraine
    COUNTERFACTUALS

    View Slide

  87. @michieltcs
    #StandWithUkraine
    COGNITIVE EFFECTS
    OF CHANGES

    View Slide

  88. @michieltcs
    #StandWithUkraine
    ROOT CAUSE
    ANALYSIS?

    View Slide

  89. @michieltcs
    #StandWithUkraine
    "What you call 'root
    cause' is simply the
    place where you stop
    looking any further."
    S I D N E Y D E K K E R

    View Slide

  90. @michieltcs
    #StandWithUkraine
    ROOT CAUSES?

    View Slide

  91. @michieltcs
    #StandWithUkraine
    5 WHYS

    View Slide

  92. @michieltcs
    #StandWithUkraine
    Figure 1 A causal event tree (adapted from ref. 21). ID, identification; IT, information technology.
    The problem with…
    Alan J. Card - The problem with '5 whys'

    BMJ quality & safety · September 2016 DOI: 10.1136/bmjqs-2016-005849

    View Slide

  93. @michieltcs
    #StandWithUkraine
    “For every complex
    problem, there is an
    answer that is clear,
    simple, and wrong.”
    H . L . M E N C K E N

    View Slide

  94. @michieltcs
    #StandWithUkraine
    DIAGNOSING AND
    FIXING PROBLEMS

    View Slide

  95. @michieltcs
    #StandWithUkraine
    PREVENTING
    INCIDENTS?

    View Slide

  96. @michieltcs
    #StandWithUkraine
    CREATING NEW
    INCIDENTS?

    View Slide

  97. @michieltcs
    #StandWithUkraine
    WHAT THEN?

    View Slide

  98. @michieltcs
    #StandWithUkraine
    WHAT INFORMATION
    WAS AVAILABLE?

    View Slide

  99. @michieltcs
    #StandWithUkraine
    WHAT INFORMATION
    MADE SENSE?

    View Slide

  100. @michieltcs
    #StandWithUkraine
    WHAT FACTORS
    CONTRIBUTED TO
    THE EVENT?

    View Slide

  101. @michieltcs
    #StandWithUkraine
    HOW DID THIS
    DIFFER FROM
    "NORMAL" OPS?

    View Slide

  102. @michieltcs
    #StandWithUkraine
    WEAK VS STRONG


    SIGNALS

    View Slide

  103. @michieltcs
    #StandWithUkraine
    HOW DID PEOPLE
    ADAPT?

    View Slide

  104. @michieltcs
    #StandWithUkraine
    OPERATOR
    EXPERIENCE

    View Slide

  105. @michieltcs
    #StandWithUkraine
    JUST CULTURE &
    PSYCHOLOGICAL
    SAFETY

    View Slide

  106. @michieltcs
    #StandWithUkraine
    INCIDENTS &
    METRICS

    View Slide

  107. @michieltcs
    #StandWithUkraine
    DAYS SINCE LAST
    INCIDENT?

    View Slide

  108. @michieltcs
    #StandWithUkraine

    View Slide

  109. @michieltcs
    #StandWithUkraine
    NUMBER OF
    INCIDENTS?

    View Slide

  110. @michieltcs
    #StandWithUkraine

    View Slide

  111. @michieltcs
    #StandWithUkraine
    TIME TO DETECT?

    View Slide

  112. @michieltcs
    #StandWithUkraine
    TIME TO RECOVER?

    View Slide

  113. @michieltcs
    #StandWithUkraine
    READERS OF
    INCIDENT REPORTS?

    View Slide

  114. @michieltcs
    #StandWithUkraine
    NEAR MISSES?

    View Slide

  115. @michieltcs
    #StandWithUkraine
    HUMANS AS AN

    (ADAPTABLE)

    ASSET TO USE

    View Slide

  116. @michieltcs
    #StandWithUkraine
    H T T P S : // V I M EO.C O M /370 0 0 8 1 57

    View Slide

  117. @michieltcs
    #StandWithUkraine
    WHERE TO GO

    FROM HERE?

    View Slide

  118. @michieltcs
    #StandWithUkraine

    View Slide

  119. @michieltcs
    #StandWithUkraine
    https:/
    /www.humanfactors.lth.se/


    https:/
    /www.learningfromincidents.io/


    https:/
    /www.thevoid.community/


    http:/
    /resiliencepapers.club/

    View Slide

  120. @michieltcs
    #StandWithUkraine
    “Everything is
    unprecedented until it
    happens for the
    fi
    rst
    time.”
    C H ES L E Y B . S U L L E N B E R G E R

    View Slide

  121. @michieltcs
    @michieltcs
    #StandWithUkraine
    THANK YOU FOR
    LISTENING!


    twitter: @michieltcs

    mastodon: @[email protected]

    email: [email protected]


    www.michielrook.nl

    View Slide