Safety science - a primer (ConFoo 2023)

SAFETY SCIENCE A PRIMER Michiel Rook   @michieltcs

LEARNING FROM INCIDENTS A PRIMER Michiel Rook   @michieltcs

@michieltcs #StandWithUkraine (JUST) AN INTRODUCTION TO

@michieltcs #StandWithUkraine SAFETY SCIENCE

@michieltcs #StandWithUkraine INCIDENTS & ACCIDENTS

@michieltcs #StandWithUkraine LEARNING, UNDERSTANDING, IMPROVING

@michieltcs #StandWithUkraine LEARNING, UNDERSTANDING, IMPROVING?

@michieltcs #StandWithUkraine IMPORTANT THEMES AND THEORIES

@michieltcs #StandWithUkraine NEWTONIAN / CARTESIAN VIEWS

@michieltcs #StandWithUkraine "Cartesians view the mind as being wholly separate
from the corporeal body"

@michieltcs #StandWithUkraine REDUCTIONIST VIEW

@michieltcs #StandWithUkraine 2ND & 3RD LAWS

@michieltcs #StandWithUkraine "The change of motion of an object is
proportional to the force impressed; and is made in the direction... in which the force is impressed."

@michieltcs #StandWithUkraine "To every action, there is always opposed an
equal reaction"

@michieltcs #StandWithUkraine CAUSALITY

@michieltcs #StandWithUkraine SYMMETRY BETWEEN   CAUSE & EFFECT

@michieltcs #StandWithUkraine “What is the cause of the accident? This
question is just as bizarre as asking what the cause is of not having an accident.” S I D N E Y D E K K E R

@michieltcs #StandWithUkraine "SWISS CHEESE MODEL" AND SAFETY BOUNDARIES

@michieltcs #StandWithUkraine

@michieltcs #StandWithUkraine "UNCONTAINED AND HARMFUL ENERGY"

@michieltcs @michieltcs #StandWithUkraine

@michieltcs #StandWithUkraine NORMAL ACCIDENTS

@michieltcs #StandWithUkraine POTENTIAL FOR CATASTROPHE = INHERENT PROPERTY OF SOME
SYSTEMS

@michieltcs #StandWithUkraine COMPLEXITY & COUPLING

@michieltcs #StandWithUkraine TIGHT COUPLING NO DELAYS & SLACK RESPONSIBILITIES CLEARLY
STATED ONE METHOD TO REACH GOAL COMPLEX SYSTEM UNFAMILIAR FEEDBACK LOOPS NON-LINEAR LIMITED UNDERSTANDING FAILURES NOT ISOLATED

@michieltcs #StandWithUkraine REDUNDANCY & AUTOMATION BEGETS COMPLEXITY?

@michieltcs #StandWithUkraine COMPLEX SYSTEMS CAN BE HIGHLY RELIABLE!

@michieltcs #StandWithUkraine NORMALIZATION OF DEVIANCE & PRACTICAL DRIFT

@michieltcs #StandWithUkraine ACCIDENTS HAVE HISTORY

@michieltcs #StandWithUkraine ACCIDENTS HAPPEN EVEN WHEN EVERY ACTOR BEHAVES AS
EXPECTED!

@michieltcs #StandWithUkraine WHAT CAN GO WRONG USUALLY GOES RIGHT

@michieltcs #StandWithUkraine HUMAN FACTORS & HUMAN ERROR

@michieltcs #StandWithUkraine “A preliminary FAA review of last week’s outage
of the Notice to Air Missions (NOTAM) system determined that contract personnel unintentionally deleted fi les while working to correct synchronization between the live primary database and a backup database” "According to the RBQ, a mechanic carried out a simple visual inspection of the lifts after a malfunction message — contrary to the manufacturer's guidelines — and authorized a restart."

@michieltcs #StandWithUkraine "LOSS OF SITUATIONAL AWARENESS"

@michieltcs #StandWithUkraine "INEFFECTIVE COMMUNICATION"

@michieltcs #StandWithUkraine "COMPLACENT"

@michieltcs #StandWithUkraine "NOT FOLLOWING THE ESTABLISHED RULES"

@michieltcs #StandWithUkraine “There is almost no human action or decision
that cannot be made to look fl awed and less sensible in the misleading light of hindsight.” S I D N E Y D E K K E R

@michieltcs #StandWithUkraine SHARP END & BLUNT END

@michieltcs #StandWithUkraine HINDSIGHT BIAS

@michieltcs #StandWithUkraine OUTCOME BIAS

@michieltcs #StandWithUkraine CONFIRMATION BIAS

@michieltcs #StandWithUkraine WORK AS IMAGINED VS. WORK AS DONE

@michieltcs #StandWithUkraine LOCAL RATIONALITY

@michieltcs #StandWithUkraine “People do things that make sense to them,
given their goals, understanding of the situation and focus of attention at that time.” E U R O C O N T R O L

@michieltcs #StandWithUkraine HUMANS AS A PROBLEM TO CONTROL

@michieltcs #StandWithUkraine VS

@michieltcs #StandWithUkraine HUMANS AS AN ASSET TO USE

@michieltcs #StandWithUkraine “Human error is both universal & inevitable.” JA
M ES R E AS O N

@michieltcs #StandWithUkraine CONSERVATIVE & PROGRESSIVE VIEWS

@michieltcs #StandWithUkraine FINDING MALFUNCTIONS IN SYSTEMS

@michieltcs #StandWithUkraine LEARNING HOW A SYSTEM WORKS IN PRACTICE

@michieltcs #StandWithUkraine HUMANS AS A CAUSE

@michieltcs #StandWithUkraine HUMANS ARE PART OF THE SYSTEM & SUBJECT
TO ITS COMPLEXITY

@michieltcs #StandWithUkraine ASKING WHY PEOPLE DO THINGS

@michieltcs #StandWithUkraine UNDERSTANDING HOW PEOPLE ADAPT

BACK TO HAWAII https:/ / fl ic.kr/p/9cJyEY

@michieltcs #StandWithUkraine TWO CLICKS OF A MOUSE?

@michieltcs #StandWithUkraine Events Leading Up to the False Alert 3
Time Events 0805 • HI-EMA’s midnight shift supervisor begins a no-notice ballistic missile defense drill at a shift change by placing a call, pretending to be U.S. Pacific Command, to the day shift warning officers. • The midnight shift supervisor plays a recording over the phone that properly includes the drill language “EXERCISE, EXERCISE, EXERCISE,” but also erroneously contains the text of an EAS message for a live ballistic missile alert, including the language, “THIS IS NOT A DRILL.” The recording does not follow the script contained in HI-EMA’s standard operating procedure for this drill. • The day shift warning officers receive this recorded message on speakerphone. • While other warning officers understand that this is a drill, the warning officer at the alert origination terminal claimed to believe, in a written statement provided to HI- EMA, that this was a real emergency, not a drill. 0807 • This day shift warning officer responds, as trained for a real event, by transmitting a live incoming ballistic missile alert to the State of Hawaii. • In doing so, the day shift warning officer selects the template for a live alert from a drop-down menu, and clicks “yes” in response to a prompt that reads, “Are you sure that you want to send this Alert?” Preliminary Report: Hawaii Emergency Management Agency’s January 13, 2018 False Ballistic Missile Alert.

@michieltcs #StandWithUkraine Preliminary Report: Hawaii Emergency Management Agency’s January 13,
2018 False Ballistic Missile Alert. Events After the False Alert 4 Time Events 0808 • Day shift warning officer receives false WEA on mobile device 0809 • HI-EMA notifies Hawaii Governor of false alert 0810 • HI-EMA to U.S. Pacific Command and Honolulu PD: no missile launch 0812 • HI-EMA issues a cancellation, ceasing retransmission over EAS, WEA 0813 • HI-EMA begins outreach, but its phone lines become congested 0820 • HI-EMA posts on Facebook, Twitter – “NO missile threat to Hawaii” 0824 • Hawaii Governor retweets notice that there is no missile threat 0827 • HI-EMA determines that an EAS, WEA Civil Emergency Message (CEM) is the best vehicle for correction 0830 • FEMA confirms HI-EMA’s view on CEM; Hawaii Governor posts correction on Facebook 0831 • HI-EMA supervisor logs into alert system, begins to create false alert correction 0845 • HI-EMA issues correction through EAS and WEA that there is no missile threat

@michieltcs #StandWithUkraine HUMAN ERROR?

@michieltcs #StandWithUkraine False Ballistic Missile Alert Investigation for January 13,
2018, Hawaii Department of Defense

@michieltcs #StandWithUkraine Stress or stressors ‣ Stress and anxiety created
by recent North Korea nuclear attack concerns. ‣ Drills were a recently added training exercise. ‣ Drills were administered spontaneously. ‣ Experience and training ‣ Employee was a 10-year veteran of the agency and a supervisor. ‣ Standard practice test conducted during each shift change three times a day were not yet routine enough to be predictable, but they were not entirely new. ‣ Employees had not been trained on the new FEMA software currently being used. ‣ No records maintained for employee training and to what degree. Ergonomics or HMI ‣ Interfaces had not been updated. ‣ File names were very similar with little distinction between the test missile alert and the genuine missile alert. Procedures ‣ The recording does not follow the script contained in HI-EMA’s standard operating procedure for this drill. ‣ Recorded message was incorrectly interpreted as part of an unscheduled drill. ‣ “The day-shift manager was not prepared to supervise the morning test,” the FCC said. ‣ Following standard procedures, the night-shift supervisor posing as Paci fi c Command played a recorded message to the emergency workers warning them of the fake threat. The message included the phrase “Exercise, exercise, exercise.” But the message inaccurately included the phrase “This is not a drill.” Fitness for duty ‣ Reports stated that Hawaii emergency management o ff i cials knew for years that the employee had problems performing his job. ‣ In the past, the employee in question had mistakenly believed drills for tsunami and fi re warnings were actual events. ‣ Colleagues reported not feeling comfortable working with the individual. Supervisors had counselled the employee repeatedly, yet retained him for a decade in a position that had to be renewed each year. ‣ During the drill, the employee reported that he did not hear the word “exercise” repeated six times. All of the fellow o ff i cers participating in the drill con fi rmed that they had clearly heard the word “exercise” during the drill. Work processes ‣ FEMA and HI-EMA did not have clear policies for issuing alerts. HI-EMA waited for permission to issue a second alert stating that the fi rst alert was a false alarm. According to FEMA, this was not a requirement. ‣ The agency had a vague checklist for missile alerts, allowing workers to interpret the steps they should follow di ff erently. ‣ HI-EMA Managers didn’t require a second person to sign o ff on alerts before they were sent. ‣ The agency lacked any preparation on how to correct a false warning. New computer software programs had been added to the agency’s computers but no training had been conducted to teach o ff i cers how to apply it. Human Reliability Considerations from the Hawaii Ballistic Missile Alert Event

@michieltcs #StandWithUkraine “another clear example of how failures occur in
complex technological systems” H E AT H E R M E D E M A , H A R O L D B L AC K M A N , K AT E RY N A SAVC H E N KO, R O N A L D B O R I N G

@michieltcs #StandWithUkraine "Incidents are a fact of life.    
How well you respond is your choice." J I M S E V E R I N O

@michieltcs #StandWithUkraine POST-INCIDENT REVIEWS

@michieltcs #StandWithUkraine (HUMAN) LIMITATIONS AND TENDENCIES

@michieltcs #StandWithUkraine STRESS & EMOTIONS

@michieltcs #StandWithUkraine BLAMELESS   VS   BLAME-AWARE

@michieltcs #StandWithUkraine BIASES

@michieltcs #StandWithUkraine COUNTERFACTUALS

@michieltcs #StandWithUkraine COGNITIVE EFFECTS OF CHANGES

@michieltcs #StandWithUkraine ROOT CAUSE ANALYSIS?

@michieltcs #StandWithUkraine "What you call 'root cause' is simply the
place where you stop looking any further." S I D N E Y D E K K E R

@michieltcs #StandWithUkraine ROOT CAUSES?

@michieltcs #StandWithUkraine 5 WHYS

@michieltcs #StandWithUkraine Figure 1 A causal event tree (adapted from
ref. 21). ID, identification; IT, information technology. The problem with… Alan J. Card - The problem with '5 whys'   BMJ quality & safety · September 2016 DOI: 10.1136/bmjqs-2016-005849

@michieltcs #StandWithUkraine “For every complex problem, there is an answer
that is clear, simple, and wrong.” H . L . M E N C K E N

@michieltcs #StandWithUkraine DIAGNOSING AND FIXING PROBLEMS

@michieltcs #StandWithUkraine PREVENTING INCIDENTS?

@michieltcs #StandWithUkraine CREATING NEW INCIDENTS?

@michieltcs #StandWithUkraine WHAT THEN?

@michieltcs #StandWithUkraine WHAT INFORMATION WAS AVAILABLE?

@michieltcs #StandWithUkraine WHAT INFORMATION MADE SENSE?

@michieltcs #StandWithUkraine WHAT FACTORS CONTRIBUTED TO THE EVENT?

@michieltcs #StandWithUkraine HOW DID THIS DIFFER FROM "NORMAL" OPS?

@michieltcs #StandWithUkraine WEAK VS STRONG SIGNALS

@michieltcs #StandWithUkraine HOW DID PEOPLE ADAPT?

@michieltcs #StandWithUkraine OPERATOR EXPERIENCE

@michieltcs #StandWithUkraine JUST CULTURE & PSYCHOLOGICAL SAFETY

@michieltcs #StandWithUkraine INCIDENTS & METRICS

@michieltcs #StandWithUkraine DAYS SINCE LAST INCIDENT?

@michieltcs #StandWithUkraine NUMBER OF INCIDENTS?

@michieltcs #StandWithUkraine TIME TO DETECT?

@michieltcs #StandWithUkraine TIME TO RECOVER?

@michieltcs #StandWithUkraine READERS OF INCIDENT REPORTS?

@michieltcs #StandWithUkraine NEAR MISSES?

@michieltcs #StandWithUkraine HUMANS AS AN   (ADAPTABLE)   ASSET TO
USE

@michieltcs #StandWithUkraine H T T P S : // V
I M EO.C O M /370 0 0 8 1 57

@michieltcs #StandWithUkraine WHERE TO GO   FROM HERE?

@michieltcs #StandWithUkraine https:/ /www.humanfactors.lth.se/ https:/ /www.learningfromincidents.io/ https:/ /www.thevoid.community/ http:/ /resiliencepapers.club/

@michieltcs #StandWithUkraine “Everything is unprecedented until it happens for the
fi rst time.” C H ES L E Y B . S U L L E N B E R G E R

@michieltcs @michieltcs #StandWithUkraine THANK YOU FOR LISTENING! twitter: @michieltcs  
mastodon: @[email protected]   email: [email protected] www.michielrook.nl

Safety science - a primer (ConFoo 2023)

Safety science - a primer (ConFoo 2023)

More Decks by Michiel Rook

Other Decks in Science

Featured

Transcript