the design of technology. • Accidents can be avoided by having more automation. • Procedures can be speciﬁed to be objective and comprehensive. • Operators just have to follow the procedures to get work done. • “Humans Are Better At” versus “Machines Are Better At” List (HABA-MABA)
modern systems, and also introduces new forms of challenges and risk. • Rules and procedures are always underspeciﬁed, so therefore can’t guarantee safety by themselves without interpreting them in local context. • Events in these environments will require operators to make decisions and take action that cannot be pre-speciﬁed. • The methods and models for “risk” that rely on “human error” categories, accounting, taxonomies, etc. are fraught.
potential for uncovering elements of expertise and related cognitive phenomena.” (Klein, Crandall, Hoffman, 2006) methods, approaches, and techniques cognitive task analysis cognitive work analysis process tracing conversation analysis Critical Decision Method Critical Incident Technique more…
observations and hypotheses others share what has been investigated thus far what’s been happening in the world (news, service provider outages, etc.) time-series data alerts tracing/observability tools recent changes in existing tech new dependencies who is on vacation, at a conference, traveling, etc. status of other ongoing work
trying to understand situations. …cues are only ‘objective’ in a limited sense. …rather, the knowledge and expectancies a person has will determine what counts as a cue and whether it will be noticed.”
are these people? what roles do they play…actually? people for “ﬁxing”…? for understanding? for ‘stemming the bleeding’? for customer support? for…? be productive via hypotheses? via past experience? via…? think
graphs, etc.) that people regularly dismiss or are suspicious of? 0 100 200 300 400 0 10% 20% 30 % 40 % 0 100 200 300 400 0 1,000 2,000 3,000 4,000 1 2 3 4 5 How do people improvise new tools to help them understand what is happening? What tricks do people or teams use to understand how otherwise opaque 3rd party services are behaving?
is critically important. (The stakes are already too high, and we’re behind.) • Doing this well will mean new language, concepts, paradigms, and practices — some of which may be unintuitive and/or controversial. • Must be driven by both research/academia and industry/practitioners. • Vendors: if you pay attention, this will be a competitive advantage for you.