Thinking Fast and Slow for Testers: Cognitive Distortions Affecting Testing

In his book Thinking Fast and Slow, Nobel winner Daniel Kahneman introduces two mental systems, one that is fast and the other slow. Together they shape our impressions of the world around us and help us make choices. System 1 is largely unconscious and makes snap judgments based upon our memory of similar events and our emotions. System 2 is painfully slow, and is the process by which we consciously check the facts and think carefully and rationally. However, System 2 is easily distracted and System 1 is wrong quite often. System 1 is easily swayed by our emotions. Examples he cites include the fact that pro golfers are more accurate when putting for par than they are for birdie (regardless of distance), and people buy more cans of soup when there's a sign on the display that says "Limit 12 per customer." In this session we will, through interactive games, learn how our minds can affect our abilities to investigate, observe, and recall events. We’ll discuss some strategies to minimize our minds erroneous impacts to those abilities.

Transcript

1. Thinking Fast and Slow for Testers Cognitive Distortions affecting Testing
2. Introducing Joseph Ours Director of Quality Assurance and Testing Services More than 15 years of IT experience spanning several industries and roles Successfully managed multi-million dollar initiatives while introducing innovative solutions for Fortune 1000 companies, resulting in improved quality, increased client satisfaction, cost avoidance, on- time delivery, and establishing a true team culture @justjoehere Joseph.ours@gmail.com www.linkedin.com/in/josephours
5. Why is it?
7. Bug Hunting Skills Investigate Search Probe Explore Interrogate Observe Detect Discover Monitor Study Recall Think of Recollect Summon Cite
8. Is affected by our mind…. Which is replete with judgment and bias
9. Understanding Observations Assimilate Interpret Observe
10. Understanding Communication Recall Interpret Describe
11. But each observations informs and changes our expectations based on the differences in expectations Working Together What we see or don’t see Is often based on what we expect to see And is stored in memory based on closest associative other memory And affects what we see in the future and what we tell others we observed
12. Filters and Biases Social Bias Decision Bias Memory Bias Belief Bias
13. Example Social Biases http://www.slideshare.net/efern211/cognitive-biases-a-visual-study-guide-by-the-royal-society-of-account-planning
14. Example Memory Biases http://www.slideshare.net/efern211/cognitive-biases-a-visual-study-guide-by-the-royal-society-of-account-planning
15. Example Decision Making Biases http://www.slideshare.net/efern211/cognitive-biases-a-visual-study-guide-by-the-royal-society-of-account-planning
16. Example Belief Biases http://www.slideshare.net/efern211/cognitive-biases-a-visual-study-guide-by-the-royal-society-of-account-planning
17. Remember I said your mind is replete with biases Don’t take my word for it, let’s go through each key area and perform some exercises to see biases in action
18. Examples Time for some FUN activities!
19. BIAS AFFECTING YOUR INVESTIGATIVE SKILLS
20. What comes Next? 2 4 6
21. Did you guess? 2 4 6 8 10 12 N=n*2 linear sequence n=(1,2,3,4,5,6)
22. Think about it?
23. Did you consider? 2 4 6 10 14 22 N=n*2 Prime sequence n=(1,2,3,5,7,11)
24. Or? 2 4 6 10 16 26 Nn =(N2 )+(N1 )
25. Non-numerical Example 2 people died in a cabin How?
26. Beizer’s Pesticide Paradox
27. Improve your investigative skills Change your search pattern Search for alternate answers and/or paths Defend your answer against antagonist Ensure gaps are not unduly filled in
28. BIAS AFFECTING YOUR OBSERVATION SKILLS
31. What are the differences?
32. Sometimes flicker helps
33. Observation Test The Flicker Test at Work!
34. Improve your observation skills Time-box your activities Focus/De-focus Don’t over rely on expected results Listen to intuition
35. BIAS AFFECTING YOUR RECALL SKILLS
36. Recall Skills Get Ready to Pick a Card from Those Shown Memorize the Card Let the Fun Begin!
37. Recall Test Is your card missing? Stare Here!
38. Recall Test I need 4 volunteers
39. Recall Test Pretend ALL of you are doing this at your seat
40. Improve your recall skillls Acknowledge your recall CAN be faulty Record as you go, keep a journal Or use a recording tool Chunk Tests into groups 5-7 Reduce cognitive load
41. Recap • Our biases affect all aspects of testing • What we look for, what we observe, and what we miss • How we see it • How we recall it
42. Minimizing Bias Investigation • Change your search patterns • Look for alternate answers • Defend your answer against antagonist • What your fill-ins Observation • Focus/De-focus • Time box your activities • Listen to intuition • Don’t over rely on documentation, think Recall • Limit what you have to recall • Use tools to assist • Record/Journal as you go • Reduce cognitive load Acknowledge Biases Exist
43. Q&A
44. OTHER THINGS TO CONSIDER
45. Time’ s Up Myth of Multitasking – Step 1 1 2 3 4 5 6 7 8
46. Time’ s Up Myth of Multitasking – Step 2 1 2 3 4 5 6 7 8
47. Time’ s Up Myth of Multitasking – Step 3 1 2 3 4 5 6 7 8
48. Bias with Metrics! You have – Urn with 1 red and 10 white marbles – Urn with 8 red and 100 white marbles Which urn will you use to try draw a red?
49. Law of Small Numbers A false conclusion based on a logical assumption with insufficient evidence • Metrics • Frequency of occurrence of a defect • Actions taken to reproduce a defect • Test Approach • Test sample size is too small to validate functionality • Equivalency class is more reliable than it is • Pairwise testing covers more than it does Where testers may encounter this
50. Meta Example • See a crime (car running from cops, crashes into another car), hear others give their accounting – You believe as others • An investigator “leads” a question (how fast was car going when smash/bump) – Your memory is altered • Telling friends of crime, suggests car driver was drunk – despite driving straight on road • Conveying driver was guilty – based on stereotype of runners