Exploratory Testing

An introduction to Exploratory Testing for trainee testers and non-testers.

Transcript

1. @dannysmith Exploratory Testing
2. Agenda Background Charters Observation Session Sheets
3. “ Marchell, One of Elisabeth Hendrickson's coworkers It's so frustrating. No matter how many tests we write, no matter how many test cases we execute, we always find the most serious bugs when we go off the script.
4. • Does the software behave as expected under the conditions it's supposed to be able to handle? • Are there any other risks? Test Strategy Aims
6. “ James Bach Exploratory testing is simultaneous learning, test design and test execution.
7. “ Elisabeth Hendrickson Simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next.
8. • Designing • Executing • Learning • Steering • Timeboxing Essential Elements ⚗
10. Tap and hold on an item causes the photo to dissapear. What next?
11. Tap and hold on an item causes the photo to dissapear. What next?
12. Charters Background → Design / Execute / Learn / Steer Charters Observation Session Sheets
22. Charters
23. • Often generated during conversations: ▶ Agile Ceremonies ▶ Requirements gathering meetings ▶ Conversations with BAs, stakeholders and developers Charters
24. • Often generated during conversations: ▶ Agile Ceremonies ▶ Requirements gathering meetings ▶ Conversations with BAs, stakeholders and developers • Often generated when executing test cases Charters
25. • Often generated during conversations: ▶ Agile Ceremonies ▶ Requirements gathering meetings ▶ Conversations with BAs, stakeholders and developers • Often generated when executing test cases • Must include: ▶ The target area ▶ The resources/techniques/tools we'll use ▶ The information we're looking for Charters
26. • Often generated during conversations: ▶ Agile Ceremonies ▶ Requirements gathering meetings ▶ Conversations with BAs, stakeholders and developers • Often generated when executing test cases • Must include: ▶ The target area ▶ The resources/techniques/tools we'll use ▶ The information we're looking for • Should not be too specific Charters
27. • Often generated during conversations: ▶ Agile Ceremonies ▶ Requirements gathering meetings ▶ Conversations with BAs, stakeholders and developers • Often generated when executing test cases • Must include: ▶ The target area ▶ The resources/techniques/tools we'll use ▶ The information we're looking for • Should not be too specific • Should not be too broad Charters
28. Talk to your Stakeholders
29. Where can we get ideas?
30. • Stakeholder Questions and comments ▶ "What will happen if we add 300 more products in the future?" ▶ "Could a user ever see another users private profile?" Where can we get ideas?
31. • Stakeholder Questions and comments ▶ "What will happen if we add 300 more products in the future?" ▶ "Could a user ever see another users private profile?" • The Source Code ▶ // TODO: Don't know why this works, but it does. Don't touch it. Need to fix at some point Where can we get ideas?
32. • Stakeholder Questions and comments ▶ "What will happen if we add 300 more products in the future?" ▶ "Could a user ever see another users private profile?" • The Source Code ▶ // TODO: Don't know why this works, but it does. Don't touch it. Need to fix at some point • Developer comments ▶ "The profile image upload is a bit flaky because the Redis write queue is being weird. It works most of the time though." Where can we get ideas?
33. • Stakeholder Questions and comments ▶ "What will happen if we add 300 more products in the future?" ▶ "Could a user ever see another users private profile?" • The Source Code ▶ // TODO: Don't know why this works, but it does. Don't touch it. Need to fix at some point • Developer comments ▶ "The profile image upload is a bit flaky because the Redis write queue is being weird. It works most of the time though." • Your own findings and knowledge of the system ▶ "That video behaved strangely when I put the iPad in landscape mode. I wonder if that problem exists anywhere else?" Where can we get ideas?
34. • Stakeholder Questions and comments ▶ "What will happen if we add 300 more products in the future?" ▶ "Could a user ever see another users private profile?" • The Source Code ▶ // TODO: Don't know why this works, but it does. Don't touch it. Need to fix at some point • Developer comments ▶ "The profile image upload is a bit flaky because the Redis write queue is being weird. It works most of the time though." • Your own findings and knowledge of the system ▶ "That video behaved strangely when I put the iPad in landscape mode. I wonder if that problem exists anywhere else?" • Existing Defects and Defect Reports ▶ Particularly areas where defects are clustered. Where can we get ideas?
35. Questions?
36. Observation Background Charters Observation Session Sheets
39. Observation
40. • Be alert to subtle cues Observation
41. • Be alert to subtle cues • Try unusual things Observation
42. • Be alert to subtle cues • Try unusual things • Use monitoring software to look for the unusual Observation
43. • Be alert to subtle cues • Try unusual things • Use monitoring software to look for the unusual ▶ High CPU/Memory usage Observation
44. • Be alert to subtle cues • Try unusual things • Use monitoring software to look for the unusual ▶ High CPU/Memory usage ▶ Unusual network traffic Observation
45. • Be alert to subtle cues • Try unusual things • Use monitoring software to look for the unusual ▶ High CPU/Memory usage ▶ Unusual network traffic ▶ Strange HTTP requests/ responses Observation
46. • Be alert to subtle cues • Try unusual things • Use monitoring software to look for the unusual ▶ High CPU/Memory usage ▶ Unusual network traffic ▶ Strange HTTP requests/ responses ▶ Strage filesystem read/writes Observation
47. • Be alert to subtle cues • Try unusual things • Use monitoring software to look for the unusual ▶ High CPU/Memory usage ▶ Unusual network traffic ▶ Strange HTTP requests/ responses ▶ Strage filesystem read/writes • Look at Logs Observation
51. Try Unusual Things
55. “ Extract from "Explore It!" In the late 80's a number of cancer patients treated with the Therac-25 radiation therapy machine died after treatment. The cause of death was radiation poisoning. One of the incidents happened because the technician entered and then edited the treatment data very quickly using keyboard shortcuts, in less than 8 seconds. Eight seconds was also the time it took for the magnetic locks to engage. This created a window of vulnerability.
56. ✍ Session Sheets Background Charters Observation Session Sheets
58. Last Questions?
59. Summary ✅ Background ✅ Charters ✅ Observation ✅ Session Sheets
60. It's a skill that can be learned, but it's also reliant on experience and intuition.
61. further reading
62. @dannysmith ✉ danny@rootpath.io rootpath.io danny.is @rootpathhq Danny Smith