Use the Scientific Method to Improve Your Testing

Presentation was given by Raye Anne Hurst and Nicole Errante at Agile Testing Days USA 2019 in Chicago, IL, USA.

Key Learnings:
1) How the scientific method can help you test more effectively

2) Observation is a skill that starts with effective questions and ends with experimentation

3) Learn to identify different types of variables in your tests and how to use them to conduct small experiments that will improve your testing

4) Why scientists make great testers

Transcript

1. Use the Scientific Method to Improve Your Testing: Why Charles Darwin Would Make an Excellent Tester. Agile Testing Days USA 2019 Nicole Errante Raye Anne Hurst @canisrufus @rayeannehurst
2. Raye Anne Hurst • QA Practice Coach @ Deluxe • B.S.G. Geology • 11 yrs. Elementary School Science Teacher • 20 yrs. Quality @rayeannehurst
3. Nicole Errante • Quality Assurance Ninja @ Species360 • B.S. Biology • M.S. Criminal Justice – Forensic Science @canisrufus
4. Our Scientific Journey Scientific Method Variables Observation Scientists = Testers @canisrufus @rayeannehurst
5. Scientific Method State Problem Formulate Hypothesis Design Experiment Collect and Analyze Data Draw Conclusion / Report Results @canisrufus @rayeannehurst
6. Preschool Scientific Method • State Problem • Observation • Ask question • Formulate Hypothesis • Educated Guess • Testable Explanation • Experiment & Collect Data • Reach Conclusion & Report Results @canisrufus @rayeannehurst
7. Tester’s Scientific Method • State Problem • Review a story • Formulate Hypothesis • Meet expected results • Experiment & Collect Data • Execute tests • Reach Conclusion & Report Results • Tests passed or failed @canisrufus @rayeannehurst
8. Misconceptions about the Scientific Method Not a rigid set of steps – continuous loop of experiments and refinement Not trying to prove theory correct – trying to generate more questions Smallpondscience.com @canisrufus @rayeannehurst
9. Variables • In computer programming, a variable is a named container used to store information • In a scientific experiment, a variable is any factor that can be changed, controlled, or measured @canisrufus @rayeannehurst
10. Independent vs. Dependent vs. Control Independent • Cause (X) • Influencer Dependent • Effect (Y) • Influenced By Control • Remain constant @canisrufus @rayeannehurst
11. Variables and Bug Troubleshooting @canisrufus @rayeannehurst
12. Variables and Bug Troubleshooting (continued) • Variables • Each input • Selection of radio button • How record is saved • How enrichment is entered • Date format • Specific data entered • HTML Editor @canisrufus @rayeannehurst
13. Variables and Bug Troubleshooting (continued) • Dependent Variable: • Outcome that the record does not save • Independent Variable: • One input at a time starting with enrichment item • Controls: • All other fields remain their default or previous value • Save button is used
14. Keep Variables in Mind! • Identify • Change one at a time • Control the rest @canisrufus @rayeannehurst
15. Exploratory Testing Learning Test Execution Test Design @canisrufus @rayeannehurst
16. Exploratory Charter • Explore target • With resources • To Discover information Time Spent Wandering Insight Exploring Lost @canisrufus @rayeannehurst
17. Exploratory Testing is Learning Observe Debrief Explore Charters, Heuristics and Oracles Notes, Record, and Spark New Ideas Inspire and Make Sense of Findings @canisrufus @rayeannehurst
18. Learn to See Learning Test Execution Test Design “Real progress comes in the field writing notes, at the office amid a litter of doodled paper, in the corridor struggling to explain something to a friend, at lunchtime, eating alone, or in a garden while walking. To have a eureka moment requires hard work. And focus.” - - Letters to a Young Scientist @canisrufus @rayeannehurst
19. Scientific Observation Question Observe Experiment Scientists question Capture observations What When Evidence Who Record the data Experiment with it @canisrufus @rayeannehurst
20. What makes a good scientist = What makes a good tester Curiosity & Questioning Why? How? "Someone described scientists, good scientists, as perpetual adolescents. They never really grow up because they keep learning. They continue to be curious. They want to understand different things.“ Peter Dougherty, Nobel Laureate in Physiology or Medicine, 1996 @canisrufus @rayeannehurst
21. What makes a good scientist = What makes a good tester (continued) Clarity Accuracy Precision Consistency Relevance Sound evidence Good reasons Depth Breadth Fairness @canisrufus @rayeannehurst
22. What makes a good scientist = What makes a good tester (continued) @canisrufus @rayeannehurst Passionate Lifelong learner Balance
23. What makes a good scientist = What makes a good tester (continued) @canisrufus @rayeannehurst Goodcommunicator Ethical
24. Journey Home @canisrufus @rayeannehurst
25. Resources Ayala, F. J. (2009). Darwin and the scientific method. Proceedings of the National Academy of Sciences 106 Suppl 1, 10033-9. doi: 10.1073/pnas.0901404106 Bach, J., & Bolton, M. (2013, March 26). Testing and Checking Refined [Blog post]. Retrieved from https://www.satisfice.com/blog/archives/856 Costa, J. T. (2017). Darwin's Backyard: How Small Experiments Led to a Big Theory. New York, NY: W. W. Norton Company. Darwin, C. (2003). The Origin of Species (150th Anniversary ed.). New York, NY: Signet Classics. Giddings, M. C. (2008). On the Process of Becoming a Great Scientist. PLoS Comput Biol. 4(2), e33. doi: 10.1371/journal.pcbi.0040033. Haverford College Introductory Psychology Fall 2015. (2016). Science Exposed [Pressbook edition]. Retrieved from http://ds-wordpress.haverford.edu/psych2015/projects/ Hendrickson, E. (2012). Explore It! Reduce Risk and Increase Confidence with Exploratory Testing. Pragmatic Bookshelf. @canisrufus @rayeannehurst
26. Resources Koopmans, K (2019, March 13). How cognitive biases influence software development [Blog post]. Retrieved from https://www.atlassian.com/blog/add-ons/how-cognitive-biases-influence-software-development Langone, J., Stutz, B., & Gianopoulos, A. (2006). Theories for Everything: An Illustrated History of Science. Washington, D.C.: National Geographic. Paulk, K. (2018). What Makes A Good Software Tester Or What Makes A Software Tester Good? The Testing Planet 2018. Retrieved from https://dojo.ministryoftesting.com/dojo/series/the-testing-planet-2018/lessons/what-makes-a-good- software-tester Prell, A. (2019, February 18). How an Understanding of the Scientific Method Can Make You A Better Practitioner Of Agile Software Development [Blog post]. Retrieved from https://teamgaslight.com/blog/how-an-understanding-of-the- scientific-method-can-make-you-a-better-practitioner-of-agile-software-development Schoots, H. (2014, March 23). Heuristics for recognizing professional testers [Blog post]. Retrieved from http://www.huibschoots.nl/wordpress/?p=1666 Stahl, M. (2018). Ethics in Software Testing. Retrieved from https://www.stickyminds.com/article/ethics-software-testing. @canisrufus @rayeannehurst
27. Resources Strager, H. (2016). A Modest Genius: The Story of Darwin's Life and How His Ideas Changed Everything. Createspace Independent Publishing Platform. T/DG Admin (2019, April 1). Do Software Testers Miss Bugs due to Cognitive Bias? [Blog post]. Retrieved from https://blog.thedigitalgroup.com/does-cognitive-bias-make-software-testers-miss-out-bugs The Foundation for Critical Thinking (n.d.). Defining Critical Thinking. Retrieved June 17, 2019 from http://www.criticalthinking.org/pages/defining-critical-thinking/766 Tomes, S. (2016). Three Digestible Diagrams to Describe Exploratory Testing. The Testing Planet 2016. Retrieved from https://dojo.ministryoftesting.com/dojo/lessons/three-digestible-diagrams-to-describe-exploratory-testing van Wyhe, J. (Ed.), (2002-). The Complete Work of Charles Darwin Online. Retrieved from http://darwin-online.org.uk/ Velasquez, M., Andre, C., Shanks, T., & Meyer, M. J. (2010). What is Ethics? Retrieved from https://www.scu.edu/ethics/ethics-resources/ethical-decision-making/what-is-ethics/ Wilson, E. O. (2013). Letters to a Young Scientist [Kindle edition]. Retrieved from http://www.amazon.com @canisrufus @rayeannehurst
28. Photo/Illustration Credits @canisrufus @rayeannehurst Goetsch, H. [Untitled photo of dandelion]. Retrieved June 2019 from https://unsplash.com/photos/SGKQh9wNgAk Pritchett, R. T. (1900). HMS Beagle in the Straits of Magellan [Online image]. Retrieved June 2019 from https://commons.wikimedia.org/wiki/File:PSM_V57_D097_Hms_beagle_in_the_straits_of_magellan.png LadyOfHats (2012). Newton portrait with apple tree [Online image]. Retrieved June 2019 from https://commons.wikimedia.org/wiki/File:Newton_portrait_with_apple_tree.svg STEMShareNSW. [Untitled photo of people at computers]. Retrieved June 2019 from https://unsplash.com/photos/MtnEvxP_4TI McGlynn, Terry. (2013). The Method of Science, as it happens in my lab [Online diagram]. Retrieved June 2019 from https://smallpondscience.com/2013/05/30/were-teaching-the-scientific-method-incorrectly/