Lead BBST Instructor for the AST • Runs AST’s webinar program • Maintains a open source list of software testing conferences at TestingConferences.org • Nominee for the Board of the AST Dwayne Green • Test Lead at 1-800 Contacts • Learning how to train testers through practice - Deliberate Practice in Testing
over time, with practice and feedback. ◦ We hope this workshop provides some practice and feedback • Identify and characterize data types used by variables in the application • Determine variables that are good candidates for domain testing ◦ When looking at an application be able to exercise variables using data • Stretch Goal: Generate best representative data and put that information into a domain testing table
Testing before? 2. What about Boundary Analysis or Equivalence Class Partitioning? 3. How many people here have applied either Domain Testing before (or Boundary Analysis or Equivalence Class Partitioning)? 4. How many people have heard of the term Quick tests or Quick attacks? 5. Has anyone taken the BBST Test Design course?
or Quick attacks are tests that don’t cost much to design, are based on some guesstimate for how the system could fail (risk-based) and don’t take much prior knowledge to apply. ◦ Great for starting your testing but can run out of steam quickly. What do you do then?
information • Techniques are a way to guide our thinking as we develop a group of tests ◦ Think of functional testing, specification testing, domain testing, et. al as techniques • Techniques have different attributes or characteristics (see right)
are different techniques but they often go hand in hand. • Complete testing is impossible ◦ Any sufficiently complex application will have too many possible values to test ◦ Too many inputs for us to test with a reasonable amount of time ◦ How do we deal with this problem?
Domain is a set of possible values for a function ◦ Input domain ◦ Output domain • In Software Development, Domain Testing is an umbrella term for: ◦ Boundary analysis ◦ Equivalence class partitioning ◦ (Depending on the test design book you are reading you might this might also be called Domain Analysis or Input Domain Analysis)
for efficiently hunting for bugs in a programs handling of data ◦ Anytime you have a very large set of data you can use and want to reduce it to a smaller sample size • We can’t run every possible test for every value ◦ Complete testing is impossible • Instead we can select a few values we think will uncover issues ◦ Power is a characteristic we are looking for in all domain tests
a program interacts with. • Data Types: Programmers define the types of their variables ◦ Integer ◦ Boolean ◦ Floating Point ◦ Strings ◦ Enumerated values ◦ Byte ◦ Char • Variables can have relationships with other variables
think they are • Think of the Goldilocks rule: ◦ Too big ◦ Too small ◦ Just right ◦ But also.. Too strange? • Example using a Data Type: ◦ Integer Value (Int32) ▪ Too Big: 2147483648 ▪ Too Small: -2147483649 ▪ Just Right: 0 or 1 or 1000 ▪ Too Strange: A#$@! or (2-1) or 0.01 or +=1
◦ Accepts a range of characters with a minimum of 5 and a maximum of 16 ◦ The size of the field has the following boundaries: ▪ 0 - 4: Too few characters ▪ 5 - 16: Just the right number of characters ▪ 17+: Too many characters
application looking for what you can change. Anything you can change is a variable. Find them all and make a list. Select one interesting variable and write down what values you think it can take. • Timebox: 20 Minutes
notational variables ◦ We analyze the program as if these variables and our understanding of them is correct but it may not be ◦ This makes the work we do subjective but valuable • Input, storage and result (output) variables ◦ Calculations should add up ◦ Where is the data going? ◦ Do any of these variables constrain or effect each other • Can you get past the input filter? ◦ Are you looking at the results?
we think the program will handle the same way ◦ If one catches a bug, they all will ◦ If one doesn’t catch a bug, the others probably won’t either • We partition (or split) a domain (input or output) based on how we think the application will handle it. ◦ Valid Values ◦ Invalid Values ◦ Risks • Much of this partitioning is based on what data type we think this variable is
tasks for Domain Testing: ◦ Partition or split a domain into an equivalence class(es) ◦ Select a few representatives from each class to test with • A best representative of an equivalence class is a boundary or another value in the class that seems more likely than other values to cause a program failure. ◦ This is the idea that all domain tests should be powerful.
MortgageCalc or ChrisReads application with the goal of identifying one variable you think is a good candidate for domain testing and identify the best representative data. Organize your work however you want, but list out the best representative data for the variable. • Timebox: 30 minutes
our Test Design overview, good test strategies will involve many techniques ◦ This means you are likely to apply domain testing combined with another technique like functional testing • For practice you might use an ET Tour focused on understanding the variables of a system during which you might try to apply this. ◦ This is exactly what we did ◦ You can create Exploratory Testing Charters to help
Identified and characterized variables based on Data Type ◦ Determined variables that are good candidates for domain testing ◦ Applied domain testing by ▪ Splitting a domain into equivalence classes ▪ Selected best Representatives • This goes wider and deeper ◦ BBST Test Design and BBST Domain Testing introduce a schema for helping with this technique that is 18 steps long
either using Exploratory Tours or by integrating what you learned into your existing test strategy. • A few classes you could take: ◦ BBST Domain Testing -> runs once or twice per year ◦ BBST Test Design -> runs more frequently • Try a risk-based approach to Domain Testing (assignment)