(or classes) that are tested and treated the same wa y • Best for testing ranges and group s • Partitions are either valid or invali d • There is no added value from retesting the same partition with different value s • A single value belongs to only one partitio n • Valid and invalid partitions must be tested to have full coverage
testing is that we assume all items in the portions will act the same. It’s acceptable risk because the time saved is worth the risk of finding a bug like this rarely . • Example : • We added some new needs to service requests under existing types . • We assumed every need under the type would act the same (properly assigned priority, closed as appropriate ) • Testing every new need would have been over-testin g • We had a typo so one need was acting incorrectly . • Should we have tested everything? NO! Finding this bug should not change our manual testing
1 - 1 0 • Valid partition: 1 - 1 0 • Invalid partitions: less than 1, more than 10, non-integer character s • A vet treats cats . • Valid partition: cat s • Invalid partition: any thing that isn’t a cat
based on how much you buy : • .1-.5 lbs: no discoun t • .6-1.0 lbs: 5% discoun t • 1.1-2.0 lbs: 10% discoun t • 2.0+ lbs: 20% discoun t • What is each partition ? •
at the boundaries of input s • Boundaries are where we often see special cases, unusual behavior—you know, edge cases • Two-value BVA: test minimum and maximum of the boundar y • Three-value BVA: test a value above the boundary, at the boundary, and below the boundary
for the first ten miles they travel on a visit. After that, they are paid $1 for each mile up to 20 miles and $.25 for any mile after that. Assume integers.
a discount based on how much you buy : • .1-.5 lbs: no discoun t • .6-1.0 lbs: 5% discoun t • 1.1-2.0 lbs: 10% discoun t • 2.0+ lbs: 20% discoun t • What is each boundary ? •
affect the resul t • Works very well for business rules, especially with some complexit y • List the inputs as a table and fill in the outcome s • Can show you where there are equivalence partitions to reduce th number of set cases needed
the first ten miles they travel on a visit. After that, they are paid $1 for each mile up to 20 miles and $.25 for any mile after that if they are in North Carolina . • A Pal is paid $1 for the first ten miles they travel on a visit. After that, they are paid $2 for each mile up to 20 miles and $1 up to 25 miles, but aren’t paid for any miles after that if they are in California . • Pals in Portland do not get paid commute miles, but the rest of Oregon follows North Carolina rules . • Assume integers.
based on how much you buy : • .1-.5 lbs: no discoun t • .6-1.0 lbs: 5% discoun t • 1.1-2.0 lbs: 10% discoun t • 2.0+ lbs: 20% discoun t • Chocolates and lollipops follow the standard discount but fudge is not discounted above 2lbs. Also, gummy candy is on sale with an additional 2% discount . • What is the decision table ?
we can test to have a certain level of confidence that the software is not presenting major defects ? • Look at all your inputs and use equivalence partitioning to find any partitions (pay attention to valid/invalid, inputs with the same outputs ) • Set up a table with all the inputs listed starting with the variable that has the most inputs and ending with the leas t • As you fill out the test cases, you want to see all pairs represented somewhere in your test cases
in person . • If the visit is in person, it could be transportation or in home . • The visit can be for a health plan Papa, a direct consumer Papa, or a Caregiver Papa . • The visit can be one time or recurring on a schedule: daily, weekly, bi-weekly, monthly . • The objectives for the visit can be cleaning, tech help, or yard work. If the visit is transportation, it could be grocery shopping or doctor’s visit. All visits should have companionship . • The visit could be set to All Pals or Preferred Pals.
720 test cases Objectives • Cleaning • Tech help • Yard work • Grocery shopping • Doctor’s visit • Companionship Schedule • One time • Daily • Weekly • Bi-weekly • Monthly Papa • Health plan • Direct consumer • Caregiver Visibility • All Pals • Preferred Pals Type • Virtual • Inperson Class • Transportation • In-home
down to 120! Objectives • Cleaning • Tech help • Yard work • Grocery shopping • Doctor’s visit • Companionship Schedule • One time • Reoccurring • Daily • Weekly • Bi-weekly • Monthly Papa • Health plan • Direct consumer • Caregiver • 3rd party pays Visibility • All Pals • Preferred Pals Type • Virtual • In person - transpo • In person - in home
• As you do, make sure that there is a representative of each pair of the previous and current column . • IE: for virtual visits, we should have a Self-Pay and 3rd Party option somewhere in the list.
cases ! • That’s doable so we’re done right ? • Nope, still too many test cases. Let’s continue to reduce it . • First: are any of these cases invalid?
! • This gets us full coverage of all pairs of inputs and will allow us to be confident that unless there is an interaction we aren’t aware of . • There are tools that will do this for you and that’s my go-to.
options! You can buy chocolates, lollipops, fudge, or gummy candy. They can be packaged to go or you can eat them in store. If you get them to go, you can also get them gift wrapped. We offer blue, yellow, and green gift wrapping. The fudge and chocolate are available in gluten free versions which are an extra $1 per half pound. Candy may be purchased in .1lb increments from .1 lb to 10lbs using the previously described discount structure.
that transition from one state to another. The system can be in only one state at a time . • There is a distinct event that results in the transition into our out of state but multiple events can move the system into the same state . • Coverage is described in switches : • One transition: 0-switch coverag e • Two transitions: 1-switch coverag e • If you are looking at more than one or two transitions, rethink your strategy or understanding of how your states affect each other . • This works best when states are influenced by the previous state and transition but not the entire chain of states
100% coverage . • We want to ensure we can make valid transitions and avoid invalid transitions . • You can use a flow diagram or a state table . • In a state table, list the states on the left and events across the top . • You need to have a clear idea of your states and the transition events.
incoming and outgoing states then add up all your values. This is your starting point. (72 in our example! ) • Now write them down and look at the resulting cases. See if you can combine any or some are invalid . • Then decide if you need a long or short chain of test s • Consider if it matters how we got into the state or if we just need to be in the right state . • If we don’t need to move through the whole flow, then we shouldn’t.
care about one particular state so focus on that . • Do we care what happened before it got into that state or do we only care about the visit being in that state? If we care about the states before, we may be making bad assumptions.
testing that lives deep in our codebas e • Generally, these run very quickly and are considered chea p • Tests the smallest unit of code possibl e • Generally written by developer s • Great place to test functionality like dates and math
• Follows a prescribed script or test cas e • No room for investigatio n • Defects mean the scripts fai l • Test cases are generally scripted black box testin g • Automated tests are scripted white box testing
featur e • No real goal—just wandering around to see what you fin d • Most of our testing is ad hoc testing around feature s • Bug bashes are an example of ad hoc testin g • This is generally considered black or grey box testin g • Cannot be automated
executio n • Has a clear mission called a charte r • Take notes as you go or record session s • Hold off putting in or chasing defects until the session is ove r • Focuses on exploring with a ma p • This is generally considered black box testing • Cannot be automated
A to point B • Generally done as a user or with a user type proces s • Tests the intersection of several modules, flows, or system s • Third party APIs, backend systems, front end systems, databases and more can be layers that need to be tested together • This can be white or black box testin g • Can be automated
as expected under given workload s • Several types of testing fall under this umbrella : • Load - expected loa d • Soak - expected load for an extended duratio n • Stress testing - higher than expected loa d • Spike testing - sudden influx of users or connection s • Volume testing - dramatic increase in data coming i n • If you don’t do it, your users wil l • This is white box testing . • Almost always scripted
are workin g • Asks the basic questions we need to answer to ensure the base functionality is unchange d • Should be very short and quic k • Smoke tests are expected to pas s • This is generally black box testing • Can be automated
since the previous iteration of the softwar e • Often done before large change s • Regression is generally a heavy burden on the team and should be avoide d • Regression should take into account planned changes and should never be ‘test everything ’ • This is generally black box testin g • Can be automated (sort of)
front of your end users and in real world scenarios to ensure it meets their need s • Generally formalized as the team asks the end users to focus on specific flow s • Seeks to harden and find unknown requirements before releas e • This is considered black box testin g • Cannot be automated
front of your end users and in real world scenarios to ensure it meets their need s • Generally formalized as the team asks the end users to focus on specific flow s • Seeks to harden and find unknown requirements before releas e • Cannot be automated
jennydoesthings
tammyeverts
smashingmag
rasmusluckow
uxyall
asonas
honzajavorek
nwiizo
tamaranovitovic
greggifford
cassininazir
helmedeiros
jct
![Questions? @jennydoesthings • [email protected] • Twitter: @jennydoesthings](https://files.speakerdeck.com/presentations/82db3e6a05474028ad7ff4ae7eeca34b/slide_56.jpg){kind=link}
![Questions? @jennydoesthings • [email protected] • Twitter: @jennydoesthings](https://files.speakerdeck.com/presentations/82db3e6a05474028ad7ff4ae7eeca34b/slide_79.jpg){kind=link}
