Quality Assurance Basics for Devs, Project Managers & Presidents

In this talk we’ll discuss the principles of QA and the value of QA for all stakeholders. We will begin by defining QA, some basic vocabulary, and discussing when & why it is needed as part of the SDLC. The discussion will cover the benefits of using QA, comparisons of automated and manual testing and when each is appropriate, and does it really cut costs?

Transcript

1. Why and When to Use QA: Brie Hoblin QA Consultant @bhoblin brie@briehoblin.com
2. Because when people ask me about QA, those are the questions I get: very basic ones.
3. OK, my glasses may not be quite that cool…
4. QUALITY ASSURANCE
5. ƒ quality checking that happens AFTER the product goes out the door (that’s Quality Control or “QC”) ƒ (Brie’s pet peeve is people telling her she does “QC”) …but it’s not what I do
6. ƒ Quality Assurance is not just testing software. ƒ Quality Assurance involves software testing, but THEY ARE NOT SYNONOMOUS.
7. ƒ It’s not just yelling ‘ship it!’ (…although we do get to do that on occasion)
8. Software QA involves the entire software development PROCESS ‐ monitoring and improving the process, making sure that any agreed‐upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'. ‐‐from http://www.softwareqatest.com/qatfaq1.html
9. ƒ Creating test plans ƒ Creating test strategies ƒ Deciding on manual or automated testing ƒ Assessing existing processes & their functionality or lack thereof & how that impacts the overall quality of the software ƒ Maintaining ticketing system ƒ Software Testing ƒ Logging bugs ƒ Yelling “Ship It!”
10. ƒ Ensuring the software produced meets the existing requirements. ƒ This is half of QA. It’s not finding big bad bugs, it’s ensuring a game of telephone didn’t ruin the software.
11. ƒ Super awesome QA involves QA people sitting in on requirements gathering & steering conception & design of new features ƒ (…I’ve only worked at one place that actually did this.)
12. WHY?
13. ƒ “I’m a dev and I am super smart and I check my code so you don’t have to.” (THANK YOU!) ƒ OK, but are you still human? ƒ Are all dev’s as awesome as you are?
14. ƒ Programmers often test to prove the application works. ƒ Testers test to find where it doesn’t.
15. ƒ “Testing adds time & expense, without enough payoff.” ƒ “We’re fine without QA, we’ve gotten this far.” ƒ ‐‐We’ll address these arguments later
16. Because otherwise you might make the software equivalent of these...
17. Missing icons anyone?
20. To avoid logic errors like not allowing a return when a non‐returnable item was shipped in place of a returnable one…
21. ƒ What would you test?
22. ƒ Temperature range each material can handle before cracking, breaking, or shrinking ƒ Insulative properties of materials used in mug—safe to hold in your hand no matter how hot the coffee is ƒ Easy to carry, hold, walk with for people with small or large hands ƒ Decent amount of coffee fits in mug ƒ Mug fits in standard car cup holder ƒ Mug does not leak even after coffee is left in it for days, with variations in temperature ƒ Coffee comes out of drink spout evenly – doesn’t spill all over your clothes ƒ Keeps coffee warm for a reasonable amount of time ƒ Advertising on mug is clear and conveys meaning ƒ Materials are durable over time and repeated use ƒ Mug is balanced and does not tip over in cup holder or on reasonably flat surface ƒ Clearly labeled dishwasher safe, or not dishwasher safe.
23. ƒ We’re all human ƒ Third party integration is a b**** ƒ Javascript libraries never fight each other the same way in local as they do in production ƒ Users ALWAYS behave exactly the way we expect them to, right? ƒ Requirements gathering can be like a game of telephone ƒ There are always lumps in the oatmeal
24. WHEN do you need QA?
25. AS SOON AS YOU HAVE CODE
26. (James Bach, QA superhero)
27. ƒ When major clients are majorly unhappy with quality, and threatening to leave because they are fed up ƒ (when someone in your office starts looking like this)
28. ƒ Stakeholders are looking over your shoulder ƒ Product or company is expanding rapidly ƒ Third party integration
29. Integrating QA into your SDLC when you don’t have dedicated QA employees
30. ƒ Pseudo code!!! ƒ Pair programming ƒ Code review ƒ 5 minutes positive testing ƒ page loads ƒ main elements in place ƒ Use good version control!!
31. ƒ Pages load (Use an automated link checker where you can) ƒ Images load ƒ Users can log in ƒ Basic use cases can be executed without a problem
32. Waterfall software development lifecycle usually has testing after requirements gathering & development.
33. In Agile development, testing will occur more frequently and is more likely to be included in the beginning planning stages.
34. ƒ Degree of documentation ƒ Higher in waterfall ƒ Degree of communication ƒ Higher in Agile ƒ Level of responsiveness ƒ Higher in Agile ƒ Need for coordination between team members ƒ Higher in Agile
35. ƒ If budget is restricted & requirements gathering process is solid, then incorporating software testing after development is ok ƒ If overall quality is problematic, and/or requirements gathering process is weak, or company is growing rapidly, include QA in beginning stages of planning
36. Assessing level of quality needed
37. ƒ How tech savvy are your users?
38. ƒ What expectations does your audience have? ƒ How much competition is there? ƒ In house, or external audience? ƒ Usability—including handicapped users? Elderly, vision‐impaired?
39. ƒ When your clients have low expectations, and are more flexible on deadlines ƒ There’s no competition ƒ You’re a start up with less than 4 people ƒ Your clients want to do (most of) their own testing
40. Common QA Mistakes
41. 1. Making it through being a start‐up without QA, waiting until huge client crisis to hire dedicated QA, and then making the mistake of thinking once that crisis is over, the company is ok without QA again. ƒ As company grows, chains of communication get longer & more convoluted & there is a greater need for QA.
42. 2. Doing software testing but not reconciling against requirements enough
43. 3. Not testing for mobile
44. 4. Not maintaining a ticketing system
45. 5. Focusing testing too much on one browser
46. 5a. Not budgeting enough time for adequate testing when setting client expectations. (and then you need eight arms & two heads to test it all in time)
47. What makes a good software tester?
48. ƒ Is logical, rational, and methodical ƒ Able to write and execute a test plan ƒ Remember every page in the application (& all the browsers) ƒ Know the steps to reproduce a bug ƒ Thinks of every possibility (or at least a lot of them) ƒ Communicates well with the team ƒ States things briefly and clearly ƒ Is sensitive to the feelings of others ƒ Can communicate priority level of bug when asked
49. ƒ The art of QA is being observant of who your customers are, what browsers & devices they’re using, what tech stack your company is using, what profitability is like, what deadlines exist & formulating a comprehensive test strategy that accounts for all those things. It’s important to test within context.
50. QA VOCAB
51. “[Software] testing is the process of executing a program with the intent of finding errors.” –The Art of Software Testing
54. ƒ Test strategy, test plan, test cases ƒ Manual vs. automated ƒ Black, gray, and white box testing ƒ Load, performance, stress, and regression testing ƒ Positive, negative, edge test cases
55. ƒ Test Strategy: the set of ideas that guide test design ƒ Test Plan: the set of ideas that guide a test project ƒ Test Case: A test case is usually a single step, or occasionally a sequence of steps, to test the correct behavior/functionality, and features of an application. An expected result or expected outcome is usually given.
56. ƒ Automated tests can be run repeatedly and save hours of manual testing. ƒ Test Driven Development / Behavioral Driven Development can improve code quality and efficiency of developers. ƒ Manual testing is more adaptable, no tests to maintain, just a test plan.
57. ƒ Black box testing: examines the functionality of the application without peering into its internal structures. ƒ White box testing (aka clear box testing, transparent box testing) uses an internal perspective of the system, as well as programming skills, to design test cases. ƒ Gray box testing is a combination of both.
58. ƒ Load testing is the process of putting demand on a system or device and measuring its response. ƒ Performance testing is the process of determining the speed or effectiveness of a computer, network or software program or device. ƒ Stress testing determines the robustness of software by testing beyond the limits of normal operation. (More important for medical software.) ƒ Regression testing is used to ensure that any new changes have not introduced new bugs—that everything remaining the same continues to function as it should.
59. ƒ A positive test case is designed to prove that what the user expects to happen, happens. ƒ Negative test cases are designed to test what will happen outside the range of normal use. ƒ Edge cases test boundary conditions. ƒ (and corner cases are where two edge cases meet!)
60. The Art of Software Testing by Glenford J. Myers Lessons Learned in Software Testing By Cem Kaner, James Bach, and Bret Pettichord http://www.qaforums.com/ James Bach’s blog: http://www.satisfice.com/blog/
61. Brie Hoblin – QA Consultant @bhoblin brie@briehoblin.com