Essential Testing Concepts

Beginner-level "dictionary of terms" talk given at Pražské Pyvo, 2016-05-18

Transcript

1. Essential testing concepts Petr Viktorin encukou@gmail.com Pyvo, 2016-05-18
2. Testing Does the program actually work? If I change it, will it still work? Beware of bugs in the above code; I have only proved it correct, not tried it. — Donald Knuth
3. Test Automation Humans are bad at boring tasks. Automate as much as possible. A fool with a tool is still a fool. — Grady Booch
4. Basic Kinds of tests * Unit tests * Integration tests * End-to-end tests Quality means doing it right even when no one is looking. — Henry Ford
5. Unit tests Test individual functions/classes Computers are good at following instructions, but not at reading your mind. — Donald Knuth
6. Positive & Negative tests + Test that it works correctly. - Test that it breaks correctly. Pay attention to zeros. If there's a zero, someone will divide by it. — @drcemkaner
7. Test Fixtures Environment for a single test Set-up and tear-down Testers don’t like to break things; they like to dispel the illusion that things work. — Kaner, Bach, Pettichord
8. Mocking A way to test strongly-coupled code. Having a testing specialist on the team is a valuable asset, but the role of a specialist isn’t to restrict that responsibility to a single person. — Trish Khoo
9. Test Coverage How much of the code is tested? Trying to improve software quality by increasing the amount of testing is like trying to lose weight by weighing yourself more often. If you want to lose weight, don’t buy a new scale; change your diet. If you want to improve your software, don’t test more. Develop better. — Steve McConnell, Code Complete
10. Test Parametrization Same test with different inputs Testing is not responsible for the bugs inserted into software any more than the sun is responsible for creating dust in the air. — Dorothy Graham
11. Fuzzing Random, auto-generated inputs Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. — Brian W. Kernighan
12. Integration tests Do the pieces work together? Program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence. — Edsger W. Dijkstra
13. Functional tests Test that the whole system works Just because you’ve counted all the trees doesn’t mean you’ve seen the forest. — Anonymous
14. Smoke tests If it doesn't even start, no sense in further testing. Q: How many testers does it take to change a lightbulb? A: None, they just tell you that the room is dark. — Anonymous
15. Environments * Development * Testing * Staging * Production I don’t care if it works on your machine! We are not shipping your machine! — Vidiu Platon
16. Gating Only tested code can get merged Untested code is buggy code. – Unknown
17. Continuous Integration (CI) Develop – Test – Deploy It does not matter how slowly you go as long as you do not stop. – Confucius
18. A/B testing Testing interaction with users? Split users into groups – one with a new feature, one without. If you think your users are idiots, only idiots will use it. — Linus Torvalds
19. Test-Driven Development (TDD) Red – Green – Refactor Walking on water and developing software from a specification are easy if both are frozen. — Edward V. Berard
20. Questions? Testing leads to failure, and failure leads to understanding. — @BurtRutan