Essential testing concepts Petr Viktorin [email protected] Pyvo, 2016-05-18 

Testing Does the program actually work? If I change it, will it still work? Beware of bugs in the above code; I have only proved it correct, not tried it. — Donald Knuth 

Test Automation Humans are bad at boring tasks. Automate as much as possible. A fool with a tool is still a fool. — Grady Booch 

Basic Kinds of tests * Unit tests * Integration tests * End-to-end tests Quality means doing it right even when no one is looking. — Henry Ford 

Unit tests Test individual functions/classes Computers are good at following instructions, but not at reading your mind. — Donald Knuth 

Positive & Negative tests + Test that it works correctly. - Test that it breaks correctly. Pay attention to zeros. If there's a zero, someone will divide by it. — @drcemkaner 

Test Fixtures Environment for a single test Set-up and tear-down Testers don’t like to break things; they like to dispel the illusion that things work. — Kaner, Bach, Pettichord 

Mocking A way to test strongly-coupled code. Having a testing specialist on the team is a valuable asset, but the role of a specialist isn’t to restrict that responsibility to a single person. — Trish Khoo 

Test Coverage How much of the code is tested? Trying to improve software quality by increasing the amount of testing is like trying to lose weight by weighing yourself more often. If you want to lose weight, don’t buy a new scale; change your diet. If you want to improve your software, don’t test more. Develop better. — Steve McConnell, Code Complete 

Test Parametrization Same test with different inputs Testing is not responsible for the bugs inserted into software any more than the sun is responsible for creating dust in the air. — Dorothy Graham 

Fuzzing Random, auto-generated inputs Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. — Brian W. Kernighan 

Integration tests Do the pieces work together? Program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence. — Edsger W. Dijkstra 

Functional tests Test that the whole system works Just because you’ve counted all the trees doesn’t mean you’ve seen the forest. — Anonymous 

Smoke tests If it doesn't even start, no sense in further testing. Q: How many testers does it take to change a lightbulb? A: None, they just tell you that the room is dark. — Anonymous 

Environments * Development * Testing * Staging * Production I don’t care if it works on your machine! We are not shipping your machine! — Vidiu Platon 

Gating Only tested code can get merged Untested code is buggy code. – Unknown 

Continuous Integration (CI) Develop – Test – Deploy It does not matter how slowly you go as long as you do not stop. – Confucius 

A/B testing Testing interaction with users? Split users into groups – one with a new feature, one without. If you think your users are idiots, only idiots will use it. — Linus Torvalds 

Test-Driven Development (TDD) Red – Green – Refactor Walking on water and developing software from a specification are easy if both are frozen. — Edward V. Berard 

Questions? Testing leads to failure, and failure leads to understanding. — @BurtRutan