Essential Testing Concepts

Beginner-level "dictionary of terms" talk given at Pražské Pyvo, 2016-05-18

Transcript

1. Essential testing concepts
   Petr Viktorin
   encukou@gmail.com
   Pyvo, 2016-05-18
   

   View Slide

2. Testing
   Does the program actually
   work?
   If I change it, will it still work?
   Beware of bugs in the above code; I have only proved it correct,
   not tried it.
   — Donald Knuth
   

   View Slide

3. Test Automation
   Humans are bad at boring tasks.
   Automate as much as possible.
   A fool with a tool is still a fool.
   — Grady Booch
   

   View Slide

4. Basic Kinds of tests
   * Unit tests
   * Integration tests
   * End-to-end tests
   Quality means doing it right even when no one is looking.
   — Henry Ford
   

   View Slide

5. Unit tests
   Test individual functions/classes
   Computers are good at following instructions, but not at reading
   your mind.
   — Donald Knuth
   

   View Slide

6. Positive & Negative tests
   + Test that it works correctly.
   - Test that it breaks correctly.
   Pay attention to zeros. If there's a zero, someone will divide by it.
   — @drcemkaner
   

   View Slide

7. Test Fixtures
   Environment for a single test
   Set-up and tear-down
   Testers don’t like to break things; they like to dispel the illusion
   that things work.
   — Kaner, Bach, Pettichord
   

   View Slide

8. Mocking
   A way to test strongly-coupled
   code.
   Having a testing specialist on the team is a valuable asset, but the
   role of a specialist isn’t to restrict that responsibility to a single
   person.
   — Trish Khoo
   

   View Slide

9. Test Coverage
   How much of the code is tested?
   Trying to improve software quality by increasing the amount of
   testing is like trying to lose weight by weighing yourself more often.
   If you want to lose weight, don’t buy a new scale; change your diet.
   If you want to improve your software, don’t test more. Develop
   better.
   — Steve McConnell, Code Complete
   

   View Slide

10. Test Parametrization
    Same test with different inputs
    Testing is not responsible for the bugs inserted into software any
    more than the sun is responsible for creating dust in the air.
    — Dorothy Graham
    

    View Slide

11. Fuzzing
    Random, auto-generated inputs
    Debugging is twice as hard as writing the code in the first place.
    Therefore, if you write the code as cleverly as possible, you are, by
    definition, not smart enough to debug it.
    — Brian W. Kernighan
    

    View Slide

12. Integration tests
    Do the pieces work together?
    Program testing can be a very effective way to show the presence of
    bugs, but is hopelessly inadequate for showing their absence.
    — Edsger W. Dijkstra
    

    View Slide

13. Functional tests
    Test that the whole system
    works
    Just because you’ve counted all the trees doesn’t mean
    you’ve seen the forest.
    — Anonymous
    

    View Slide

14. Smoke tests
    If it doesn't even start, no sense
    in further testing.
    Q: How many testers does it take to change a lightbulb?
    A: None, they just tell you that the room is dark.
    — Anonymous
    

    View Slide

15. Environments
    * Development
    * Testing
    * Staging
    * Production
    I don’t care if it works on your machine! We are not shipping
    your machine!
    — Vidiu Platon
    

    View Slide

16. Gating
    Only tested code can get
    merged
    Untested code is buggy code.
    – Unknown
    

    View Slide

17. Continuous Integration (CI)
    Develop – Test – Deploy
    It does not matter how slowly you go as long as you do not stop.
    – Confucius
    

    View Slide

18. A/B testing
    Testing interaction with users?
    Split users into groups – one
    with a new feature, one without.
    If you think your users are idiots, only idiots will use it.
    — Linus Torvalds
    

    View Slide

19. Test-Driven Development
    (TDD)
    Red – Green – Refactor
    Walking on water and developing software from a specification are
    easy if both are frozen.
    — Edward V. Berard
    

    View Slide

20. Questions?
    Testing leads to failure, and failure leads to understanding.
    — @BurtRutan
    

    View Slide