$30 off During Our Annual Pro Sale. View Details »

FFS - Great test automation does not absolve you from manual / exploratory testing

FFS - Great test automation does not absolve you from manual / exploratory testing

In recent years we've seen a movement to more test automation replacing manual/exploratory testing and even replacing dedicated testers altogether.
Many programmers don't like to manually test things. But not everything can (or should) be automated. Some things are hard if not impossible to automate, others are simply not worth the effort.
I'll illustrate this with a tale of a new system built by my team where we still found bugs in e2e testing even though we had solid unit and integration test coverage.

Marit van Dijk

September 19, 2018
Tweet

More Decks by Marit van Dijk

Other Decks in Programming

Transcript

  1. @MaritvanDijk77

    github.com/mlvandijk

    medium.com/@mlvandijk
    FFS - Great test automation
    does not absolve you from
    manual / exploratory testing

    View Slide

  2. Recent years
    •More test automation
    •No more dedicated testers
    •No more manual testing - at all!
    @MaritvanDijk77

    View Slide

  3. Automate all the things
    (meme)
    @MaritvanDijk77

    View Slide

  4. But…
    •We cannot automate everything
    (we shouldn’t want to)
    •Automation cannot tell you what it’s
    like to actually use it
    @MaritvanDijk77

    View Slide

  5. “If you don’t test it, your users will.”
    ~ @lanettecream
    @MaritvanDijk77

    View Slide

  6. Tested components
    @MaritvanDijk77
    Unit Unit Unit
    Integration
    Integration

    View Slide

  7. Tested components
    @MaritvanDijk77
    Unit Unit Unit
    Integration
    Integration

    View Slide

  8. Tested whole?
    @MaritvanDijk77
    Unit Unit Unit
    Integration
    Integration

    View Slide

  9. Here be bugs
    Tested whole?
    @MaritvanDijk77
    Unit Unit Unit
    Integration
    Integration












    View Slide

  10. Example: Rebuilding a workflow system
    + Familiar with process
    + Close cooperation with BA
    + Unit tests
    + Integration tests
    @MaritvanDijk77

    View Slide

  11. Overview
    Workflow
    System
    Other
    System
    Other
    System
    Other
    System

    View Slide

  12. E2E Test
    •Notoriously difficult:
    •Consistent state in several systems
    •Collaboration between teams/depts
    •Hard to automate / maintain
    @MaritvanDijk77

    View Slide

  13. Pair testing
    •Technical knowledge (dev)
    •Functional knowledge (test)
    •Less error prone
    •No time wasted on reproduction
    •Quickly fixed
    @MaritvanDijk77

    View Slide

  14. Test your code!
    https://www.redbubble.com/people/hogfish
    @MaritvanDijk77

    View Slide

  15. @MaritvanDijk77

    github.com/mlvandijk

    medium.com/@mlvandijk
    FFS - Great test automation
    does not absolve you from
    manual / exploratory testing

    View Slide