Upgrade to Pro — share decks privately, control downloads, hide ads and more …

FFS - Great test automation does not absolve you from manual / exploratory testing

FFS - Great test automation does not absolve you from manual / exploratory testing

In recent years we've seen a movement to more test automation replacing manual/exploratory testing and even replacing dedicated testers altogether.
Many programmers don't like to manually test things. But not everything can (or should) be automated. Some things are hard if not impossible to automate, others are simply not worth the effort.
I'll illustrate this with a tale of a new system built by my team where we still found bugs in e2e testing even though we had solid unit and integration test coverage.

62b3ae0dcd8e12a5eedd1fbe417bc6c8?s=128

Marit van Dijk

September 19, 2018
Tweet

Transcript

  1. @MaritvanDijk77
 github.com/mlvandijk
 medium.com/@mlvandijk FFS - Great test automation does not

    absolve you from manual / exploratory testing
  2. Recent years •More test automation •No more dedicated testers •No

    more manual testing - at all! @MaritvanDijk77
  3. Automate all the things (meme) @MaritvanDijk77

  4. But… •We cannot automate everything (we shouldn’t want to) •Automation

    cannot tell you what it’s like to actually use it @MaritvanDijk77
  5. “If you don’t test it, your users will.” ~ @lanettecream

    @MaritvanDijk77
  6. Tested components @MaritvanDijk77 Unit Unit Unit Integration Integration

  7. Tested components @MaritvanDijk77 Unit Unit Unit Integration Integration

  8. Tested whole? @MaritvanDijk77 Unit Unit Unit Integration Integration

  9. Here be bugs Tested whole? @MaritvanDijk77 Unit Unit Unit Integration

    Integration
  10. Example: Rebuilding a workflow system + Familiar with process +

    Close cooperation with BA + Unit tests + Integration tests @MaritvanDijk77
  11. Overview Workflow System Other System Other System Other System

  12. E2E Test •Notoriously difficult: •Consistent state in several systems •Collaboration

    between teams/depts •Hard to automate / maintain @MaritvanDijk77
  13. Pair testing •Technical knowledge (dev) •Functional knowledge (test) •Less error

    prone •No time wasted on reproduction •Quickly fixed @MaritvanDijk77
  14. Test your code! https://www.redbubble.com/people/hogfish @MaritvanDijk77

  15. @MaritvanDijk77
 github.com/mlvandijk
 medium.com/@mlvandijk FFS - Great test automation does not

    absolve you from manual / exploratory testing