Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automated Tests Aren't Enough

Automated Tests Aren't Enough

Talk given at MinneBar 14.

Recording available here: https://www.youtube.com/watch?v=Ih1-y2Vdvic

Daniel Lew

April 27, 2019

More Decks by Daniel Lew

Other Decks in Programming


  1. Automation Can Be Expensive • 100% code coverage is nearly

    impossible • UI testing sucks • Tests require maintenance
  2. Nomenclature • Testing is the process of evaluating a product

    by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc. • Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product. • James Bach & Michael Bolton - satisfice.com/blog/archives/856
  3. Manual Testing Strengths Weaknesses • Exploratory • Tests correctness •

    Tests goodness • Easy to “write” and “update” • Black-box testing only • Unreliable • Time intensive • Difficult reproductions
  4. Automated Checks Strengths Weaknesses • Test any layer • Repeatable

    • Fast • Easy reproduction • Cannot explore • Cannot prove correctness • Cannot prove goodness • Expensive to maintain