Share
Embed
Sensible Testing
Conferences Box
June 23, 2012
Transcript
29 slides
-
Justin Leitgeb Stack Builders June 23, 2012 GoRuCo Sensible Testing
Saturday, June 23, 12 -
Quantitative concepts: How much should I test? • TATFT •
DHH • Kent Beck Saturday, June 23, 12 -
DHH • "Don’t aim for 100% coverage." • "Code-to-test ratios
above 1:2 is a smell, above 1:3 is a stink." • "You’re probably doing it wrong if testing is taking more than 1/3 of your time. You’re definitely doing it wrong if it’s taking up more than half." http://37signals.com/svn/posts/3159-testing-like-the-tsa Saturday, June 23, 12 -
Kent Beck I get paid for code that works, not
for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). http://stackoverflow.com/questions/153234/how-deep-are-your-unit-tests Saturday, June 23, 12 -
Moving beyond quantitative concepts • We have lots of concepts
for production code (DRY, SOLID, design patterns) • Much testing talk stays on the quantitative level - what is our “coverage”? • Instead, what are some test concepts that help to make our code more “habitable”? Saturday, June 23, 12 -
Habitability Habitability is one of the most important characteristics of
software. It enables developers to live comfortably in and repair or modify code and design. Patterns of software: Tales from the Software Community Richard Gabriel Saturday, June 23, 12 -
Let’s build concepts • What concepts do we mobilize to
evaluate whether tests are good or not? • We can create more sensible tests by changing our concepts Saturday, June 23, 12 -
C: Consistent Distance • If tests are fully integrative (end-to-end),
don’t stub - only relate to your system by the endpoints • If tests are unit tests, don’t integrate - make your objects’ relationships to others clear by mocking Saturday, June 23, 12 -
U: Unstubbed • Don’t stub private methods • Prefer things
like dependency injection to add dependencies • Don’t stub anything that isn’t yours Saturday, June 23, 12 -
I: Idempotent • Specs should always pass, regardless of order
• Your tests should not have side-effects • Run your specs with --random Saturday, June 23, 12 -
D: Distinct • See how much behavior you’re testing in
both unit tests and integration tests, and choose one or the other • You should not be afraid to delete tests • How many things fail when you change one piece of code? Saturday, June 23, 12 -
CUPID (review) • Consistent distance • Unstubbed • Pyramidal •
Idempotent • Distinct Saturday, June 23, 12 -
Problem 1: Violating consistent distance • Unit tests should isolate
a class under test • Acceptance tests should be end-to-end • VCR stubs - used for stubbing in acceptance tests Saturday, June 23, 12 -
Stubbing isn’t end-to- end • Better to have specific tests
that connect to external endpoints, or provide a fake service • http://github.com/jsl/shamrock Saturday, June 23, 12 -
Unstub: Use mock objects instead • Mock objects are underused
in most tests, we stub too frequently • Stubbing leads to complex test setup, and doesn’t help to make dependencies explicit Saturday, June 23, 12 -
Don’t be scared of DI • Don’t need a framework
in Ruby • Good for “harder” dependencies of classes • Provide default so that other classes don’t have to specify the dependency, and provide a mock in your tests Saturday, June 23, 12 -
Problem 5 - Duplicate coverage In a project that I
worked on, one single bug caused 124 tests to fail. I was so perplexed that I couldn't figure out which bridge to jump from. Alexandre de Oliveira - http://alexsquest.com/texts/1 Saturday, June 23, 12 -
Why is testing difficult? ... the world is messier in
general. There’s massive amounts of data floating around, and the kinds of problems that we’re trying to solve are much sloppier, and the solutions a lot less discrete than they used to be. Gerald Sussman http://cemerick.com/2009/03/24/why-mit-now-uses-python-instead-of-scheme-for-its- undergraduate-cs-program/ Saturday, June 23, 12 -
Let’s improve it ... we regularly reflect on how well
TDD is working for us, identify any weaknesses, and adapt our testing strategy. Freeman, Steve; Pryce, Nat (2009-10-12). Growing Object-Oriented Software, Guided by Tests (Kindle Locations 1161-1162). Pearson Education (USA). Kindle Edition. Saturday, June 23, 12 -
Sensible testing is reflexive What concepts are you mobilizing in
your testing? Justin Leitgeb justinleitgeb.com @justinleitgeb github.com/jsl Saturday, June 23, 12