Sensible Testing

Transcript

1. Justin Leitgeb Stack Builders June 23, 2012 GoRuCo Sensible Testing Saturday, June 23, 12
2. Quantitative concepts: How much should I test? • TATFT • DHH • Kent Beck Saturday, June 23, 12
3. TATFT Test all the time? Saturday, June 23, 12
4. DHH • "Don’t aim for 100% coverage." • "Code-to-test ratios above 1:2 is a smell, above 1:3 is a stink." • "You’re probably doing it wrong if testing is taking more than 1/3 of your time. You’re definitely doing it wrong if it’s taking up more than half." http://37signals.com/svn/posts/3159-testing-like-the-tsa Saturday, June 23, 12
5. Kent Beck I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). http://stackoverflow.com/questions/153234/how-deep-are-your-unit-tests Saturday, June 23, 12
6. Moving beyond quantitative concepts • We have lots of concepts for production code (DRY, SOLID, design patterns) • Much testing talk stays on the quantitative level - what is our “coverage”? • Instead, what are some test concepts that help to make our code more “habitable”? Saturday, June 23, 12
7. Habitability Habitability is one of the most important characteristics of software. It enables developers to live comfortably in and repair or modify code and design. Patterns of software: Tales from the Software Community Richard Gabriel Saturday, June 23, 12
8. Let’s build concepts • What concepts do we mobilize to evaluate whether tests are good or not? • We can create more sensible tests by changing our concepts Saturday, June 23, 12
9. CUPID Show your tests some love. Saturday, June 23, 12
10. C: Consistent Distance • If tests are fully integrative (end-to-end), don’t stub - only relate to your system by the endpoints • If tests are unit tests, don’t integrate - make your objects’ relationships to others clear by mocking Saturday, June 23, 12
11. U: Unstubbed • Don’t stub private methods • Prefer things like dependency injection to add dependencies • Don’t stub anything that isn’t yours Saturday, June 23, 12
12. P: Pyramidal Saturday, June 23, 12
13. I: Idempotent • Specs should always pass, regardless of order • Your tests should not have side-effects • Run your specs with --random Saturday, June 23, 12
14. D: Distinct • See how much behavior you’re testing in both unit tests and integration tests, and choose one or the other • You should not be afraid to delete tests • How many things fail when you change one piece of code? Saturday, June 23, 12
15. CUPID (review) • Consistent distance • Unstubbed • Pyramidal • Idempotent • Distinct Saturday, June 23, 12
16. Applying CUPID Saturday, June 23, 12
17. Problem 1: Violating consistent distance • Unit tests should isolate a class under test • Acceptance tests should be end-to-end • VCR stubs - used for stubbing in acceptance tests Saturday, June 23, 12
18. Stubbing isn’t end-to- end • Better to have specific tests that connect to external endpoints, or provide a fake service • http://github.com/jsl/shamrock Saturday, June 23, 12
19. Problem 2: Stubbing instead of mocking Saturday, June 23, 12
20. Unstub: Use mock objects instead • Mock objects are underused in most tests, we stub too frequently • Stubbing leads to complex test setup, and doesn’t help to make dependencies explicit Saturday, June 23, 12
21. Unstub continued Don’t stub things you don’t own Saturday, June 23, 12
22. Use dependency injection Saturday, June 23, 12
23. Don’t be scared of DI • Don’t need a framework in Ruby • Good for “harder” dependencies of classes • Provide default so that other classes don’t have to specify the dependency, and provide a mock in your tests Saturday, June 23, 12
24. Problem 3: Reverse pyramid Saturday, June 23, 12
25. Problem 4: Non- idempotent tests Saturday, June 23, 12
26. Problem 5 - Duplicate coverage In a project that I worked on, one single bug caused 124 tests to fail. I was so perplexed that I couldn't figure out which bridge to jump from. Alexandre de Oliveira - http://alexsquest.com/texts/1 Saturday, June 23, 12
27. Why is testing difficult? ... the world is messier in general. There’s massive amounts of data floating around, and the kinds of problems that we’re trying to solve are much sloppier, and the solutions a lot less discrete than they used to be. Gerald Sussman http://cemerick.com/2009/03/24/why-mit-now-uses-python-instead-of-scheme-for-its- undergraduate-cs-program/ Saturday, June 23, 12
28. Let’s improve it ... we regularly reflect on how well TDD is working for us, identify any weaknesses, and adapt our testing strategy. Freeman, Steve; Pryce, Nat (2009-10-12). Growing Object-Oriented Software, Guided by Tests (Kindle Locations 1161-1162). Pearson Education (USA). Kindle Edition. Saturday, June 23, 12
29. Sensible testing is reflexive What concepts are you mobilizing in your testing? Justin Leitgeb justinleitgeb.com @justinleitgeb github.com/jsl Saturday, June 23, 12