Share
Embed
Seb Rose - Common Objections to TDD (and their refutations)
This is not a session about how or why to practice TDD. Based upon research conducted during the first quart of 2012, I will outline the most common objections to TDD and describe in detail, with examples where appropriate, how to refute, avoid or mitigate each of them. The coverage will acknowledge that there are risks inherent to all techniques and will not promote the idea that TDD is some kind of silver bullet.
Lean Agile Scotland
November 02, 2012
Transcript
50 slides
-
IBM Software Group © 2009 IBM Corporation Common Objections to
TDD (and their refutations) Seb Rose Twitter: @sebrose Blog: claysnow.blogspot.com E-mail: [email protected] Phone: 01721 788178 -
IBM Software Group | Rational software Innovation for a smarter
planet 2 Agenda Motivation Education Inappropriate project Cultural Time pressures Unclear benefits -
IBM Software Group | Rational software Innovation for a smarter
planet 3 Refutations. Really? re·fute /rɪˈfyut/ Show Spelled verb (used with object), re·fut·ed, re·fut·ing. 1. to prove to be false or erroneous, as an opinion or charge. 2. to prove (a person) to be in error. -
IBM Software Group | Rational software Innovation for a smarter
planet 4 TDD – in one slide Test First predates XP and Agile TDD has been around for a while One of XP’s core practices Widely referenced Still quite controversial I am not going to describe TDD in any detail Red/Green/Refactor (and variants) ‘T’ for Test has unfortunate connotations First ‘D’ definitely stands for Driven Second ‘D’ may stand for Development or Design -
IBM Software Group | Rational software Innovation for a smarter
planet 6 Survey Simple survey ran for 3 months Jan – Mar 2012 260 unique respondents 896 objections “What reasons are there not to practice TDD” “The question wreaks of elitism - it should instead be ‘what reasons are there to practice TDD’”. Free text made for a wide range of responses Harder to analyse Split into 18 types, grouped into 5 categories -
IBM Software Group | Rational software Innovation for a smarter
planet 7 Breakdown Education (223) Lack of Practice, Lack of Investment Project (194) Domain, Legacy, Environment, Tooling Cultural (179) Management, Team, Fanaticism, Demarcation, Egotism Time (174) Slowness of development, maintenance and execution Benefits (120) Lack of Proof & Experience, Alternatives -
IBM Software Group | Rational software Innovation for a smarter
planet 8 Results Education Time Benefits Project Cultural -
IBM Software Group | Rational software Innovation for a smarter
planet 9 The required techniques of effective reasoning are pretty formal, but as long as programming is done by people that don't master them, the software crisis will remain with us and will be considered an incurable disease. And you know what incurable diseases do: they invite the quacks and charlatans in, who in this case take the form of Software Engineering gurus. - Dijkstra (ACM Turing Award Lecture 1972) Software Crisis! -
IBM Software Group | Rational software Innovation for a smarter
planet 10 Fanaticism and Fluff The steps of FDD are simple: 1. Take a tiny piece of fluff from the plate and put it on your head, holding your head quite still to ensure that the fluff does not fall off your hair. 2. Write a line of code. 3. Say, “I am the Fluff Lord, within the Dominion Of The Fluffists.” 4. Repeat. That’s it. Seriously, that’s all there is to it. -
IBM Software Group | Rational software Innovation for a smarter
planet 11 Agenda Motivation Education Inappropriate project Cultural Time pressures Unclear benefits Wrap up -
IBM Software Group | Rational software Innovation for a smarter
planet 12 Hard to learn “TDD is hard.” Yes, it is Any technique is hard to master Fundamental changes are hard to adopt 10,000 hours of practice Did you stop trying to ride a bicycle when you fell off? -
IBM Software Group | Rational software Innovation for a smarter
planet 13 Where to start “Sheer lack of knowledge on how to approach it appropriately. There are too many bowling examples and not enough practicality.” How do you usually learn something new? Unit Testing skills are foundational Coding Dojo to practice TDD http://cyber-dojo.com Better used as a group, but still useful solo -
IBM Software Group | Rational software Innovation for a smarter
planet 14 Tests before code “The compiler complains if I write tests before the code” First step to get to green is to make the test compile Only then can you move on to make the test pass -
IBM Software Group | Rational software Innovation for a smarter
planet 15 Tests will be buggy “Your test code is just your production code, written from the other end (i.e. just as complex and likely to have bugs)” No process is infallible “To err is human …” Safety in numbers Pair programming Peer review Static analysis -
IBM Software Group | Rational software Innovation for a smarter
planet 16 Unfamiliar architectural style “How will we ever know what’s going on if we can't see all of the code at once?” TDD tends to lead to small, concise implementations Low coupling High cohesion “Proliferation” of interfaces Literate-ish programming -
IBM Software Group | Rational software Innovation for a smarter
planet 17 Discipline “TDD is a discipline and a work habit. It's very difficult to establish the habit.” Habits are hard to form, but also hard to break. Nothing works better than positive feedback There are many self-help guides available Switch – Chip & Dan Heath Drive – Dan Pink -
IBM Software Group | Rational software Innovation for a smarter
planet 18 Agenda Motivation Education Inappropriate project Cultural Time pressures Unclear benefits Wrap up -
IBM Software Group | Rational software Innovation for a smarter
planet 19 Too simple "This code is too simple to need a test“ Then the tests will be simple too What’s simple to you may be opaque to others As the code evolves, tests can help keep it simple -
IBM Software Group | Rational software Innovation for a smarter
planet 20 Just a spike “Because you're prototyping an idea and it's much faster to spike without tests. If you do end up using the code then you can write tests and refactor.” Use the right tool – TDD is not mandatory Discipline to ensure spike does not mutate “Write one to throw away” TDD the spike anyway! -
IBM Software Group | Rational software Innovation for a smarter
planet 21 Not useful for all aspects of design “I don't think that deriving a design by writing tests is a useful practice. Tests by themselves cannot cover many aspects of a design (designing for concurrency and performance in particular by writing tests is something that I've never seen anyone do).” TDD is not the only tool in the toolbox Probably wrong tool for performance testing Can be applied to concurrency testing (but not recommended) -
IBM Software Group | Rational software Innovation for a smarter
planet 22 Legacy code “Difficulty starting with legacy code. A simple change done through TDD can take orders of magnitude longer due to a need to redesign toward testability. ” Working Effectively With Legacy Code – Feathers Small, conservative steps, eventually tame fear and doubt Consider less intrusive approaches initially e.g. TextTest – texttest.org -
IBM Software Group | Rational software Innovation for a smarter
planet 23 Insufficient tooling “There is no unit-test framework for the language I'm using, and I don't have time/inclination to develop one myself.” There probably is. You might not need one “TDD in C” – Olve Maudal -
IBM Software Group | Rational software Innovation for a smarter
planet 24 GUI “I do a lot of UI code and don't really know how to properly do that with TDD.” “Subcutaneous” testing Presenter First pattern Based on MVC/MVP patterns Very shallow view, with no business logic Stateless presenter orchestrates interaction between view and model -
IBM Software Group | Rational software Innovation for a smarter
planet 25 Agenda Motivation Education Inappropriate project Cultural Time pressures Unclear benefits Wrap up -
IBM Software Group | Rational software Innovation for a smarter
planet 26 Management antipathy “Management want results NOW; very much willing to clean up small oversights later. 'Close is good enough' attitude.” Management are generally result focused Previous bad experiences affect appetite for change Both ways! Start small Need to demonstrate benefit to ‘bottom line’ -
IBM Software Group | Rational software Innovation for a smarter
planet 27 Team resistance “Teammates are not prepared. They do not have enough knowledge or experience in unit testing.” “No one tells me how to program!” Changing behaviours is hard “Fearless Change” has many patterns for introducing change -
IBM Software Group | Rational software Innovation for a smarter
planet 28 I never make mistakes “I already know what the code needs to do and it's low risk. I don't need a test for it.” “My first design idea is always perfectly good.” “My code always works the first time.” Do people really believe this? Even if they do, is everyone that ever touches the code going to be so talented? -
IBM Software Group | Rational software Innovation for a smarter
planet 29 Testing is for testers “We don't need TDD, we've got a QA department. They'll find the bugs for us.” TDD is not just about testing Drives design Refactoring/regression safety net Write tests to explore EVERY defect you find Living documentation Helps future developers understand intended behaviours -
IBM Software Group | Rational software Innovation for a smarter
planet 30 All TDD-ers are fanatics “It has become a cult. Its advocates have made it antithetical to ‘Individuals and interactions over processes and tools’. It is evangelized through coercion and browbeating -- necessarily, as it isn't compelling on it's own.” “Someone needs to tell unclebob that he might be right, but he's part of the problem - to non-modern coders (which felt like the majority last time I looked) he comes across like a raving loony.” -
IBM Software Group | Rational software Innovation for a smarter
planet 31 Fluff (continued) If at any time you even THINK about writing a line of code before putting fluff on your head, then you’ve to delete all your code, shake all the fluff from your head onto the plate and start all over again. I have been practicing FDD for 12 years now; sometimes in the office, I have so much fluff on my head that my boss thinks I’m a hay-stack, only made of fluff. A sort-of fluff-stack, if you will. But one thing is beyond doubt: the code I write, when I’m in this fluff-zone, is the most high-quality code that anyone has ever seen. -
IBM Software Group | Rational software Innovation for a smarter
planet 32 Agenda Motivation Education Inappropriate project Cultural Time pressures Unclear benefits Wrap up -
IBM Software Group | Rational software Innovation for a smarter
planet 33 More code to write “Takes too much time to write test.” How will your code get tested? Test team? Customers & Stack Trace Driven Development? How much time will be spent figuring out how to make changes later? What proportion of your time do you actually spend coding? -
IBM Software Group | Rational software Innovation for a smarter
planet 34 More code to maintain “If I change something it will break a whole bunch of tests that I will have to fix and it will be more work for me in the end than just verifying my changes manually.” It is USEFUL to know when behaviour changes Foundational Unit Testing skills reduce brittleness -
IBM Software Group | Rational software Innovation for a smarter
planet 35 Mock-based tests rot “Unit-test mock/stub assumptions rots” Use collaboration and contract tests – J.B. Rainsberger Collaboration tests make assumptions about the contract; contract tests try to justify those assumptions A stub in a collaboration test must correspond to an expected result in a contract test An expectation in a collaboration test must correspond to an action in a contract test -
IBM Software Group | Rational software Innovation for a smarter
planet 36 That library is third party “API layers above and below you don't provide adequate mocks. Testing against the ‘real thing’ is hard/impossible.” “Don't mock types you don't own” – Joe Walnes Write adapters that provide a domain specific API Write acceptance tests that verify the library’s behaviour Run whenever adopting a new version -
IBM Software Group | Rational software Innovation for a smarter
planet 37 Slow execution “Running the tests takes too long.” For TDD to work tests need to (build and) execute in seconds Some environments need careful configuration C/C++ Rails Are they really Unit Tests? Partition your test suites -
IBM Software Group | Rational software Innovation for a smarter
planet 38 Agenda Motivation Education Inappropriate project Cultural Time pressures Unclear benefits Wrap up -
IBM Software Group | Rational software Innovation for a smarter
planet 39 Absence of research “No scientific proof of it actually having any benefit (compared to just code reviews, pair programming or etc.) for the same amount of time” “It's not practical for the kind of work I do. By which I mean, it cannot be conclusively shown to provide tangible benefits that outweigh the perceived costs.” “In an industry where todays must have offering is tomorrows dustbin liner, it's often prudent to wait before trying the latest snake oil!” -
IBM Software Group | Rational software Innovation for a smarter
planet 40 Absence of research (cont.) “We found that test-first students on average wrote more tests and, in turn, students who wrote more tests tended to be more productive. We also observed that the minimum quality increased linearly with the number of programmer tests, independent of the development strategy employed.” On the Effectiveness of Test-first Approach to Programming Proceedings of the IEEE Transactions on Software Engineering 2005 Plenty of papers: http://research.microsoft.com/en-us/groups/ese/nagappan_tdd.pdf http://bradapp.blogspot.co.uk/2009/08/studies-on-effectiveness-of-tdd.html -
IBM Software Group | Rational software Innovation for a smarter
planet 41 Manual testing “It is much faster to just think about the proper implementation, write the code and manually test it.” Until you need to do it again … and again Manual scripts Error prone Time consuming Expensive -
IBM Software Group | Rational software Innovation for a smarter
planet 42 BDD “Why practice TDD when you can practice BDD?” BDD extends TDD Involves stakeholders, not just developers Even harder to implement. -
IBM Software Group | Rational software Innovation for a smarter
planet 43 ATDD, BDD, GOOS Many shades of outside-in, test-first development -
IBM Software Group | Rational software Innovation for a smarter
planet 44 “And check that nothing happens that shouldn’t happen and everything else happens that should happen for all variations of this scenario and all possible states of the ATM and all possible states of the customer’s account and all possible states of the rest of the database and all possible states of the system as a whole, and anything happening in the cloud that should not matter but might matter.” Given the account is in credit And the card is valid And the dispenser contains cash When the customer requests cash Then check that the account is debited And check that cash is dispensed And check that the card is returned None of these are enough either -
IBM Software Group | Rational software Innovation for a smarter
planet 45 Agenda Motivation Education Inappropriate project Cultural Time pressures Unclear benefits Wrap up -
IBM Software Group | Rational software Innovation for a smarter
planet 46 Not sufficient, but necessary? TDD is not enough. Consider: Acceptance Usability Performance / Stress Mutation Exploratory Systems have been, and continue to be, successfully delivered without TDD. But they have also been delivered without designs, documentation, or …. testing of any kind. -
IBM Software Group | Rational software Innovation for a smarter
planet 47 Conclusions There are many impediments to adopting TDD Benefits are not universally accepted TDD is not a “Silver Bullet” -
IBM Software Group | Rational software Innovation for a smarter
planet 48 Questions? [email protected] @sebrose www.claysnow.co.uk Pick up a business card! -
IBM Software Group | Rational software Innovation for a smarter
planet 49 Essential Reading Test Driven Development by Example – Kent Beck Growing Object Oriented Software Guided By Tests – Steve Freeman/Nat Pryce Working With Legacy Code – Michael Feathers Fearless Change – Mary Lynn Manns/Linda Rising JUnit Recipes – J.B. Rainsberger -
IBM Software Group | Rational software Innovation for a smarter
planet 50 Other references Fluff Driven Development: http://www.threeriversinstitute.org/blog/?p=594 Presenter First: http://atomicobject.com/files/PresenterFirstAgile2006.pdf TDD in C: http://www.pvv.org/~oma/TDDinC_Smidig2007.pdf On the effectiveness of Test-first approach to Programming: http://nparc.cisti-icist.nrc- cnrc.gc.ca/npsi/ctrl?action=rtdoc&an=5763742&article=0&lang= en&ext=pdf