The Cartography of Testing: CodeMash 2015

The Cartography of Testing: CodeMash 2015

Developers writing tests is now very very common. But testing remains a big, confusing, and controversial topic. This talk attempts to take a fresh look at testing, and try to get at a deeper more nuanced understanding of it. So even if you do TDD every day, hopefully you’ll come away with a refreshed and invigorated perspective.

35e74c48a612d8a6786f8ab6424b49a1?s=128

Kevin Berridge

January 08, 2015
Tweet

Transcript

  1. THE CARTOGRAPHY OF TESTING BY KEVIN BERRIDGE From Matrix Pointe

    Software
  2. THE CARTOGRAPHY OF TESTING BY KEVIN BERRIDGE From Matrix Pointe

    Software
  3. CONTENTIOUS ARGUMENTS

  4. EASY TO GET LOST

  5. MAP OUT THE TESTING SPACE

  6. WHY WRITE TESTS? HOW DO WE WRITE “GOOD” TESTS?

  7. WHY WRITE TESTS?

  8. BECAUSE.

  9. Uncle Bob Martin

  10. REASONS

  11. –Pragmatic Programmer “It ends up being cheaper in the long

    run.”
  12. –Practical Object-Oriented Design In Ruby “The true purpose of testing,

    just like the true purpose of design, is to reduce costs.”
  13. TESTING REDUCES COSTS?

  14. I DON’T HAVE TIME TO TEST

  15. COST == TIME

  16. TESTING REDUCES COSTS?

  17. TESTING REDUCES TIME?

  18. No Tests 0 5000 10000 15000 20000 Production Test

  19. 2x Test LOC 0 5000 10000 15000 20000 Production Test

  20. –Practical Object-Oriented Design In Ruby “Fixing bugs early always lowers

    costs.”
  21. –Code Complete “It is cheaper to build high- quality software

    than it is to build and fix low-quality software.”
  22. –Software Test Automation “Run more often. Runs tests that are

    difficult or impossible manually. Better use of resources. Consistency & repeatability.”
  23. –Practical Object-Oriented Design In Ruby “Tests provide the only reliable

    documentation of design.”
  24. –Test-Driven Development “Be sure to factor debugging, integrating, and explaining

    time into your metrics, though.”
  25. –Code Complete “The debugging job is easier.”

  26. –Refactoring “These days I hardly ever spend more than a

    few minutes debugging.”
  27. –Software Test Automation “The true value of automated tests is

    often in providing confidence rather than finding defects.”
  28. wk1 wk2 wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10

    wk11 wk12 PRODUCTIVITY
  29. wk1 wk2 wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10

    wk11 wk12 PRODUCTIVITY
  30. SOFTWARE ROT

  31. –@ChrisOldwood, Twitter “Not refactoring after making a code change is

    like putting an empty cereal box back in the cupboard.”
  32. REFACTORING

  33. BIG CHANGES

  34. SMALL CHANGES

  35. None
  36. SOFTWARE ROT

  37. WHAT DOES THIS HAVE TO DO WITH TESTING?

  38. FEAR OF CHANGING CODE SHOE HORNING IN CHANGES SOFTWARE ROT

  39. –Refactoring “If you want to refactor, the essential precondition is

    having solid tests.”
  40. –Refactoring “Whenever I do a refactoring, the first step is

    always the same. I need to build a solid set of tests for that section of code. The tests are essential because even though I follow refactorings structured to avoid most of the opportunities for introducing bugs, I'm still human and still make mistakes. Thus I need solid tests.”
  41. –Working Effectively With Legacy Code “To me, legacy code is

    simply code without tests.”
  42. Martin Fowler

  43. IF YOU DON’T HAVE TESTS, YOU WON’T CHANGE THE CODE

  44. –Clean Code “It is unit tests that keep our code

    flexible, maintainable, and reusable." "If you have tests, you do not fear making changes to the code!”
  45. –Refactoring “The tests give you the security to change the

    program later.”
  46. –Clean Code “Code without tests is bad code. It doesn't

    matter how well written it is; it doesn't matter how pretty or object-oriented or well- encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don't know if our code is getting better or worse.”
  47. –Growing Object-Oriented Software “We need to keep the code as

    simple as possible, so it's easier to understand and modify. …Simplicity takes effort, so we constantly refactor our code as we work with it. …The test suites in the feedback loops protect us against our own mistakes as we improve (and therefore change) the code.”
  48. –Practical Object-Oriented Design In Ruby “The art of writing changeable

    code requires the ability to write high-value tests. Tests give you confidence to refactor constantly.”
  49. WHY WRITE TESTS?

  50. TESTS ARE THE BEST AND CHEAPEST TOOL WE HAVE TO

    MANAGE FEAR AND REPLACE IT WITH CONFIDENCE
  51. WRITE “GOOD” TESTS SO YOU CAN (AND WILL) REFACTOR SO

    YOUR CODE DOESN’T ROT
  52. –Test-Driven Development “My goal is to feel better about a

    project after a year than I did in the starry-eyed beginning, and TDD helps me achieve this.”
  53. HOW TO WRITE “GOOD” TESTS?

  54. PREFER “SMALLER” TESTS

  55. MORE THAN ONE KIND OF TEST

  56. UNIT INTEGRATION SYSTEM ACCEPTANCE VALIDATION/ VERIFICATION PERFORMANCE RESOURCE-EXHAUSTION USABILITY FUNCTIONAL

    DEVELOPER BLACK BOX GLASS BOX ISOLATED CHARACTERIZATION END-TO-END
  57. None
  58. SYSTEM ACCEPTANCE FUNCTIONAL BLACK BOX END-TO-END

  59. END-TO-END

  60. INTEGRATION DEVELOPER GLASS BOX ISOLATED CHARACTERIZATION UNIT

  61. UNIT

  62. Unit End-To-End

  63. T O O O D S UI

  64. T O O O D S UI

  65. T O I V D S UI

  66. PREFER “SMALLER” TESTS

  67. FAST VS. SLOW

  68. FEEDBACK

  69. –Working Effectively With Legacy Code “Do you want your feedback

    in a minute or overnight?”
  70. –Working Effectively With Legacy Code “Unit testing is one of

    the most important components in legacy code work. System-level regression tests are great, but small, localized tests are invaluable. They can give you feedback as you develop and allow you to refactor with much more safety.”
  71. –Working Effectively With Legacy Code “The feedback we get from

    [testing] is very useful. It pays to [test] at a finer- grained level.”
  72. COMPLEXITY

  73. T O O O

  74. –Growing Object-Oriented Software “If we test at too large a

    grain, the combinatorial explosion of trying all the possible paths through the code will bring development to a halt.”
  75. –Working Effectively With Legacy Code “There are a few problems

    with large tests: Error localization, Execution time, Coverage”
  76. http://blog.thecodewhisperer.com/2010/10/16/integrated-tests-are-a-scam/

  77. 310 59,049

  78. T O O O

  79. T O I V

  80. –Working Effectively With Legacy Code “There are a few problems

    with large tests: Error localization, Execution time, Coverage”
  81. USE “LARGER” TESTS APPROPRIATELY

  82. Acceptance Integration Unit 10% 20% 70% http://jamescrisp.org/2011/05/30/automated-testing-and-the-test-pyramid/

  83. None
  84. THE GOOS APPROACH

  85. –Growing Object-Oriented Software “We start work on a new feature

    by writing failing acceptance tests that demonstrate that the system does not yet have the feature we're about to write and track our progress towards completion of the feature.”
  86. END-TO-END: PROVE THE EXISTENCE OF A FEATURE NOT THE CORRECTNESS

  87. “INTEGRATION” TESTS

  88. –Mike Bland, http://martinfowler.com/articles/testing-culture.html “You should not be catching bugs at

    the integration or system levels that could have been caught at the unit level.”
  89. THE EXTRACT CLASS REFACTORING

  90. T O

  91. T O O

  92. T T O O

  93. T T O O

  94. T T O O

  95. PREVENT TEST ROT

  96. –Clean Code “Test code is just as important as production

    code.”
  97. –Sandi Metz “I examine the original test and morph it

    from an integration test into an actual unit test.”
  98. T T O O

  99. TEST DOUBLES MOCKS STUBS FAKES SUBSTITUTES

  100. ALTERNATE REALITY

  101. bit.ly/kwb-oop

  102. –Growing Object-Oriented Software “…investigate why the test is hard to

    write and refactor the code to improve its structure. We call this 'listening to the tests.'”
  103. RESPOND TO THE TESTS

  104. –Practical Object-Oriented Design In Ruby “Tests are the canary in

    the coal mine; when the design is bad, testing is hard.”
  105. –Growing Object-Oriented Software “We've found that the qualities that make

    an object easy to test also make our code responsive to change.”
  106. DEPENDENCY

  107. –Working Effectively With Legacy Code “Dependency is one of the

    most critical problems in software development. Much legacy code work involves breaking dependencies so that change can be easier.”
  108. –Test-Driven Development “Dependency is the key problem in software development

    at all scales.”
  109. –Working Effectively With Legacy Code “When classes depend directly on

    things that are hard to use in a test, they are hard to modify and hard to work with.”
  110. –Practical Object-Oriented Design In Ruby “Dealing with objects as if

    they are only and exactly the messages to which they respond lets you design a changeable application, and it is your understanding of the importance of this perspective that allows you to create tests that provide maximum benefit at minimum cost.”
  111. TEST BEHAVIOR NOT IMPLEMENTATION

  112. –Kent Beck “Separate interface from implementation thinking. I have a

    tendency to pollute API design decisions with implementation speculation.”
  113. T O C Q

  114. T I O C Q

  115. T I O C Q

  116. T O I C Q

  117. –Practical Object-Oriented Design in Ruby “Some outgoing messages have no

    side effects… Outgoing messages like this are known as queries and they need not be tested by the sending object.”
  118. –Practical Object-Oriented Design in Ruby “Many outgoing messages do have

    side effects… These messages are commands and it is the responsibility of the sending object to prove that they are properly sent.”
  119. WRITE CLEAR TESTS

  120. POORLY WRITTEN TESTS ARE A HINDRANCE

  121. –Growing Object-Oriented Software “For TDD to be sustainable, the tests

    must do more than verify the behavior of the code; they must also express that behavior clearly -- they must be readable.”
  122. –Clean Code “The dirtier the tests, the harder they are

    to change… So the tests become viewed as an ever- increasing liability.”
  123. –Test-Driven Development “"Test methods should be easy to read.”

  124. –Test-Driven Development “The first criterion for your tests is confidence.

    …The second criterion is communication.”
  125. –Growing Object-Oriented Software “We want our test code to read

    like a declarative description of what is being testing.”
  126. –Growing Object-Oriented Software “We want to make sure the tests

    pull their weight by making them expressive, so that we can tell what's important when we read them and when they fail”
  127. MAKE EVERY SYMBOL IN THE TEST ESSENTIAL

  128. –Clean Code “…the technique of building a domain-specific language for

    your tests”
  129. CARE ABOUT THE FAILURE MESSAGE

  130. –Software Test Automation “Test cases must be designed with debugging

    in mind by asking 'What would I want to know when this test fails.’”
  131. –Growing Object-Oriented Software “When we get the 'right' failure, we

    check that the diagnostics are helpful.”
  132. WHY WRITE TESTS?

  133. TESTING REDUCES COSTS

  134. SOFTWARE ROT

  135. HOW TO WRITE “GOOD” TESTS?

  136. PREFER “SMALLER” TESTS

  137. USE “LARGER” TESTS APPROPRIATELY

  138. PREVENT TEST ROT

  139. RESPOND TO THE TESTS

  140. TEST BEHAVIOR NOT IMPLEMENTATION

  141. WRITE CLEAR TESTS

  142. MAKE EVERY SYMBOL IN THE TEST ESSENTIAL

  143. CARE ABOUT THE FAILURE MESSAGE

  144. None
  145. –Kevin Berridge, @kberridge “Thanks.”