Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Cartography of Testing: CodeMash 2015

The Cartography of Testing: CodeMash 2015

Developers writing tests is now very very common. But testing remains a big, confusing, and controversial topic. This talk attempts to take a fresh look at testing, and try to get at a deeper more nuanced understanding of it. So even if you do TDD every day, hopefully you’ll come away with a refreshed and invigorated perspective.

Kevin Berridge

January 08, 2015
Tweet

More Decks by Kevin Berridge

Other Decks in Programming

Transcript

  1. THE
    CARTOGRAPHY
    OF
    TESTING
    BY KEVIN BERRIDGE
    From Matrix Pointe Software

    View full-size slide

  2. THE
    CARTOGRAPHY
    OF
    TESTING
    BY KEVIN BERRIDGE
    From Matrix Pointe Software

    View full-size slide

  3. CONTENTIOUS
    ARGUMENTS

    View full-size slide

  4. EASY TO
    GET LOST

    View full-size slide

  5. MAP OUT THE
    TESTING SPACE

    View full-size slide

  6. WHY WRITE TESTS?
    HOW DO WE WRITE
    “GOOD” TESTS?

    View full-size slide

  7. WHY WRITE TESTS?

    View full-size slide

  8. Uncle Bob Martin

    View full-size slide

  9. –Pragmatic Programmer
    “It ends up being cheaper
    in the long run.”

    View full-size slide

  10. –Practical Object-Oriented Design In
    Ruby
    “The true purpose of testing,
    just like the true purpose of
    design, is to reduce costs.”

    View full-size slide

  11. TESTING
    REDUCES
    COSTS?

    View full-size slide

  12. I DON’T HAVE TIME TO TEST

    View full-size slide

  13. COST == TIME

    View full-size slide

  14. TESTING
    REDUCES
    COSTS?

    View full-size slide

  15. TESTING
    REDUCES
    TIME?

    View full-size slide

  16. No Tests
    0
    5000
    10000
    15000
    20000
    Production Test

    View full-size slide

  17. 2x Test LOC
    0
    5000
    10000
    15000
    20000
    Production Test

    View full-size slide

  18. –Practical Object-Oriented Design In
    Ruby
    “Fixing bugs early always
    lowers costs.”

    View full-size slide

  19. –Code Complete
    “It is cheaper to build high-
    quality software than it is to
    build and fix low-quality
    software.”

    View full-size slide

  20. –Software Test Automation
    “Run more often.
    Runs tests that are difficult or
    impossible manually.
    Better use of resources.
    Consistency & repeatability.”

    View full-size slide

  21. –Practical Object-Oriented Design In
    Ruby
    “Tests provide the only
    reliable documentation of
    design.”

    View full-size slide

  22. –Test-Driven Development
    “Be sure to factor
    debugging, integrating, and
    explaining time into your
    metrics, though.”

    View full-size slide

  23. –Code Complete
    “The debugging job is
    easier.”

    View full-size slide

  24. –Refactoring
    “These days I hardly ever
    spend more than a few
    minutes debugging.”

    View full-size slide

  25. –Software Test Automation
    “The true value of
    automated tests is often in
    providing confidence rather
    than finding defects.”

    View full-size slide

  26. wk1 wk2 wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10 wk11 wk12
    PRODUCTIVITY

    View full-size slide

  27. wk1 wk2 wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10 wk11 wk12
    PRODUCTIVITY

    View full-size slide

  28. –@ChrisOldwood, Twitter
    “Not refactoring after making a
    code change is like putting an
    empty cereal box back in the
    cupboard.”

    View full-size slide

  29. SMALL
    CHANGES

    View full-size slide

  30. WHAT DOES THIS HAVE
    TO DO WITH TESTING?

    View full-size slide

  31. FEAR OF
    CHANGING
    CODE
    SHOE HORNING
    IN CHANGES
    SOFTWARE
    ROT

    View full-size slide

  32. –Refactoring
    “If you want to refactor, the
    essential precondition is
    having solid tests.”

    View full-size slide

  33. –Refactoring
    “Whenever I do a refactoring, the first step is
    always the same. I need to build a solid set of
    tests for that section of code. The tests are
    essential because even though I follow
    refactorings structured to avoid most of the
    opportunities for introducing bugs, I'm still human
    and still make mistakes. Thus I need solid tests.”

    View full-size slide

  34. –Working Effectively With Legacy Code
    “To me, legacy code is
    simply code without tests.”

    View full-size slide

  35. Martin Fowler

    View full-size slide

  36. IF YOU DON’T HAVE TESTS,
    YOU WON’T CHANGE THE
    CODE

    View full-size slide

  37. –Clean Code
    “It is unit tests that keep our
    code flexible, maintainable,
    and reusable."
    "If you have tests, you do
    not fear making changes to
    the code!”

    View full-size slide

  38. –Refactoring
    “The tests give you the
    security to change the
    program later.”

    View full-size slide

  39. –Clean Code
    “Code without tests is bad code. It doesn't
    matter how well written it is; it doesn't matter
    how pretty or object-oriented or well-
    encapsulated it is. With tests, we can change
    the behavior of our code quickly and
    verifiably. Without them, we really don't know
    if our code is getting better or worse.”

    View full-size slide

  40. –Growing Object-Oriented Software
    “We need to keep the code as simple as
    possible, so it's easier to understand and
    modify. …Simplicity takes effort, so we
    constantly refactor our code as we work with
    it. …The test suites in the feedback loops
    protect us against our own mistakes as we
    improve (and therefore change) the code.”

    View full-size slide

  41. –Practical Object-Oriented Design In
    Ruby
    “The art of writing changeable
    code requires the ability to write
    high-value tests. Tests give you
    confidence to refactor
    constantly.”

    View full-size slide

  42. WHY WRITE TESTS?

    View full-size slide

  43. TESTS ARE THE BEST AND
    CHEAPEST TOOL WE HAVE TO
    MANAGE FEAR AND REPLACE IT
    WITH CONFIDENCE

    View full-size slide

  44. WRITE
    “GOOD” TESTS
    SO YOU CAN
    (AND WILL)
    REFACTOR
    SO YOUR
    CODE DOESN’T
    ROT

    View full-size slide

  45. –Test-Driven Development
    “My goal is to feel better about a project after
    a year than I did in the starry-eyed
    beginning, and TDD helps me achieve this.”

    View full-size slide

  46. HOW TO WRITE
    “GOOD” TESTS?

    View full-size slide

  47. PREFER
    “SMALLER”
    TESTS

    View full-size slide

  48. MORE THAN ONE
    KIND OF TEST

    View full-size slide

  49. UNIT
    INTEGRATION
    SYSTEM
    ACCEPTANCE
    VALIDATION/
    VERIFICATION
    PERFORMANCE
    RESOURCE-EXHAUSTION
    USABILITY
    FUNCTIONAL
    DEVELOPER
    BLACK BOX
    GLASS BOX
    ISOLATED
    CHARACTERIZATION
    END-TO-END

    View full-size slide

  50. SYSTEM
    ACCEPTANCE
    FUNCTIONAL
    BLACK BOX
    END-TO-END

    View full-size slide

  51. INTEGRATION
    DEVELOPER
    GLASS BOX
    ISOLATED
    CHARACTERIZATION
    UNIT

    View full-size slide

  52. Unit End-To-End

    View full-size slide

  53. PREFER
    “SMALLER”
    TESTS

    View full-size slide

  54. FAST VS. SLOW

    View full-size slide

  55. –Working Effectively With Legacy Code
    “Do you want your feedback
    in a minute or overnight?”

    View full-size slide

  56. –Working Effectively With Legacy Code
    “Unit testing is one of the most important components
    in legacy code work. System-level regression tests are
    great, but small, localized tests are invaluable. They
    can give you feedback as you develop and allow you
    to refactor with much more safety.”

    View full-size slide

  57. –Working Effectively With Legacy Code
    “The feedback we get from
    [testing] is very useful. It
    pays to [test] at a finer-
    grained level.”

    View full-size slide

  58. –Growing Object-Oriented Software
    “If we test at too large a grain,
    the combinatorial explosion of
    trying all the possible paths
    through the code will bring
    development to a halt.”

    View full-size slide

  59. –Working Effectively With Legacy Code
    “There are a few problems with
    large tests: Error localization,
    Execution time, Coverage”

    View full-size slide

  60. http://blog.thecodewhisperer.com/2010/10/16/integrated-tests-are-a-scam/

    View full-size slide

  61. –Working Effectively With Legacy Code
    “There are a few problems with
    large tests: Error localization,
    Execution time, Coverage”

    View full-size slide

  62. USE “LARGER”
    TESTS
    APPROPRIATELY

    View full-size slide

  63. Acceptance
    Integration
    Unit
    10%
    20%
    70%
    http://jamescrisp.org/2011/05/30/automated-testing-and-the-test-pyramid/

    View full-size slide

  64. THE GOOS
    APPROACH

    View full-size slide

  65. –Growing Object-Oriented Software
    “We start work on a new feature by
    writing failing acceptance tests that
    demonstrate that the system does not
    yet have the feature we're about to
    write and track our progress towards
    completion of the feature.”

    View full-size slide

  66. END-TO-END:
    PROVE THE
    EXISTENCE OF A
    FEATURE NOT THE
    CORRECTNESS

    View full-size slide

  67. “INTEGRATION”
    TESTS

    View full-size slide

  68. –Mike Bland, http://martinfowler.com/articles/testing-culture.html
    “You should not be catching bugs
    at the integration or system levels
    that could have been caught at
    the unit level.”

    View full-size slide

  69. THE EXTRACT CLASS
    REFACTORING

    View full-size slide

  70. PREVENT
    TEST
    ROT

    View full-size slide

  71. –Clean Code
    “Test code is just as
    important as production
    code.”

    View full-size slide

  72. –Sandi Metz
    “I examine the original test and
    morph it from an integration test
    into an actual unit test.”

    View full-size slide

  73. TEST DOUBLES
    MOCKS
    STUBS
    FAKES
    SUBSTITUTES

    View full-size slide

  74. ALTERNATE REALITY

    View full-size slide

  75. bit.ly/kwb-oop

    View full-size slide

  76. –Growing Object-Oriented Software
    “…investigate why the test is hard to
    write and refactor the code to improve
    its structure. We call this 'listening to
    the tests.'”

    View full-size slide

  77. RESPOND
    TO THE
    TESTS

    View full-size slide

  78. –Practical Object-Oriented Design In
    Ruby
    “Tests are the canary in the coal mine;
    when the design is bad, testing is
    hard.”

    View full-size slide

  79. –Growing Object-Oriented Software
    “We've found that the qualities that
    make an object easy to test also make
    our code responsive to change.”

    View full-size slide

  80. –Working Effectively With Legacy Code
    “Dependency is one of the most critical
    problems in software development.
    Much legacy code work involves
    breaking dependencies so that change
    can be easier.”

    View full-size slide

  81. –Test-Driven Development
    “Dependency is the key problem in
    software development at all scales.”

    View full-size slide

  82. –Working Effectively With Legacy Code
    “When classes depend directly
    on things that are hard to use in
    a test, they are hard to modify
    and hard to work with.”

    View full-size slide

  83. –Practical Object-Oriented Design In
    Ruby
    “Dealing with objects as if they are only
    and exactly the messages to which they
    respond lets you design a changeable
    application, and it is your understanding
    of the importance of this perspective that
    allows you to create tests that provide
    maximum benefit at minimum cost.”

    View full-size slide

  84. TEST BEHAVIOR
    NOT
    IMPLEMENTATION

    View full-size slide

  85. –Kent Beck
    “Separate interface from
    implementation thinking. I have
    a tendency to pollute API
    design decisions with
    implementation speculation.”

    View full-size slide

  86. –Practical Object-Oriented Design in
    Ruby
    “Some outgoing messages have
    no side effects… Outgoing
    messages like this are known as
    queries and they need not be
    tested by the sending object.”

    View full-size slide

  87. –Practical Object-Oriented Design in
    Ruby
    “Many outgoing messages do have
    side effects… These messages are
    commands and it is the
    responsibility of the sending object
    to prove that they are properly sent.”

    View full-size slide

  88. WRITE
    CLEAR
    TESTS

    View full-size slide

  89. POORLY WRITTEN TESTS
    ARE A HINDRANCE

    View full-size slide

  90. –Growing Object-Oriented Software
    “For TDD to be sustainable, the
    tests must do more than verify the
    behavior of the code; they must
    also express that behavior clearly --
    they must be readable.”

    View full-size slide

  91. –Clean Code
    “The dirtier the tests, the harder
    they are to change… So the tests
    become viewed as an ever-
    increasing liability.”

    View full-size slide

  92. –Test-Driven Development
    “"Test methods should be
    easy to read.”

    View full-size slide

  93. –Test-Driven Development
    “The first criterion for your tests
    is confidence. …The second
    criterion is communication.”

    View full-size slide

  94. –Growing Object-Oriented Software
    “We want our test code to read
    like a declarative description of
    what is being testing.”

    View full-size slide

  95. –Growing Object-Oriented Software
    “We want to make sure the tests
    pull their weight by making them
    expressive, so that we can tell
    what's important when we read
    them and when they fail”

    View full-size slide

  96. MAKE EVERY
    SYMBOL IN THE
    TEST ESSENTIAL

    View full-size slide

  97. –Clean Code
    “…the technique of building
    a domain-specific language
    for your tests”

    View full-size slide

  98. CARE ABOUT
    THE FAILURE
    MESSAGE

    View full-size slide

  99. –Software Test Automation
    “Test cases must be designed
    with debugging in mind by
    asking 'What would I want to
    know when this test fails.’”

    View full-size slide

  100. –Growing Object-Oriented Software
    “When we get the 'right'
    failure, we check that the
    diagnostics are helpful.”

    View full-size slide

  101. WHY WRITE TESTS?

    View full-size slide

  102. TESTING
    REDUCES
    COSTS

    View full-size slide

  103. HOW TO WRITE
    “GOOD” TESTS?

    View full-size slide

  104. PREFER
    “SMALLER”
    TESTS

    View full-size slide

  105. USE “LARGER”
    TESTS
    APPROPRIATELY

    View full-size slide

  106. PREVENT
    TEST
    ROT

    View full-size slide

  107. RESPOND
    TO THE
    TESTS

    View full-size slide

  108. TEST BEHAVIOR
    NOT
    IMPLEMENTATION

    View full-size slide

  109. WRITE
    CLEAR
    TESTS

    View full-size slide

  110. MAKE EVERY
    SYMBOL IN THE
    TEST ESSENTIAL

    View full-size slide

  111. CARE ABOUT
    THE FAILURE
    MESSAGE

    View full-size slide

  112. –Kevin Berridge, @kberridge
    “Thanks.”

    View full-size slide