Upgrade to Pro — share decks privately, control downloads, hide ads and more …

reality is expensive: a better way of thinking about mock objects

Justin Searls
November 10, 2012

reality is expensive: a better way of thinking about mock objects

Videos of this talk:
Rocky Mountain Ruby '12 — http://www.youtube.com/watch?v=TU3glG08BJI
Øredev 2012 — http://vimeo.com/53276460
SCNA 2012 — http://vimeo.com/54045166

This talk is an introduction to the different ways people use test doubles (be they mocks, fakes, stubs, or spies). Our goal will be to establish a more sophisticated means of communicating on the topic. This talk discusses our need to understand that reality is a variable under our control, the pros & cons of the different testing strategies, and the smells of test double abuse.

Justin Searls

November 10, 2012
Tweet

More Decks by Justin Searls

Other Decks in Programming

Transcript

  1. None
  2. imagine: testing landing gear

  3. if our tests prove that our code works

  4. then "fake" means "cheat," right?

  5. we're all cheaters

  6. selenium

  7. selenium A user who moves the mouse & keyboard exactly

    how you tell it to.
  8. selenium A user who moves the mouse & keyboard exactly

    how you tell it to. Over and over.
  9. selenium A user who moves the mouse & keyboard exactly

    how you tell it to. Over and over. Every time.
  10. test data

  11. test data Runner cleans the database on every test run.

  12. test data Runner cleans the database on every test run.

    Got years of data migrations?
  13. test data Runner cleans the database on every test run.

    Got years of data migrations? No problem!
  14. stamina

  15. stamina Runtime is always started fresh for tests.

  16. stamina Runtime is always started fresh for tests. Subtle memory

    leak?
  17. stamina Runtime is always started fresh for tests. Subtle memory

    leak? Tests won't mind!
  18. web scale

  19. web scale Tests run against a single-process runner or server.

  20. web scale Tests run against a single-process runner or server.

    Race conditions?
  21. web scale Tests run against a single-process runner or server.

    Race conditions? No deadlocks here!
  22. reality is expensive

  23. why not budget it? reality is expensive

  24. so, instead of asking

  25. is this test too fake?

  26. perhaps, we should ask

  27. how much reality does this test need?

  28. unknown fakeness we talked about

  29. known fakeness? but what about

  30. 1. identify fake things we already consciously do in our

    tests know thy fakeness
  31. 2. understand those fake things well enough to appraise their

    costs & benefits know thy fakeness
  32. 3. write more valuable tests by strategically deciding how much

    reality to give them know thy fakeness
  33. oft-overheard at conferences

  34. "Mocks are Good!" oft-overheard at conferences

  35. "Mocks are Good!" Bad! oft-overheard at conferences

  36. "Only Mock What You Own!" oft-overheard at conferences

  37. "Only Mock What You Own!" external systems! oft-overheard at conferences

  38. "Mock everything!" oft-overheard at conferences

  39. "Mock everything!" avoid over-mocking! oft-overheard at conferences

  40. I'd some

  41. nuance I'd some

  42. rigor I'd some

  43. thoughtfulness I'd some

  44. types of tests

  45. Dog App Tester Dog App end-to-end test

  46. DogFeederTest DogRepo Database DogFeeder integration test BoneRepo

  47. DogFeederTest DogRepo Fake Database DogFeeder "unit" test BoneRepo

  48. DogFeederTest Fake DogRepo DogFeeder isolation test Fake BoneRepo

  49. mock The popular term for a fake object that takes

    the place of a real object when a test is executed. n. \ˈmäk, ˈmȯk\
  50. test double The precise term for a fake object that

    takes the place of a real object when a test is executed. n. \ˈtest də-bəl\
  51. think "stunt double"

  52. our silly catch-all term test double

  53. an alternative implementation to stand-in for something you depend on

    fake test double
  54. replies to certain messages with responses that help you write

    tests fake test double stub
  55. ensures certain messages are received (and explodes upon receiving any

    unexpected messages) fake test double stub mock
  56. stealthily records all interactions, allowing you to make assertions about

    them later fake test double stub mock spy
  57. how should we use them?

  58. hold your fake horses!

  59. before the

  60. why should we use them?

  61. why do we test at all?

  62. ACCEPTANCE prove the app works as promised

  63. ACCEPTANCE prove the app works as promised

  64. SPECIFICATION examples of how the code behaves

  65. SPECIFICATION examples of how the code behaves

  66. REGRESSION prevent bugs from coming back

  67. REGRESSION prevent bugs from coming back

  68. DESIGN shape code by listening to tests

  69. DESIGN shape code by listening to tests

  70. CHARACTERIZE safely improve legacy code

  71. CHARACTERIZE safely improve legacy code

  72. this test provide?" ask, "what value will

  73. value-oriented test strategies

  74. prove it works

  75. prove it works minimize fakeness

  76. prove it works only write end-to-end & integration tests minimize

    fakeness
  77. value cost prove it works

  78. value cost prove it works • confidence that passing equals

    working
  79. value cost prove it works • confidence that passing equals

    working • fewer tests → less effort spent writing tests
  80. value cost prove it works • confidence that passing equals

    working • fewer tests → less effort spent writing tests • tests can't provide many design cues
  81. value cost prove it works • confidence that passing equals

    working • fewer tests → less effort spent writing tests • tests can't provide many design cues • slow feedback
  82. value cost prove it works • confidence that passing equals

    working • fewer tests → less effort spent writing tests • tests can't provide many design cues • slow feedback • high coverage is infeasible
  83. passage of time build duration

  84. passage of time build duration what we expect

  85. passage of time build duration

  86. passage of time build duration what actually happens once you

    account for growth of the system
  87. mocking boundaries

  88. mocking boundaries fake remote systems, never your app's code

  89. mocking boundaries fake remote systems, never your app's code primarily

    write unit & end-to-end tests
  90. value cost mocking boundaries

  91. value cost mocking boundaries • each object is exercised by

    many tests
  92. value cost mocking boundaries • each object is exercised by

    many tests • practical tests without deep knowledge of test doubles
  93. value cost mocking boundaries • each object is exercised by

    many tests • practical tests without deep knowledge of test doubles • tests give less feedback about interactions
  94. value cost mocking boundaries • each object is exercised by

    many tests • practical tests without deep knowledge of test doubles • tests give less feedback about interactions • one change → many test fixes
  95. value cost mocking boundaries • each object is exercised by

    many tests • practical tests without deep knowledge of test doubles • tests give less feedback about interactions • one change → many test fixes • end-to-end is extra redundant
  96. case-by-case

  97. case-by-case cater your testing approach to the needs of each

    situation
  98. value cost case-by-case

  99. value cost case-by-case • developers free to choose best approach

  100. value cost case-by-case • developers free to choose best approach

    • your team is likely already doing it!
  101. value cost case-by-case • developers free to choose best approach

    • your team is likely already doing it! • test doubles easily abused
  102. value cost case-by-case • developers free to choose best approach

    • your team is likely already doing it! • test doubles easily abused • no strategy → murky test value
  103. value cost case-by-case • developers free to choose best approach

    • your team is likely already doing it! • test doubles easily abused • no strategy → murky test value • "wanna mock that?" timesink
  104. the GOOS way

  105. the GOOS way isolation tests to drive design

  106. the GOOS way isolation tests to drive design end-to-end tests

    to prove it works
  107. value cost the GOOS way

  108. value cost the GOOS way • TDD gives rich feedback

    about interactions
  109. value cost the GOOS way • TDD gives rich feedback

    about interactions • consistent, fast, complete
  110. value cost the GOOS way • TDD gives rich feedback

    about interactions • consistent, fast, complete • easy to extract from app to libs
  111. value cost the GOOS way • TDD gives rich feedback

    about interactions • consistent, fast, complete • easy to extract from app to libs • requires discipline & practice
  112. value cost the GOOS way • TDD gives rich feedback

    about interactions • consistent, fast, complete • easy to extract from app to libs • requires discipline & practice • awkward "in- between" tests
  113. value cost the GOOS way • TDD gives rich feedback

    about interactions • consistent, fast, complete • easy to extract from app to libs • requires discipline & practice • awkward "in- between" tests • frameworks dislike isolation
  114. my favorite strategy

  115. 1. agree on literally any clear strategy my favorite strategy

  116. 1. agree on literally any clear strategy 2. follow that

    strategy consistently my favorite strategy
  117. test double smells

  118. smell # 1 test doubles are used in integration tests

  119. perhaps test integrity was compromised to address painful setup or

    assertions
  120. suggestion respond to pain by changing your code, not your

    tests
  121. smell #2 test replaces methods on the subject-under-test with test

    doubles
  122. perhaps subject has too many responsibilities

  123. suggestion extract a new object as a collaborator of the

    subject
  124. smell #3 your objects extend framework classes that you don't

    own
  125. perhaps inherited behavior inhibits good test isolation

  126. suggestion avoid such extension or forgo isolation testing of those

    classes
  127. smell #4 test contains lots of code just to configure

    test doubles
  128. perhaps the unit's number of collaborators or their collective surface

    area is too large
  129. suggestion reduce the count of methods on which the subject

    depends
  130. smell #5 test doubles are used to stand-in for 3rd

    party APIs
  131. perhaps test pain isn't actionable, because design of 3rd party

    code can't be changed
  132. suggestion wrap third-party libraries in small abstractions you own

  133. blame the code before the test

  134. blame the code before the test double ^

  135. www.testdouble.com @andrewvida @jasonkarns @dmosher @joelhelbling @searls @kbaribeau @timwingfield @toddkaufman