reality is expensive: a better way of thinking about mock objects

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

imagine: testing landing gear

Slide 3

Slide 3 text

if our tests prove that our code works

Slide 4

Slide 4 text

then "fake" means "cheat," right?

Slide 5

Slide 5 text

we're all cheaters

Slide 6

Slide 6 text

selenium

Slide 7

Slide 7 text

selenium A user who moves the mouse & keyboard exactly how you tell it to.

Slide 8

Slide 8 text

selenium A user who moves the mouse & keyboard exactly how you tell it to. Over and over.

Slide 9

Slide 9 text

selenium A user who moves the mouse & keyboard exactly how you tell it to. Over and over. Every time.

Slide 10

Slide 10 text

test data

Slide 11

Slide 11 text

test data Runner cleans the database on every test run.

Slide 12

Slide 12 text

test data Runner cleans the database on every test run. Got years of data migrations?

Slide 13

Slide 13 text

test data Runner cleans the database on every test run. Got years of data migrations? No problem!

Slide 14

Slide 14 text

stamina

Slide 15

Slide 15 text

stamina Runtime is always started fresh for tests.

Slide 16

Slide 16 text

stamina Runtime is always started fresh for tests. Subtle memory leak?

Slide 17

Slide 17 text

stamina Runtime is always started fresh for tests. Subtle memory leak? Tests won't mind!

Slide 18

Slide 18 text

web scale

Slide 19

Slide 19 text

web scale Tests run against a single-process runner or server.

Slide 20

Slide 20 text

web scale Tests run against a single-process runner or server. Race conditions?

Slide 21

Slide 21 text

web scale Tests run against a single-process runner or server. Race conditions? No deadlocks here!

Slide 22

Slide 22 text

reality is expensive

Slide 23

Slide 23 text

why not budget it? reality is expensive

Slide 24

Slide 24 text

so, instead of asking

Slide 25

Slide 25 text

is this test too fake?

Slide 26

Slide 26 text

perhaps, we should ask

Slide 27

Slide 27 text

how much reality does this test need?

Slide 28

Slide 28 text

unknown fakeness we talked about

Slide 29

Slide 29 text

known fakeness? but what about

Slide 30

Slide 30 text

1. identify fake things we already consciously do in our tests know thy fakeness

Slide 31

Slide 31 text

2. understand those fake things well enough to appraise their costs & beneﬁts know thy fakeness

Slide 32

Slide 32 text

3. write more valuable tests by strategically deciding how much reality to give them know thy fakeness

Slide 33

Slide 33 text

oft-overheard at conferences

Slide 34

Slide 34 text

"Mocks are Good!" oft-overheard at conferences

Slide 35

Slide 35 text

"Mocks are Good!" Bad! oft-overheard at conferences

Slide 36

Slide 36 text

"Only Mock What You Own!" oft-overheard at conferences

Slide 37

Slide 37 text

"Only Mock What You Own!" external systems! oft-overheard at conferences

Slide 38

Slide 38 text

"Mock everything!" oft-overheard at conferences

Slide 39

Slide 39 text

"Mock everything!" avoid over-mocking! oft-overheard at conferences

Slide 40

Slide 40 text

I'd some

Slide 41

Slide 41 text

nuance I'd some

Slide 42

Slide 42 text

rigor I'd some

Slide 43

Slide 43 text

thoughtfulness I'd some

Slide 44

Slide 44 text

types of tests

Slide 45

Slide 45 text

Dog App Tester Dog App end-to-end test

Slide 46

Slide 46 text

DogFeederTest DogRepo Database DogFeeder integration test BoneRepo

Slide 47

Slide 47 text

DogFeederTest DogRepo Fake Database DogFeeder "unit" test BoneRepo

Slide 48

Slide 48 text

DogFeederTest Fake DogRepo DogFeeder isolation test Fake BoneRepo

Slide 49

Slide 49 text

mock The popular term for a fake object that takes the place of a real object when a test is executed. n. \ˈmäk, ˈmȯk\

Slide 50

Slide 50 text

test double The precise term for a fake object that takes the place of a real object when a test is executed. n. \ˈtest də-bəl\

Slide 51

Slide 51 text

think "stunt double"

Slide 52

Slide 52 text

our silly catch-all term test double

Slide 53

Slide 53 text

an alternative implementation to stand-in for something you depend on fake test double

Slide 54

Slide 54 text

replies to certain messages with responses that help you write tests fake test double stub

Slide 55

Slide 55 text

ensures certain messages are received (and explodes upon receiving any unexpected messages) fake test double stub mock

Slide 56

Slide 56 text

stealthily records all interactions, allowing you to make assertions about them later fake test double stub mock spy

Slide 57

Slide 57 text

how should we use them?

Slide 58

Slide 58 text

hold your fake horses!

Slide 59

Slide 59 text

before the

Slide 60

Slide 60 text

why should we use them?

Slide 61

Slide 61 text

why do we test at all?

Slide 62

Slide 62 text

ACCEPTANCE prove the app works as promised

Slide 63

Slide 63 text

ACCEPTANCE prove the app works as promised

Slide 64

Slide 64 text

SPECIFICATION examples of how the code behaves

Slide 65

Slide 65 text

SPECIFICATION examples of how the code behaves

Slide 66

Slide 66 text

REGRESSION prevent bugs from coming back

Slide 67

Slide 67 text

REGRESSION prevent bugs from coming back

Slide 68

Slide 68 text

DESIGN shape code by listening to tests

Slide 69

Slide 69 text

DESIGN shape code by listening to tests

Slide 70

Slide 70 text

CHARACTERIZE safely improve legacy code

Slide 71

Slide 71 text

CHARACTERIZE safely improve legacy code

Slide 72

Slide 72 text

this test provide?" ask, "what value will

Slide 73

Slide 73 text

value-oriented test strategies

Slide 74

Slide 74 text

prove it works

Slide 75

Slide 75 text

prove it works minimize fakeness

Slide 76

Slide 76 text

prove it works only write end-to-end & integration tests minimize fakeness

Slide 77

Slide 77 text

value cost prove it works

Slide 78

Slide 78 text

value cost prove it works • conﬁdence that passing equals working

Slide 79

Slide 79 text

value cost prove it works • conﬁdence that passing equals working • fewer tests → less eﬀort spent writing tests

Slide 80

Slide 80 text

value cost prove it works • conﬁdence that passing equals working • fewer tests → less eﬀort spent writing tests • tests can't provide many design cues

Slide 81

Slide 81 text

value cost prove it works • conﬁdence that passing equals working • fewer tests → less eﬀort spent writing tests • tests can't provide many design cues • slow feedback

Slide 82

Slide 82 text

value cost prove it works • conﬁdence that passing equals working • fewer tests → less eﬀort spent writing tests • tests can't provide many design cues • slow feedback • high coverage is infeasible

Slide 83

Slide 83 text

passage of time build duration

Slide 84

Slide 84 text

passage of time build duration what we expect

Slide 85

Slide 85 text

passage of time build duration

Slide 86

Slide 86 text

passage of time build duration what actually happens once you account for growth of the system

Slide 87

Slide 87 text

mocking boundaries

Slide 88

Slide 88 text

mocking boundaries fake remote systems, never your app's code

Slide 89

Slide 89 text

mocking boundaries fake remote systems, never your app's code primarily write unit & end-to-end tests

Slide 90

Slide 90 text

value cost mocking boundaries

Slide 91

Slide 91 text

value cost mocking boundaries • each object is exercised by many tests

Slide 92

Slide 92 text

value cost mocking boundaries • each object is exercised by many tests • practical tests without deep knowledge of test doubles

Slide 93

Slide 93 text

value cost mocking boundaries • each object is exercised by many tests • practical tests without deep knowledge of test doubles • tests give less feedback about interactions

Slide 94

Slide 94 text

value cost mocking boundaries • each object is exercised by many tests • practical tests without deep knowledge of test doubles • tests give less feedback about interactions • one change → many test ﬁxes

Slide 95

Slide 95 text

Slide 96

Slide 96 text

case-by-case

Slide 97

Slide 97 text

case-by-case cater your testing approach to the needs of each situation

Slide 98

Slide 98 text

value cost case-by-case

Slide 99

Slide 99 text

value cost case-by-case • developers free to choose best approach

Slide 100

Slide 100 text

value cost case-by-case • developers free to choose best approach • your team is likely already doing it!

Slide 101

Slide 101 text

value cost case-by-case • developers free to choose best approach • your team is likely already doing it! • test doubles easily abused

Slide 102

Slide 102 text

value cost case-by-case • developers free to choose best approach • your team is likely already doing it! • test doubles easily abused • no strategy → murky test value

Slide 103

Slide 103 text

value cost case-by-case • developers free to choose best approach • your team is likely already doing it! • test doubles easily abused • no strategy → murky test value • "wanna mock that?" timesink

Slide 104

Slide 104 text

the GOOS way

Slide 105

Slide 105 text

the GOOS way isolation tests to drive design

Slide 106

Slide 106 text

the GOOS way isolation tests to drive design end-to-end tests to prove it works

Slide 107

Slide 107 text

value cost the GOOS way

Slide 108

Slide 108 text

value cost the GOOS way • TDD gives rich feedback about interactions

Slide 109

Slide 109 text

value cost the GOOS way • TDD gives rich feedback about interactions • consistent, fast, complete

Slide 110

Slide 110 text

value cost the GOOS way • TDD gives rich feedback about interactions • consistent, fast, complete • easy to extract from app to libs

Slide 111

Slide 111 text

value cost the GOOS way • TDD gives rich feedback about interactions • consistent, fast, complete • easy to extract from app to libs • requires discipline & practice

Slide 112

Slide 112 text

value cost the GOOS way • TDD gives rich feedback about interactions • consistent, fast, complete • easy to extract from app to libs • requires discipline & practice • awkward "in- between" tests

Slide 113

Slide 113 text

Slide 114

Slide 114 text

my favorite strategy

Slide 115

Slide 115 text

1. agree on literally any clear strategy my favorite strategy

Slide 116

Slide 116 text

1. agree on literally any clear strategy 2. follow that strategy consistently my favorite strategy

Slide 117

Slide 117 text

test double smells

Slide 118

Slide 118 text

smell # 1 test doubles are used in integration tests

Slide 119

Slide 119 text

perhaps test integrity was compromised to address painful setup or assertions

Slide 120

Slide 120 text

suggestion respond to pain by changing your code, not your tests

Slide 121

Slide 121 text

smell #2 test replaces methods on the subject-under-test with test doubles

Slide 122

Slide 122 text

perhaps subject has too many responsibilities

Slide 123

Slide 123 text

suggestion extract a new object as a collaborator of the subject

Slide 124

Slide 124 text

smell #3 your objects extend framework classes that you don't own

Slide 125

Slide 125 text

perhaps inherited behavior inhibits good test isolation

Slide 126

Slide 126 text

suggestion avoid such extension or forgo isolation testing of those classes

Slide 127

Slide 127 text

smell #4 test contains lots of code just to conﬁgure test doubles

Slide 128

Slide 128 text

perhaps the unit's number of collaborators or their collective surface area is too large

Slide 129

Slide 129 text

suggestion reduce the count of methods on which the subject depends

Slide 130

Slide 130 text

smell #5 test doubles are used to stand-in for 3rd party APIs

Slide 131

Slide 131 text

perhaps test pain isn't actionable, because design of 3rd party code can't be changed

Slide 132

Slide 132 text

suggestion wrap third-party libraries in small abstractions you own

Slide 133

Slide 133 text

blame the code before the test

Slide 134

Slide 134 text

blame the code before the test double ^

Slide 135

Slide 135 text

www.testdouble.com @andrewvida @jasonkarns @dmosher @joelhelbling @searls @kbaribeau @timwingﬁeld @toddkaufman