The Cartography of Testing: CodeMash 2015

Slide 1

Slide 1 text

THE CARTOGRAPHY OF TESTING BY KEVIN BERRIDGE From Matrix Pointe Software

Slide 2

Slide 2 text

THE CARTOGRAPHY OF TESTING BY KEVIN BERRIDGE From Matrix Pointe Software

Slide 3

Slide 3 text

CONTENTIOUS ARGUMENTS

Slide 4

Slide 4 text

EASY TO GET LOST

Slide 5

Slide 5 text

MAP OUT THE TESTING SPACE

Slide 6

Slide 6 text

WHY WRITE TESTS? HOW DO WE WRITE “GOOD” TESTS?

Slide 7

Slide 7 text

WHY WRITE TESTS?

Slide 8

Slide 8 text

BECAUSE.

Slide 9

Slide 9 text

Uncle Bob Martin

Slide 10

Slide 10 text

REASONS

Slide 11

Slide 11 text

–Pragmatic Programmer “It ends up being cheaper in the long run.”

Slide 12

Slide 12 text

–Practical Object-Oriented Design In Ruby “The true purpose of testing, just like the true purpose of design, is to reduce costs.”

Slide 13

Slide 13 text

TESTING REDUCES COSTS?

Slide 14

Slide 14 text

I DON’T HAVE TIME TO TEST

Slide 15

Slide 15 text

COST == TIME

Slide 16

Slide 16 text

TESTING REDUCES COSTS?

Slide 17

Slide 17 text

TESTING REDUCES TIME?

Slide 18

Slide 18 text

No Tests 0 5000 10000 15000 20000 Production Test

Slide 19

Slide 19 text

2x Test LOC 0 5000 10000 15000 20000 Production Test

Slide 20

Slide 20 text

–Practical Object-Oriented Design In Ruby “Fixing bugs early always lowers costs.”

Slide 21

Slide 21 text

–Code Complete “It is cheaper to build high- quality software than it is to build and ﬁx low-quality software.”

Slide 22

Slide 22 text

–Software Test Automation “Run more often. Runs tests that are difﬁcult or impossible manually. Better use of resources. Consistency & repeatability.”

Slide 23

Slide 23 text

–Practical Object-Oriented Design In Ruby “Tests provide the only reliable documentation of design.”

Slide 24

Slide 24 text

–Test-Driven Development “Be sure to factor debugging, integrating, and explaining time into your metrics, though.”

Slide 25

Slide 25 text

–Code Complete “The debugging job is easier.”

Slide 26

Slide 26 text

–Refactoring “These days I hardly ever spend more than a few minutes debugging.”

Slide 27

Slide 27 text

–Software Test Automation “The true value of automated tests is often in providing conﬁdence rather than ﬁnding defects.”

Slide 28

Slide 28 text

wk1 wk2 wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10 wk11 wk12 PRODUCTIVITY

Slide 29

Slide 29 text

wk1 wk2 wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10 wk11 wk12 PRODUCTIVITY

Slide 30

Slide 30 text

SOFTWARE ROT

Slide 31

Slide 31 text

–@ChrisOldwood, Twitter “Not refactoring after making a code change is like putting an empty cereal box back in the cupboard.”

Slide 32

Slide 32 text

REFACTORING

Slide 33

Slide 33 text

BIG CHANGES

Slide 34

Slide 34 text

SMALL CHANGES

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

SOFTWARE ROT

Slide 37

Slide 37 text

WHAT DOES THIS HAVE TO DO WITH TESTING?

Slide 38

Slide 38 text

FEAR OF CHANGING CODE SHOE HORNING IN CHANGES SOFTWARE ROT

Slide 39

Slide 39 text

–Refactoring “If you want to refactor, the essential precondition is having solid tests.”

Slide 40

Slide 40 text

–Refactoring “Whenever I do a refactoring, the ﬁrst step is always the same. I need to build a solid set of tests for that section of code. The tests are essential because even though I follow refactorings structured to avoid most of the opportunities for introducing bugs, I'm still human and still make mistakes. Thus I need solid tests.”

Slide 41

Slide 41 text

–Working Effectively With Legacy Code “To me, legacy code is simply code without tests.”

Slide 42

Slide 42 text

Martin Fowler

Slide 43

Slide 43 text

IF YOU DON’T HAVE TESTS, YOU WON’T CHANGE THE CODE

Slide 44

Slide 44 text

–Clean Code “It is unit tests that keep our code ﬂexible, maintainable, and reusable." "If you have tests, you do not fear making changes to the code!”

Slide 45

Slide 45 text

–Refactoring “The tests give you the security to change the program later.”

Slide 46

Slide 46 text

–Clean Code “Code without tests is bad code. It doesn't matter how well written it is; it doesn't matter how pretty or object-oriented or well- encapsulated it is. With tests, we can change the behavior of our code quickly and veriﬁably. Without them, we really don't know if our code is getting better or worse.”

Slide 47

Slide 47 text

–Growing Object-Oriented Software “We need to keep the code as simple as possible, so it's easier to understand and modify. …Simplicity takes effort, so we constantly refactor our code as we work with it. …The test suites in the feedback loops protect us against our own mistakes as we improve (and therefore change) the code.”

Slide 48

Slide 48 text

–Practical Object-Oriented Design In Ruby “The art of writing changeable code requires the ability to write high-value tests. Tests give you conﬁdence to refactor constantly.”

Slide 49

Slide 49 text

WHY WRITE TESTS?

Slide 50

Slide 50 text

TESTS ARE THE BEST AND CHEAPEST TOOL WE HAVE TO MANAGE FEAR AND REPLACE IT WITH CONFIDENCE

Slide 51

Slide 51 text

WRITE “GOOD” TESTS SO YOU CAN (AND WILL) REFACTOR SO YOUR CODE DOESN’T ROT

Slide 52

Slide 52 text

–Test-Driven Development “My goal is to feel better about a project after a year than I did in the starry-eyed beginning, and TDD helps me achieve this.”

Slide 53

Slide 53 text

HOW TO WRITE “GOOD” TESTS?

Slide 54

Slide 54 text

PREFER “SMALLER” TESTS

Slide 55

Slide 55 text

MORE THAN ONE KIND OF TEST

Slide 56

Slide 56 text

UNIT INTEGRATION SYSTEM ACCEPTANCE VALIDATION/ VERIFICATION PERFORMANCE RESOURCE-EXHAUSTION USABILITY FUNCTIONAL DEVELOPER BLACK BOX GLASS BOX ISOLATED CHARACTERIZATION END-TO-END

Slide 57

Slide 57 text

No content

Slide 58

Slide 58 text

SYSTEM ACCEPTANCE FUNCTIONAL BLACK BOX END-TO-END

Slide 59

Slide 59 text

END-TO-END

Slide 60

Slide 60 text

INTEGRATION DEVELOPER GLASS BOX ISOLATED CHARACTERIZATION UNIT

Slide 61

Slide 61 text

UNIT

Slide 62

Slide 62 text

Unit End-To-End

Slide 63

Slide 63 text

T O O O D S UI

Slide 64

Slide 64 text

T O O O D S UI

Slide 65

Slide 65 text

T O I V D S UI

Slide 66

Slide 66 text

PREFER “SMALLER” TESTS

Slide 67

Slide 67 text

FAST VS. SLOW

Slide 68

Slide 68 text

FEEDBACK

Slide 69

Slide 69 text

–Working Effectively With Legacy Code “Do you want your feedback in a minute or overnight?”

Slide 70

Slide 70 text

–Working Effectively With Legacy Code “Unit testing is one of the most important components in legacy code work. System-level regression tests are great, but small, localized tests are invaluable. They can give you feedback as you develop and allow you to refactor with much more safety.”

Slide 71

Slide 71 text

–Working Effectively With Legacy Code “The feedback we get from [testing] is very useful. It pays to [test] at a ﬁner- grained level.”

Slide 72

Slide 72 text

COMPLEXITY

Slide 73

Slide 73 text

T O O O

Slide 74

Slide 74 text

–Growing Object-Oriented Software “If we test at too large a grain, the combinatorial explosion of trying all the possible paths through the code will bring development to a halt.”

Slide 75

Slide 75 text

–Working Effectively With Legacy Code “There are a few problems with large tests: Error localization, Execution time, Coverage”

Slide 76

Slide 76 text

http://blog.thecodewhisperer.com/2010/10/16/integrated-tests-are-a-scam/

Slide 77

Slide 77 text

310 59,049

Slide 78

Slide 78 text

T O O O

Slide 79

Slide 79 text

T O I V

Slide 80

Slide 80 text

–Working Effectively With Legacy Code “There are a few problems with large tests: Error localization, Execution time, Coverage”

Slide 81

Slide 81 text

USE “LARGER” TESTS APPROPRIATELY

Slide 82

Slide 82 text

Acceptance Integration Unit 10% 20% 70% http://jamescrisp.org/2011/05/30/automated-testing-and-the-test-pyramid/

Slide 83

Slide 83 text

No content

Slide 84

Slide 84 text

THE GOOS APPROACH

Slide 85

Slide 85 text

–Growing Object-Oriented Software “We start work on a new feature by writing failing acceptance tests that demonstrate that the system does not yet have the feature we're about to write and track our progress towards completion of the feature.”

Slide 86

Slide 86 text

END-TO-END: PROVE THE EXISTENCE OF A FEATURE NOT THE CORRECTNESS

Slide 87

Slide 87 text

“INTEGRATION” TESTS

Slide 88

Slide 88 text

–Mike Bland, http://martinfowler.com/articles/testing-culture.html “You should not be catching bugs at the integration or system levels that could have been caught at the unit level.”

Slide 89

Slide 89 text

THE EXTRACT CLASS REFACTORING

Slide 90

Slide 90 text

T O

Slide 91

Slide 91 text

T O O

Slide 92

Slide 92 text

T T O O

Slide 93

Slide 93 text

T T O O

Slide 94

Slide 94 text

T T O O

Slide 95

Slide 95 text

PREVENT TEST ROT

Slide 96

Slide 96 text

–Clean Code “Test code is just as important as production code.”

Slide 97

Slide 97 text

–Sandi Metz “I examine the original test and morph it from an integration test into an actual unit test.”

Slide 98

Slide 98 text

T T O O

Slide 99

Slide 99 text

TEST DOUBLES MOCKS STUBS FAKES SUBSTITUTES

Slide 100

Slide 100 text

ALTERNATE REALITY

Slide 101

Slide 101 text

bit.ly/kwb-oop

Slide 102

Slide 102 text

–Growing Object-Oriented Software “…investigate why the test is hard to write and refactor the code to improve its structure. We call this 'listening to the tests.'”

Slide 103

Slide 103 text

RESPOND TO THE TESTS

Slide 104

Slide 104 text

–Practical Object-Oriented Design In Ruby “Tests are the canary in the coal mine; when the design is bad, testing is hard.”

Slide 105

Slide 105 text

–Growing Object-Oriented Software “We've found that the qualities that make an object easy to test also make our code responsive to change.”

Slide 106

Slide 106 text

DEPENDENCY

Slide 107

Slide 107 text

–Working Effectively With Legacy Code “Dependency is one of the most critical problems in software development. Much legacy code work involves breaking dependencies so that change can be easier.”

Slide 108

Slide 108 text

–Test-Driven Development “Dependency is the key problem in software development at all scales.”

Slide 109

Slide 109 text

–Working Effectively With Legacy Code “When classes depend directly on things that are hard to use in a test, they are hard to modify and hard to work with.”

Slide 110

Slide 110 text

–Practical Object-Oriented Design In Ruby “Dealing with objects as if they are only and exactly the messages to which they respond lets you design a changeable application, and it is your understanding of the importance of this perspective that allows you to create tests that provide maximum beneﬁt at minimum cost.”

Slide 111

Slide 111 text

TEST BEHAVIOR NOT IMPLEMENTATION

Slide 112

Slide 112 text

–Kent Beck “Separate interface from implementation thinking. I have a tendency to pollute API design decisions with implementation speculation.”

Slide 113

Slide 113 text

T O C Q

Slide 114

Slide 114 text

T I O C Q

Slide 115

Slide 115 text

T I O C Q

Slide 116

Slide 116 text

T O I C Q

Slide 117

Slide 117 text

–Practical Object-Oriented Design in Ruby “Some outgoing messages have no side effects… Outgoing messages like this are known as queries and they need not be tested by the sending object.”

Slide 118

Slide 118 text

–Practical Object-Oriented Design in Ruby “Many outgoing messages do have side effects… These messages are commands and it is the responsibility of the sending object to prove that they are properly sent.”

Slide 119

Slide 119 text

WRITE CLEAR TESTS

Slide 120

Slide 120 text

POORLY WRITTEN TESTS ARE A HINDRANCE

Slide 121

Slide 121 text

–Growing Object-Oriented Software “For TDD to be sustainable, the tests must do more than verify the behavior of the code; they must also express that behavior clearly -- they must be readable.”

Slide 122

Slide 122 text

–Clean Code “The dirtier the tests, the harder they are to change… So the tests become viewed as an ever- increasing liability.”

Slide 123

Slide 123 text

–Test-Driven Development “"Test methods should be easy to read.”

Slide 124

Slide 124 text

–Test-Driven Development “The ﬁrst criterion for your tests is conﬁdence. …The second criterion is communication.”

Slide 125

Slide 125 text

–Growing Object-Oriented Software “We want our test code to read like a declarative description of what is being testing.”

Slide 126

Slide 126 text

–Growing Object-Oriented Software “We want to make sure the tests pull their weight by making them expressive, so that we can tell what's important when we read them and when they fail”

Slide 127

Slide 127 text

MAKE EVERY SYMBOL IN THE TEST ESSENTIAL

Slide 128

Slide 128 text

–Clean Code “…the technique of building a domain-speciﬁc language for your tests”

Slide 129

Slide 129 text

CARE ABOUT THE FAILURE MESSAGE

Slide 130

Slide 130 text

–Software Test Automation “Test cases must be designed with debugging in mind by asking 'What would I want to know when this test fails.’”

Slide 131

Slide 131 text

–Growing Object-Oriented Software “When we get the 'right' failure, we check that the diagnostics are helpful.”