Slide 1

Slide 1 text

This slide saves me time in the long run even though it took time to create Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 2

Slide 2 text

Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 3

Slide 3 text

When I get a feature request Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 4

Slide 4 text

Inventory management software Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 5

Slide 5 text

I need to decide how to test Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 6

Slide 6 text

I could Outside-In TDD Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 7

Slide 7 text

I could just write integration tests Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 8

Slide 8 text

I could not test Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 9

Slide 9 text

I'm making a decision Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 10

Slide 10 text

Testing will be worth it Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 11

Slide 11 text

What does worth it mean? ! Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 12

Slide 12 text

How do I know? Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 13

Slide 13 text

High Cost Tests & High Value Tests Noel Rappin (@noelrap) https://www.tablexi.com http://techdoneright.io http://www.noelrappin.com http://pragprog.com/book/nrtest3 Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 14

Slide 14 text

How can you measure cost and value? Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 15

Slide 15 text

Tests are at the intersection of: Code Process Design Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 16

Slide 16 text

Time is our metric Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 17

Slide 17 text

How do tests cost time? Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 18

Slide 18 text

You have to write the test Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 19

Slide 19 text

The test runs A lot Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 20

Slide 20 text

The test needs to be understood Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 21

Slide 21 text

The test needs to be fixed Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 22

Slide 22 text

How do tests save time? Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 23

Slide 23 text

Writing the test improves code design Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 24

Slide 24 text

Running the test is faster than manual testing Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 25

Slide 25 text

The test validates the code Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 26

Slide 26 text

Catches Bugs Faster Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 27

Slide 27 text

Cost ! vs Value ! Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 28

Slide 28 text

Dev: ! " Forever: ! " # Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 29

Slide 29 text

Spoiler alert: There is no right answer Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 30

Slide 30 text

Strategy not Tactics Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 31

Slide 31 text

Some data Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 32

Slide 32 text

End-to-End Integration Capybara Starts With Ends With User Input HTML Output Write Time Run Time 30 mins 0.5 - 3 seconds Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 33

Slide 33 text

Workflow Intermediate Object Starts Ends Params and workflow.run Database changes Write Time Run Time 15 mins 0.05 - 0.3 seconds Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 34

Slide 34 text

Unit One method Starts Ends Call a method Output of that method Write Time Run Time 1-5 mins 0.001 - 0.04 seconds Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 35

Slide 35 text

Type Specs Total run time Avg run time Write time System 22 12.72 0.570 ~11 hrs Workflow 40 2.36 0.059 ~10 hrs Unit 119 1.86 0.015 ~10 hrs Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 36

Slide 36 text

System tests are 12% of the tests and 75% of the run time Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 37

Slide 37 text

The slowest 4 tests are 40% of the run time Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 38

Slide 38 text

The run times have a wider range Kind Min Max Variance Write 1 min 30 min 30x Run 0.001 sec 3 sec 3000x Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 39

Slide 39 text

Another project Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 40

Slide 40 text

Type Specs Total time Avg time System 409 579 1.42 Workflow 534 206 0.38 Unit 773 93 0.12 Total 1716 878 0.51 Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 41

Slide 41 text

End to end tests are 23% of tests and 66% of run time Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 42

Slide 42 text

What does that suggest? Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 43

Slide 43 text

Balance time spent Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 44

Slide 44 text

As you write similar tests, costs go down Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 45

Slide 45 text

Short term cost ! " is not related to test type Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 46

Slide 46 text

Long term cost ! " # is Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 47

Slide 47 text

Long term cost: runtime failure Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 48

Slide 48 text

In other words: complexity Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 49

Slide 49 text

Long term savings come from focus Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 50

Slide 50 text

A small fraction of your tests can be the bulk of your cost Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 51

Slide 51 text

Big payoff in avoiding the slowest tests Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 52

Slide 52 text

That's a long way to get to "don't write slow tests" Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 53

Slide 53 text

No individual test causes a slow suite Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 54

Slide 54 text

It's an aggregate set of decisions Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 55

Slide 55 text

Is a slow test suite inevitable? Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 56

Slide 56 text

Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 57

Slide 57 text

! Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 58

Slide 58 text

Test suites get longer as the code gets more complicated. Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 59

Slide 59 text

Only CI runs all the tests... we can throw hardware at that Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 60

Slide 60 text

As long as I can run the tests I'm working on quickly, I'm fine Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 61

Slide 61 text

A short history of the Rails community's thoughts about testing... Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 62

Slide 62 text

1. Testing is great Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 63

Slide 63 text

2. Testing is slow Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 64

Slide 64 text

3. Let's try and make testing faster Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 65

Slide 65 text

4. That's hard. Let's throw CI at it. Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 66

Slide 66 text

There's still a cost to a long suite Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 67

Slide 67 text

Giving up causes you to lose the value of tests in improving code Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 68

Slide 68 text

Inventory management software Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 69

Slide 69 text

First test: Capybara integration Fails on: • View failure • Controller failure • Handoff from controller to logic failure • Logic failure • Database access failure Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 70

Slide 70 text

Next test: workflow object Fails on: • Logic failure • Database access failure Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 71

Slide 71 text

Maybe: Unit test Fails on • specific bit of logic • database access failure Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 72

Slide 72 text

Failure paths: Bad Input "A", -3, "" Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 73

Slide 73 text

System Test Workflow Test Unit test Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 74

Slide 74 text

Unit test A partial test of the workflow Or a model test Or a logic object Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 75

Slide 75 text

New feature blank row that can become a new item Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 76

Slide 76 text

Workflow test Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 77

Slide 77 text

New bug a new row that duplicates a name Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 78

Slide 78 text

Not a system test workflow or partial workflow test Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 79

Slide 79 text

What if you don't like unit tests? Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 80

Slide 80 text

"TDD is Dead" Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 81

Slide 81 text

Argument against unit tests: Unit tests cost too much to write Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 82

Slide 82 text

Often true in a legacy context Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 83

Slide 83 text

True if framework doesn't handle tests Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 84

Slide 84 text

You might hear: Unit tests cause hard to understand designs Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 85

Slide 85 text

True if you don't like small units Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 86

Slide 86 text

You might just be writing bad unit tests Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 87

Slide 87 text

A lot of Copy/Paste Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 88

Slide 88 text

A lot of unrelated assertions Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 89

Slide 89 text

Logic change is far away from method under test Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 90

Slide 90 text

Capybara is not a unit test framework Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 91

Slide 91 text

Lack of unit tests: Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 92

Slide 92 text

Good legacy code strategy Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 93

Slide 93 text

At the cost of harder to diagnose tests Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 94

Slide 94 text

Strategies Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 95

Slide 95 text

What will make a test fail? Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 96

Slide 96 text

If it can't fail uniquely Do you need it? Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 97

Slide 97 text

Create the minimum amount of objects needed to trigger the failure Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 98

Slide 98 text

Use multiple test failures as an opportunity Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 99

Slide 99 text

Sometimes you can delete tests Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 100

Slide 100 text

Use integration tests to save development time Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 101

Slide 101 text

Tests have costs ! " Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 102

Slide 102 text

Tests have value ! " Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap

Slide 103

Slide 103 text

Noel Rappin (@noelrap) http://pragprog.com/book/nrtest3 [email protected] http://techdoneright.io/ Noel Rappin | High Cost Tests and High Value Tests | GOTO Chicago 2017 | https://www.tablexi.com | @noelrap