Slide 1

Slide 1 text

This slide saves me time in the long run even though it took time to create Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 2

Slide 2 text

When I get a feature request Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 3

Slide 3 text

Inventory management software Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 4

Slide 4 text

I need to decide how to test Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 5

Slide 5 text

I could Outside-In TDD Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 6

Slide 6 text

I could just write integration tests Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 7

Slide 7 text

I could not test Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 8

Slide 8 text

I'm making a decision Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 9

Slide 9 text

Testing will be worth it Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 10

Slide 10 text

What does worth it mean? ! Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 11

Slide 11 text

How do I know? Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 12

Slide 12 text

High Cost Tests & High Value Tests Noel Rappin (@noelrap) https://www.tablexi.com http://techdoneright.io http://www.noelrappin.com http://pragprog.com/book/nrtest3 Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 13

Slide 13 text

How can you measure cost and value? Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 14

Slide 14 text

Tests are at the intersection of: Code Process Design Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 15

Slide 15 text

Time is our metric Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 16

Slide 16 text

How do tests cost time? Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 17

Slide 17 text

You have to write the test Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 18

Slide 18 text

The test runs A lot Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 19

Slide 19 text

The test needs to be understood Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 20

Slide 20 text

The test needs to be fixed Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 21

Slide 21 text

How do tests save time? Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 22

Slide 22 text

Writing the test improves code design Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 23

Slide 23 text

Running the test is faster than manual testing Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 24

Slide 24 text

The test validates the code Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 25

Slide 25 text

Catches Bugs Faster Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 26

Slide 26 text

Cost ! vs Value ! Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 27

Slide 27 text

Dev: ! " Forever: ! " # Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 28

Slide 28 text

Spoiler alert: There is no right answer Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 29

Slide 29 text

Strategy not Tactics Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 30

Slide 30 text

Some data Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 31

Slide 31 text

End-to-End Integration Capybara Starts With Ends With User Input HTML Output Write Time Run Time 30 mins 0.5 - 3 seconds Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 32

Slide 32 text

Workflow Intermediate Object Starts Ends Params and workflow.run Database changes Write Time Run Time 15 mins 0.05 - 0.3 seconds Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 33

Slide 33 text

Unit One method Starts Ends Call a method Output of that method Write Time Run Time 1-5 mins 0.001 - 0.04 seconds Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 34

Slide 34 text

Type Specs Total run time Avg run time Write time System 22 12.72 0.570 ~11 hrs Workflow 40 2.36 0.059 ~10 hrs Unit 119 1.86 0.015 ~10 hrs Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 35

Slide 35 text

System tests are 12% of the code and 75% of the output Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 36

Slide 36 text

The slowest 4 tests are 40% of the run time Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 37

Slide 37 text

The run times have a wider range Kind Min Max Variance Write 1 min 30 min 30x Run 0.001 sec 3 sec 3000x Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 38

Slide 38 text

Another project Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 39

Slide 39 text

Type Specs Total time Avg time Cucumber 116 520 4.48 System 175 110 0.62 Workflow 493 209 0.42 Unit 772 92 0.19 Total 1571 987 6.28 Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 40

Slide 40 text

End to end tests are 18% of tests and 67% of run time Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 41

Slide 41 text

What does that suggest? Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 42

Slide 42 text

Balance time spent Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 43

Slide 43 text

As you write similar tests, costs go down Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 44

Slide 44 text

Short term cost ! " is not related to test type Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 45

Slide 45 text

Long term cost ! " # is Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 46

Slide 46 text

Long term cost: runtime failure Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 47

Slide 47 text

In other words: complexity Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 48

Slide 48 text

Long term savings come from focus Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 49

Slide 49 text

A small fraction of your tests can be the bulk of your cost Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 50

Slide 50 text

Big payoff in avoiding the slowest tests Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 51

Slide 51 text

That's a long way to get to "don't write slow tests" Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 52

Slide 52 text

No individual test causes a slow suite Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 53

Slide 53 text

It's an aggregate set of decisions Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 54

Slide 54 text

Inventory management software Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 55

Slide 55 text

First test: Capybara integration Fails on: • View failure • Controller failure • Handoff from controller to logic failure • Logic failure • Database access failure Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 56

Slide 56 text

Next test: workflow object Fails on: • Logic failure • Database access failure Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 57

Slide 57 text

Maybe: Unit test Fails on • specific bit of logic • database access failure Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 58

Slide 58 text

Failure paths: Bad Input "A", -3, "" Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 59

Slide 59 text

System Test Workflow Test Unit test Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 60

Slide 60 text

Unit test A partial test of the workflow Or a model test Or a logic object Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 61

Slide 61 text

New feature blank row that can become a new item Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 62

Slide 62 text

Workflow test Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 63

Slide 63 text

New bug a new row that duplicates a name Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 64

Slide 64 text

Not a system test workflow or partial workflow test Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 65

Slide 65 text

Signs your test is too big? Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 66

Slide 66 text

A lot of Copy/Paste Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 67

Slide 67 text

A lot of unrelated assertions Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 68

Slide 68 text

Logic change is far away from method under test Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 69

Slide 69 text

Capybara is not a unit test framework Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 70

Slide 70 text

What if you don't like unit tests? Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 71

Slide 71 text

"TDD is Dead" Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 72

Slide 72 text

Saw this as JS testing advice: Write tests Not Too Many Mostly integration Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 73

Slide 73 text

Unit tests cost too much to write Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 74

Slide 74 text

Often true in a legacy context Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 75

Slide 75 text

True if framework doesn't handle tests Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 76

Slide 76 text

Unit tests force hard to understand designs Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 77

Slide 77 text

True if you don't like small units Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 78

Slide 78 text

! matters less if you are not running tests while developing Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 79

Slide 79 text

Really good legacy code strategy Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 80

Slide 80 text

Cost of harder to diagnose tests Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 81

Slide 81 text

Strategies Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 82

Slide 82 text

What will make a test fail? Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 83

Slide 83 text

It it can't fail uniquely Do you need it? Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 84

Slide 84 text

Create the minimum amount of objects needed to trigger the failure Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 85

Slide 85 text

Use multiple test failures as an opportunity Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 86

Slide 86 text

Sometimes you can delete tests Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 87

Slide 87 text

Use integration tests to save development time Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 88

Slide 88 text

Tests have costs ! " Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 89

Slide 89 text

Tests have value ! " Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap

Slide 90

Slide 90 text

Noel Rappin (@noelrap) http://pragprog.com/book/nrtest3 (15% off with code "RubyConf NOLA") & http://techdoneright.io/ Noel Rappin | High Cost Tests and High Value Tests | RubyConf 2017 | https://www.tablexi.com | @noelrap