Testing Strategy: New Model, Better Outcomes

by David Laribee

Slide 1

Slide 1 text

@#$&!? NEW MODEL, BETTER OUTCOMES! TESTING STRATEGY:

Slide 2

Slide 2 text

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Hi. I’m Dave. Coaching & Workshops Continuous Delivery, Test Driven Development, Technical Debt Management 20 Years, Former C#/Arch MVP, Former Coach @ VersionOne, ALT.NET Organizer, Playing, Full Stack Coach @laribee [email protected]

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

@#$&!?

Slide 9

Slide 9 text

PRETTY SURE GUSTA! I love my debugger. Unconsciously incompetent and life is good. The maintenance team will handle it. @#$&!?

Slide 10

Slide 10 text

PRETTY SURE GUSTA! I love my debugger. Unconsciously incompetent and life is good. The maintenance team will handle it. FML - This is getting complicated. This app was supposed to be small. It was a one off! Did you know that 5 “Marlboro Men” died of smoking-related diseases? #notsustainable EEEEEEEEEEEEE @#$&!?

Slide 11

Slide 11 text

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

Alistair Scott

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

PRETTY SURE GUSTA! Explains past mistakes. End-to-end tests are costly and brittle. Isolation yields more pointed feedback. Developers test too!

Slide 16

Slide 16 text

PRETTY SURE GUSTA! Explains past mistakes. End-to-end tests are costly and brittle. Isolation yields more pointed feedback. Developers test too! What’s 10% mean? 90%? Count? Runtime? How big a pyramid are we talking? Uh… I’m a test automation person… what does this mean for me? Where do all of these unit tests come from? EEEEEEEEEEEEE

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Fabio Pereira

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

PRETTY SURE GUSTA! That explains some of the angst I feel. Our organization/culture has a huge influence on our testing strategy. Helps explain the “testing pyramid” model.

Slide 25

Slide 25 text

PRETTY SURE GUSTA! That explains some of the angst I feel. Our organization/culture has a huge influence on our testing strategy. Helps explain the “testing pyramid” model. This describes us. THIS DESCRIBES US. Changing this threatens my role. “We have a major investment in Test Center.” Why do we keep ignoring those 500 cucumber test failures? Cupcakes are delicious. I prefer mine kitten-free. EEEEEEEEEEEEE

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Alistair Scott

Slide 29

Slide 29 text

Alistair Scott

Slide 30

Slide 30 text

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Slide 33

Slide 33 text

PRETTY SURE GUSTA! Tests have an audience. Different tests serve different concerns. Maybe we shouldn’t invent and/or automate everything ourselves (performance testing, etc.)? Manual testing ain’t dead. Business folks testing?! I don’t get the “support the team” vs. “critique the product” divide. Business-technology is kind of a limiting divide, don’t you think? How do tests tell us if we’re building something people want / will use? EEEEEEEEEEEEE

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

@laribee #testingmodels © 2016 Nerd/Noir, LLC. •Extreme Programming - Chrysler C3 Project •Red-Green-Refactor •Black Box: drive out implementation from public API. •Heavy emphasis on refactoring. •AKA - “Classicist” (Fowler) “DETROIT SCHOOL” TDD

Slide 38

Slide 38 text

Slide 39

Slide 39 text

@laribee #testingmodels © 2016 Nerd/Noir, LLC. •“Growing Object Oriented Software Guided By Tests” •Heavy use of Mock Objects (a stub with an assert). •White Box: discover interfaces and interactions. •Tests are a industrial by-product of design. •AKA - “Mockist” (Fowler) “LONDON SCHOOL” TDD

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Slide 46

Slide 46 text

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Controller External API / Library Write an integrated test after Application Service Mock / Stub Adapter Mock / Stub Business Object / Entity Actual Application Service Controller

Slide 47

Slide 47 text

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

Slide 50

Slide 50 text

PRETTY SURE GUSTA! Test-first and, to a lesser extent, test-closely ensure unit tests get written and not forgotten / put off. Design your code by treating everything as an API. Your tests are simply the first client! Scales with level of detail: product, iteration, code artifact. A nice tool, but you know what they say about silver bullets. Discipline is hard. I forget to refactor sometimes. Rugged and steep learning curve. Mock objects are weird. London school tests seem redundant (and weird), especially so with code first. Practicing both Detroit and London (state- vs. interaction- based verification) blurs the line of unit. EEEEEEEEEEEEE

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

Slide 58

Slide 58 text

PRETTY SURE GUSTA! A whole team testing mindset makes such a huge difference in product and code quality. “The secret of genius is to carry the spirit of the child into old age, which means never losing your enthusiasm.” - Huxley Is execution bias creating a team fixation? What other things could we do to improve our product and/or code quality? “Enthusiasm just creates bubbles; it doesn't keep them from popping.” - Svitak EEEEEEEEEEEEE

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Yeah… I’m gonna go ahead and need you to get us to 50% coverage… That’d be great….

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Slide 64

Slide 64 text

Slide 65

Slide 65 text

Slide 66

Slide 66 text

PRETTY SURE GUSTA! Code coverage can be handy while unit testing (for the people creating the tests). A coverage tool can help fortify refactoring resolve and be a useful, if not temporary, tool on the team- wide test-driven path. Manager: “How else do you expect me to measure quality! If I can’t measure it, I can’t manage it.” Developer to Manager: “Do we get to measure you by useless minutes spent in meetings?” Coverage tells me nothing about test suite quality. Maybe start with an engaged senior engineer? EEEEEEEEEEEEE

Slide 67

Slide 67 text

Slide 68

Slide 68 text

Slide 69

Slide 69 text

Slide 70

Slide 70 text

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

Slide 73

Slide 73 text

PRETTY SURE GUSTA! “People support a world they help to create” - Carnegie It’s an all hands on deck approach. A sound alternative to maintaining extensive DB of manual test scripts (which often go stale). Developers develop. Testers test. Stay off our lawns. What about my test scripts?! As a manager of testers, engineers, or product(s) - how do I know we haven’t regressed? EEEEEEEEEEEEE

Slide 74

Slide 74 text

Slide 75

Slide 75 text

Slide 76

Slide 76 text

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Gestalt |ɡƧˈSHtält,-ˈSHtôlt| noun 1. an organized whole that is perceived as more than the sum of its parts. https://en.wikipedia.org/wiki/Gestalt_psychology

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Slide 79

Slide 79 text

Slide 80

Slide 80 text

Slide 81

Slide 81 text

Slide 82

Slide 82 text

Slide 83

Slide 83 text

Slide 84

Slide 84 text

Slide 85

Slide 85 text

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Safety Specs Design Confidence the software is working. Find regressions before customers. Smoke tests. Automated acceptance tests. Feature tests. Exploratory testing. Curation over collection.

Slide 86

Slide 86 text

Slide 87

Slide 87 text

Slide 88

Slide 88 text

Slide 89

Slide 89 text

Slide 90

Slide 90 text

Slide 91

Slide 91 text

Slide 92

Slide 92 text

Slide 93

Slide 93 text

Slide 94

Slide 94 text

Slide 95

Slide 95 text

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Safety Specs Examples Design Discovery Developer tests. Millisecond feedback… FAST! High value feedback! State-based verification. Black box. Refactoring Dependent.

Slide 96

Slide 96 text

Slide 97

Slide 97 text

Slide 98

Slide 98 text

Slide 99

Slide 99 text

Slide 100

Slide 100 text

Slide 101

Slide 101 text

Slide 102

Slide 102 text

Slide 103

Slide 103 text

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Safety Specs Examples Design Discovery Also unit tests. Also: FAST FEEDBACK. Heavy use of test doubles (mocks, stubs) to design object interactions. Helps us discover boundaries. White-box. That’s OK!

Slide 104

Slide 104 text

Slide 105

Slide 105 text

Slide 106

Slide 106 text

Slide 107

Slide 107 text

Slide 108

Slide 108 text

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Product-level testing! Whole team testing! Testing culture. Exploratory prompts automation. Mindful of its own size. Advanced: Passionate users. Advanced: Session Testing Advanced: The Testing Shard +/- Advanced: Contract Testing Safety