Testing Strategy: New Model, Better Outcomes

@#$&!? NEW MODEL, BETTER OUTCOMES! TESTING STRATEGY:

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Hi. I’m Dave. Coaching
& Workshops Continuous Delivery, Test Driven Development, Technical Debt Management 20 Years, Former C#/Arch MVP, Former Coach @ VersionOne, ALT.NET Organizer, Playing, Full Stack Coach @laribee [email protected]

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Model |ˈmädl| noun 1.
a system or thing used as an example to follow or imitate.

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Every time you end
a debugging session… Monsters devour a kitten.

@#$&!?

PRETTY SURE GUSTA! I love my debugger. Unconsciously incompetent and
life is good. The maintenance team will handle it. @#$&!?

PRETTY SURE GUSTA! I love my debugger. Unconsciously incompetent and
life is good. The maintenance team will handle it. FML - This is getting complicated. This app was supposed to be small. It was a one off! Did you know that 5 “Marlboro Men” died of smoking-related diseases? #notsustainable EEEEEEEEEEEEE @#$&!?

Alistair Scott

PRETTY SURE GUSTA! Explains past mistakes. End-to-end tests are costly
and brittle. Isolation yields more pointed feedback. Developers test too!

PRETTY SURE GUSTA! Explains past mistakes. End-to-end tests are costly
and brittle. Isolation yields more pointed feedback. Developers test too! What’s 10% mean? 90%? Count? Runtime? How big a pyramid are we talking? Uh… I’m a test automation person… what does this mean for me? Where do all of these unit tests come from? EEEEEEEEEEEEE

Fabio Pereira

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Org / Team Test
Inventory

PRETTY SURE GUSTA! That explains some of the angst I
feel. Our organization/culture has a huge influence on our testing strategy. Helps explain the “testing pyramid” model.

PRETTY SURE GUSTA! That explains some of the angst I
feel. Our organization/culture has a huge influence on our testing strategy. Helps explain the “testing pyramid” model. This describes us. THIS DESCRIBES US. Changing this threatens my role. “We have a major investment in Test Center.” Why do we keep ignoring those 500 cucumber test failures? Cupcakes are delicious. I prefer mine kitten-free. EEEEEEEEEEEEE

Alistair Scott

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Gojko Adzic Safety Innovating
w/ Data

PRETTY SURE GUSTA! Tests have an audience. Different tests serve
different concerns. Maybe we shouldn’t invent and/or automate everything ourselves (performance testing, etc.)? Manual testing ain’t dead.

PRETTY SURE GUSTA! Tests have an audience. Different tests serve
different concerns. Maybe we shouldn’t invent and/or automate everything ourselves (performance testing, etc.)? Manual testing ain’t dead. Business folks testing?! I don’t get the “support the team” vs. “critique the product” divide. Business-technology is kind of a limiting divide, don’t you think? How do tests tell us if we’re building something people want / will use? EEEEEEEEEEEEE

@laribee #testingmodels © 2016 Nerd/Noir, LLC. •Extreme Programming - Chrysler
C3 Project •Red-Green-Refactor •Black Box: drive out implementation from public API. •Heavy emphasis on refactoring. •AKA - “Classicist” (Fowler) “DETROIT SCHOOL” TDD

@laribee #testingmodels © 2016 Nerd/Noir, LLC. •“Growing Object Oriented Software
Guided By Tests” •Heavy use of Mock Objects (a stub with an assert). •White Box: discover interfaces and interactions. •Tests are a industrial by-product of design. •AKA - “Mockist” (Fowler) “LONDON SCHOOL” TDD

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Controller Adapter Mock /
Stub

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Controller Application Service Mock
/ Stub Adapter Mock / Stub

/ Stub Adapter Mock / Stub Business Object / Entity Actual

/ Stub Adapter Mock / Stub Business Object / Entity Actual Application Service Controller

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Controller External API /
Library Write an integrated test after Application Service Mock / Stub Adapter Mock / Stub Business Object / Entity Actual Application Service Controller

PRETTY SURE GUSTA! Test-first and, to a lesser extent, test-closely
ensure unit tests get written and not forgotten / put off. Design your code by treating everything as an API. Your tests are simply the first client! Scales with level of detail: product, iteration, code artifact. A nice tool, but you know what they say about silver bullets.

PRETTY SURE GUSTA! Test-first and, to a lesser extent, test-closely
ensure unit tests get written and not forgotten / put off. Design your code by treating everything as an API. Your tests are simply the first client! Scales with level of detail: product, iteration, code artifact. A nice tool, but you know what they say about silver bullets. Discipline is hard. I forget to refactor sometimes. Rugged and steep learning curve. Mock objects are weird. London school tests seem redundant (and weird), especially so with code first. Practicing both Detroit and London (state- vs. interaction- based verification) blurs the line of unit. EEEEEEEEEEEEE

@laribee #testingmodels © 2016 Nerd/Noir, LLC. When do you test?
ALL THE F-ING TIME!

PRETTY SURE GUSTA! A whole team testing mindset makes such
a huge difference in product and code quality. “The secret of genius is to carry the spirit of the child into old age, which means never losing your enthusiasm.” - Huxley

PRETTY SURE GUSTA! A whole team testing mindset makes such
a huge difference in product and code quality. “The secret of genius is to carry the spirit of the child into old age, which means never losing your enthusiasm.” - Huxley Is execution bias creating a team fixation? What other things could we do to improve our product and/or code quality? “Enthusiasm just creates bubbles; it doesn't keep them from popping.” - Svitak EEEEEEEEEEEEE

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Code Coverage (Through
Tests)

Yeah… I’m gonna go ahead and need you to get
us to 50% coverage… That’d be great….

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Tyler Vigen - Spurious
Coorelations

PRETTY SURE GUSTA! Code coverage can be handy while unit
testing (for the people creating the tests). A coverage tool can help fortify refactoring resolve and be a useful, if not temporary, tool on the team- wide test-driven path.

PRETTY SURE GUSTA! Code coverage can be handy while unit
testing (for the people creating the tests). A coverage tool can help fortify refactoring resolve and be a useful, if not temporary, tool on the team- wide test-driven path. Manager: “How else do you expect me to measure quality! If I can’t measure it, I can’t manage it.” Developer to Manager: “Do we get to measure you by useless minutes spent in meetings?” Coverage tells me nothing about test suite quality. Maybe start with an engaged senior engineer? EEEEEEEEEEEEE

@laribee #testingmodels © 2016 Nerd/Noir, LLC. “Exploratory testing is simultaneous
learning, test design, and test execution.” James Bach

PRETTY SURE GUSTA! “People support a world they help to
create” - Carnegie It’s an all hands on deck approach. A sound alternative to maintaining extensive DB of manual test scripts (which often go stale).

PRETTY SURE GUSTA! “People support a world they help to
create” - Carnegie It’s an all hands on deck approach. A sound alternative to maintaining extensive DB of manual test scripts (which often go stale). Developers develop. Testers test. Stay off our lawns. What about my test scripts?! As a manager of testers, engineers, or product(s) - how do I know we haven’t regressed? EEEEEEEEEEEEE

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Strategy |ˈstradƧjē| noun 1.
a high level plan to achieve one or more goals under conditions of uncertainty.

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Gestalt |ɡƧˈSHtält,-ˈSHtôlt| noun 1.
an organized whole that is perceived as more than the sum of its parts. https://en.wikipedia.org/wiki/Gestalt_psychology

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Safety Specs Examples Design
Discovery

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Safety Specs Design Confidence
the software is working.

the software is working. Find regressions before customers.

the software is working. Find regressions before customers. Smoke tests.

the software is working. Find regressions before customers. Smoke tests. Automated acceptance tests.

the software is working. Find regressions before customers. Smoke tests. Automated acceptance tests. Feature tests.

the software is working. Find regressions before customers. Smoke tests. Automated acceptance tests. Feature tests. Exploratory testing.

the software is working. Find regressions before customers. Smoke tests. Automated acceptance tests. Feature tests. Exploratory testing. Curation over collection.

the software is working. Find regressions before customers. Smoke tests. Automated acceptance tests. Feature tests. Exploratory testing. Curation over collection. Written & understood by team!

the software is working. Find regressions before customers. Smoke tests. Automated acceptance tests. Feature tests. Exploratory testing. Curation over collection. Written & understood by team! Time boxed (< 30 minutes? < 1 Hour?) S.A.F.E. - Justin Searls

Discovery

Discovery Developer tests.

Discovery Developer tests. Millisecond feedback… FAST!

Discovery Developer tests. Millisecond feedback… FAST! High value feedback!

Discovery Developer tests. Millisecond feedback… FAST! High value feedback! State-based verification.

Discovery Developer tests. Millisecond feedback… FAST! High value feedback! State-based verification. Black box.

Discovery Developer tests. Millisecond feedback… FAST! High value feedback! State-based verification. Black box. Refactoring Dependent.

Discovery Developer tests. Millisecond feedback… FAST! High value feedback! State-based verification. Black box. Refactoring Dependent. “Detroit School.” Old school.

Discovery Developer tests. Millisecond feedback… FAST! High value feedback! State-based verification. Black box. Refactoring Dependent. “Detroit School.” Old school. Small graphs of independent objects.

Discovery

Discovery Also unit tests. Also: FAST FEEDBACK.

Discovery Also unit tests. Also: FAST FEEDBACK. Heavy use of test doubles (mocks, stubs) to design object interactions.

Discovery Also unit tests. Also: FAST FEEDBACK. Heavy use of test doubles (mocks, stubs) to design object interactions. Helps us discover boundaries.

Discovery Also unit tests. Also: FAST FEEDBACK. Heavy use of test doubles (mocks, stubs) to design object interactions. Helps us discover boundaries. White-box. That’s OK!

Discovery Also unit tests. Also: FAST FEEDBACK. Heavy use of test doubles (mocks, stubs) to design object interactions. Helps us discover boundaries. White-box. That’s OK! “London School.” New-old school.

Discovery Also unit tests. Also: FAST FEEDBACK. Heavy use of test doubles (mocks, stubs) to design object interactions. Helps us discover boundaries. White-box. That’s OK! “London School.” New-old school. Deletion is OK! No sacred cows here.

Discovery

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Product-level testing! Whole team
testing! Testing culture. Exploratory prompts automation. Mindful of its own size. Advanced: Passionate users. Advanced: Session Testing Advanced: The Testing Shard +/- Advanced: Contract Testing Safety

Discovery

@laribee #testingmodels © 2016 Nerd/Noir, LLC. Thank you, Nashville! @laribee
[email protected] Download slides - nerdnoir.com/testing-models

Testing Strategy: New Model, Better Outcomes

Testing Strategy: New Model, Better Outcomes

More Decks by David Laribee

Other Decks in Technology

Featured

Transcript