Uh, Does This Thing Actually Work? Modern Microservices Testing

Holly Cummins Red Hat #JSpring uh, does this thing actually
work? modern microservices testing

Disclaimer: I am not a tester. … but that’s ok.
(Testing is every developer’s job.)

@holly_cummins #RedHat what does testing mean?

@holly_cummins #RedHat writing tests?

@holly_cummins #RedHat writing lots of tests?

@holly_cummins #RedHat 95% test coverage!

@holly_cummins #RedHat actually having assertions in tests

@holly_cummins #RedHat confidence the thing works

@holly_cummins #RedHat confidence to make changes

@holly_cummins #RedHat “we are moving to microservices so we can
change things more easily.”

@holly_cummins #RedHat “If your microservice only does one thing, it
can’t have an error. That would be an additional thing.” – Matt Stratton (not being serious!)

@holly_cummins #RedHat if you’re going to do microservices, you need
to get good at automation. … and testing.

@holly_cummins #RedHat fallacies of microservices

@holly_cummins #RedHat fallacies of microservices “things we wish were true”

#Quarkus @holly_cummins “microservices makes testing easy! the thing I have
to test is so small!”

#Quarkus @holly_cummins Netflix microservice architecture “microservices makes testing easy! the
thing I have to test is so small!”

#Quarkus @holly_cummins Netflix microservice architecture “microservices makes testing easy! the
thing I have to test is so small!” if you only test the points, and not the lines … does the thing work?

@holly_cummins #RedHat “my microservice is perfectly tested, even though the
system doesn’t actually work.”

@holly_cummins #RedHat “we don’t need to test the lines… we’re
decoupled!”

@holly_cummins #RedHat distributed != decoupled

@holly_cummins #RedHat “uh, what do you mean you corrected the
typo in your json?”

@holly_cummins #RedHat “ohhhh, we weren’t expecting your service to do
that … “

@holly_cummins #RedHat

@holly_cummins #RedHat “you’re putting a value in the wrong field
when you call our API.

@holly_cummins #RedHat “you’re putting a value in the wrong field
when you call our API. yes, I know there are 15 fields and the names aren’t particularly descriptive.”

@holly_cummins #RedHat how do you know if you’re decoupled?

can you deploy independently? v5.1.4 v5.1.4 v5.1.4 v5.1.4

can you deploy independently? v6.2.8 v65.7 v5.1.4 v1.0.5

@holly_cummins #RedHat TDD

@holly_cummins #RedHat TDD: improves quality

@holly_cummins #RedHat TDD: an excellent design tool

@holly_cummins #RedHat TDD: an excellent design tool (if testing is
hard, that’s a hint the design is wrong)

@holly_cummins #RedHat the real reason to do TDD:

@holly_cummins #RedHat the real reason to do TDD: gamified testing

@holly_cummins #RedHat the test pyramid

@holly_cummins #RedHat the test pyramid (you can TDD at every
level)

@holly_cummins #RedHat the test pyramid

@holly_cummins #RedHat the test pyramid end-to-end tests

@holly_cummins #RedHat the test pyramid end-to-end tests integration tests

@holly_cummins #RedHat the test pyramid end-to-end tests unit tests integration
tests

tests low effort

tests low effort high realism

tests low effort high realism high value

@holly_cummins #RedHat all models are wrong some are useful

@holly_cummins #RedHat tools evolve end-to-end tests unit tests integration tests
low effort high realism

low effort high realism tests with application server

low effort high realism tests with application server test REST endpoints

@holly_cummins #RedHat unit tests low effort

@holly_cummins #RedHat “why have I spent half a day writing
mocks just so I can write a unit test?” the limitations of unit tests

@holly_cummins #RedHat the limitations of unit tests “why do my
colleagues tell me my code is broken, even though my unit tests all pass?”

@holly_cummins #RedHat their code our code why unit tests aren’t
enough

@holly_cummins #RedHat their code our code why mocks aren’t enough

@holly_cummins #RedHat their code our code why stubs aren’t enough

@holly_cummins #RedHat our code our mock why mocks aren’t enough

@holly_cummins #RedHat our code our mock tests ✔ why mocks
aren’t enough

@holly_cummins #RedHat our code our mock our code their actual
code tests ✔ why mocks aren’t enough

code tests ✔ reality ✘ why mocks aren’t enough

@holly_cummins #RedHat what needs integration tests? our service

@holly_cummins #RedHat what needs integration tests? our service all code
that serialises or deserialises data

@holly_cummins #RedHat what needs integration tests? our service rest api
usage by consumers all code that serialises or deserialises data

usage by consumers db calls all code that serialises or deserialises data

usage by consumers db calls queues ETL all code that serialises or deserialises data

usage by consumers db calls filesystem config, etc queues ETL all code that serialises or deserialises data

usage by consumers db calls external applications colleague’s services other services filesystem config, etc queues ETL all code that serialises or deserialises data

usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data

@holly_cummins #RedHat end-to-end tests unit tests integration tests low effort
high realism

@holly_cummins #RedHat the test trophy end-to-end tests unit tests integration

@holly_cummins #RedHat the test trophy end-to-end tests high realism

@holly_cummins #RedHat the test trophy end-to-end tests high realism in
staging

@holly_cummins #RedHat the test trophy end-to-end tests high realism in
staging ?

@holly_cummins #RedHat “why do I keep getting failures in production
even though everything looked fine in staging?” Photo by Eggs&Beer, Flickr

@holly_cummins #RedHat staging staging ✔ why staging is not enough
our service other services other services other services other services

@holly_cummins #RedHat staging staging ✔ production ✘ why staging is
not enough our service other services other services other services other services our service production other services other services other services other services

@holly_cummins #RedHat “test in prod or live a lie” -
Charity Majors

@holly_cummins #RedHat testing in production, done well: observability ability to
deploy quickly (& safely)

tests low effort high realism testing in production

tests low effort high realism static tests testing in production

@holly_cummins #RedHat confidence effort

@holly_cummins #RedHat confidence effort quality

@holly_cummins #RedHat confidence effort quality expense

@holly_cummins #RedHat confidence effort quality expense cheap and cheerful

@holly_cummins #RedHat confidence effort quality expense hard work, but valuable
cheap and cheerful

@holly_cummins #RedHat winning confidence effort quality expense hard work, but
valuable cheap and cheerful

@holly_cummins #RedHat a waste winning confidence effort quality expense hard
work, but valuable cheap and cheerful

work, but valuable cheap and cheerful unit tests

work, but valuable cheap and cheerful unit tests integration tests

work, but valuable cheap and cheerful unit tests integration tests e2e tests

@holly_cummins #RedHat a waste confidence effort quality expense hard work,
but valuable cheap and cheerful unit tests integration tests e2e tests static tests

but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests

but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests YMMV

but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests

but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests constantly question the value of your tests

@holly_cummins #RedHat integration tests

@holly_cummins #RedHat confidence effort unit tests integration tests e2e tests
testing in production static tests

testing in production static tests all integration tests are not created equal

testing in production static tests integration tests all integration tests are not created equal

testing in production static tests integration tests all integration tests are not created equal integration tests

testing in production static tests integration tests all integration tests are not created equal integration tests integration tests

testing in production static tests integration tests all integration tests are not created equal integration tests integration tests integration tests

testing in production static tests all integration tests are not created equal integration tests contract tests integration tests integration tests

testing in production static tests all integration tests are not created equal contract tests sandbox tests integration tests integration tests

testing in production static tests all integration tests are not created equal contract tests sandbox tests testcontainers tests integration tests

@holly_cummins #RedHat approaches to integration tests our service rest api
external applications filesystem colleague’s services colleague’s services colleague’s services external applications

external applications filesystem colleague’s services colleague’s services colleague’s services external applications mock all the things

@holly_cummins #RedHat “why have I spent half a day writing
mocks just so I can write a cleanly-scoped integration test?”

db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications

db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications containerise all the things

@holly_cummins #RedHat “I can’t bring up my dev stack

@holly_cummins #RedHat “I can’t bring up my dev stack …
on my brand new Apple laptop with a M1 chip and 64GB of RAM.” - fintech developer

@holly_cummins #RedHat “I can’t bring up my dev stack …
on my brand new Apple laptop with a M1 chip and 64GB of RAM.” digression: if this is you • maybe testing is only part of your problem • … and Quarkus could help :) - fintech developer

@holly_cummins #RedHat “I have to do all this shell scripting
and manual process and orchestration before I can write a test or do development.”

@holly_cummins #RedHat testing is a useful design tool

@holly_cummins #RedHat testing is a useful design tool (if testing
your micro service is hard, that’s a hint the domain boundary is wrong)

external applications filesystem colleague’s services colleague’s services colleague’s services external applications remote sandbox remote system + remote deploy

@holly_cummins #RedHat “I miss developing on my actual laptop.”

external applications filesystem colleague’s services colleague’s services colleague’s services external applications remote system with telepresence

external applications filesystem colleague’s services colleague’s services colleague’s services external applications remocal sandbox remote system with telepresence

@holly_cummins #RedHat pitfalls: • all services need to be configured
to forward headers • relies on rest; will not play well with kafka queues or other protocols • can be tricky to get to work

external applications filesystem colleague’s services colleague’s services colleague’s services external applications shared remocal sandbox remote system with telepresence and per-request isolation

@holly_cummins #RedHat pitfalls: • it all works great – until
a test changes some state, eg • writing to a db • adding a user to a security provider • getting IP whitelists when running locally is tricky

@holly_cummins #RedHat “could I be … a bit less realistic?
and a bit more offline?”

@holly_cummins #RedHat networks: just say no

db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications

db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications testcontainers

to the code! @holly_cummins #RedHat

@holly_cummins #RedHat drawback:

@holly_cummins #RedHat drawback: • the service still uses CPU

@holly_cummins #RedHat drawback: • the service still uses CPU •
only works for well-known libraries

@holly_cummins #RedHat bonus: •programmatic control of dependencies •seamless with some
frameworks (esp. Quarkus)

external applications filesystem colleague’s services colleague’s services colleague’s services external applications

external applications filesystem colleague’s services colleague’s services colleague’s services external applications contract test (for services who will talk to you)

code tests ✔ why mocks aren’t enough

code tests ✔ reality ✘ why mocks aren’t enough

@holly_cummins #RedHat our code their code why mocks aren’t enough

@holly_cummins #RedHat our code their code contract test why mocks
aren’t enough

@holly_cummins #RedHat our code their code contract test mock why
mocks aren’t enough

@holly_cummins #RedHat our code their code contract test mock functional
test why mocks aren’t enough

test our tests ✔ their tests ✔ reality ✔ why mocks aren’t enough

test our tests ✔ their tests ✘ reality ✘ why mocks aren’t enough

test our tests ✘ their tests ✔ reality ✘ why mocks aren’t enough

@holly_cummins #RedHat contract-first is a great way to TDD

@holly_cummins #RedHat contract-first is a great way to TDD (and
a great design tool)

to the code! @holly_cummins #RedHat

#RedHat @holly_cummins demo recap: https://github.com/holly-cummins/house-of-microservices-quarkus-contract-testing-sample

#RedHat @holly_cummins demo recap: • consumer-driven contract testing can save
your bacon • pact is a mock for the consumer • pact is a functional test for the producer • shared json contracts aligns expectations across services

@holly_cummins #RedHat pitfalls of contract testing: • we have to
talk to the other team • our consumers can break our CI?! • but I’ve already written mocks and functional tests

@holly_cummins #RedHat unexpected benefits of contract testing: preserving backwards compatibility
YAGNI YAWIBNCSDI*

@holly_cummins #RedHat unexpected benefits of contract testing: preserving backwards compatibility
YAGNI YAWIBNCSDI* * (you already wrote it but now can safely delete it)

@holly_cummins #RedHat contract testing options: • pact.io • Microcks •
spring contract • OpenAPI schema validation + mock generation • prism • schemathesis

@holly_cummins #RedHat ok but what should I do?

@holly_cummins #RedHat confidence effort integration testing landscape mock all the
services testcontainers remote sandbox remocal sandbox contract tests local orchestration

services testcontainers remote sandbox remocal sandbox contract tests local orchestration if you don’t have a big platform team

services testcontainers remote sandbox remocal sandbox contract tests local orchestration if you have a platform team

@holly_cummins #RedHat so, does the thing work? test early, test
often testcontainers rocks contract testing can prevent pain think both locally and system-level

® thank you @holly_cummins slides + code

Uh, Does This Thing Actually Work? Modern Microservices Testing

Uh, Does This Thing Actually Work? Modern Microservices Testing

More Decks by Holly Cummins

Other Decks in Programming

Featured

Transcript