Uh, Does This Thing Actually Work? Modern Microservices Testing

Holly Cummins Red Hat #SkillsMatter uh, does this thing actually
work? modern microservices testing

disclaimer: I am not a tester

disclaimer: I am not a tester … or even a
test specialist.

disclaimer: I am not a tester … or even a
test specialist. but that’s ok! testing is every developer’s job.

@holly_cummins #RedHat fallacies of microservices

#Quarkus @holly_cummins “microservices makes testing easy! the thing I have
to test is so small!”

#Quarkus @holly_cummins Netflix microservice architecture “microservices makes testing easy! the
thing I have to test is so small!”

#Quarkus @holly_cummins Netflix microservice architecture “microservices makes testing easy! the
thing I have to test is so small!” if you only test the points, and not the lines … does the thing work?

#Quarkus @holly_cummins “That’s no moon. It’s our architecture.” Netflix microservice
architecture

@holly_cummins #RedHat “microservices makes testing easy. we’re decoupled!”

@holly_cummins #RedHat distributed != decoupled

@holly_cummins #RedHat “ohhhh, we weren’t expecting your service to do
that … “

@holly_cummins #RedHat “uh, what do you mean you corrected the
typo in your json?”

@holly_cummins #RedHat how do you know if you’re decoupled?

can you deploy independently? v5.1.4 v5.1.4 v5.1.4 v5.1.4

can you deploy independently? v6.2.8 v65.7 v5.1.4 v1.0.5

@holly_cummins #RedHat if you’re going to do microservices, you need
to get good at automation. … and testing.

@holly_cummins #RedHat TDD

@holly_cummins #RedHat TDD: improves quality

@holly_cummins #RedHat TDD: an excellent design tool

@holly_cummins #RedHat

@holly_cummins #RedHat the real reason to do TDD:

@holly_cummins #RedHat the real reason to do TDD: gamified testing

@holly_cummins #RedHat the test pyramid

@holly_cummins #RedHat the test pyramid (you can TDD at every
level)

@holly_cummins #RedHat the test pyramid

@holly_cummins #RedHat the test pyramid end-to-end tests

@holly_cummins #RedHat the test pyramid end-to-end tests integration tests

@holly_cummins #RedHat the test pyramid end-to-end tests unit tests integration
tests

tests low effort

tests low effort high realism

tests low effort high realism high value

@holly_cummins #RedHat all models are wrong some are useful

@holly_cummins #RedHat tools evolve end-to-end tests unit tests integration tests
low effort high realism

low effort high realism tests with application server

low effort high realism tests with application server test REST endpoints

@holly_cummins #RedHat unit tests low effort

@holly_cummins #RedHat “why have I spent half a day writing
mocks just so I can write a unit test?” the limitations of unit tests

@holly_cummins #RedHat the limitations of unit tests “why do my
colleagues tell me my code is broken, even though my unit tests all pass?”

@holly_cummins #RedHat their code our code why unit tests aren’t
enough

@holly_cummins #RedHat their code our code why mocks aren’t enough

@holly_cummins #RedHat their code our code why stubs aren’t enough

@holly_cummins #RedHat our code our mock why mocks aren’t enough

@holly_cummins #RedHat our code our mock tests ✔ why mocks
aren’t enough

@holly_cummins #RedHat our code our mock our code their actual
code tests ✔ why mocks aren’t enough

code tests ✔ reality ✘ why mocks aren’t enough

@holly_cummins #RedHat what needs integration tests? our service

@holly_cummins #RedHat what needs integration tests? our service all code
that serialises or deserialises data

@holly_cummins #RedHat what needs integration tests? our service rest api
usage by consumers all code that serialises or deserialises data

usage by consumers db calls all code that serialises or deserialises data

usage by consumers db calls queues ETL all code that serialises or deserialises data

usage by consumers db calls filesystem config, etc queues ETL all code that serialises or deserialises data

usage by consumers db calls external applications colleague’s services other services filesystem config, etc queues ETL all code that serialises or deserialises data

usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data

@holly_cummins #RedHat end-to-end tests unit tests integration tests low effort
high realism

@holly_cummins #RedHat the test trophy end-to-end tests unit tests integration

@holly_cummins #RedHat the test trophy end-to-end tests high realism

@holly_cummins #RedHat the test trophy end-to-end tests high realism in
staging

@holly_cummins #RedHat the test trophy end-to-end tests high realism in
staging ?

@holly_cummins #RedHat “why do I keep getting failures in production
even though everything looked fine in staging?” Photo by Eggs&Beer, Flickr

@holly_cummins #RedHat staging staging ✔ why staging is not enough
our service other services other services other services other services

@holly_cummins #RedHat staging staging ✔ production ✘ why staging is
not enough our service other services other services other services other services our service production other services other services other services other services

@holly_cummins #RedHat “test in prod or live a lie” -
Charity Majors

tests low effort high realism testing in production

tests low effort high realism static tests testing in production

@holly_cummins #RedHat confidence effort

@holly_cummins #RedHat confidence effort quality

@holly_cummins #RedHat confidence effort quality expense

@holly_cummins #RedHat confidence effort quality expense cheap and cheerful

@holly_cummins #RedHat confidence effort quality expense hard work, but valuable
cheap and cheerful

@holly_cummins #RedHat winning confidence effort quality expense hard work, but
valuable cheap and cheerful

@holly_cummins #RedHat a waste winning confidence effort quality expense hard
work, but valuable cheap and cheerful

work, but valuable cheap and cheerful unit tests

work, but valuable cheap and cheerful unit tests integration tests

work, but valuable cheap and cheerful unit tests integration tests e2e tests

@holly_cummins #RedHat a waste confidence effort quality expense hard work,
but valuable cheap and cheerful unit tests integration tests e2e tests static tests

but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests

but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests YMMV

but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests

but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests constantly question the value of your tests

@holly_cummins #RedHat integration tests

@holly_cummins #RedHat confidence effort unit tests integration tests e2e tests
testing in production static tests

testing in production static tests all integration tests are not created equal

testing in production static tests integration tests all integration tests are not created equal

testing in production static tests integration tests all integration tests are not created equal integration tests

testing in production static tests integration tests all integration tests are not created equal integration tests integration tests

testing in production static tests integration tests all integration tests are not created equal integration tests integration tests integration tests

testing in production static tests all integration tests are not created equal integration tests contract tests integration tests integration tests

testing in production static tests all integration tests are not created equal contract tests sandbox tests integration tests integration tests

testing in production static tests all integration tests are not created equal contract tests sandbox tests testcontainers tests integration tests

@holly_cummins #RedHat approaches to integration tests our service rest api
external applications filesystem colleague’s services colleague’s services colleague’s services external applications

external applications filesystem colleague’s services colleague’s services colleague’s services external applications mock all the things

@holly_cummins #RedHat “why have I spent half a day writing
mocks just so I can write a cleanly-scoped integration test?”

db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications

db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications containerise all the things

@holly_cummins #RedHat “I can’t bring up my dev stack

@holly_cummins #RedHat “I can’t bring up my dev stack …
on my brand new Apple laptop with a M1 chip and 64GB of RAM.” - fintech developer

@holly_cummins #RedHat “I can’t bring up my dev stack …
on my brand new Apple laptop with a M1 chip and 64GB of RAM.” digression: if this is you • maybe testing is only part of your problem • … and Quarkus could help :) - fintech developer

@holly_cummins #RedHat “I have to do all this shell scripting
and manual process and orchestration before I can write a test or do development.”

external applications filesystem colleague’s services colleague’s services colleague’s services external applications remote sandbox remote system + remote deploy

@holly_cummins #RedHat “I miss developing on my actual laptop.”

external applications filesystem colleague’s services colleague’s services colleague’s services external applications remote system with telepresence

external applications filesystem colleague’s services colleague’s services colleague’s services external applications remocal sandbox remote system with telepresence

external applications filesystem colleague’s services colleague’s services colleague’s services external applications shared remocal sandbox remote system with telepresence and per-request isolation

@holly_cummins #RedHat pitfalls: • it all works great – until
a test changes some state, eg • writing to a db • adding a user to a security provider • getting IP whitelists when running locally is tricky

@holly_cummins #RedHat “could I be … a bit less realistic?
and a bit more offline?”

@holly_cummins #RedHat networks: just say no

db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications

db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications testcontainers

@holly_cummins #RedHat drawback:

@holly_cummins #RedHat drawback: • the service still uses CPU

@holly_cummins #RedHat drawback: • the service still uses CPU •
only works for well-known libraries

external applications filesystem colleague’s services colleague’s services colleague’s services external applications

external applications filesystem colleague’s services colleague’s services colleague’s services external applications contract test (for services who will talk to you)

@holly_cummins #RedHat their code our code why mocks aren’t enough

@holly_cummins #RedHat our code our mock why mocks aren’t enough

@holly_cummins #RedHat our code our mock tests ✔ why mocks
aren’t enough

code tests ✔ why mocks aren’t enough

code tests ✔ reality ✘ why mocks aren’t enough

@holly_cummins #RedHat our code their code why mocks aren’t enough

@holly_cummins #RedHat our code their code contract test why mocks
aren’t enough

@holly_cummins #RedHat our code their code contract test mock why
mocks aren’t enough

@holly_cummins #RedHat our code their code contract test mock functional
test why mocks aren’t enough

test our tests ✔ their tests ✔ reality ✔ why mocks aren’t enough

test our tests ✔ their tests ✘ reality ✘ why mocks aren’t enough

test our tests ✘ their tests ✔ reality ✘ why mocks aren’t enough

@holly_cummins #RedHat contract-first is a great way to TDD

@holly_cummins #RedHat contract-first is a great way to TDD (and
a great design tool)

to the code! @holly_cummins #RedHat

#RedHat @holly_cummins demo recap: https://github.com/holly-cummins/house-of-microservices-quarkus-contract-testing-sample

#RedHat @holly_cummins demo recap: • consumer-driven contract testing can save
your bacon • pact is a mock for the consumer • pact is a functional test for the producer • shared json contracts aligns expectations across services

@holly_cummins #RedHat pitfalls of contract testing: • we have to
talk to the other team • our consumers can break our CI?! • versions • pact is hard to grok • testing the mock • but I’ve invested in wiremock • duplicated tests is overhead

@holly_cummins #RedHat contract testing options: • pact.io • spring contract
• OpenAPI schema validation + mock generation • prism • schemathesis

@holly_cummins #RedHat confidence effort integration testing landscape mock all the
services testcontainers remote sandbox remocal sandbox contract tests local orchestration

® thank you @holly_cummins https://github.com/holly-cummins/house-of- microservices-quarkus-contract-testing-sample

Uh, Does This Thing Actually Work? Modern Micro...

Uh, Does This Thing Actually Work? Modern Microservices Testing

More Decks by Holly Cummins

Other Decks in Programming

Featured

Transcript