Slide 1
Slide 1 text
Holly Cummins
Sr. Principal Software Engineer
Red Hat
@holly_cummins
Eric Deandrea
Sr. Principal Developer Advocate
Red Hat
@edeandrea
Avoiding common pitfalls with
modern microservices testing
Slide 2
Slide 2 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“Every time we touch one microservice,
all the others break.”
Slide 3
Slide 3 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“My end-to-end test environment is only up the
third weekend every month.”
Slide 4
Slide 4 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Imposter Disclaimer!
Slide 5
Slide 5 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Imposter Disclaimer!
We are not testers.
(or even test experts)
Slide 6
Slide 6 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Imposter Disclaimer!
We are not testers.
(or even test experts)
That’s ok– testing is
every developer’s job!
Slide 7
Slide 7 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Why is testing microservices so hard?
Slide 8
Slide 8 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Netflix microservice
architecture
Slide 9
Slide 9 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Death star
“That’s no moon.
It’s our architecture.”
Slide 10
Slide 10 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Do microservices solve all problems?
Slide 11
Slide 11 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
A microservice system: a portrait
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Netflix microservice
architecture
Slide 15
Slide 15 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Netflix microservice
architecture
Testing with microservices is easy!
Slide 16
Slide 16 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Netflix microservice
architecture
Testing with microservices is easy!
My services are so small!
Slide 17
Slide 17 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Netflix microservice
architecture
Testing with microservices is easy!
My services are so small!
Everything is decoupled!
But…..
Slide 18
Slide 18 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Netflix microservice
architecture
Testing with microservices is easy!
My services are so small!
Everything is decoupled!
But…..
If you only test the points,
And not the lines….
Does the thing actually work?
Slide 19
Slide 19 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
distributed != decoupled
Slide 20
Slide 20 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Understand the coupling
Slide 21
Slide 21 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“It is all that is left unsaid upon
which tragedies are built.”
- Kreia, Star Wars Knights of the Old Republic II: The Sith Lords
Slide 22
Slide 22 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Slide 23
Slide 23 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
This is not
the answer.
Slide 24
Slide 24 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“You lied to me.
How many other lies have I been
told by the Council?
And how do you know that you
even have the truth?”
– Anakin Skywalker
Slide 25
Slide 25 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“In my book, experience
outranks everything.”
— Captain Rex, The Clone Wars
Slide 26
Slide 26 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Ok, so how do we try these out?
Slide 27
Slide 27 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Get good at testing.
Slide 28
Slide 28 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Get good at testing.
And automation.
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
end-to-end tests
high realism
high value
Slide 31
Slide 31 text
end-to-end tests
unit tests
low effort
high realism
high value
Slide 32
Slide 32 text
end-to-end tests
unit tests
integration tests
low effort
high realism
high value
Slide 33
Slide 33 text
end-to-end tests
unit tests
integration tests
low effort
high realism
Slide 34
Slide 34 text
end-to-end tests
unit tests
integration tests
low effort
high realism
tests with
application
server
test REST
endpoints
tests with a
database
Slide 35
Slide 35 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
their code
our code
Why mocks aren’t enough
Slide 36
Slide 36 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
our mock
our code
Why mocks aren’t enough
Slide 37
Slide 37 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
our mock
our code
✔ tests
Why mocks aren’t enough
Slide 38
Slide 38 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
our mock
our code
Why mocks aren’t enough
Slide 39
Slide 39 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
our mock
our code
✔ tests
✘ reality
our code their actual code
Why mocks aren’t enough
Slide 40
Slide 40 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
a mock
stormtrooper
Slide 41
Slide 41 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
a mock
stormtrooper
“Aren’t you a little short
for a stormtrooper?”
– Princess Leia
Slide 42
Slide 42 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
https://www.flickr.com/photos/jdhancock/4811282106
Source: https://medium.com/jung-skywalker/star-wars-symbology-luke-han-and-how-to-face-the-abyss-e9fa966319b5
Slide 43
Slide 43 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Demo
The limitations of unit tests
https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
Slide 44
Slide 44 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“Will someone get
this big walking
carpet out of my
way?”
– Princess Leia
Demo
The limitations of unit tests
https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
Slide 45
Slide 45 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“Will someone get
this big walking
carpet out of my
way?”
– Princess Leia
Demo
The limitations of unit tests
https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
Slide 46
Slide 46 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
A potential solution
Slide 47
Slide 47 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
How to test a fire alarm?
Slide 48
Slide 48 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Slide 49
Slide 49 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
How NOT to
test a fire alarm!
Slide 50
Slide 50 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
Unit testing a
fire alarm
Slide 53
Slide 53 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
uh … is that enough?
Slide 54
Slide 54 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Slide 55
Slide 55 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract testing a
fire alarm
Slide 56
Slide 56 text
Photo: https://www.westcoastfirepits.com/shop/death-star
Slide 57
Slide 57 text
Photo: https://www.westcoastfirepits.com/shop/death-star
How to SAFELY burn a Death Star!
Slide 58
Slide 58 text
end-to-end tests
unit tests
integration tests
low effort
high realism
high value
Slide 59
Slide 59 text
end-to-end tests
unit tests
integration tests
low effort
high realism
high value
contract tests
Slide 60
Slide 60 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
our code their code
A better mock
Slide 61
Slide 61 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
our code their code
contract test
mock functional test
A better mock
Slide 62
Slide 62 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
✔ our tests
✔ their tests
✔ reality
our code their code
contract test
mock functional test
A better mock
Slide 63
Slide 63 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
our code their code
A better mock
Slide 64
Slide 64 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
our code their code
contract test
mock functional test
A better mock
Slide 65
Slide 65 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
✔ our tests
✘ their tests
✘ reality
our code their code
contract test
mock functional test
A better mock
Slide 66
Slide 66 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
A better mock
our code their code
Slide 67
Slide 67 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
A better mock
our code their code
contract test
mock functional test
Slide 68
Slide 68 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
✘ our tests
✔ their tests
✘ reality
A better mock
our code their code
contract test
mock functional test
Slide 69
Slide 69 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Pact Contract Testing
Slide 70
Slide 70 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Demo
Slide 71
Slide 71 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“Help me, Pact!
You’re my only hope!”
– Princess Leia
(almost)
Demo
Slide 72
Slide 72 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“Uh, you’re telling me that consumers
can break a provider’s tests?”
Slide 73
Slide 73 text
No content
Slide 74
Slide 74 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Consumer-driven contract testing
Slide 75
Slide 75 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
consumer
Consumer-driven contract testing
Slide 76
Slide 76 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
declare (Pact)
Pact file
Consumer-driven contract testing
Spring Cloud Contract
Slide 77
Slide 77 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
mock
declare (Pact)
Pact file
Consumer-driven contract testing
Spring Cloud Contract
Pact
Slide 78
Slide 78 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
declare (Pact)
Pact file
Consumer-driven contract testing
Spring Cloud Contract
Pact
Slide 79
Slide 79 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
declare (Pact)
contract
tests
verify
Pact file
Consumer-driven contract testing
Spring Cloud Contract
Pact
Pact
Slide 80
Slide 80 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
declare (Pact)
contract
tests
verify
Pact file
Consumer-driven contract testing
☑ rich semantic testing
Spring Cloud Contract
Pact
Pact
Slide 81
Slide 81 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
declare (Pact)
contract
tests
verify
Pact file
Consumer-driven contract testing
☑ rich semantic testing
☑ provider can ‘develop to the contract’ instead of
developing speculatively (TDD)
Spring Cloud Contract
Pact
Pact
Slide 82
Slide 82 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
declare (Pact)
contract
tests
verify
Pact file
Consumer-driven contract testing
☑ rich semantic testing
☑ provider can ‘develop to the contract’ instead of
developing speculatively (TDD)
⛔ existing spec assets cannot be re-purposed
Spring Cloud Contract
Pact
Pact
Slide 83
Slide 83 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
declare (Pact)
contract
tests
verify
Pact file
Consumer-driven contract testing
☑ rich semantic testing
☑ provider can ‘develop to the contract’ instead of
developing speculatively (TDD)
⛔ existing spec assets cannot be re-purposed
⛔ provider needs to know who consumers are
Spring Cloud Contract
Pact
Pact
Slide 84
Slide 84 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
declare (Pact)
contract
tests
verify
Pact file
Consumer-driven contract testing
☑ rich semantic testing
☑ provider can ‘develop to the contract’ instead of
developing speculatively (TDD)
⛔ existing spec assets cannot be re-purposed
⛔ provider needs to know who consumers are
⛔ consumers can break provider CI
Spring Cloud Contract
Pact
Pact
Slide 85
Slide 85 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
…but I want to just use Swagger/OpenAPI
Slide 86
Slide 86 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Is Pact always the best choice?
Slide 87
Slide 87 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Provider-driven contract testing
Slide 88
Slide 88 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
provider
Provider-driven contract testing
Slide 89
Slide 89 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
provider
OpenAPI
Provider-driven contract testing
generate
(OpenAPI snapshot of
current implementation)
other protocol specs
Slide 90
Slide 90 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
provider
contract
tests
verify
OpenAPI
Provider-driven contract testing
generate
(OpenAPI snapshot of
current implementation)
other protocol specs
Schemathesis
Microcks
Slide 91
Slide 91 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer provider
contract
tests
verify
OpenAPI
Provider-driven contract testing
generate
(OpenAPI snapshot of
current implementation)
other protocol specs
Schemathesis
Microcks
Slide 92
Slide 92 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
contract
tests
verify
OpenAPI
Provider-driven contract testing
generate
(OpenAPI snapshot of
current implementation)
other protocol specs
Prism
Microcks
Schemathesis
Microcks
Slide 93
Slide 93 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
contract
tests
verify
OpenAPI
Provider-driven contract testing
generate
(OpenAPI snapshot of
current implementation)
other protocol specs
☑ familiar contract format
Prism
Microcks
Schemathesis
Microcks
Slide 94
Slide 94 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
contract
tests
verify
OpenAPI
Provider-driven contract testing
generate
(OpenAPI snapshot of
current implementation)
other protocol specs
☑ familiar contract format
☑ provider does not need comms with consumers
Prism
Microcks
Schemathesis
Microcks
Slide 95
Slide 95 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
contract
tests
verify
OpenAPI
Provider-driven contract testing
generate
(OpenAPI snapshot of
current implementation)
other protocol specs
☑ familiar contract format
☑ provider does not need comms with consumers
⛔ provider does not get insight into how API is being
used (deprecation is harder)
Prism
Microcks
Schemathesis
Microcks
Slide 96
Slide 96 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
contract
consumer
tests
provider
mock
contract
tests
verify
OpenAPI
Provider-driven contract testing
generate
(OpenAPI snapshot of
current implementation)
other protocol specs
☑ familiar contract format
☑ provider does not need comms with consumers
⛔ provider does not get insight into how API is being
used (deprecation is harder)
⛔ spec-based tests lack functional depth
Prism
Microcks
Schemathesis
Microcks
Slide 97
Slide 97 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Is OpenAPI enough?
It has the same syntax, but the
semantics are different.
Slide 98
Slide 98 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Is OpenAPI enough?
It has the same syntax, but the
semantics are different.
It looks the same, but the
behaviour is different.
Slide 99
Slide 99 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Is OpenAPI enough?
It has the same syntax, but the
semantics are different.
It looks the same, but the
behaviour is different.
Looks like a storm trooper.
Behaves like Luke.
(because it it is Luke)
Slide 100
Slide 100 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Demo:
Semantic testing
with Pact
Slide 101
Slide 101 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
how annoying, the provider is returning 204. the contract
expects 404, so my test is broken.
quarkus will return 204 by default.
204 vs 404 - it’s more about the theory behind REST...if a
caller asks for a resource the server doesn't have, it should
be a 404 - same way if a user types in an invalid URL into
the browser
ugh, it’s too late at night to care about REST theory.
I’m just going to update the expectations in the test to
match reality.
what?! why is my other contract test broken now?
oh … because what I’m changing is a mock….
Based on a true story….
Slide 102
Slide 102 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Slide 103
Slide 103 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“contract-first”
Slide 104
Slide 104 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“contract-first”
Slide 105
Slide 105 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“contract-first”
It’s like TDD …
Slide 106
Slide 106 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“contract-first”
It’s like TDD …
… between teams for consumer-driven
Slide 107
Slide 107 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
“contract-first”
It’s like TDD …
… between teams for consumer-driven
… with yourself, for provider-driven
Slide 108
Slide 108 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Slide 109
Slide 109 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
soo …
Slide 110
Slide 110 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
soo …
Slide 111
Slide 111 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
soo …
microservices testing is hard
Slide 112
Slide 112 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
soo …
microservices testing is hard
limit end-to-end testing
Slide 113
Slide 113 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
soo …
microservices testing is hard
limit end-to-end testing
mocks will not be enough
Slide 114
Slide 114 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
soo …
microservices testing is hard
limit end-to-end testing
mocks will not be enough
contract testing can rescue you
Slide 115
Slide 115 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat ®
Thank you!
@holly_cummins
@edeandrea
slides
Slide 116
Slide 116 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Additional Resources
Slide 117
Slide 117 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
https://github.com/quarkusio/quarkus-super-heroes
More realistic sample application
(fewer wookies)
Slide 118
Slide 118 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
Kubernetes Native Java with Quarkus
Deployment
Developer Joy
● Quickly & easily build Kubernetes-Native Java applications
● Industry leader in startup time and memory utilization for native and
JVM applications
● One step native executable generation
● Leverage existing Java, Jakarta, MicroProfile, & Spring expertise
● Imperative & reactive APIs side-by-side
● Live coding with zero config
https://www.infoq.com/articles/native-java-quarkus
Slide 119
Slide 119 text
@holly_cummins @edeandrea
#DevoxxUK #RedHat
https://www.oreilly.com/library/view/reactive-systems-in/9781492091714
https://developers.redhat.com/e-books/understanding-quarkus
https://developers.redhat.com/e-books/practising-quarkus