Slide 1
Slide 1 text
Holly Cummins
Sr. Principal Software Engineer
Red Hat
@holly_cummins
Eric Deandrea
Sr. Principal Developer Advocate
Red Hat
@edeandrea
Avoiding common pitfalls with
modern microservices testing
Slide 2
Slide 2 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“Every time we touch one microservice,
all the others break.”
Slide 3
Slide 3 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“My end-to-end test environment is only up the
third weekend every month.”
Slide 4
Slide 4 text
@edeandre
a
for Developers
1:30 pm today!
Slide 5
Slide 5 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Free copy (while they last)!
Come see Eric at the Red Hat booth
(at 2:45 PM today, not now)
https://red.ht/quarkus-spring-devs
Slide 6
Slide 6 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Disclaimer!
Slide 7
Slide 7 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Disclaimer!
We are not testers.
(or even test experts)
Slide 8
Slide 8 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Disclaimer!
We are not testers.
(or even test experts)
That’s ok– testing is
every developer’s job!
Slide 9
Slide 9 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Why is testing microservices so hard?
Slide 10
Slide 10 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Netflix microservice
architecture
Slide 11
Slide 11 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Death star
“That’s no moon.
It’s our architecture.”
Slide 12
Slide 12 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Do microservices solve all problems?
Slide 13
Slide 13 text
@holly_cummins @edeandrea
#DevNexus #RedHat
A microservice system: a portrait
Slide 14
Slide 14 text
No content
Slide 15
Slide 15 text
No content
Slide 16
Slide 16 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Netflix microservice
architecture
Testing with microservices is easy!
The thing I have to test is so small!
Slide 17
Slide 17 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Netflix microservice
architecture
Testing with microservices is easy!
Everything is decoupled!
Slide 18
Slide 18 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Netflix microservice
architecture
Testing with microservices is easy!
Everything is decoupled!
Slide 19
Slide 19 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Netflix microservice
architecture
Testing with microservices is easy!
Everything is decoupled!
But…..
If you only test the points,
And not the lines….
Does the thing actually work?
Slide 20
Slide 20 text
@holly_cummins @edeandrea
#DevNexus #RedHat
distributed != decoupled
Slide 21
Slide 21 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Understand the coupling
Slide 22
Slide 22 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“It is all that is left unsaid upon
which tragedies are built.”
- Kreia, Star Wars Knights of the Old Republic II: The Sith Lords
Slide 23
Slide 23 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“You lied to me.
How many other lies have I been
told by the Council?
And how do you know that you
even have the truth?”
– Anakin Skywalker
Slide 24
Slide 24 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Slide 25
Slide 25 text
@holly_cummins @edeandrea
#DevNexus #RedHat
This is not
the answer.
Slide 26
Slide 26 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“Many of the truths that
we cling to depend on our
viewpoint.”
— Obi-Wan Kenobi, Return of the Jedi
Slide 27
Slide 27 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“In my book, experience
outranks everything.”
— Captain Rex, The Clone Wars
Slide 28
Slide 28 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Ok, so how do we try these out?
Slide 29
Slide 29 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Get good at testing.
Slide 30
Slide 30 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Get good at testing.
And automation.
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
end-to-end tests
high realism
high value
Slide 33
Slide 33 text
end-to-end tests
unit tests
low effort
high realism
high value
Slide 34
Slide 34 text
end-to-end tests
unit tests
integration tests
low effort
high realism
high value
Slide 35
Slide 35 text
end-to-end tests
unit tests
integration tests
low effort
high realism
Slide 36
Slide 36 text
end-to-end tests
unit tests
integration tests
low effort
high realism
tests with
application
server
test REST
endpoints
tests with a
database
Slide 37
Slide 37 text
@holly_cummins @edeandrea
#DevNexus #RedHat
their code
our code
Why mocks aren’t enough
Slide 38
Slide 38 text
@holly_cummins @edeandrea
#DevNexus #RedHat
our mock
our code
Why mocks aren’t enough
Slide 39
Slide 39 text
@holly_cummins @edeandrea
#DevNexus #RedHat
our mock
our code
✔ tests
Why mocks aren’t enough
Slide 40
Slide 40 text
@holly_cummins @edeandrea
#DevNexus #RedHat
our mock
our code
Why mocks aren’t enough
Slide 41
Slide 41 text
@holly_cummins @edeandrea
#DevNexus #RedHat
our mock
our code
✔ tests
✘ reality
our code their actual code
Why mocks aren’t enough
Slide 42
Slide 42 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Slide 43
Slide 43 text
@holly_cummins @edeandrea
#DevNexus #RedHat
https://www.flickr.com/photos/jdhancock/4811282106
Source: https://medium.com/jung-skywalker/star-wars-symbology-luke-han-and-how-to-face-the-abyss-e9fa966319b5
Slide 44
Slide 44 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Demo
The limitations of unit tests
https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
Slide 45
Slide 45 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“Will someone get
this big walking
carpet out of my
way?”
– Princess Leia
Demo
The limitations of unit tests
https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
Slide 46
Slide 46 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“Will someone get
this big walking
carpet out of my
way?”
– Princess Leia
Demo
The limitations of unit tests
https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
Slide 47
Slide 47 text
@holly_cummins @edeandrea
#DevNexus #RedHat
A potential solution
Slide 48
Slide 48 text
@holly_cummins @edeandrea
#DevNexus #RedHat
How to test a fire alarm?
Slide 49
Slide 49 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Slide 50
Slide 50 text
@holly_cummins @edeandrea
#DevNexus #RedHat
How NOT to
test a fire alarm!
Slide 51
Slide 51 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
Unit testing a
fire alarm
Slide 54
Slide 54 text
@holly_cummins @edeandrea
#DevNexus #RedHat
uh … is that enough?
Slide 55
Slide 55 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Slide 56
Slide 56 text
@holly_cummins @edeandrea
#DevNexus #RedHat
contract testing a
fire alarm
Slide 57
Slide 57 text
Photo: https://www.westcoastfirepits.com/shop/death-star
Slide 58
Slide 58 text
Photo: https://www.westcoastfirepits.com/shop/death-star
How to SAFELY burn a Death Star!
Slide 59
Slide 59 text
end-to-end tests
unit tests
integration tests
low effort
high realism
high value
Slide 60
Slide 60 text
end-to-end tests
unit tests
integration tests
low effort
high realism
high value
contract tests
Slide 61
Slide 61 text
@holly_cummins @edeandrea
#DevNexus #RedHat
our code their code
A better mock
Slide 62
Slide 62 text
@holly_cummins @edeandrea
#DevNexus #RedHat
our code their code
contract test
mock functional test
A better mock
Slide 63
Slide 63 text
@holly_cummins @edeandrea
#DevNexus #RedHat
✔ our tests
✔ their tests
✔ reality
our code their code
contract test
mock functional test
A better mock
Slide 64
Slide 64 text
@holly_cummins @edeandrea
#DevNexus #RedHat
our code their code
A better mock
Slide 65
Slide 65 text
@holly_cummins @edeandrea
#DevNexus #RedHat
our code their code
contract test
mock functional test
A better mock
Slide 66
Slide 66 text
@holly_cummins @edeandrea
#DevNexus #RedHat
✔ our tests
✘ their tests
✘ reality
our code their code
contract test
mock functional test
A better mock
Slide 67
Slide 67 text
@holly_cummins @edeandrea
#DevNexus #RedHat
A better mock
our code their code
Slide 68
Slide 68 text
@holly_cummins @edeandrea
#DevNexus #RedHat
A better mock
our code their code
contract test
mock functional test
Slide 69
Slide 69 text
@holly_cummins @edeandrea
#DevNexus #RedHat
✘ our tests
✔ their tests
✘ reality
A better mock
our code their code
contract test
mock functional test
Slide 70
Slide 70 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Some contract
testing options
Slide 71
Slide 71 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Some contract
testing options
● Pact
● Spring Cloud Contract
● OpenAPI schema validation + mock
generation
● Prism
● Schemathesis
Slide 72
Slide 72 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Pact Contract Testing
Slide 73
Slide 73 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Demo
Slide 74
Slide 74 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“Help me, Pact!
You’re my only hope!”
– Princess Leia
(almost)
Demo
Slide 75
Slide 75 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Consumer-driven contract testing
Slide 76
Slide 76 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“Uh, you’re telling me that consumers
can break a provider’s tests?”
Slide 77
Slide 77 text
No content
Slide 78
Slide 78 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“contract-first”
(It’s like TDD, but between teams)
Slide 79
Slide 79 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Is Pact always the best choice?
Slide 80
Slide 80 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Consumer and provider do not have good communication
Slide 81
Slide 81 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Consumer and provider do not have good communication
Slide 82
Slide 82 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Team maintaining the “other side” is not using Pact
Consumer and provider do not have good communication
P ! P
Slide 83
Slide 83 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Consumers can’t be identified (i.e. for public APIs)
Consumer and provider do not have good communication
Team maintaining the “other side” is not using Pact
Slide 84
Slide 84 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Can’t control data being used for provider responses
Consumer and provider do not have good communication
Team maintaining the “other side” is not using Pact
Consumers can’t be identified (i.e. for public APIs)
Slide 85
Slide 85 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“Pass-through” APIs
Consumer and provider do not have good communication
Team maintaining the “other side” is not using Pact
Consumers can’t be identified (i.e. for public APIs)
Can’t control data being used for provider responses
Slide 86
Slide 86 text
@holly_cummins @edeandrea
#DevNexus #RedHat
“Pass-through” APIs
Consumer and provider do not have good communication
Team maintaining the “other side” is not using Pact
Consumers can’t be identified (i.e. for public APIs)
Can’t control data being used for provider responses
Slide 87
Slide 87 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Functional, performance, or load testing
Consumer and provider do not have good communication
Team maintaining the “other side” is not using Pact
Consumers can’t be identified (i.e. for public APIs)
Can’t control data being used for provider responses
“Pass-through” APIs
Slide 88
Slide 88 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Functional, performance, or load testing
Consumer and provider do not have good communication
Team maintaining the “other side” is not using Pact
Consumers can’t be identified (i.e. for public APIs)
Can’t control data being used for provider responses
“Pass-through” APIs
Slide 89
Slide 89 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Pact vs Spring
Cloud Contract?
Slide 90
Slide 90 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Pact vs Spring
Cloud Contract?
● Pact is polyglot!
● Pact generates JSON acceptance
contracts (pacts)
● Pact is consumer-driven whereas SCC is
provider-driven
● Pact allows consumers to specify
provider test states
● Pact broker
Slide 91
Slide 91 text
@holly_cummins @edeandrea
#DevNexus #RedHat
…but I use Swagger/OpenAPI
It has the same syntax, but the
semantics are different.
Slide 92
Slide 92 text
@holly_cummins @edeandrea
#DevNexus #RedHat
…but I use Swagger/OpenAPI
It has the same syntax, but the
semantics are different.
It looks the same, but the
behaviour is different.
Slide 93
Slide 93 text
@holly_cummins @edeandrea
#DevNexus #RedHat
…but I use Swagger/OpenAPI
It has the same syntax, but the
semantics are different.
It looks the same, but the
behaviour is different.
Looks like a storm trooper.
Behaves like Luke.
(because it it is Luke)
Slide 94
Slide 94 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Demo:
Semantic testing
with Pact
Slide 95
Slide 95 text
@holly_cummins @edeandrea
#DevNexus #RedHat
how annoying, the provider is returning 204. the contract
expects 404, so my test is broken.
quarkus will return 204 by default.
204 vs 404 - it’s more about the theory behind REST...if a
caller asks for a resource the server doesn't have, it should
be a 404 - same way if a user types in an invalid URL into
the browser
ugh, it’s too late at night to care about REST theory.
I’m just going to update the expectations in the test to
match reality.
what?! why is my other contract test broken now?
oh … because what I’m changing is a mock….
Based on a true story….
Slide 96
Slide 96 text
@holly_cummins @edeandrea
#DevNexus #RedHat ®
Thank you!
@holly_cummins
@edeandrea
Slide 97
Slide 97 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Additional Resources
Slide 98
Slide 98 text
@holly_cummins @edeandrea
#DevNexus #RedHat
https://github.com/quarkusio/quarkus-super-heroes
More realistic sample application
(fewer wookies)
Slide 99
Slide 99 text
@holly_cummins @edeandrea
#DevNexus #RedHat
Kubernetes Native Java with Quarkus
Deployment
Developer Joy
● Quickly & easily build Kubernetes-Native Java applications
● Industry leader in startup time and memory utilization for native and
JVM applications
● One step native executable generation
● Leverage existing Java, Jakarta, MicroProfile, & Spring expertise
● Imperative & reactive APIs side-by-side
● Live coding with zero config
https://www.infoq.com/articles/native-java-quarkus
Slide 100
Slide 100 text
@holly_cummins @edeandrea
#DevNexus #RedHat
https://www.oreilly.com/library/view/reactive-systems-in/9781492091714
https://developers.redhat.com/e-books/understanding-quarkus
https://developers.redhat.com/e-books/practising-quarkus