Avoiding common pitfalls with modern microservices testing

Have you ever wondered why your microservices break in production after all the tests have passed during CI/CD? Have you ever wondered if you’re testing too much or too little? If you want to learn about common pitfalls and how to catch them at build time rather than at runtime, this session is for you!

In this session we’ll share some common battle scars from our experiences in the field. Additionally, we will introduce the testing pyramid and explain why it is important in helping to continuously deploy microservices independently without fear. We will demonstrate, through a test-driven development approach, how the combination of Pact contract testing combined with Quarkus developer joy can help prevent your microservices from breaking in production.

Transcript

1. Holly Cummins Sr. Principal Software Engineer Red Hat @holly_cummins Eric Deandrea Sr. Principal Developer Advocate Red Hat @edeandrea Avoiding common pitfalls with modern microservices testing
2. @holly_cummins @edeandrea #DevNexus #RedHat “Every time we touch one microservice, all the others break.”
3. @holly_cummins @edeandrea #DevNexus #RedHat “My end-to-end test environment is only up the third weekend every month.”
4. @edeandre a for Developers 1:30 pm today!
5. @holly_cummins @edeandrea #DevNexus #RedHat Free copy (while they last)! Come see Eric at the Red Hat booth 
 (at 2:45 PM today, not now) https://red.ht/quarkus-spring-devs
6. @holly_cummins @edeandrea #DevNexus #RedHat Disclaimer!
7. @holly_cummins @edeandrea #DevNexus #RedHat Disclaimer! We are not testers. (or even test experts)
8. @holly_cummins @edeandrea #DevNexus #RedHat Disclaimer! We are not testers. (or even test experts) That’s ok– testing is every developer’s job!
9. @holly_cummins @edeandrea #DevNexus #RedHat Why is testing microservices so hard?
10. @holly_cummins @edeandrea #DevNexus #RedHat Netflix microservice architecture
11. @holly_cummins @edeandrea #DevNexus #RedHat Death star “That’s no moon. It’s our architecture.”
12. @holly_cummins @edeandrea #DevNexus #RedHat Do microservices solve all problems?
13. @holly_cummins @edeandrea #DevNexus #RedHat A microservice system: a portrait
16. @holly_cummins @edeandrea #DevNexus #RedHat Netflix microservice architecture Testing with microservices is easy! The thing I have to test is so small!
17. @holly_cummins @edeandrea #DevNexus #RedHat Netflix microservice architecture Testing with microservices is easy! Everything is decoupled!
18. @holly_cummins @edeandrea #DevNexus #RedHat Netflix microservice architecture Testing with microservices is easy! Everything is decoupled!
19. @holly_cummins @edeandrea #DevNexus #RedHat Netflix microservice architecture Testing with microservices is easy! Everything is decoupled! But….. If you only test the points, And not the lines…. Does the thing actually work?
20. @holly_cummins @edeandrea #DevNexus #RedHat distributed != decoupled
21. @holly_cummins @edeandrea #DevNexus #RedHat Understand the coupling
22. @holly_cummins @edeandrea #DevNexus #RedHat “It is all that is left unsaid upon which tragedies are built.” - Kreia, Star Wars Knights of the Old Republic II: The Sith Lords
23. @holly_cummins @edeandrea #DevNexus #RedHat “You lied to me. How many other lies have I been told by the Council? And how do you know that you even have the truth?” – Anakin Skywalker
24. @holly_cummins @edeandrea #DevNexus #RedHat
25. @holly_cummins @edeandrea #DevNexus #RedHat This is not the answer.
26. @holly_cummins @edeandrea #DevNexus #RedHat “Many of the truths that we cling to depend on our viewpoint.” — Obi-Wan Kenobi, Return of the Jedi
27. @holly_cummins @edeandrea #DevNexus #RedHat “In my book, experience outranks everything.” — Captain Rex, The Clone Wars
28. @holly_cummins @edeandrea #DevNexus #RedHat Ok, so how do we try these out?
29. @holly_cummins @edeandrea #DevNexus #RedHat Get good at testing.
30. @holly_cummins @edeandrea #DevNexus #RedHat Get good at testing. And automation.
32. end-to-end tests high realism high value
33. end-to-end tests unit tests low effort high realism high value
34. end-to-end tests unit tests integration tests low effort high realism high value
35. end-to-end tests unit tests integration tests low effort high realism
36. end-to-end tests unit tests integration tests low effort high realism tests with application server test REST endpoints tests with a database
37. @holly_cummins @edeandrea #DevNexus #RedHat their code our code Why mocks aren’t enough
38. @holly_cummins @edeandrea #DevNexus #RedHat our mock our code Why mocks aren’t enough
39. @holly_cummins @edeandrea #DevNexus #RedHat our mock our code ✔ tests Why mocks aren’t enough
40. @holly_cummins @edeandrea #DevNexus #RedHat our mock our code Why mocks aren’t enough
41. @holly_cummins @edeandrea #DevNexus #RedHat our mock our code ✔ tests ✘ reality our code their actual code Why mocks aren’t enough
42. @holly_cummins @edeandrea #DevNexus #RedHat
43. @holly_cummins @edeandrea #DevNexus #RedHat https://www.flickr.com/photos/jdhancock/4811282106 Source: https://medium.com/jung-skywalker/star-wars-symbology-luke-han-and-how-to-face-the-abyss-e9fa966319b5
44. @holly_cummins @edeandrea #DevNexus #RedHat Demo The limitations of unit tests https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
45. @holly_cummins @edeandrea #DevNexus #RedHat “Will someone get this big walking carpet out of my way?” – Princess Leia Demo The limitations of unit tests https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
46. @holly_cummins @edeandrea #DevNexus #RedHat “Will someone get this big walking carpet out of my way?” – Princess Leia Demo The limitations of unit tests https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
47. @holly_cummins @edeandrea #DevNexus #RedHat A potential solution
48. @holly_cummins @edeandrea #DevNexus #RedHat How to test a fire alarm?
49. @holly_cummins @edeandrea #DevNexus #RedHat
50. @holly_cummins @edeandrea #DevNexus #RedHat How NOT to test a fire alarm!
51. @holly_cummins @edeandrea #DevNexus #RedHat
53. Unit testing a fire alarm
54. @holly_cummins @edeandrea #DevNexus #RedHat uh … is that enough?
55. @holly_cummins @edeandrea #DevNexus #RedHat
56. @holly_cummins @edeandrea #DevNexus #RedHat contract testing a fire alarm
57. Photo: https://www.westcoastfirepits.com/shop/death-star
58. Photo: https://www.westcoastfirepits.com/shop/death-star How to SAFELY burn a Death Star!
59. end-to-end tests unit tests integration tests low effort high realism high value
60. end-to-end tests unit tests integration tests low effort high realism high value contract tests
61. @holly_cummins @edeandrea #DevNexus #RedHat our code their code A better mock
62. @holly_cummins @edeandrea #DevNexus #RedHat our code their code contract test mock functional test A better mock
63. @holly_cummins @edeandrea #DevNexus #RedHat ✔ our tests ✔ their tests ✔ reality our code their code contract test mock functional test A better mock
64. @holly_cummins @edeandrea #DevNexus #RedHat our code their code A better mock
65. @holly_cummins @edeandrea #DevNexus #RedHat our code their code contract test mock functional test A better mock
66. @holly_cummins @edeandrea #DevNexus #RedHat ✔ our tests ✘ their tests ✘ reality our code their code contract test mock functional test A better mock
67. @holly_cummins @edeandrea #DevNexus #RedHat A better mock our code their code
68. @holly_cummins @edeandrea #DevNexus #RedHat A better mock our code their code contract test mock functional test
69. @holly_cummins @edeandrea #DevNexus #RedHat ✘ our tests ✔ their tests ✘ reality A better mock our code their code contract test mock functional test
70. @holly_cummins @edeandrea #DevNexus #RedHat Some contract testing options
71. @holly_cummins @edeandrea #DevNexus #RedHat Some contract testing options • Pact • Spring Cloud Contract • OpenAPI schema validation + mock generation • Prism • Schemathesis
72. @holly_cummins @edeandrea #DevNexus #RedHat Pact Contract Testing
73. @holly_cummins @edeandrea #DevNexus #RedHat Demo
74. @holly_cummins @edeandrea #DevNexus #RedHat “Help me, Pact! You’re my only hope!” – Princess Leia (almost) Demo
75. @holly_cummins @edeandrea #DevNexus #RedHat Consumer-driven contract testing
76. @holly_cummins @edeandrea #DevNexus #RedHat “Uh, you’re telling me that consumers can break a provider’s tests?”
78. @holly_cummins @edeandrea #DevNexus #RedHat “contract-first” (It’s like TDD, but between teams)
79. @holly_cummins @edeandrea #DevNexus #RedHat Is Pact always the best choice?
80. @holly_cummins @edeandrea #DevNexus #RedHat Consumer and provider do not have good communication
81. @holly_cummins @edeandrea #DevNexus #RedHat Consumer and provider do not have good communication
82. @holly_cummins @edeandrea #DevNexus #RedHat Team maintaining the “other side” is not using Pact Consumer and provider do not have good communication P ! P
83. @holly_cummins @edeandrea #DevNexus #RedHat Consumers can’t be identified (i.e. for public APIs) Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact
84. @holly_cummins @edeandrea #DevNexus #RedHat Can’t control data being used for provider responses Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs)
85. @holly_cummins @edeandrea #DevNexus #RedHat “Pass-through” APIs Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses
86. @holly_cummins @edeandrea #DevNexus #RedHat “Pass-through” APIs Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses
87. @holly_cummins @edeandrea #DevNexus #RedHat Functional, performance, or load testing Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses “Pass-through” APIs
88. @holly_cummins @edeandrea #DevNexus #RedHat Functional, performance, or load testing Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses “Pass-through” APIs
89. @holly_cummins @edeandrea #DevNexus #RedHat Pact vs Spring Cloud Contract?
90. @holly_cummins @edeandrea #DevNexus #RedHat Pact vs Spring Cloud Contract? • Pact is polyglot! • Pact generates JSON acceptance contracts (pacts) • Pact is consumer-driven whereas SCC is provider-driven • Pact allows consumers to specify provider test states • Pact broker
91. @holly_cummins @edeandrea #DevNexus #RedHat …but I use Swagger/OpenAPI It has the same syntax, but the semantics are different.
92. @holly_cummins @edeandrea #DevNexus #RedHat …but I use Swagger/OpenAPI It has the same syntax, but the semantics are different. It looks the same, but the behaviour is different.
93. @holly_cummins @edeandrea #DevNexus #RedHat …but I use Swagger/OpenAPI It has the same syntax, but the semantics are different. It looks the same, but the behaviour is different. Looks like a storm trooper. Behaves like Luke. (because it it is Luke)
94. @holly_cummins @edeandrea #DevNexus #RedHat Demo: Semantic testing with Pact
95. @holly_cummins @edeandrea #DevNexus #RedHat how annoying, the provider is returning 204. the contract expects 404, so my test is broken. quarkus will return 204 by default. 204 vs 404 - it’s more about the theory behind REST...if a caller asks for a resource the server doesn't have, it should be a 404 - same way if a user types in an invalid URL into the browser ugh, it’s too late at night to care about REST theory. I’m just going to update the expectations in the test to match reality. what?! why is my other contract test broken now? 
 oh … because what I’m changing is a mock…. Based on a true story….
96. @holly_cummins @edeandrea #DevNexus #RedHat ® Thank you! @holly_cummins 
 @edeandrea
97. @holly_cummins @edeandrea #DevNexus #RedHat Additional Resources
98. @holly_cummins @edeandrea #DevNexus #RedHat https://github.com/quarkusio/quarkus-super-heroes More realistic sample application (fewer wookies)
99. @holly_cummins @edeandrea #DevNexus #RedHat Kubernetes Native Java with Quarkus Deployment Developer Joy • Quickly & easily build Kubernetes-Native Java applications • Industry leader in startup time and memory utilization for native and JVM applications • One step native executable generation • Leverage existing Java, Jakarta, MicroProfile, & Spring expertise • Imperative & reactive APIs side-by-side • Live coding with zero config https://www.infoq.com/articles/native-java-quarkus
100. @holly_cummins @edeandrea #DevNexus #RedHat https://www.oreilly.com/library/view/reactive-systems-in/9781492091714 https://developers.redhat.com/e-books/understanding-quarkus https://developers.redhat.com/e-books/practising-quarkus