6-29-23 - [DevNation] - Avoiding common pitfalls with modern microservices testing

Have you ever wondered why your microservices break in production after all the tests have passed during CI/CD? Have you ever wondered if you’re testing too much or too little? If you want to learn about common pitfalls and how to catch them at build time rather than at runtime, this session is for you!

In this session, I’ll share some common battle scars from my experiences in the field. Additionally, I will introduce the testing pyramid and explain why it is important in helping to continuously deploy microservices independently without fear. I will demonstrate, through a test-driven development approach, how the combination of Pact contract testing combined with Quarkus developer joy can help prevent your microservices from breaking in production.

Transcript

1. Eric Deandrea Sr. Principal Developer Advocate Red Hat @edeandrea Avoiding common pitfalls with modern microservices testing Drawings courtesy of Holly Cummins
2. @edeandrea #RedHat #DevNation
3. @edeandrea #RedHat #DevNation “Every time we touch one microservice, all the others break.”
4. @edeandrea #RedHat #DevNation
5. @edeandrea #RedHat #DevNation “My end-to-end test environment is only up the third weekend every month.”
6. @edeandrea #RedHat #DevNation Disclaimer!
7. @edeandrea #RedHat #DevNation Disclaimer! I am not a tester. (or even test expert)
8. @edeandrea #RedHat #DevNation Disclaimer! I am not a tester. (or even test expert) That’s ok– testing is every developer’s job!
9. @edeandrea #RedHat #DevNation Why is testing microservices so hard?
10. @edeandrea #RedHat #DevNation Netflix microservice architecture
11. @edeandrea #RedHat #DevNation Death star “That’s no moon. It’s our architecture.”
12. @edeandrea #RedHat #DevNation Do microservices solve all problems?
13. @edeandrea #RedHat #DevNation A microservice system: a portrait
16. @edeandrea #RedHat #DevNation Netflix microservice architecture
17. @edeandrea #RedHat #DevNation Netflix microservice architecture Testing with microservices is easy!
18. @edeandrea #RedHat #DevNation Netflix microservice architecture Testing with microservices is easy! My services are so small! Everything is decoupled!
19. @edeandrea #RedHat #DevNation Netflix microservice architecture Testing with microservices is easy! My services are so small! Everything is decoupled! But….. If you only test the points, And not the lines…. Does the thing actually work?
20. @edeandrea #RedHat #DevNation distributed != decoupled
21. @edeandrea #RedHat #DevNation Understand the coupling
22. @edeandrea #RedHat #DevNation “It is all that is left unsaid upon which tragedies are built.” - Kreia, Star Wars Knights of the Old Republic II: The Sith Lords
23. @edeandrea #RedHat #DevNation
24. @edeandrea #RedHat #DevNation This is not the answer.
25. @edeandrea #RedHat #DevNation “You lied to me. How many other lies have I been told by the Council? And how do you know that you even have the truth?” – Anakin Skywalker
26. @edeandrea #RedHat #DevNation “Many of the truths that we cling to depend on our viewpoint.” — Obi-Wan Kenobi, Return of the Jedi
27. @edeandrea #RedHat #DevNation “In my book, experience outranks everything.” — Captain Rex, The Clone Wars
28. @edeandrea #RedHat #DevNation Ok, so how do we try these out?
29. @edeandrea #RedHat #DevNation Get good at testing.
30. @edeandrea #RedHat #DevNation Get good at testing. And automation.
32. end-to-end tests high realism high value
33. end-to-end tests unit tests low effort high realism high value
34. end-to-end tests unit tests integration tests low effort high realism high value
35. end-to-end tests unit tests integration tests low effort high realism
36. end-to-end tests unit tests integration tests low effort high realism tests with application server test REST endpoints tests with a database
37. @edeandrea #RedHat #DevNation their code our code Why mocks aren’t enough
38. @edeandrea #RedHat #DevNation our mock our code Why mocks aren’t enough
39. @edeandrea #RedHat #DevNation our mock our code ✔ tests Why mocks aren’t enough
40. @edeandrea #RedHat #DevNation our mock our code Why mocks aren’t enough
41. @edeandrea #RedHat #DevNation our mock our code ✔ tests ✘ reality our code their actual code Why mocks aren’t enough
42. @edeandrea #RedHat #DevNation a mock stormtrooper
43. @edeandrea #RedHat #DevNation a mock stormtrooper “Aren’t you a little short for a stormtrooper?” – Princess Leia
44. @edeandrea #RedHat #DevNation https://www.flickr.com/photos/jdhancock/4811282106 Source: https://medium.com/jung-skywalker/star-wars-symbology-luke-han-and-how-to-face-the-abyss-e9fa966319b5
45. @edeandrea #RedHat #DevNation Demo The limitations of unit tests https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
46. @edeandrea #RedHat #DevNation “Will someone get this big walking carpet out of my way?” – Princess Leia Demo The limitations of unit tests https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
47. @edeandrea #RedHat #DevNation “Will someone get this big walking carpet out of my way?” – Princess Leia Demo The limitations of unit tests https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
48. @edeandrea #RedHat #DevNation A potential solution
49. @edeandrea #RedHat #DevNation How to test a fire alarm?
50. @edeandrea #RedHat #DevNation
51. @edeandrea #RedHat #DevNation How NOT to test a fire alarm!
52. @edeandrea #RedHat #DevNation
54. Unit testing a fire alarm
55. @edeandrea #RedHat #DevNation uh … is that enough?
56. @edeandrea #RedHat #DevNation
57. @edeandrea #RedHat #DevNation contract testing a fire alarm
58. end-to-end tests unit tests integration tests low effort high realism high value
59. end-to-end tests unit tests integration tests low effort high realism high value contract tests
60. @edeandrea #RedHat #DevNation our code their code A better mock
61. @edeandrea #RedHat #DevNation our code their code contract test mock functional test A better mock
62. @edeandrea #RedHat #DevNation ✔ our tests ✔ their tests ✔ reality our code their code contract test mock functional test A better mock
63. @edeandrea #RedHat #DevNation our code their code A better mock
64. @edeandrea #RedHat #DevNation our code their code contract test mock functional test A better mock
65. @edeandrea #RedHat #DevNation ✔ our tests ✘ their tests ✘ reality our code their code contract test mock functional test A better mock
66. @edeandrea #RedHat #DevNation A better mock our code their code
67. @edeandrea #RedHat #DevNation A better mock our code their code contract test mock functional test
68. @edeandrea #RedHat #DevNation ✘ our tests ✔ their tests ✘ reality A better mock our code their code contract test mock functional test
69. @edeandrea #RedHat #DevNation Pact Contract Testing
70. @edeandrea #RedHat #DevNation Demo
71. @edeandrea #RedHat #DevNation “Help me, Pact! You’re my only hope!” – Princess Leia (almost) Demo
72. @edeandrea #RedHat #DevNation “Uh, you’re telling me that consumers can break a provider’s tests?”
74. @edeandrea #RedHat #DevNation Consumer-driven contract testing
75. @edeandrea #RedHat #DevNation consumer Consumer-driven contract testing
76. @edeandrea #RedHat #DevNation contract consumer tests mock declare (Pact) Pact file Consumer-driven contract testing Spring Cloud Contract Pact
77. @edeandrea #RedHat #DevNation contract consumer tests provider mock declare (Pact) contract tests verify Pact file Consumer-driven contract testing Spring Cloud Contract Pact Pact
78. @edeandrea #RedHat #DevNation contract consumer tests provider mock declare (Pact) contract tests verify Pact file Consumer-driven contract testing ☑ rich semantic testing Spring Cloud Contract Pact Pact
79. @edeandrea #RedHat #DevNation contract consumer tests provider mock declare (Pact) contract tests verify Pact file Consumer-driven contract testing ☑ rich semantic testing ☑ provider can ‘develop to the contract’ instead of developing speculatively (TDD) Spring Cloud Contract Pact Pact
80. @edeandrea #RedHat #DevNation contract consumer tests provider mock declare (Pact) contract tests verify Pact file Consumer-driven contract testing ☑ rich semantic testing ☑ provider can ‘develop to the contract’ instead of developing speculatively (TDD) ⛔ existing spec assets cannot be re-purposed Spring Cloud Contract Pact Pact
81. @edeandrea #RedHat #DevNation contract consumer tests provider mock declare (Pact) contract tests verify Pact file Consumer-driven contract testing ☑ rich semantic testing ☑ provider can ‘develop to the contract’ instead of developing speculatively (TDD) ⛔ existing spec assets cannot be re-purposed ⛔ provider needs to know who consumers are Spring Cloud Contract Pact Pact
82. @edeandrea #RedHat #DevNation contract consumer tests provider mock declare (Pact) contract tests verify Pact file Consumer-driven contract testing ☑ rich semantic testing ☑ provider can ‘develop to the contract’ instead of developing speculatively (TDD) ⛔ existing spec assets cannot be re-purposed ⛔ provider needs to know who consumers are ⛔ consumers can break provider CI Spring Cloud Contract Pact Pact
83. @edeandrea #RedHat #DevNation …but I want to just use Swagger/OpenAPI
84. @edeandrea #RedHat #DevNation Is Pact always the best choice?
85. @edeandrea #RedHat #DevNation Provider-driven contract testing
86. @edeandrea #RedHat #DevNation consumer provider Provider-driven contract testing
87. @edeandrea #RedHat #DevNation contract consumer provider OpenAPI Provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs
88. @edeandrea #RedHat #DevNation contract consumer provider contract tests verify OpenAPI Provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs Schemathesis Microcks
89. @edeandrea #RedHat #DevNation contract consumer tests provider mock contract tests verify OpenAPI Provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs Prism Microcks Schemathesis Microcks
90. @edeandrea #RedHat #DevNation contract consumer tests provider mock contract tests verify OpenAPI Provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs ☑ familiar contract format Prism Microcks Schemathesis Microcks
91. @edeandrea #RedHat #DevNation contract consumer tests provider mock contract tests verify OpenAPI Provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs ☑ familiar contract format ☑ provider does not need comms with consumers Prism Microcks Schemathesis Microcks
92. @edeandrea #RedHat #DevNation contract consumer tests provider mock contract tests verify OpenAPI Provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs ☑ familiar contract format ☑ provider does not need comms with consumers ⛔ provider does not get insight into how API is being used (deprecation is harder) Prism Microcks Schemathesis Microcks
93. @edeandrea #RedHat #DevNation contract consumer tests provider mock contract tests verify OpenAPI Provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs ☑ familiar contract format ☑ provider does not need comms with consumers ⛔ provider does not get insight into how API is being used (deprecation is harder) ⛔ spec-based tests lack functional depth Prism Microcks Schemathesis Microcks
94. @edeandrea #RedHat #DevNation Is OpenAPI enough? It has the same syntax, but the semantics are different.
95. @edeandrea #RedHat #DevNation Is OpenAPI enough? It has the same syntax, but the semantics are different. It looks the same, but the behaviour is different.
96. @edeandrea #RedHat #DevNation Is OpenAPI enough? It has the same syntax, but the semantics are different. It looks the same, but the behaviour is different. Looks like a storm trooper. Behaves like Luke. (because it it is Luke)
97. @edeandrea #RedHat #DevNation Demo: Semantic testing with Pact
98. @edeandrea #RedHat #DevNation
99. @edeandrea #RedHat #DevNation soo … microservices testing is hard limit end-to-end testing mocks will not be enough contract testing can rescue you
100. @edeandrea #RedHat #DevNation ® Thank you! @edeandrea
101. @edeandrea #RedHat #DevNation Additional Resources
102. @edeandrea #RedHat #DevNation https://github.com/quarkusio/quarkus-super-heroes More realistic sample application (fewer wookies)
103. @edeandrea #RedHat #DevNation Kubernetes Native Java with Quarkus Deployment Developer Joy • Quickly & easily build Kubernetes-Native Java applications • Industry leader in startup time and memory utilization for native and JVM applications • One step native executable generation • Leverage existing Java, Jakarta, MicroProfile, & Spring expertise • Imperative & reactive APIs side-by-side • Live coding with zero config https://www.infoq.com/articles/native-java-quarkus
104. @edeandrea #RedHat #DevNation https://www.oreilly.com/library/view/reactive-systems-in/9781492091714 https://developers.redhat.com/e-books/understanding-quarkus https://developers.redhat.com/e-books/practising-quarkus