How to avoid common pitfalls with modern microservices testing

Have you ever wondered why your microservices break in production after all the tests have passed during CI/CD? Have you ever wondered if you’re testing too much or too little? If you want to learn about common pitfalls and how to catch them at build time rather than at runtime, this session is for you!

In this session we’ll share some common battle scars from our experiences in the field. Additionally, we will introduce the testing pyramid and explain why it is important in helping to continuously deploy microservices independently without fear. We will demonstrate, through a test-driven development approach, how the combination of Pact contract testing combined with Quarkus developer joy can help prevent your microservices from breaking in production.

Transcript

1. @holly_cummins @edeandrea #Devoxx #RedHat Avoiding common pitfalls with modern microservices testing Holly Cummins Sr. Principal Software Engineer Red Hat @holly_cummins Eric Deandrea Sr. Principal Developer Advocate Red Hat @edeandrea
2. @holly_cummins @edeandrea #Devoxx #RedHat Free copy (while they last)! Come see Eric at the Red Hat booth 
 (after the talk, not now) https://red.ht/quarkus-spring-devs
3. @holly_cummins @edeandrea #Devoxx #RedHat Disclaimer
4. @holly_cummins @edeandrea #Devoxx #RedHat Disclaimer We are not testers or even test experts
5. @holly_cummins @edeandrea #Devoxx #RedHat Disclaimer We are not testers or even test experts But that’s ok - testing is every developer’s job!
6. @holly_cummins @edeandrea #Devoxx #RedHat So what’s this talk about?
7. @holly_cummins @edeandrea #Devoxx #RedHat So what’s this talk about? Little Red Riding Hood visits the Death Star while trying not to burn your house down on the way to Egypt!
8. @holly_cummins @edeandrea #Devoxx #RedHat Image: https://www.freepik.com/free-photo/close-up-little-red-riding-hood_20031614.htm
9. @holly_cummins @edeandrea #Devoxx #RedHat
10. @holly_cummins @edeandrea #Devoxx #RedHat Death star
11. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture “That’s no moon. It’s our architecture.”
12. @holly_cummins @edeandrea #Devoxx #RedHat Fallacies of microservices
13. @holly_cummins @edeandrea #Devoxx #RedHat “Our architecture is modern so it can never fail.”
14. @holly_cummins @edeandrea #Devoxx #RedHat “Our architecture is modern so it can never fail.” "We've analyzed their attack, sir, and there is a danger. Should I have your ship standing by?” “Evacuate? In our moment of triumph? I think you overestimate their chances."
15. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices is easy! The thing I have to test is so small!
16. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices is easy! Everything is decoupled!
17. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices is easy! Everything is decoupled!
18. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices is easy! Everything is decoupled! But….. if you only test the points, and not the lines … does the thing actually work?
19. @holly_cummins @edeandrea #Devoxx #RedHat distributed != decoupled
20. @holly_cummins @edeandrea #Devoxx #RedHat “Ohhhh, we weren’t expecting your service to do that….“
21. @holly_cummins @edeandrea #Devoxx #RedHat
22. @holly_cummins @edeandrea #Devoxx #RedHat “Every time we touch one microservice, all the others break.”
23. @holly_cummins @edeandrea #Devoxx #RedHat
24. @holly_cummins @edeandrea #Devoxx #RedHat Ok, so how do we do microservices?
25. @holly_cummins @edeandrea #Devoxx #RedHat Get good at automation!
26. @holly_cummins @edeandrea #Devoxx #RedHat Get good at automation! And testing!
27. IBM Cloud © 2020 IBM Corporation
28. IBM Cloud © 2020 IBM Corporation The test pyramid
29. IBM Cloud © 2020 IBM Corporation The test pyramid end-to-end tests high realism high value
30. IBM Cloud © 2020 IBM Corporation The test pyramid end-to-end tests unit tests low effort high realism high value
31. IBM Cloud © 2020 IBM Corporation The test pyramid end-to-end tests unit tests integration tests low effort high realism high value
32. IBM Cloud © 2020 IBM Corporation The test pyramid (you can TDD at every level) end-to-end tests unit tests integration tests low effort high realism high value
33. IBM Cloud © 2020 IBM Corporation tools evolve end-to-end tests unit tests integration tests low effort high realism
34. IBM Cloud © 2020 IBM Corporation tools evolve end-to-end tests unit tests integration tests low effort high realism tests with application server test REST endpoints tests with a database
35. @holly_cummins @edeandrea #Devoxx #RedHat their code our code Why mocks aren’t enough
36. @holly_cummins @edeandrea #Devoxx #RedHat our mock our code Why mocks aren’t enough
37. @holly_cummins @edeandrea #Devoxx #RedHat our mock our code ✔ tests Why mocks aren’t enough
38. @holly_cummins @edeandrea #Devoxx #RedHat our mock our code Why mocks aren’t enough
39. @holly_cummins @edeandrea #Devoxx #RedHat our mock our code ✔ tests ✘ reality our code their actual code Why mocks aren’t enough
40. @holly_cummins @edeandrea #Devoxx #RedHat Demo: The limitations of unit tests
41. @holly_cummins @edeandrea #Devoxx #RedHat Quarkus Superheroes https://github.com/quarkusio/quarkus-super-heroes
42. @holly_cummins @edeandrea #Devoxx #RedHat Quarkus Superheroes https://github.com/quarkusio/quarkus-super-heroes
43. @holly_cummins @edeandrea #Devoxx #RedHat A potential solution
44. @holly_cummins @edeandrea #Devoxx #RedHat How to test a fire alarm?
45. @holly_cummins @edeandrea #Devoxx #RedHat
46. @holly_cummins @edeandrea #Devoxx #RedHat How NOT to test a fire alarm!
47. @holly_cummins @edeandrea #Devoxx #RedHat
48. @holly_cummins @edeandrea #Devoxx #RedHat Unit testing a fire alarm
49. @holly_cummins @edeandrea #Devoxx #RedHat Unit testing a fire alarm
50. @holly_cummins @edeandrea #Devoxx #RedHat uh … is that enough?
51. @holly_cummins @edeandrea #Devoxx #RedHat
52. @holly_cummins @edeandrea #Devoxx #RedHat contract testing a fire alarm
53. IBM Cloud © 2020 IBM Corporation end-to-end tests unit tests integration tests low effort high realism high value
54. IBM Cloud © 2020 IBM Corporation end-to-end tests unit tests integration tests low effort high realism high value contract tests
55. @holly_cummins @edeandrea #Devoxx #RedHat our code their code Why mocks aren’t enough
56. @holly_cummins @edeandrea #Devoxx #RedHat our code their code contract test mock functional test Why mocks aren’t enough
57. @holly_cummins @edeandrea #Devoxx #RedHat ✔ our tests ✔ their tests ✔ reality our code their code contract test mock functional test Why mocks aren’t enough
58. @holly_cummins @edeandrea #Devoxx #RedHat our code their code Why mocks aren’t enough
59. @holly_cummins @edeandrea #Devoxx #RedHat our code their code contract test mock functional test Why mocks aren’t enough
60. @holly_cummins @edeandrea #Devoxx #RedHat ✔ our tests ✘ their tests ✘ reality our code their code contract test mock functional test Why mocks aren’t enough
61. @holly_cummins @edeandrea #Devoxx #RedHat Why mocks aren’t enough our code their code
62. @holly_cummins @edeandrea #Devoxx #RedHat Why mocks aren’t enough our code their code contract test mock functional test
63. @holly_cummins @edeandrea #Devoxx #RedHat ✘ our tests ✔ their tests ✘ reality Why mocks aren’t enough our code their code contract test mock functional test
64. @holly_cummins @edeandrea #Devoxx #RedHat Some Contract Testing options
65. @holly_cummins @edeandrea #Devoxx #RedHat Some Contract Testing options • Pact • Spring Cloud Contract • OpenAPI schema validation + mock generation • Prism • Schemathesis
66. @holly_cummins @edeandrea #Devoxx #RedHat Pact Contract Testing
67. @holly_cummins @edeandrea #Devoxx #RedHat Demo: How Pact saves our bacon!
68. @holly_cummins @edeandrea #Devoxx #RedHat Consumer-driven contract testing
69. @holly_cummins @edeandrea #Devoxx #RedHat uh, you’re telling me that consumers can break a provider’s tests?
70. IBM Cloud © 2020 IBM Corporation
71. @holly_cummins @edeandrea #Devoxx #RedHat “contract-first” (it’s like TDD, but between teams)
72. @holly_cummins @edeandrea #Devoxx #RedHat Is Pact always the best choice?
73. @holly_cummins @edeandrea #Devoxx #RedHat Consumer and provider do not have good communication
74. @holly_cummins @edeandrea #Devoxx #RedHat Consumer and provider do not have good communication
75. @holly_cummins @edeandrea #Devoxx #RedHat Team maintaining the “other side” is not using Pact Consumer and provider do not have good communication P ! P
76. @holly_cummins @edeandrea #Devoxx #RedHat Consumers can’t be identified (i.e. for public APIs) Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact
77. @holly_cummins @edeandrea #Devoxx #RedHat Can’t control data being used for provider responses Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs)
78. @holly_cummins @edeandrea #Devoxx #RedHat “Pass-through” APIs Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses
79. @holly_cummins @edeandrea #Devoxx #RedHat “Pass-through” APIs Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses
80. @holly_cummins @edeandrea #Devoxx #RedHat Functional, performance, or load testing Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses “Pass-through” APIs
81. @holly_cummins @edeandrea #Devoxx #RedHat Functional, performance, or load testing Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses “Pass-through” APIs
82. @holly_cummins @edeandrea #Devoxx #RedHat Pact vs Spring Cloud Contract?
83. @holly_cummins @edeandrea #Devoxx #RedHat Pact vs Spring Cloud Contract? • Pact is polyglot! • Pact generates JSON acceptance contracts (Pacts) • Pact is consumer-driven whereas SCC is provider- driven • Pact allows consumers to specify provider test states • Pact broker
84. @holly_cummins @edeandrea #Devoxx #RedHat …but I use Swagger/OpenAPI “My, what large teeth you have, Grandma” It looks the same, but the semantics are different.
85. @holly_cummins @edeandrea #Devoxx #RedHat Demo: More complex testing with Pact
86. @holly_cummins @edeandrea #Devoxx #RedHat how annoying, the provider is returning 204. the contract expects 404, so my test is broken. quarkus will return 204 by default. 204 vs 404 - it’s more about the theory behind REST...if a caller asks for a resource the server doesn't have, it should be a 404 - same way if a user types in an invalid URL into the browser ugh, it’s too late at night to care about REST theory. I’m just going to update the expectations in the test to match reality. what?! why is my other contract test broken now? 
 oh … because what I’m changing is a mock…. Based on a true story….
87. @holly_cummins @edeandrea #Devoxx #RedHat
88. @holly_cummins @edeandrea #Devoxx #RedHat ® Thank you! @holly_cummins 
 @edeandrea
89. @holly_cummins @edeandrea #Devoxx #RedHat Additional Resources
90. @holly_cummins @edeandrea #Devoxx #RedHat Kubernetes Native Java with Quarkus Deployment Developer Joy • Quickly & easily build Kubernetes-Native Java applications • Industry leader in startup time and memory utilization for native and JVM applications • One step native executable generation • Leverage existing Java, Jakarta, MicroProfile, & Spring expertise • Imperative & reactive APIs side-by-side • Live coding with zero config https://www.infoq.com/articles/native-java-quarkus
91. @holly_cummins @edeandrea #Devoxx #RedHat https://www.oreilly.com/library/view/reactive-systems-in/9781492091714 https://developers.redhat.com/e-books/understanding-quarkus https://developers.redhat.com/e-books/practising-quarkus
92. @holly_cummins @edeandrea #Devoxx #RedHat