Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to avoid common pitfalls with modern microservices testing

How to avoid common pitfalls with modern microservices testing

Have you ever wondered why your microservices break in production after all the tests have passed during CI/CD? Have you ever wondered if you’re testing too much or too little? If you want to learn about common pitfalls and how to catch them at build time rather than at runtime, this session is for you!

In this session we’ll share some common battle scars from our experiences in the field. Additionally, we will introduce the testing pyramid and explain why it is important in helping to continuously deploy microservices independently without fear. We will demonstrate, through a test-driven development approach, how the combination of Pact contract testing combined with Quarkus developer joy can help prevent your microservices from breaking in production.

Holly Cummins

October 13, 2022
Tweet

More Decks by Holly Cummins

Other Decks in Programming

Transcript

  1. @holly_cummins @edeandrea #Devoxx #RedHat Avoiding common pitfalls with modern microservices

    testing Holly Cummins Sr. Principal Software Engineer Red Hat @holly_cummins Eric Deandrea Sr. Principal Developer Advocate Red Hat @edeandrea
  2. @holly_cummins @edeandrea #Devoxx #RedHat Free copy (while they last)! Come

    see Eric at the Red Hat booth 
 (after the talk, not now) https://red.ht/quarkus-spring-devs
  3. @holly_cummins @edeandrea #Devoxx #RedHat Disclaimer

  4. @holly_cummins @edeandrea #Devoxx #RedHat Disclaimer We are not testers or

    even test experts
  5. @holly_cummins @edeandrea #Devoxx #RedHat Disclaimer We are not testers or

    even test experts But that’s ok - testing is every developer’s job!
  6. @holly_cummins @edeandrea #Devoxx #RedHat So what’s this talk about?

  7. @holly_cummins @edeandrea #Devoxx #RedHat So what’s this talk about? Little

    Red Riding Hood visits the Death Star while trying not to burn your house down on the way to Egypt!
  8. @holly_cummins @edeandrea #Devoxx #RedHat Image: https://www.freepik.com/free-photo/close-up-little-red-riding-hood_20031614.htm

  9. @holly_cummins @edeandrea #Devoxx #RedHat

  10. @holly_cummins @edeandrea #Devoxx #RedHat Death star

  11. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture “That’s no moon.

    It’s our architecture.”
  12. @holly_cummins @edeandrea #Devoxx #RedHat Fallacies of microservices

  13. @holly_cummins @edeandrea #Devoxx #RedHat “Our architecture is modern so it

    can never fail.”
  14. @holly_cummins @edeandrea #Devoxx #RedHat “Our architecture is modern so it

    can never fail.” "We've analyzed their attack, sir, and there is a danger. Should I have your ship standing by?” “Evacuate? In our moment of triumph? I think you overestimate their chances."
  15. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices

    is easy! The thing I have to test is so small!
  16. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices

    is easy! Everything is decoupled!
  17. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices

    is easy! Everything is decoupled!
  18. @holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices

    is easy! Everything is decoupled! But….. if you only test the points, and not the lines … does the thing actually work?
  19. @holly_cummins @edeandrea #Devoxx #RedHat distributed != decoupled

  20. @holly_cummins @edeandrea #Devoxx #RedHat “Ohhhh, we weren’t expecting your service

    to do that….“
  21. @holly_cummins @edeandrea #Devoxx #RedHat

  22. @holly_cummins @edeandrea #Devoxx #RedHat “Every time we touch one microservice,

    all the others break.”
  23. @holly_cummins @edeandrea #Devoxx #RedHat

  24. @holly_cummins @edeandrea #Devoxx #RedHat Ok, so how do we do

    microservices?
  25. @holly_cummins @edeandrea #Devoxx #RedHat Get good at automation!

  26. @holly_cummins @edeandrea #Devoxx #RedHat Get good at automation! And testing!

  27. IBM Cloud © 2020 IBM Corporation

  28. IBM Cloud © 2020 IBM Corporation The test pyramid

  29. IBM Cloud © 2020 IBM Corporation The test pyramid end-to-end

    tests high realism high value
  30. IBM Cloud © 2020 IBM Corporation The test pyramid end-to-end

    tests unit tests low effort high realism high value
  31. IBM Cloud © 2020 IBM Corporation The test pyramid end-to-end

    tests unit tests integration tests low effort high realism high value
  32. IBM Cloud © 2020 IBM Corporation The test pyramid (you

    can TDD at every level) end-to-end tests unit tests integration tests low effort high realism high value
  33. IBM Cloud © 2020 IBM Corporation tools evolve end-to-end tests

    unit tests integration tests low effort high realism
  34. IBM Cloud © 2020 IBM Corporation tools evolve end-to-end tests

    unit tests integration tests low effort high realism tests with application server test REST endpoints tests with a database
  35. @holly_cummins @edeandrea #Devoxx #RedHat their code our code Why mocks

    aren’t enough
  36. @holly_cummins @edeandrea #Devoxx #RedHat our mock our code Why mocks

    aren’t enough
  37. @holly_cummins @edeandrea #Devoxx #RedHat our mock our code ✔ tests

    Why mocks aren’t enough
  38. @holly_cummins @edeandrea #Devoxx #RedHat our mock our code Why mocks

    aren’t enough
  39. @holly_cummins @edeandrea #Devoxx #RedHat our mock our code ✔ tests

    ✘ reality our code their actual code Why mocks aren’t enough
  40. @holly_cummins @edeandrea #Devoxx #RedHat Demo: The limitations of unit tests

  41. @holly_cummins @edeandrea #Devoxx #RedHat Quarkus Superheroes https://github.com/quarkusio/quarkus-super-heroes

  42. @holly_cummins @edeandrea #Devoxx #RedHat Quarkus Superheroes https://github.com/quarkusio/quarkus-super-heroes

  43. @holly_cummins @edeandrea #Devoxx #RedHat A potential solution

  44. @holly_cummins @edeandrea #Devoxx #RedHat How to test a fire alarm?

  45. @holly_cummins @edeandrea #Devoxx #RedHat

  46. @holly_cummins @edeandrea #Devoxx #RedHat How NOT to test a fire

    alarm!
  47. @holly_cummins @edeandrea #Devoxx #RedHat

  48. @holly_cummins @edeandrea #Devoxx #RedHat Unit testing a fire alarm

  49. @holly_cummins @edeandrea #Devoxx #RedHat Unit testing a fire alarm

  50. @holly_cummins @edeandrea #Devoxx #RedHat uh … is that enough?

  51. @holly_cummins @edeandrea #Devoxx #RedHat

  52. @holly_cummins @edeandrea #Devoxx #RedHat contract testing a fire alarm

  53. IBM Cloud © 2020 IBM Corporation end-to-end tests unit tests

    integration tests low effort high realism high value
  54. IBM Cloud © 2020 IBM Corporation end-to-end tests unit tests

    integration tests low effort high realism high value contract tests
  55. @holly_cummins @edeandrea #Devoxx #RedHat our code their code Why mocks

    aren’t enough
  56. @holly_cummins @edeandrea #Devoxx #RedHat our code their code contract test

    mock functional test Why mocks aren’t enough
  57. @holly_cummins @edeandrea #Devoxx #RedHat ✔ our tests ✔ their tests

    ✔ reality our code their code contract test mock functional test Why mocks aren’t enough
  58. @holly_cummins @edeandrea #Devoxx #RedHat our code their code Why mocks

    aren’t enough
  59. @holly_cummins @edeandrea #Devoxx #RedHat our code their code contract test

    mock functional test Why mocks aren’t enough
  60. @holly_cummins @edeandrea #Devoxx #RedHat ✔ our tests ✘ their tests

    ✘ reality our code their code contract test mock functional test Why mocks aren’t enough
  61. @holly_cummins @edeandrea #Devoxx #RedHat Why mocks aren’t enough our code

    their code
  62. @holly_cummins @edeandrea #Devoxx #RedHat Why mocks aren’t enough our code

    their code contract test mock functional test
  63. @holly_cummins @edeandrea #Devoxx #RedHat ✘ our tests ✔ their tests

    ✘ reality Why mocks aren’t enough our code their code contract test mock functional test
  64. @holly_cummins @edeandrea #Devoxx #RedHat Some Contract Testing options

  65. @holly_cummins @edeandrea #Devoxx #RedHat Some Contract Testing options • Pact

    • Spring Cloud Contract • OpenAPI schema validation + mock generation • Prism • Schemathesis
  66. @holly_cummins @edeandrea #Devoxx #RedHat Pact Contract Testing

  67. @holly_cummins @edeandrea #Devoxx #RedHat Demo: How Pact saves our bacon!

  68. @holly_cummins @edeandrea #Devoxx #RedHat Consumer-driven contract testing

  69. @holly_cummins @edeandrea #Devoxx #RedHat uh, you’re telling me that consumers

    can break a provider’s tests?
  70. IBM Cloud © 2020 IBM Corporation

  71. @holly_cummins @edeandrea #Devoxx #RedHat “contract-first” (it’s like TDD, but between

    teams)
  72. @holly_cummins @edeandrea #Devoxx #RedHat Is Pact always the best choice?

  73. @holly_cummins @edeandrea #Devoxx #RedHat Consumer and provider do not have

    good communication
  74. @holly_cummins @edeandrea #Devoxx #RedHat Consumer and provider do not have

    good communication
  75. @holly_cummins @edeandrea #Devoxx #RedHat Team maintaining the “other side” is

    not using Pact Consumer and provider do not have good communication P ! P
  76. @holly_cummins @edeandrea #Devoxx #RedHat Consumers can’t be identified (i.e. for

    public APIs) Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact
  77. @holly_cummins @edeandrea #Devoxx #RedHat Can’t control data being used for

    provider responses Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs)
  78. @holly_cummins @edeandrea #Devoxx #RedHat “Pass-through” APIs Consumer and provider do

    not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses
  79. @holly_cummins @edeandrea #Devoxx #RedHat “Pass-through” APIs Consumer and provider do

    not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses
  80. @holly_cummins @edeandrea #Devoxx #RedHat Functional, performance, or load testing Consumer

    and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses “Pass-through” APIs
  81. @holly_cummins @edeandrea #Devoxx #RedHat Functional, performance, or load testing Consumer

    and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses “Pass-through” APIs
  82. @holly_cummins @edeandrea #Devoxx #RedHat Pact vs Spring Cloud Contract?

  83. @holly_cummins @edeandrea #Devoxx #RedHat Pact vs Spring Cloud Contract? •

    Pact is polyglot! • Pact generates JSON acceptance contracts (Pacts) • Pact is consumer-driven whereas SCC is provider- driven • Pact allows consumers to specify provider test states • Pact broker
  84. @holly_cummins @edeandrea #Devoxx #RedHat …but I use Swagger/OpenAPI “My, what

    large teeth you have, Grandma” It looks the same, but the semantics are different.
  85. @holly_cummins @edeandrea #Devoxx #RedHat Demo: More complex testing with Pact

  86. @holly_cummins @edeandrea #Devoxx #RedHat how annoying, the provider is returning

    204. the contract expects 404, so my test is broken. quarkus will return 204 by default. 204 vs 404 - it’s more about the theory behind REST...if a caller asks for a resource the server doesn't have, it should be a 404 - same way if a user types in an invalid URL into the browser ugh, it’s too late at night to care about REST theory. I’m just going to update the expectations in the test to match reality. what?! why is my other contract test broken now? 
 oh … because what I’m changing is a mock…. Based on a true story….
  87. @holly_cummins @edeandrea #Devoxx #RedHat

  88. @holly_cummins @edeandrea #Devoxx #RedHat ® Thank you! @holly_cummins 
 @edeandrea

  89. @holly_cummins @edeandrea #Devoxx #RedHat Additional Resources

  90. @holly_cummins @edeandrea #Devoxx #RedHat Kubernetes Native Java with Quarkus Deployment

    Developer Joy • Quickly & easily build Kubernetes-Native Java applications • Industry leader in startup time and memory utilization for native and JVM applications • One step native executable generation • Leverage existing Java, Jakarta, MicroProfile, & Spring expertise • Imperative & reactive APIs side-by-side • Live coding with zero config https://www.infoq.com/articles/native-java-quarkus
  91. @holly_cummins @edeandrea #Devoxx #RedHat https://www.oreilly.com/library/view/reactive-systems-in/9781492091714 https://developers.redhat.com/e-books/understanding-quarkus https://developers.redhat.com/e-books/practising-quarkus

  92. @holly_cummins @edeandrea #Devoxx #RedHat