Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Uh, Does This Thing Actually Work? Modern Microservices Testing

Uh, Does This Thing Actually Work? Modern Microservices Testing

To realise the promise of microservices, individual services must be independently deployable, and they should be deployed often. But that’s easier said than done. Releases take confidence. How does a business achieve enough confidence to release often without getting bogged down in manual QA bottlenecks? Is automating tests too expensive? How do services know if they’ve made a breaking change? If the individual microservices all work, does that mean the system works? This talk discuss microservices testing strategies and explore some of the modern tools which make testing easier and better. And there’s a demo: Holly will show Pact contract testing, test containers, and Quarkus’s continuous testing and dev services support.

Holly Cummins

August 31, 2022
Tweet

More Decks by Holly Cummins

Other Decks in Programming

Transcript

  1. Holly Cummins Red Hat #SkillsMatter uh, does this thing actually

    work? modern microservices testing
  2. None
  3. disclaimer: I am not a tester

  4. disclaimer: I am not a tester … or even a

    test specialist.
  5. disclaimer: I am not a tester … or even a

    test specialist. but that’s ok! testing is every developer’s job.
  6. @holly_cummins #RedHat fallacies of microservices

  7. None
  8. None
  9. None
  10. None
  11. None
  12. None
  13. None
  14. #Quarkus @holly_cummins “microservices makes testing easy! the thing I have

    to test is so small!”
  15. #Quarkus @holly_cummins Netflix microservice architecture “microservices makes testing easy! the

    thing I have to test is so small!”
  16. #Quarkus @holly_cummins Netflix microservice architecture “microservices makes testing easy! the

    thing I have to test is so small!” if you only test the points, and not the lines … does the thing work?
  17. #Quarkus @holly_cummins “That’s no moon. It’s our architecture.” Netflix microservice

    architecture
  18. @holly_cummins #RedHat “microservices makes testing easy. we’re decoupled!”

  19. @holly_cummins #RedHat “microservices makes testing easy. we’re decoupled!”

  20. @holly_cummins #RedHat distributed != decoupled

  21. @holly_cummins #RedHat “ohhhh, we weren’t expecting your service to do

    that … “
  22. @holly_cummins #RedHat “uh, what do you mean you corrected the

    typo in your json?”
  23. @holly_cummins #RedHat how do you know if you’re decoupled?

  24. can you deploy independently? v5.1.4 v5.1.4 v5.1.4 v5.1.4

  25. can you deploy independently? v6.2.8 v65.7 v5.1.4 v1.0.5

  26. @holly_cummins #RedHat if you’re going to do microservices, you need

    to get good at automation. … and testing.
  27. None
  28. @holly_cummins #RedHat TDD

  29. @holly_cummins #RedHat TDD: improves quality

  30. @holly_cummins #RedHat TDD: an excellent design tool

  31. @holly_cummins #RedHat

  32. @holly_cummins #RedHat the real reason to do TDD:

  33. @holly_cummins #RedHat the real reason to do TDD:

  34. @holly_cummins #RedHat the real reason to do TDD: gamified testing

  35. @holly_cummins #RedHat the test pyramid

  36. @holly_cummins #RedHat the test pyramid (you can TDD at every

    level)
  37. @holly_cummins #RedHat the test pyramid

  38. @holly_cummins #RedHat the test pyramid end-to-end tests

  39. @holly_cummins #RedHat the test pyramid end-to-end tests integration tests

  40. @holly_cummins #RedHat the test pyramid end-to-end tests unit tests integration

    tests
  41. @holly_cummins #RedHat the test pyramid end-to-end tests unit tests integration

    tests low effort
  42. @holly_cummins #RedHat the test pyramid end-to-end tests unit tests integration

    tests low effort high realism
  43. @holly_cummins #RedHat the test pyramid end-to-end tests unit tests integration

    tests low effort high realism high value
  44. @holly_cummins #RedHat all models are wrong some are useful

  45. @holly_cummins #RedHat tools evolve end-to-end tests unit tests integration tests

    low effort high realism
  46. @holly_cummins #RedHat tools evolve end-to-end tests unit tests integration tests

    low effort high realism tests with application server
  47. @holly_cummins #RedHat tools evolve end-to-end tests unit tests integration tests

    low effort high realism tests with application server test REST endpoints
  48. @holly_cummins #RedHat tools evolve end-to-end tests unit tests integration tests

    low effort high realism tests with application server test REST endpoints
  49. @holly_cummins #RedHat unit tests low effort

  50. @holly_cummins #RedHat “why have I spent half a day writing

    mocks just so I can write a unit test?” the limitations of unit tests
  51. @holly_cummins #RedHat the limitations of unit tests “why do my

    colleagues tell me my code is broken, even though my unit tests all pass?”
  52. @holly_cummins #RedHat their code our code why unit tests aren’t

    enough
  53. @holly_cummins #RedHat their code our code why mocks aren’t enough

  54. @holly_cummins #RedHat their code our code why stubs aren’t enough

  55. @holly_cummins #RedHat our code our mock why mocks aren’t enough

  56. @holly_cummins #RedHat our code our mock tests ✔ why mocks

    aren’t enough
  57. @holly_cummins #RedHat our code our mock our code their actual

    code tests ✔ why mocks aren’t enough
  58. @holly_cummins #RedHat our code our mock our code their actual

    code tests ✔ reality ✘ why mocks aren’t enough
  59. @holly_cummins #RedHat what needs integration tests? our service

  60. @holly_cummins #RedHat what needs integration tests? our service all code

    that serialises or deserialises data
  61. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers all code that serialises or deserialises data
  62. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls all code that serialises or deserialises data
  63. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls queues ETL all code that serialises or deserialises data
  64. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls filesystem config, etc queues ETL all code that serialises or deserialises data
  65. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services other services filesystem config, etc queues ETL all code that serialises or deserialises data
  66. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data
  67. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data
  68. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data
  69. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data
  70. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data
  71. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data
  72. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data
  73. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data
  74. @holly_cummins #RedHat what needs integration tests? our service rest api

    usage by consumers db calls external applications colleague’s services filesystem config, etc queues ETL all code that serialises or deserialises data
  75. @holly_cummins #RedHat end-to-end tests unit tests integration tests low effort

    high realism
  76. @holly_cummins #RedHat end-to-end tests unit tests integration tests low effort

    high realism
  77. @holly_cummins #RedHat end-to-end tests unit tests integration tests low effort

    high realism
  78. @holly_cummins #RedHat the test trophy end-to-end tests unit tests integration

    tests low effort high realism
  79. @holly_cummins #RedHat the test trophy end-to-end tests high realism

  80. @holly_cummins #RedHat the test trophy end-to-end tests high realism in

    staging
  81. @holly_cummins #RedHat the test trophy end-to-end tests high realism in

    staging ?
  82. @holly_cummins #RedHat “why do I keep getting failures in production

    even though everything looked fine in staging?” Photo by Eggs&Beer, Flickr
  83. @holly_cummins #RedHat staging staging ✔ why staging is not enough

    our service other services other services other services other services
  84. @holly_cummins #RedHat staging staging ✔ production ✘ why staging is

    not enough our service other services other services other services other services our service production other services other services other services other services
  85. @holly_cummins #RedHat “test in prod or live a lie” -

    Charity Majors
  86. @holly_cummins #RedHat the test trophy end-to-end tests unit tests integration

    tests low effort high realism
  87. @holly_cummins #RedHat the test trophy end-to-end tests unit tests integration

    tests low effort high realism testing in production
  88. @holly_cummins #RedHat the test trophy end-to-end tests unit tests integration

    tests low effort high realism static tests testing in production
  89. @holly_cummins #RedHat confidence effort

  90. @holly_cummins #RedHat confidence effort quality

  91. @holly_cummins #RedHat confidence effort quality expense

  92. @holly_cummins #RedHat confidence effort quality expense cheap and cheerful

  93. @holly_cummins #RedHat confidence effort quality expense hard work, but valuable

    cheap and cheerful
  94. @holly_cummins #RedHat winning confidence effort quality expense hard work, but

    valuable cheap and cheerful
  95. @holly_cummins #RedHat a waste winning confidence effort quality expense hard

    work, but valuable cheap and cheerful
  96. @holly_cummins #RedHat a waste winning confidence effort quality expense hard

    work, but valuable cheap and cheerful
  97. @holly_cummins #RedHat a waste winning confidence effort quality expense hard

    work, but valuable cheap and cheerful unit tests
  98. @holly_cummins #RedHat a waste winning confidence effort quality expense hard

    work, but valuable cheap and cheerful unit tests integration tests
  99. @holly_cummins #RedHat a waste winning confidence effort quality expense hard

    work, but valuable cheap and cheerful unit tests integration tests e2e tests
  100. @holly_cummins #RedHat a waste confidence effort quality expense hard work,

    but valuable cheap and cheerful unit tests integration tests e2e tests static tests
  101. @holly_cummins #RedHat a waste confidence effort quality expense hard work,

    but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests
  102. @holly_cummins #RedHat a waste confidence effort quality expense hard work,

    but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests YMMV
  103. @holly_cummins #RedHat a waste confidence effort quality expense hard work,

    but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests YMMV
  104. @holly_cummins #RedHat a waste confidence effort quality expense hard work,

    but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests YMMV
  105. @holly_cummins #RedHat a waste confidence effort quality expense hard work,

    but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests YMMV
  106. @holly_cummins #RedHat a waste confidence effort quality expense hard work,

    but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests
  107. @holly_cummins #RedHat a waste confidence effort quality expense hard work,

    but valuable cheap and cheerful unit tests integration tests e2e tests testing in production static tests constantly question the value of your tests
  108. @holly_cummins #RedHat integration tests

  109. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests
  110. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests
  111. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests all integration tests are not created equal
  112. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests integration tests all integration tests are not created equal
  113. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests integration tests all integration tests are not created equal integration tests
  114. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests integration tests all integration tests are not created equal integration tests integration tests
  115. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests integration tests all integration tests are not created equal integration tests integration tests integration tests
  116. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests all integration tests are not created equal integration tests contract tests integration tests integration tests
  117. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests all integration tests are not created equal contract tests sandbox tests integration tests integration tests
  118. @holly_cummins #RedHat confidence effort unit tests integration tests e2e tests

    testing in production static tests all integration tests are not created equal contract tests sandbox tests testcontainers tests integration tests
  119. @holly_cummins #RedHat approaches to integration tests our service rest api

    external applications filesystem colleague’s services colleague’s services colleague’s services external applications
  120. @holly_cummins #RedHat approaches to integration tests our service rest api

    external applications filesystem colleague’s services colleague’s services colleague’s services external applications mock all the things
  121. @holly_cummins #RedHat “why have I spent half a day writing

    mocks just so I can write a cleanly-scoped integration test?”
  122. @holly_cummins #RedHat approaches to integration tests our service rest api

    db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications
  123. @holly_cummins #RedHat approaches to integration tests our service rest api

    db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications
  124. @holly_cummins #RedHat approaches to integration tests our service rest api

    db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications containerise all the things
  125. @holly_cummins #RedHat approaches to integration tests our service rest api

    db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications containerise all the things
  126. @holly_cummins #RedHat

  127. @holly_cummins #RedHat “I can’t bring up my dev stack

  128. @holly_cummins #RedHat “I can’t bring up my dev stack …

    on my brand new Apple laptop with a M1 chip and 64GB of RAM.” - fintech developer
  129. @holly_cummins #RedHat “I can’t bring up my dev stack …

    on my brand new Apple laptop with a M1 chip and 64GB of RAM.” digression: if this is you • maybe testing is only part of your problem • … and Quarkus could help :) - fintech developer
  130. @holly_cummins #RedHat “I have to do all this shell scripting

    and manual process and orchestration before I can write a test or do development.”
  131. @holly_cummins #RedHat approaches to integration tests our service rest api

    external applications filesystem colleague’s services colleague’s services colleague’s services external applications remote sandbox remote system + remote deploy
  132. @holly_cummins #RedHat “I miss developing on my actual laptop.”

  133. @holly_cummins #RedHat approaches to integration tests our service rest api

    external applications filesystem colleague’s services colleague’s services colleague’s services external applications remote system with telepresence
  134. @holly_cummins #RedHat approaches to integration tests our service rest api

    external applications filesystem colleague’s services colleague’s services colleague’s services external applications remocal sandbox remote system with telepresence
  135. @holly_cummins #RedHat approaches to integration tests our service rest api

    external applications filesystem colleague’s services colleague’s services colleague’s services external applications remocal sandbox remote system with telepresence
  136. @holly_cummins #RedHat approaches to integration tests our service rest api

    external applications filesystem colleague’s services colleague’s services colleague’s services external applications shared remocal sandbox remote system with telepresence and per-request isolation
  137. @holly_cummins #RedHat pitfalls: • it all works great – until

    a test changes some state, eg • writing to a db • adding a user to a security provider • getting IP whitelists when running locally is tricky
  138. @holly_cummins #RedHat

  139. @holly_cummins #RedHat “could I be … a bit less realistic?

    and a bit more offline?”
  140. @holly_cummins #RedHat networks: just say no

  141. @holly_cummins #RedHat approaches to integration tests our service rest api

    db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications
  142. @holly_cummins #RedHat approaches to integration tests our service rest api

    db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications
  143. @holly_cummins #RedHat approaches to integration tests our service rest api

    db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications testcontainers
  144. @holly_cummins #RedHat approaches to integration tests our service rest api

    db calls external applications filesystem config, etc queues ETL colleague’s services colleague’s services colleague’s services external applications testcontainers
  145. @holly_cummins #RedHat

  146. @holly_cummins #RedHat drawback:

  147. @holly_cummins #RedHat drawback: • the service still uses CPU

  148. @holly_cummins #RedHat drawback: • the service still uses CPU •

    only works for well-known libraries
  149. @holly_cummins #RedHat approaches to integration tests our service rest api

    external applications filesystem colleague’s services colleague’s services colleague’s services external applications
  150. @holly_cummins #RedHat approaches to integration tests our service rest api

    external applications filesystem colleague’s services colleague’s services colleague’s services external applications contract test (for services who will talk to you)
  151. @holly_cummins #RedHat their code our code why mocks aren’t enough

  152. @holly_cummins #RedHat our code our mock why mocks aren’t enough

  153. @holly_cummins #RedHat our code our mock tests ✔ why mocks

    aren’t enough
  154. @holly_cummins #RedHat our code our mock our code their actual

    code tests ✔ why mocks aren’t enough
  155. @holly_cummins #RedHat our code our mock our code their actual

    code tests ✔ reality ✘ why mocks aren’t enough
  156. @holly_cummins #RedHat our code their code why mocks aren’t enough

  157. @holly_cummins #RedHat our code their code contract test why mocks

    aren’t enough
  158. @holly_cummins #RedHat our code their code contract test mock why

    mocks aren’t enough
  159. @holly_cummins #RedHat our code their code contract test mock functional

    test why mocks aren’t enough
  160. @holly_cummins #RedHat our code their code contract test mock functional

    test our tests ✔ their tests ✔ reality ✔ why mocks aren’t enough
  161. @holly_cummins #RedHat our code their code contract test mock functional

    test why mocks aren’t enough
  162. @holly_cummins #RedHat our code their code contract test mock functional

    test our tests ✔ their tests ✘ reality ✘ why mocks aren’t enough
  163. @holly_cummins #RedHat our code their code contract test mock functional

    test why mocks aren’t enough
  164. @holly_cummins #RedHat our code their code contract test mock functional

    test our tests ✘ their tests ✔ reality ✘ why mocks aren’t enough
  165. @holly_cummins #RedHat

  166. @holly_cummins #RedHat contract-first is a great way to TDD

  167. @holly_cummins #RedHat contract-first is a great way to TDD (and

    a great design tool)
  168. to the code! @holly_cummins #RedHat

  169. #RedHat @holly_cummins demo recap: https://github.com/holly-cummins/house-of-microservices-quarkus-contract-testing-sample

  170. #RedHat @holly_cummins demo recap: https://github.com/holly-cummins/house-of-microservices-quarkus-contract-testing-sample

  171. #RedHat @holly_cummins demo recap: • consumer-driven contract testing can save

    your bacon • pact is a mock for the consumer • pact is a functional test for the producer • shared json contracts aligns expectations across services
  172. @holly_cummins #RedHat pitfalls of contract testing: • we have to

    talk to the other team • our consumers can break our CI?! • versions • pact is hard to grok • testing the mock • but I’ve invested in wiremock • duplicated tests is overhead
  173. @holly_cummins #RedHat contract testing options: • pact.io • spring contract

    • OpenAPI schema validation + mock generation • prism • schemathesis
  174. @holly_cummins #RedHat confidence effort integration testing landscape mock all the

    services testcontainers remote sandbox remocal sandbox contract tests local orchestration
  175. ® thank you @holly_cummins https://github.com/holly-cummins/house-of- microservices-quarkus-contract-testing-sample