Avoiding common pitfalls with modern microservices testing

Have you ever wondered why your microservices break in production after all the tests have passed during CI/CD? Have you ever wondered if you’re testing too much or too little? If you want to learn about common pitfalls and how to catch them at build time rather than at runtime, this session is for you!

In this session we’ll share some common battle scars from our experiences in the field. Additionally, we will introduce the testing pyramid and explain why it is important in helping to continuously deploy microservices independently without fear. We will demonstrate, through a test-driven development approach, how the combination of Pact contract testing combined with Quarkus developer joy can help prevent your microservices from breaking in production.

Transcript

1. Holly Cummins
   
   
   Sr. Principal Software Engineer
   
   
   Red Hat
   
   
   @holly_cummins
   Eric Deandrea
   
   
   Sr. Principal Developer Advocate
   
   
   Red Hat
   
   
   @edeandrea
   Avoiding common pitfalls with
   
   
   modern microservices testing
   
   
   

   View Slide

2. @holly_cummins @edeandrea
   #DevNexus #RedHat
   “Every time we touch one microservice,
   all the others break.”
   

   View Slide

3. @holly_cummins @edeandrea
   #DevNexus #RedHat
   “My end-to-end test environment is only up the
   
   
   third weekend every month.”
   

   View Slide

4. @edeandre
   a
   for Developers
   1:30 pm today!
   

   View Slide

5. @holly_cummins @edeandrea
   #DevNexus #RedHat
   Free copy (while they last)!
   
   
   Come see Eric at the Red Hat booth
   

   (at 2:45 PM today, not now)
   https://red.ht/quarkus-spring-devs
   

   View Slide

6. @holly_cummins @edeandrea
   #DevNexus #RedHat
   Disclaimer!
   

   View Slide

7. @holly_cummins @edeandrea
   #DevNexus #RedHat
   Disclaimer!
   We are not testers.
   (or even test experts)
   

   View Slide

8. @holly_cummins @edeandrea
   #DevNexus #RedHat
   Disclaimer!
   We are not testers.
   (or even test experts)
   That’s ok– testing is
   every developer’s job!
   

   View Slide

9. @holly_cummins @edeandrea
   #DevNexus #RedHat
   Why is testing microservices so hard?
   

   View Slide

10. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Netflix microservice
    architecture
    

    View Slide

11. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Death star
    “That’s no moon.
    It’s our architecture.”
    

    View Slide

12. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Do microservices solve all problems?
    

    View Slide

13. @holly_cummins @edeandrea
    #DevNexus #RedHat
    A microservice system: a portrait
    

    View Slide

14. View Slide

15. View Slide

16. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Netflix microservice
    architecture
    Testing with microservices is easy!
    
    
    The thing I have to test is so small!
    

    View Slide

17. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Netflix microservice
    architecture
    Testing with microservices is easy!
    
    
    Everything is decoupled!
    

    View Slide

18. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Netflix microservice
    architecture
    Testing with microservices is easy!
    
    
    Everything is decoupled!
    

    View Slide

19. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Netflix microservice
    architecture
    Testing with microservices is easy!
    
    
    Everything is decoupled!
    But…..
    
    
    If you only test the points,
    
    
    And not the lines….
    
    
    Does the thing actually work?
    

    View Slide

20. @holly_cummins @edeandrea
    #DevNexus #RedHat
    distributed != decoupled
    

    View Slide

21. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Understand the coupling
    

    View Slide

22. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “It is all that is left unsaid upon
    which tragedies are built.”
    
    
    - Kreia, Star Wars Knights of the Old Republic II: The Sith Lords
    

    View Slide

23. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “You lied to me.
    
    
    How many other lies have I been
    told by the Council?
    
    
    And how do you know that you
    even have the truth?”
    
    
    – Anakin Skywalker
    

    View Slide

24. @holly_cummins @edeandrea
    #DevNexus #RedHat
    

    View Slide

25. @holly_cummins @edeandrea
    #DevNexus #RedHat
    This is not
    the answer.
    

    View Slide

26. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “Many of the truths that
    we cling to depend on our
    viewpoint.”
    
    
    — Obi-Wan Kenobi, Return of the Jedi
    

    View Slide

27. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “In my book, experience
    outranks everything.”
    
    
    — Captain Rex, The Clone Wars
    

    View Slide

28. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Ok, so how do we try these out?
    

    View Slide

29. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Get good at testing.
    

    View Slide

30. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Get good at testing.
    And automation.
    

    View Slide

31. View Slide

32. end-to-end tests
    high realism
    high value
    

    View Slide

33. end-to-end tests
    unit tests
    low effort
    high realism
    high value
    

    View Slide

34. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism
    high value
    

    View Slide

35. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism
    

    View Slide

36. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism
    tests with
    application
    server
    test REST
    endpoints
    tests with a
    database
    

    View Slide

37. @holly_cummins @edeandrea
    #DevNexus #RedHat
    their code
    our code
    Why mocks aren’t enough
    

    View Slide

38. @holly_cummins @edeandrea
    #DevNexus #RedHat
    our mock
    our code
    Why mocks aren’t enough
    

    View Slide

39. @holly_cummins @edeandrea
    #DevNexus #RedHat
    our mock
    our code
    ✔ tests
    Why mocks aren’t enough
    

    View Slide

40. @holly_cummins @edeandrea
    #DevNexus #RedHat
    our mock
    our code
    Why mocks aren’t enough
    

    View Slide

41. @holly_cummins @edeandrea
    #DevNexus #RedHat
    our mock
    our code
    ✔ tests
    
    
    ✘ reality
    our code their actual code
    Why mocks aren’t enough
    

    View Slide

42. @holly_cummins @edeandrea
    #DevNexus #RedHat
    

    View Slide

43. @holly_cummins @edeandrea
    #DevNexus #RedHat
    https://www.flickr.com/photos/jdhancock/4811282106
    Source: https://medium.com/jung-skywalker/star-wars-symbology-luke-han-and-how-to-face-the-abyss-e9fa966319b5
    

    View Slide

44. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Demo
    
    
    The limitations of unit tests
    https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
    

    View Slide

45. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “Will someone get
    this big walking
    carpet out of my
    way?”
    
    
    – Princess Leia
    Demo
    
    
    The limitations of unit tests
    https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
    

    View Slide

46. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “Will someone get
    this big walking
    carpet out of my
    way?”
    
    
    – Princess Leia
    Demo
    
    
    The limitations of unit tests
    https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo
    

    View Slide

47. @holly_cummins @edeandrea
    #DevNexus #RedHat
    A potential solution
    

    View Slide

48. @holly_cummins @edeandrea
    #DevNexus #RedHat
    How to test a fire alarm?
    

    View Slide

49. @holly_cummins @edeandrea
    #DevNexus #RedHat
    

    View Slide

50. @holly_cummins @edeandrea
    #DevNexus #RedHat
    How NOT to
    test a fire alarm!
    

    View Slide

51. @holly_cummins @edeandrea
    #DevNexus #RedHat
    

    View Slide

52. View Slide

53. Unit testing a
    
    
    fire alarm
    

    View Slide

54. @holly_cummins @edeandrea
    #DevNexus #RedHat
    uh … is that enough?
    

    View Slide

55. @holly_cummins @edeandrea
    #DevNexus #RedHat
    

    View Slide

56. @holly_cummins @edeandrea
    #DevNexus #RedHat
    contract testing a
    fire alarm
    

    View Slide

57. Photo: https://www.westcoastfirepits.com/shop/death-star
    

    View Slide

58. Photo: https://www.westcoastfirepits.com/shop/death-star
    How to SAFELY burn a Death Star!
    

    View Slide

59. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism
    high value
    

    View Slide

60. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism
    high value
    contract tests
    

    View Slide

61. @holly_cummins @edeandrea
    #DevNexus #RedHat
    our code their code
    A better mock
    

    View Slide

62. @holly_cummins @edeandrea
    #DevNexus #RedHat
    our code their code
    contract test
    mock functional test
    A better mock
    

    View Slide

63. @holly_cummins @edeandrea
    #DevNexus #RedHat
    ✔ our tests
    
    
    ✔ their tests
    
    
    ✔ reality
    our code their code
    contract test
    mock functional test
    A better mock
    

    View Slide

64. @holly_cummins @edeandrea
    #DevNexus #RedHat
    our code their code
    A better mock
    

    View Slide

65. @holly_cummins @edeandrea
    #DevNexus #RedHat
    our code their code
    contract test
    mock functional test
    A better mock
    

    View Slide

66. @holly_cummins @edeandrea
    #DevNexus #RedHat
    ✔ our tests
    
    
    ✘ their tests
    
    
    ✘ reality
    our code their code
    contract test
    mock functional test
    A better mock
    

    View Slide

67. @holly_cummins @edeandrea
    #DevNexus #RedHat
    A better mock
    our code their code
    

    View Slide

68. @holly_cummins @edeandrea
    #DevNexus #RedHat
    A better mock
    our code their code
    contract test
    mock functional test
    

    View Slide

69. @holly_cummins @edeandrea
    #DevNexus #RedHat
    ✘ our tests
    
    
    ✔ their tests
    
    
    ✘ reality
    A better mock
    our code their code
    contract test
    mock functional test
    

    View Slide

70. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Some contract
    testing options
    

    View Slide

71. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Some contract
    testing options
    ● Pact
    ● Spring Cloud Contract
    ● OpenAPI schema validation + mock
    generation
    
    
    ● Prism
    
    
    ● Schemathesis
    

    View Slide

72. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Pact Contract Testing
    

    View Slide

73. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Demo
    

    View Slide

74. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “Help me, Pact!
    
    
    You’re my only hope!”
    
    
    – Princess Leia
    
    
    (almost)
    Demo
    

    View Slide

75. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Consumer-driven contract testing
    

    View Slide

76. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “Uh, you’re telling me that consumers
    can break a provider’s tests?”
    

    View Slide

77. View Slide

78. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “contract-first”
    
    
    (It’s like TDD, but between teams)
    

    View Slide

79. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Is Pact always the best choice?
    

    View Slide

80. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Consumer and provider do not have good communication
    

    View Slide

81. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Consumer and provider do not have good communication
    

    View Slide

82. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Team maintaining the “other side” is not using Pact
    Consumer and provider do not have good communication
    P ! P
    

    View Slide

83. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Consumers can’t be identified (i.e. for public APIs)
    
    
    Consumer and provider do not have good communication
    Team maintaining the “other side” is not using Pact
    

    View Slide

84. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Can’t control data being used for provider responses
    Consumer and provider do not have good communication
    Team maintaining the “other side” is not using Pact
    Consumers can’t be identified (i.e. for public APIs)
    
    
    

    View Slide

85. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “Pass-through” APIs
    Consumer and provider do not have good communication
    Team maintaining the “other side” is not using Pact
    Consumers can’t be identified (i.e. for public APIs)
    
    
    Can’t control data being used for provider responses
    

    View Slide

86. @holly_cummins @edeandrea
    #DevNexus #RedHat
    “Pass-through” APIs
    Consumer and provider do not have good communication
    Team maintaining the “other side” is not using Pact
    Consumers can’t be identified (i.e. for public APIs)
    
    
    Can’t control data being used for provider responses
    

    View Slide

87. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Functional, performance, or load testing
    Consumer and provider do not have good communication
    Team maintaining the “other side” is not using Pact
    Consumers can’t be identified (i.e. for public APIs)
    
    
    Can’t control data being used for provider responses
    “Pass-through” APIs
    

    View Slide

88. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Functional, performance, or load testing
    Consumer and provider do not have good communication
    Team maintaining the “other side” is not using Pact
    Consumers can’t be identified (i.e. for public APIs)
    
    
    Can’t control data being used for provider responses
    “Pass-through” APIs
    

    View Slide

89. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Pact vs Spring
    Cloud Contract?
    

    View Slide

90. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Pact vs Spring
    Cloud Contract?
    ● Pact is polyglot!
    ● Pact generates JSON acceptance
    contracts (pacts)
    ● Pact is consumer-driven whereas SCC is
    provider-driven
    ● Pact allows consumers to specify
    provider test states
    ● Pact broker
    

    View Slide

91. @holly_cummins @edeandrea
    #DevNexus #RedHat
    …but I use Swagger/OpenAPI
    It has the same syntax, but the
    semantics are different.
    

    View Slide

92. @holly_cummins @edeandrea
    #DevNexus #RedHat
    …but I use Swagger/OpenAPI
    It has the same syntax, but the
    semantics are different.
    It looks the same, but the
    behaviour is different.
    

    View Slide

93. @holly_cummins @edeandrea
    #DevNexus #RedHat
    …but I use Swagger/OpenAPI
    It has the same syntax, but the
    semantics are different.
    It looks the same, but the
    behaviour is different.
    Looks like a storm trooper.
    
    
    Behaves like Luke.
    (because it it is Luke)
    

    View Slide

94. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Demo:
    
    
    Semantic testing
    with Pact
    

    View Slide

95. @holly_cummins @edeandrea
    #DevNexus #RedHat
    how annoying, the provider is returning 204. the contract
    expects 404, so my test is broken.
    quarkus will return 204 by default.
    204 vs 404 - it’s more about the theory behind REST...if a
    caller asks for a resource the server doesn't have, it should
    be a 404 - same way if a user types in an invalid URL into
    the browser
    ugh, it’s too late at night to care about REST theory.
    
    
    I’m just going to update the expectations in the test to
    match reality.
    what?! why is my other contract test broken now?
    

    oh … because what I’m changing is a mock….
    Based on a true story….
    

    View Slide

96. @holly_cummins @edeandrea
    #DevNexus #RedHat ®
    Thank you!
    
    
    @holly_cummins
    

    @edeandrea
    

    View Slide

97. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Additional Resources
    

    View Slide

98. @holly_cummins @edeandrea
    #DevNexus #RedHat
    https://github.com/quarkusio/quarkus-super-heroes
    More realistic sample application
    
    
    (fewer wookies)
    

    View Slide

99. @holly_cummins @edeandrea
    #DevNexus #RedHat
    Kubernetes Native Java with Quarkus
    Deployment
    Developer Joy
    ● Quickly & easily build Kubernetes-Native Java applications
    
    
    ● Industry leader in startup time and memory utilization for native and
    JVM applications
    
    
    ● One step native executable generation
    ● Leverage existing Java, Jakarta, MicroProfile, & Spring expertise
    
    
    ● Imperative & reactive APIs side-by-side
    
    
    ● Live coding with zero config
    https://www.infoq.com/articles/native-java-quarkus
    

    View Slide

100. @holly_cummins @edeandrea
     #DevNexus #RedHat
     https://www.oreilly.com/library/view/reactive-systems-in/9781492091714
     
     
     https://developers.redhat.com/e-books/understanding-quarkus
     
     
     https://developers.redhat.com/e-books/practising-quarkus
     

     View Slide