Upgrade to Pro — share decks privately, control downloads, hide ads and more …

7-11-23 - [RivieraDev] - Avoiding Common Pitfalls with Modern Microservices Testing

7-11-23 - [RivieraDev] - Avoiding Common Pitfalls with Modern Microservices Testing

Have you ever wondered why your microservices break in production after all the tests have passed during CI/CD? Have you ever wondered if you’re testing too much or too little? If you want to learn about common pitfalls and how to catch them at build time rather than at runtime, this session is for you!

In this session, I’ll share some common battle scars from my experiences in the field. Additionally, I will introduce the testing pyramid and explain why it is important in helping to continuously deploy microservices independently without fear. I will demonstrate, through a test-driven development approach, how the combination of Pact contract testing combined with Quarkus developer joy can help prevent your microservices from breaking in production.

Eric Deandrea

July 10, 2023
Tweet

More Decks by Eric Deandrea

Other Decks in Technology

Transcript

  1. Eric Deandrea


    Sr. Principal Developer Advocate


    Red Hat


    @edeandrea
    Avoiding common pitfalls with


    modern microservices testing


    Drawings courtesy of Holly Cummins

    View full-size slide

  2. @edeandrea
    #RedHat #RivieraDev2023

    View full-size slide

  3. @edeandrea
    #RedHat #RivieraDev2023
    “Every time we touch one microservice,
    all the others break.”

    View full-size slide

  4. @edeandrea
    #RedHat #RivieraDev2023

    View full-size slide

  5. @edeandrea
    #RedHat #RivieraDev2023
    “My end-to-end test environment is only up the
    third weekend every month.”

    View full-size slide

  6. @edeandrea
    #RedHat #RivieraDev2023
    Disclaimer!

    View full-size slide

  7. @edeandrea
    #RedHat #RivieraDev2023
    Disclaimer!
    I am not a tester.
    (or even test expert)

    View full-size slide

  8. @edeandrea
    #RedHat #RivieraDev2023
    Disclaimer!
    I am not a tester.
    (or even test expert)
    That’s ok– testing is
    every developer’s job!

    View full-size slide

  9. @edeandrea
    #RedHat #RivieraDev2023
    Why is testing microservices so hard?

    View full-size slide

  10. @edeandrea
    #RedHat #RivieraDev2023
    Netflix microservice
    architecture

    View full-size slide

  11. @edeandrea
    #RedHat #RivieraDev2023
    Death star
    “That’s no moon.
    It’s our architecture.”

    View full-size slide

  12. @edeandrea
    #RedHat #RivieraDev2023
    Do microservices solve all problems?

    View full-size slide

  13. @edeandrea
    #RedHat #RivieraDev2023
    A microservice system: a portrait

    View full-size slide

  14. @edeandrea
    #RedHat #RivieraDev2023
    Netflix microservice
    architecture

    View full-size slide

  15. @edeandrea
    #RedHat #RivieraDev2023
    Netflix microservice
    architecture
    Testing with microservices is easy!

    View full-size slide

  16. @edeandrea
    #RedHat #RivieraDev2023
    Netflix microservice
    architecture
    Testing with microservices is easy!
    My services are so small!
    Everything is decoupled!

    View full-size slide

  17. @edeandrea
    #RedHat #RivieraDev2023
    Netflix microservice
    architecture
    Testing with microservices is easy!
    My services are so small!
    Everything is decoupled!
    But…..
    If you only test the points,
    And not the lines….
    Does the thing actually work?

    View full-size slide

  18. @edeandrea
    #RedHat #RivieraDev2023
    distributed != decoupled

    View full-size slide

  19. @edeandrea
    #RedHat #RivieraDev2023
    Understand the coupling

    View full-size slide

  20. @edeandrea
    #RedHat #RivieraDev2023
    “It is all that is left unsaid upon
    which tragedies are built.”


    - Kreia, Star Wars Knights of the Old Republic II: The Sith Lords

    View full-size slide

  21. @edeandrea
    #RedHat #RivieraDev2023

    View full-size slide

  22. @edeandrea
    #RedHat #RivieraDev2023
    This is not
    the answer.

    View full-size slide

  23. @edeandrea
    #RedHat #RivieraDev2023
    “You lied to me.


    How many other lies have I been
    told by the Council?


    And how do you know that you
    even have the truth?”


    – Anakin Skywalker

    View full-size slide

  24. @edeandrea
    #RedHat #RivieraDev2023
    “Many of the truths that
    we cling to depend on our
    viewpoint.”


    — Obi-Wan Kenobi, Return of the Jedi

    View full-size slide

  25. @edeandrea
    #RedHat #RivieraDev2023
    “In my book, experience
    outranks everything.”


    — Captain Rex, The Clone Wars

    View full-size slide

  26. @edeandrea
    #RedHat #RivieraDev2023
    Ok, so how do we try these out?

    View full-size slide

  27. @edeandrea
    #RedHat #RivieraDev2023
    Get good at testing.

    View full-size slide

  28. @edeandrea
    #RedHat #RivieraDev2023
    Get good at testing.
    And automation.

    View full-size slide

  29. end-to-end tests
    high realism
    high value

    View full-size slide

  30. end-to-end tests
    unit tests
    low effort
    high realism
    high value

    View full-size slide

  31. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism
    high value

    View full-size slide

  32. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism

    View full-size slide

  33. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism
    tests with
    application
    server
    test REST
    endpoints
    tests with a
    database

    View full-size slide

  34. @edeandrea
    #RedHat #RivieraDev2023
    their code
    our code
    Why mocks aren’t enough

    View full-size slide

  35. @edeandrea
    #RedHat #RivieraDev2023
    our mock
    our code
    Why mocks aren’t enough

    View full-size slide

  36. @edeandrea
    #RedHat #RivieraDev2023
    our mock
    our code
    ✔ tests
    Why mocks aren’t enough

    View full-size slide

  37. @edeandrea
    #RedHat #RivieraDev2023
    our mock
    our code
    Why mocks aren’t enough

    View full-size slide

  38. @edeandrea
    #RedHat #RivieraDev2023
    our mock
    our code
    ✔ tests


    ✘ reality
    our code their actual code
    Why mocks aren’t enough

    View full-size slide

  39. @edeandrea
    #RedHat #RivieraDev2023
    a mock
    stormtrooper

    View full-size slide

  40. @edeandrea
    #RedHat #RivieraDev2023
    a mock
    stormtrooper
    “Aren’t you a little short
    for a stormtrooper?”


    – Princess Leia

    View full-size slide

  41. @edeandrea
    #RedHat #RivieraDev2023
    https://www.flickr.com/photos/jdhancock/4811282106
    Source: https://medium.com/jung-skywalker/star-wars-symbology-luke-han-and-how-to-face-the-abyss-e9fa966319b5

    View full-size slide

  42. @edeandrea
    #RedHat #RivieraDev2023
    Demo


    The limitations of unit tests
    https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo

    View full-size slide

  43. @edeandrea
    #RedHat #RivieraDev2023
    “Will someone get
    this big walking
    carpet out of my
    way?”


    – Princess Leia
    Demo


    The limitations of unit tests
    https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo

    View full-size slide

  44. @edeandrea
    #RedHat #RivieraDev2023
    “Will someone get
    this big walking
    carpet out of my
    way?”


    – Princess Leia
    Demo


    The limitations of unit tests
    https://github.com/edeandrea/pact-quarkus-wookie-carpet-demo

    View full-size slide

  45. @edeandrea
    #RedHat #RivieraDev2023
    A potential solution

    View full-size slide

  46. @edeandrea
    #RedHat #RivieraDev2023
    How to test a fire alarm?

    View full-size slide

  47. @edeandrea
    #RedHat #RivieraDev2023

    View full-size slide

  48. @edeandrea
    #RedHat #RivieraDev2023
    How NOT to
    test a fire alarm!

    View full-size slide

  49. @edeandrea
    #RedHat #RivieraDev2023

    View full-size slide

  50. Unit testing a


    fire alarm

    View full-size slide

  51. @edeandrea
    #RedHat #RivieraDev2023
    uh … is that enough?

    View full-size slide

  52. @edeandrea
    #RedHat #RivieraDev2023

    View full-size slide

  53. @edeandrea
    #RedHat #RivieraDev2023
    contract testing a
    fire alarm

    View full-size slide

  54. Photo: https://www.westcoastfirepits.com/shop/death-star

    View full-size slide

  55. Photo: https://www.westcoastfirepits.com/shop/death-star
    How to SAFELY burn a Death Star!

    View full-size slide

  56. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism
    high value

    View full-size slide

  57. end-to-end tests
    unit tests
    integration tests
    low effort
    high realism
    high value
    contract tests

    View full-size slide

  58. @edeandrea
    #RedHat #RivieraDev2023
    our code their code
    A better mock

    View full-size slide

  59. @edeandrea
    #RedHat #RivieraDev2023
    our code their code
    contract test
    mock functional test
    A better mock

    View full-size slide

  60. @edeandrea
    #RedHat #RivieraDev2023
    ✔ our tests


    ✔ their tests


    ✔ reality
    our code their code
    contract test
    mock functional test
    A better mock

    View full-size slide

  61. @edeandrea
    #RedHat #RivieraDev2023
    our code their code
    A better mock

    View full-size slide

  62. @edeandrea
    #RedHat #RivieraDev2023
    our code their code
    contract test
    mock functional test
    A better mock

    View full-size slide

  63. @edeandrea
    #RedHat #RivieraDev2023
    ✔ our tests


    ✘ their tests


    ✘ reality
    our code their code
    contract test
    mock functional test
    A better mock

    View full-size slide

  64. @edeandrea
    #RedHat #RivieraDev2023
    A better mock
    our code their code

    View full-size slide

  65. @edeandrea
    #RedHat #RivieraDev2023
    A better mock
    our code their code
    contract test
    mock functional test

    View full-size slide

  66. @edeandrea
    #RedHat #RivieraDev2023
    ✘ our tests


    ✔ their tests


    ✘ reality
    A better mock
    our code their code
    contract test
    mock functional test

    View full-size slide

  67. @edeandrea
    #RedHat #RivieraDev2023
    Pact Contract Testing

    View full-size slide

  68. @edeandrea
    #RedHat #RivieraDev2023
    Demo

    View full-size slide

  69. @edeandrea
    #RedHat #RivieraDev2023
    “Help me, Pact!


    You’re my only hope!”


    – Princess Leia


    (almost)
    Demo

    View full-size slide

  70. @edeandrea
    #RedHat #RivieraDev2023
    “Uh, you’re telling me that consumers
    can break a provider’s tests?”

    View full-size slide

  71. @edeandrea
    #RedHat #RivieraDev2023
    Consumer-driven contract testing

    View full-size slide

  72. @edeandrea
    #RedHat #RivieraDev2023
    consumer
    Consumer-driven contract testing

    View full-size slide

  73. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    mock
    declare (Pact)
    Pact file
    Consumer-driven contract testing
    Spring Cloud Contract
    Pact

    View full-size slide

  74. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    declare (Pact)
    contract
    tests
    verify
    Pact file
    Consumer-driven contract testing
    Spring Cloud Contract
    Pact
    Pact

    View full-size slide

  75. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    declare (Pact)
    contract
    tests
    verify
    Pact file
    Consumer-driven contract testing
    ☑ rich semantic testing
    Spring Cloud Contract
    Pact
    Pact

    View full-size slide

  76. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    declare (Pact)
    contract
    tests
    verify
    Pact file
    Consumer-driven contract testing
    ☑ rich semantic testing
    ☑ provider can ‘develop to the contract’ instead of
    developing speculatively (TDD)
    Spring Cloud Contract
    Pact
    Pact

    View full-size slide

  77. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    declare (Pact)
    contract
    tests
    verify
    Pact file
    Consumer-driven contract testing
    ☑ rich semantic testing
    ☑ provider can ‘develop to the contract’ instead of
    developing speculatively (TDD)
    ⛔ existing spec assets cannot be re-purposed
    Spring Cloud Contract
    Pact
    Pact

    View full-size slide

  78. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    declare (Pact)
    contract
    tests
    verify
    Pact file
    Consumer-driven contract testing
    ☑ rich semantic testing
    ☑ provider can ‘develop to the contract’ instead of
    developing speculatively (TDD)
    ⛔ existing spec assets cannot be re-purposed
    ⛔ provider needs to know who consumers are
    Spring Cloud Contract
    Pact
    Pact

    View full-size slide

  79. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    declare (Pact)
    contract
    tests
    verify
    Pact file
    Consumer-driven contract testing
    ☑ rich semantic testing
    ☑ provider can ‘develop to the contract’ instead of
    developing speculatively (TDD)
    ⛔ existing spec assets cannot be re-purposed
    ⛔ provider needs to know who consumers are
    ⛔ consumers can break provider CI
    Spring Cloud Contract
    Pact
    Pact

    View full-size slide

  80. @edeandrea
    #RedHat #RivieraDev2023
    …but I want to just use Swagger/OpenAPI

    View full-size slide

  81. @edeandrea
    #RedHat #RivieraDev2023
    Is Pact always the best choice?

    View full-size slide

  82. @edeandrea
    #RedHat #RivieraDev2023
    Provider-driven contract testing

    View full-size slide

  83. @edeandrea
    #RedHat #RivieraDev2023
    consumer provider
    Provider-driven contract testing

    View full-size slide

  84. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer provider
    OpenAPI
    Provider-driven contract testing
    generate


    (OpenAPI snapshot of
    current implementation)
    other protocol specs

    View full-size slide

  85. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer provider
    contract
    tests
    verify
    OpenAPI
    Provider-driven contract testing
    generate


    (OpenAPI snapshot of
    current implementation)
    other protocol specs
    Schemathesis


    Microcks

    View full-size slide

  86. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    contract
    tests
    verify
    OpenAPI
    Provider-driven contract testing
    generate


    (OpenAPI snapshot of
    current implementation)
    other protocol specs
    Prism
    Microcks
    Schemathesis


    Microcks

    View full-size slide

  87. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    contract
    tests
    verify
    OpenAPI
    Provider-driven contract testing
    generate


    (OpenAPI snapshot of
    current implementation)
    other protocol specs
    ☑ familiar contract format
    Prism
    Microcks
    Schemathesis


    Microcks

    View full-size slide

  88. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    contract
    tests
    verify
    OpenAPI
    Provider-driven contract testing
    generate


    (OpenAPI snapshot of
    current implementation)
    other protocol specs
    ☑ familiar contract format
    ☑ provider does not need comms with consumers
    Prism
    Microcks
    Schemathesis


    Microcks

    View full-size slide

  89. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    contract
    tests
    verify
    OpenAPI
    Provider-driven contract testing
    generate


    (OpenAPI snapshot of
    current implementation)
    other protocol specs
    ☑ familiar contract format
    ☑ provider does not need comms with consumers
    ⛔ provider does not get insight into how API is being
    used (deprecation is harder)
    Prism
    Microcks
    Schemathesis


    Microcks

    View full-size slide

  90. @edeandrea
    #RedHat #RivieraDev2023
    contract


    consumer
    tests
    provider
    mock
    contract
    tests
    verify
    OpenAPI
    Provider-driven contract testing
    generate


    (OpenAPI snapshot of
    current implementation)
    other protocol specs
    ☑ familiar contract format
    ☑ provider does not need comms with consumers
    ⛔ provider does not get insight into how API is being
    used (deprecation is harder)
    ⛔ spec-based tests lack functional depth
    Prism
    Microcks
    Schemathesis


    Microcks

    View full-size slide

  91. @edeandrea
    #RedHat #RivieraDev2023
    Is OpenAPI enough?
    It has the same syntax, but the
    semantics are different.

    View full-size slide

  92. @edeandrea
    #RedHat #RivieraDev2023
    Is OpenAPI enough?
    It has the same syntax, but the
    semantics are different.
    It looks the same, but the
    behaviour is different.

    View full-size slide

  93. @edeandrea
    #RedHat #RivieraDev2023
    Is OpenAPI enough?
    It has the same syntax, but the
    semantics are different.
    It looks the same, but the
    behaviour is different.
    Looks like a storm trooper.


    Behaves like Luke.
    (because it it is Luke)

    View full-size slide

  94. @edeandrea
    #RedHat #RivieraDev2023
    Demo:


    Semantic testing
    with Pact

    View full-size slide

  95. @edeandrea
    #RedHat #RivieraDev2023
    how annoying, the provider is returning 204. the contract
    expects 404, so my test is broken.
    quarkus will return 204 by default.
    204 vs 404 - it’s more about the theory behind REST...if a
    caller asks for a resource the server doesn't have, it should
    be a 404 - same way if a user types in an invalid URL into
    the browser
    ugh, it’s too late at night to care about REST theory.


    I’m just going to update the expectations in the test to
    match reality.
    what?! why is my other contract test broken now?

    oh … because what I’m changing is a mock….
    Based on a true story….

    View full-size slide

  96. @edeandrea
    #RedHat #RivieraDev2023

    View full-size slide

  97. @edeandrea
    #RedHat #RivieraDev2023
    soo …
    microservices testing is hard
    limit end-to-end testing
    mocks will not be enough
    contract testing can rescue you

    View full-size slide

  98. @edeandrea
    #RedHat #RivieraDev2023 ®
    Thank you!


    Merci!


    @edeandrea

    View full-size slide

  99. @edeandrea
    #RedHat #RivieraDev2023
    Feedback Slides

    View full-size slide

  100. @edeandrea
    #RedHat #RivieraDev2023
    Additional Resources

    View full-size slide

  101. @edeandrea
    #RedHat #RivieraDev2023
    https://github.com/quarkusio/quarkus-super-heroes
    More realistic sample application


    (fewer wookies)

    View full-size slide

  102. @edeandrea
    #RedHat #RivieraDev2023
    Kubernetes Native Java with Quarkus
    Deployment
    Developer Joy
    ● Quickly & easily build Kubernetes-Native Java applications


    ● Industry leader in startup time and memory utilization for native and
    JVM applications


    ● One step native executable generation
    ● Leverage existing Java, Jakarta, MicroProfile, & Spring expertise


    ● Imperative & reactive APIs side-by-side


    ● Live coding with zero config
    https://www.infoq.com/articles/native-java-quarkus

    View full-size slide

  103. @edeandrea
    #RedHat #RivieraDev2023
    https://www.oreilly.com/library/view/reactive-systems-in/9781492091714


    https://developers.redhat.com/e-books/understanding-quarkus


    https://developers.redhat.com/e-books/practising-quarkus

    View full-size slide