How to avoid common pitfalls with modern microservices testing

Slide 1

Slide 1 text

@holly_cummins @edeandrea #Devoxx #RedHat Avoiding common pitfalls with modern microservices testing Holly Cummins Sr. Principal Software Engineer Red Hat @holly_cummins Eric Deandrea Sr. Principal Developer Advocate Red Hat @edeandrea

Slide 2

Slide 2 text

@holly_cummins @edeandrea #Devoxx #RedHat Free copy (while they last)! Come see Eric at the Red Hat booth   (after the talk, not now) https://red.ht/quarkus-spring-devs

Slide 3

Slide 3 text

@holly_cummins @edeandrea #Devoxx #RedHat Disclaimer

Slide 4

Slide 4 text

@holly_cummins @edeandrea #Devoxx #RedHat Disclaimer We are not testers or even test experts

Slide 5

Slide 5 text

@holly_cummins @edeandrea #Devoxx #RedHat Disclaimer We are not testers or even test experts But that’s ok - testing is every developer’s job!

Slide 6

Slide 6 text

@holly_cummins @edeandrea #Devoxx #RedHat So what’s this talk about?

Slide 7

Slide 7 text

@holly_cummins @edeandrea #Devoxx #RedHat So what’s this talk about? Little Red Riding Hood visits the Death Star while trying not to burn your house down on the way to Egypt!

Slide 8

Slide 8 text

@holly_cummins @edeandrea #Devoxx #RedHat Image: https://www.freepik.com/free-photo/close-up-little-red-riding-hood_20031614.htm

Slide 9

Slide 9 text

@holly_cummins @edeandrea #Devoxx #RedHat

Slide 10

Slide 10 text

@holly_cummins @edeandrea #Devoxx #RedHat Death star

Slide 11

Slide 11 text

@holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture “That’s no moon. It’s our architecture.”

Slide 12

Slide 12 text

@holly_cummins @edeandrea #Devoxx #RedHat Fallacies of microservices

Slide 13

Slide 13 text

@holly_cummins @edeandrea #Devoxx #RedHat “Our architecture is modern so it can never fail.”

Slide 14

Slide 14 text

@holly_cummins @edeandrea #Devoxx #RedHat “Our architecture is modern so it can never fail.” "We've analyzed their attack, sir, and there is a danger. Should I have your ship standing by?” “Evacuate? In our moment of triumph? I think you overestimate their chances."

Slide 15

Slide 15 text

@holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices is easy! The thing I have to test is so small!

Slide 16

Slide 16 text

@holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices is easy! Everything is decoupled!

Slide 17

Slide 17 text

@holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices is easy! Everything is decoupled!

Slide 18

Slide 18 text

@holly_cummins @edeandrea #Devoxx #RedHat Netflix microservice architecture Testing with microservices is easy! Everything is decoupled! But….. if you only test the points, and not the lines … does the thing actually work?

Slide 19

Slide 19 text

@holly_cummins @edeandrea #Devoxx #RedHat distributed != decoupled

Slide 20

Slide 20 text

@holly_cummins @edeandrea #Devoxx #RedHat “Ohhhh, we weren’t expecting your service to do that….“

Slide 21

Slide 21 text

@holly_cummins @edeandrea #Devoxx #RedHat

Slide 22

Slide 22 text

@holly_cummins @edeandrea #Devoxx #RedHat “Every time we touch one microservice, all the others break.”

Slide 23

Slide 23 text

@holly_cummins @edeandrea #Devoxx #RedHat

Slide 24

Slide 24 text

@holly_cummins @edeandrea #Devoxx #RedHat Ok, so how do we do microservices?

Slide 25

Slide 25 text

@holly_cummins @edeandrea #Devoxx #RedHat Get good at automation!

Slide 26

Slide 26 text

@holly_cummins @edeandrea #Devoxx #RedHat Get good at automation! And testing!

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

@holly_cummins @edeandrea #Devoxx #RedHat their code our code Why mocks aren’t enough

Slide 36

Slide 36 text

@holly_cummins @edeandrea #Devoxx #RedHat our mock our code Why mocks aren’t enough

Slide 37

Slide 37 text

@holly_cummins @edeandrea #Devoxx #RedHat our mock our code ✔ tests Why mocks aren’t enough

Slide 38

Slide 38 text

@holly_cummins @edeandrea #Devoxx #RedHat our mock our code Why mocks aren’t enough

Slide 39

Slide 39 text

@holly_cummins @edeandrea #Devoxx #RedHat our mock our code ✔ tests ✘ reality our code their actual code Why mocks aren’t enough

Slide 40

Slide 40 text

@holly_cummins @edeandrea #Devoxx #RedHat Demo: The limitations of unit tests

Slide 41

Slide 41 text

@holly_cummins @edeandrea #Devoxx #RedHat Quarkus Superheroes https://github.com/quarkusio/quarkus-super-heroes

Slide 42

Slide 42 text

@holly_cummins @edeandrea #Devoxx #RedHat Quarkus Superheroes https://github.com/quarkusio/quarkus-super-heroes

Slide 43

Slide 43 text

@holly_cummins @edeandrea #Devoxx #RedHat A potential solution

Slide 44

Slide 44 text

@holly_cummins @edeandrea #Devoxx #RedHat How to test a fire alarm?

Slide 45

Slide 45 text

@holly_cummins @edeandrea #Devoxx #RedHat

Slide 46

Slide 46 text

@holly_cummins @edeandrea #Devoxx #RedHat How NOT to test a fire alarm!

Slide 47

Slide 47 text

@holly_cummins @edeandrea #Devoxx #RedHat

Slide 48

Slide 48 text

@holly_cummins @edeandrea #Devoxx #RedHat Unit testing a fire alarm

Slide 49

Slide 49 text

@holly_cummins @edeandrea #Devoxx #RedHat Unit testing a fire alarm

Slide 50

Slide 50 text

@holly_cummins @edeandrea #Devoxx #RedHat uh … is that enough?

Slide 51

Slide 51 text

@holly_cummins @edeandrea #Devoxx #RedHat

Slide 52

Slide 52 text

@holly_cummins @edeandrea #Devoxx #RedHat contract testing a fire alarm

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

@holly_cummins @edeandrea #Devoxx #RedHat our code their code Why mocks aren’t enough

Slide 56

Slide 56 text

@holly_cummins @edeandrea #Devoxx #RedHat our code their code contract test mock functional test Why mocks aren’t enough

Slide 57

Slide 57 text

@holly_cummins @edeandrea #Devoxx #RedHat ✔ our tests ✔ their tests ✔ reality our code their code contract test mock functional test Why mocks aren’t enough

Slide 58

Slide 58 text

@holly_cummins @edeandrea #Devoxx #RedHat our code their code Why mocks aren’t enough

Slide 59

Slide 59 text

@holly_cummins @edeandrea #Devoxx #RedHat our code their code contract test mock functional test Why mocks aren’t enough

Slide 60

Slide 60 text

@holly_cummins @edeandrea #Devoxx #RedHat ✔ our tests ✘ their tests ✘ reality our code their code contract test mock functional test Why mocks aren’t enough

Slide 61

Slide 61 text

@holly_cummins @edeandrea #Devoxx #RedHat Why mocks aren’t enough our code their code

Slide 62

Slide 62 text

@holly_cummins @edeandrea #Devoxx #RedHat Why mocks aren’t enough our code their code contract test mock functional test

Slide 63

Slide 63 text

@holly_cummins @edeandrea #Devoxx #RedHat ✘ our tests ✔ their tests ✘ reality Why mocks aren’t enough our code their code contract test mock functional test

Slide 64

Slide 64 text

@holly_cummins @edeandrea #Devoxx #RedHat Some Contract Testing options

Slide 65

Slide 65 text

@holly_cummins @edeandrea #Devoxx #RedHat Some Contract Testing options ● Pact ● Spring Cloud Contract ● OpenAPI schema validation + mock generation ● Prism ● Schemathesis

Slide 66

Slide 66 text

@holly_cummins @edeandrea #Devoxx #RedHat Pact Contract Testing

Slide 67

Slide 67 text

@holly_cummins @edeandrea #Devoxx #RedHat Demo: How Pact saves our bacon!

Slide 68

Slide 68 text

@holly_cummins @edeandrea #Devoxx #RedHat Consumer-driven contract testing

Slide 69

Slide 69 text

@holly_cummins @edeandrea #Devoxx #RedHat uh, you’re telling me that consumers can break a provider’s tests?

Slide 70

Slide 70 text

Slide 71

Slide 71 text

@holly_cummins @edeandrea #Devoxx #RedHat “contract-first” (it’s like TDD, but between teams)

Slide 72

Slide 72 text

@holly_cummins @edeandrea #Devoxx #RedHat Is Pact always the best choice?

Slide 73

Slide 73 text

@holly_cummins @edeandrea #Devoxx #RedHat Consumer and provider do not have good communication

Slide 74

Slide 74 text

@holly_cummins @edeandrea #Devoxx #RedHat Consumer and provider do not have good communication

Slide 75

Slide 75 text

@holly_cummins @edeandrea #Devoxx #RedHat Team maintaining the “other side” is not using Pact Consumer and provider do not have good communication P ! P

Slide 76

Slide 76 text

@holly_cummins @edeandrea #Devoxx #RedHat Consumers can’t be identified (i.e. for public APIs) Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact

Slide 77

Slide 77 text

@holly_cummins @edeandrea #Devoxx #RedHat Can’t control data being used for provider responses Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs)

Slide 78

Slide 78 text

@holly_cummins @edeandrea #Devoxx #RedHat “Pass-through” APIs Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses

Slide 79

Slide 79 text

Slide 80

Slide 80 text

@holly_cummins @edeandrea #Devoxx #RedHat Functional, performance, or load testing Consumer and provider do not have good communication Team maintaining the “other side” is not using Pact Consumers can’t be identified (i.e. for public APIs) Can’t control data being used for provider responses “Pass-through” APIs

Slide 81

Slide 81 text

Slide 82

Slide 82 text

@holly_cummins @edeandrea #Devoxx #RedHat Pact vs Spring Cloud Contract?

Slide 83

Slide 83 text

@holly_cummins @edeandrea #Devoxx #RedHat Pact vs Spring Cloud Contract? ● Pact is polyglot! ● Pact generates JSON acceptance contracts (Pacts) ● Pact is consumer-driven whereas SCC is provider- driven ● Pact allows consumers to specify provider test states ● Pact broker

Slide 84

Slide 84 text

@holly_cummins @edeandrea #Devoxx #RedHat …but I use Swagger/OpenAPI “My, what large teeth you have, Grandma” It looks the same, but the semantics are different.

Slide 85

Slide 85 text

@holly_cummins @edeandrea #Devoxx #RedHat Demo: More complex testing with Pact

Slide 86

Slide 86 text

@holly_cummins @edeandrea #Devoxx #RedHat how annoying, the provider is returning 204. the contract expects 404, so my test is broken. quarkus will return 204 by default. 204 vs 404 - it’s more about the theory behind REST...if a caller asks for a resource the server doesn't have, it should be a 404 - same way if a user types in an invalid URL into the browser ugh, it’s too late at night to care about REST theory. I’m just going to update the expectations in the test to match reality. what?! why is my other contract test broken now?   oh … because what I’m changing is a mock…. Based on a true story….

Slide 87

Slide 87 text

@holly_cummins @edeandrea #Devoxx #RedHat

Slide 88

Slide 88 text

@holly_cummins @edeandrea #Devoxx #RedHat ® Thank you! @holly_cummins   @edeandrea

Slide 89

Slide 89 text

@holly_cummins @edeandrea #Devoxx #RedHat Additional Resources

Slide 90

Slide 90 text

@holly_cummins @edeandrea #Devoxx #RedHat Kubernetes Native Java with Quarkus Deployment Developer Joy ● Quickly & easily build Kubernetes-Native Java applications ● Industry leader in startup time and memory utilization for native and JVM applications ● One step native executable generation ● Leverage existing Java, Jakarta, MicroProfile, & Spring expertise ● Imperative & reactive APIs side-by-side ● Live coding with zero config https://www.infoq.com/articles/native-java-quarkus

Slide 91

Slide 91 text

@holly_cummins @edeandrea #Devoxx #RedHat https://www.oreilly.com/library/view/reactive-systems-in/9781492091714 https://developers.redhat.com/e-books/understanding-quarkus https://developers.redhat.com/e-books/practising-quarkus

Slide 92

Slide 92 text

@holly_cummins @edeandrea #Devoxx #RedHat