Contract testing with Pact and Quarkus

Every time you change one microservice, the others break. But you had unit tests! What’s going on? Unit tests aren’t enough to give system-level confidence, even with a microservices architecture. What’s the solution? Integration testing is annoyingly expensive, flaky, and fries your laptop. Remocal development environments are complex to manage. Testing in production is important, but the feedback loop is longer, so it’s not a substitute for local inner-loop testing. Enter contract tests. Contract tests combine the lightness of unit tests with the confidence of integration tests, and should be part of your development toolkit. This session will demo how to use the Pact contract testing framework to catch integration issues early. The demos will be run on Quarkus, but Pact runs on all Java frameworks. In fact, Pact has libraries for almost all of the popular programming languages.

Transcript

1. contract testing with pact and quarkus Holly Cummins Red Hat @holly_cummins@hachyderm.io
2. @holly_cummins the big microservices question
3. @holly_cummins “how do I know the whole system works? the big microservices question
4. @holly_cummins “how do I know the whole system works? … without losing the goodness of microservices?” the big microservices question
5. can you deploy independently? v5.1.4 v5.1.4 v5.1.4 v5.1.4
6. can you deploy independently? v6.2.8 v65.7 v5.1.4 v1.0.5
7. do you have confidence in how individual services are tested? v6.2.8 v65.7 v5.1.4 v1.0.5
8. @holly_cummins #RedHat “we made our pipeline enforce releasing all our microservices at the same time.” true story
9. @holly_cummins #RedHat “we made our pipeline enforce releasing all our microservices at the same time.” true story are you dependent on end-to-end testing?
10. @holly_cummins testing in isolation is not enough
11. @holly_cummins distributed != decoupled
12. @holly_cummins “every time we touch one microservice, all the others break.”
13. @holly_cummins “ohhhh, we weren’t expecting your service to do that … “
14. @holly_cummins “uh, what do you mean you corrected the typo in your json?”
15. @holly_cummins “surely the right way to handle missing data is a 204?”
16. @holly_cummins #RedHat their code our code why mocks aren’t enough
17. @holly_cummins #RedHat our code our mock why mocks aren’t enough
18. @holly_cummins #RedHat our code our mock tests ✔ why mocks aren’t enough
19. @holly_cummins #RedHat our code our mock our code their actual code tests ✔ why mocks aren’t enough
20. @holly_cummins #RedHat our code our mock our code their actual code tests ✔ reality ✘ why mocks aren’t enough
21. @holly_cummins demo: the limitations of unit tests
22. @holly_cummins #RedHat our code their code why mocks aren’t enough
23. @holly_cummins #RedHat our code their code contract test why mocks aren’t enough
24. @holly_cummins #RedHat our code their code contract test mock why mocks aren’t enough
25. @holly_cummins #RedHat our code their code contract test mock functional test why mocks aren’t enough
26. @holly_cummins #RedHat our code their code contract test mock functional test our tests ✔ their tests ✔ reality ✔ why mocks aren’t enough
27. @holly_cummins #RedHat our code their code contract test mock functional test why mocks aren’t enough
28. @holly_cummins #RedHat our code their code contract test mock functional test our tests ✔ their tests ✘ reality ✘ why mocks aren’t enough
29. @holly_cummins #RedHat our code their code contract test mock functional test why mocks aren’t enough
30. @holly_cummins #RedHat our code their code contract test mock functional test our tests ✘ their tests ✔ reality ✘ why mocks aren’t enough
31. IBM Cloud © 2020 IBM Corporation
32. IBM Cloud © 2020 IBM Corporation the test pyramid
33. IBM Cloud © 2020 IBM Corporation the test pyramid end-to-end tests
34. IBM Cloud © 2020 IBM Corporation the test pyramid end-to-end tests unit tests
35. IBM Cloud © 2020 IBM Corporation the test pyramid end-to-end tests unit tests contract tests
36. #RedHat @holly_cummins How to test a fire alarm?
37. IBM Cloud © 2020 IBM Corporation how not to test a fire alarm
38. IBM Cloud © 2020 IBM Corporation how not to test a fire alarm
39. #RedHat @holly_cummins
40. #RedHat @holly_cummins unit testing a fire alarm
41. #RedHat @holly_cummins uh … is that enough?
42. #RedHat @holly_cummins contract testing a fire alarm
43. #RedHat @holly_cummins contract testing a fire alarm
44. @holly_cummins demo: how pact helps
45. @holly_cummins #RedHat if you remember one thing …
46. @holly_cummins #RedHat if you remember one thing … • don’t test the mock! • your consumer test should be testing your code, not what’s coming back from the other side • lots of examples get this wrong :(
47. @holly_cummins #RedHat don’t test the mock.
48. @holly_cummins #RedHat but we use openapi?
49. @holly_cummins demo: semantic testing with pact
50. @holly_cummins #RedHat cold person knitter sheep farmer semantics vs syntax (why sharing expectations about behaviour is important)
51. @holly_cummins #RedHat cold person knitter sheep farmer “white sweater, please” semantics vs syntax (why sharing expectations about behaviour is important)
52. @holly_cummins #RedHat cold person knitter sheep farmer “white sweater, please” semantics vs syntax (why sharing expectations about behaviour is important)
53. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” semantics vs syntax (why sharing expectations about behaviour is important)
54. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” white wool semantics vs syntax (why sharing expectations about behaviour is important)
55. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” white wool semantics vs syntax (why sharing expectations about behaviour is important)
56. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” white wool semantics vs syntax (why sharing expectations about behaviour is important)
57. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” white wool black sweater semantics vs syntax (why sharing expectations about behaviour is important)
58. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” white wool black sweater black wool semantics vs syntax (why sharing expectations about behaviour is important)
59. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” “pink sweater, please” white wool black sweater black wool semantics vs syntax (why sharing expectations about behaviour is important)
60. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” “pink sweater, please” white wool black sweater black wool semantics vs syntax (why sharing expectations about behaviour is important)
61. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” “pink sweater, please” white wool black sweater black wool pink sweater semantics vs syntax (why sharing expectations about behaviour is important)
62. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” “pink sweater, please” white wool black sweater black wool pink sweater pink wool semantics vs syntax (why sharing expectations about behaviour is important)
63. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” “pink sweater, please” white wool black sweater black wool pink sweater pink wool semantics vs syntax (why sharing expectations about behaviour is important) what?! pink sheep don’t exist
64. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” “pink sweater, please” white wool black sweater black wool pink sweater pink wool semantics vs syntax (why sharing expectations about behaviour is important) should the knitter ask for white wool and bring in a dyeing service? what?! pink sheep don’t exist
65. @holly_cummins #RedHat cold person white sweater knitter sheep farmer “white sweater, please” “black sweater, please” “pink sweater, please” white wool black sweater black wool pink sweater pink wool semantics vs syntax (why sharing expectations about behaviour is important) should the knitter ask for white wool and bring in a dyeing service? should the farmer dye the wool? what?! pink sheep don’t exist
66. @holly_cummins #RedHat was this a realistic demo?
67. @holly_cummins #RedHat Photo of sheep at the Latitude Festival by Tim Parkinson, https://www.flickr.com/photos/timparkinson
68. @holly_cummins #RedHat Photo of sheep at the Latitude Festival by Tim Parkinson, https://www.flickr.com/photos/timparkinson “pink sheep don’t exist”
69. @holly_cummins #RedHat how annoying, the provider is returning 204. the contract expects 404, so my test is broken. quarkus will return 204 by default. 204 vs 404 - it’s more about the theory behind REST...if a caller asks for a resource the server doesn't have, it should be a 404 - same way if a user types in an invalid URL into the browser it’s too late at night to care about REST theory. I’m just going to update the expectations in the test to match reality. what?! why is my other contract test broken now? 
 oh … because what I’m changing is a mock…. Based on a true story….
70. @holly_cummins #RedHat contract testing landscape
71. @holly_cummins #RedHat provider contract testing landscape
72. @holly_cummins #RedHat consumer provider contract testing landscape
73. @holly_cummins #RedHat contract consumer provider contract testing landscape
74. @holly_cummins #RedHat contract consumer provider contract tests verify Pact (functional tests) contract testing landscape
75. @holly_cummins #RedHat contract consumer tests provider mock Pact contract tests verify Pact (functional tests) contract testing landscape
76. @holly_cummins #RedHat contract consumer tests provider mock Pact declare (Pact) contract tests verify Pact (functional tests) contract testing landscape Pact file
77. @holly_cummins #RedHat contract consumer tests provider mock Pact declare (Pact) contract tests verify Pact (functional tests) OpenAPI contract testing landscape Pact file
78. @holly_cummins #RedHat contract consumer tests provider mock Pact declare (Pact) contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file
79. @holly_cummins #RedHat contract consumer tests provider mock Pact declare (Pact) contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file Microcks
80. @holly_cummins #RedHat contract consumer tests provider mock Pact Prism (OpenAPI) declare (Pact) contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file Microcks
81. @holly_cummins #RedHat contract consumer tests provider mock Pact Prism (OpenAPI) Microcks (OpenAPI) declare (Pact) contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file Microcks
82. @holly_cummins #RedHat contract consumer tests provider mock Pact Prism (OpenAPI) Microcks (OpenAPI) declare (Pact) contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file mock server OpenAPI + Prism Microcks Microcks
83. @holly_cummins #RedHat contract consumer tests provider mock Pact Prism (OpenAPI) Microcks (OpenAPI) declare (Pact) contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file generate (OpenAPI snapshot of current implementation) mock server OpenAPI + Prism Microcks Microcks
84. @holly_cummins #RedHat contract consumer tests provider mock Pact Prism (OpenAPI) Microcks (OpenAPI) declare (Pact) generate skeleton from OpenAPI contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file generate (OpenAPI snapshot of current implementation) mock server OpenAPI + Prism Microcks Microcks
85. @holly_cummins #RedHat contract consumer tests provider mock Pact Prism (OpenAPI) Microcks (OpenAPI) declare (Pact) generate skeleton from OpenAPI contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file dashboard generate (OpenAPI snapshot of current implementation) mock server OpenAPI + Prism Microcks Microcks
86. @holly_cummins #RedHat contract consumer tests provider mock Pact Prism (OpenAPI) Microcks (OpenAPI) declare (Pact) generate skeleton from OpenAPI contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file dashboard Pact broker generate (OpenAPI snapshot of current implementation) mock server OpenAPI + Prism Microcks Microcks
87. @holly_cummins #RedHat contract consumer tests provider mock Pact Prism (OpenAPI) Microcks (OpenAPI) declare (Pact) generate skeleton from OpenAPI contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file dashboard Pact broker Microcks generate (OpenAPI snapshot of current implementation) mock server OpenAPI + Prism Microcks Microcks
88. @holly_cummins #RedHat contract consumer tests provider mock Pact Prism (OpenAPI) Microcks (OpenAPI) declare (Pact) generate skeleton from OpenAPI contract tests verify Pact (functional tests) OpenAPI OpenAPI + Schemathesis (syntax only, weaker connection to consumer expectations) contract testing landscape Pact file dashboard Pact broker Microcks generate (OpenAPI snapshot of current implementation) mock server OpenAPI + Prism Microcks Microcks other protocol specs
89. @holly_cummins #RedHat provider-driven contract testing
90. @holly_cummins #RedHat provider provider-driven contract testing
91. @holly_cummins #RedHat contract provider provider-driven contract testing generate (OpenAPI snapshot of current implementation)
92. @holly_cummins #RedHat contract provider contract tests verify provider-driven contract testing generate (OpenAPI snapshot of current implementation)
93. @holly_cummins #RedHat contract consumer provider contract tests verify provider-driven contract testing generate (OpenAPI snapshot of current implementation)
94. @holly_cummins #RedHat contract consumer tests provider mock contract tests verify provider-driven contract testing generate (OpenAPI snapshot of current implementation)
95. @holly_cummins #RedHat contract consumer tests provider mock contract tests verify OpenAPI provider-driven contract testing generate (OpenAPI snapshot of current implementation)
96. @holly_cummins #RedHat contract consumer tests provider mock contract tests verify OpenAPI provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs
97. @holly_cummins #RedHat contract consumer tests provider mock contract tests verify OpenAPI provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs + familiar contract format
98. @holly_cummins #RedHat contract consumer tests provider mock contract tests verify OpenAPI provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs + familiar contract format + provider does not need comms with consumers
99. @holly_cummins #RedHat contract consumer tests provider mock contract tests verify OpenAPI provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs + familiar contract format + provider does not need comms with consumers - provider does not get insight into how API is being used
100. @holly_cummins #RedHat contract consumer tests provider mock contract tests verify OpenAPI provider-driven contract testing generate (OpenAPI snapshot of current implementation) other protocol specs + familiar contract format + provider does not need comms with consumers - provider does not get insight into how API is being used - spec-based tests lack depth
101. @holly_cummins #RedHat consumer-driven contract testing
102. @holly_cummins #RedHat consumer consumer-driven contract testing
103. @holly_cummins #RedHat contract consumer tests declare (Pact) Pact file consumer-driven contract testing
104. @holly_cummins #RedHat contract consumer tests mock declare (Pact) Pact file consumer-driven contract testing
105. @holly_cummins #RedHat contract consumer tests provider mock declare (Pact) Pact file consumer-driven contract testing
106. @holly_cummins #RedHat contract consumer tests provider mock declare (Pact) contract tests verify Pact file consumer-driven contract testing
107. @holly_cummins #RedHat contract consumer tests provider mock declare (Pact) contract tests verify OpenAPI Pact file consumer-driven contract testing
108. @holly_cummins #RedHat contract consumer tests provider mock declare (Pact) contract tests verify OpenAPI Pact file other protocol specs consumer-driven contract testing
109. @holly_cummins #RedHat contract consumer tests provider mock declare (Pact) contract tests verify OpenAPI Pact file other protocol specs consumer-driven contract testing + rich semantic testing
110. @holly_cummins #RedHat contract consumer tests provider mock declare (Pact) contract tests verify OpenAPI Pact file other protocol specs consumer-driven contract testing + rich semantic testing + provider can ‘develop to the test’ instead of developing speculatively
111. @holly_cummins #RedHat contract consumer tests provider mock declare (Pact) contract tests verify OpenAPI Pact file other protocol specs consumer-driven contract testing + rich semantic testing + provider can ‘develop to the test’ instead of developing speculatively - existing spec assets cannot be re-purposed
112. @holly_cummins #RedHat contract consumer tests provider mock declare (Pact) contract tests verify OpenAPI Pact file other protocol specs consumer-driven contract testing + rich semantic testing + provider can ‘develop to the test’ instead of developing speculatively - existing spec assets cannot be re-purposed - provider needs to identify consumers
113. @holly_cummins #RedHat contract consumer tests provider mock declare (Pact) contract tests verify OpenAPI Pact file other protocol specs consumer-driven contract testing + rich semantic testing + provider can ‘develop to the test’ instead of developing speculatively - existing spec assets cannot be re-purposed - provider needs to identify consumers - consumers can break provider CI
114. @holly_cummins #RedHat “contract-first”
115. @holly_cummins #RedHat
116. @holly_cummins #RedHat some contract testing options
117. @holly_cummins #RedHat some contract testing options • Pact
118. @holly_cummins #RedHat some contract testing options • Pact • Microcks
119. @holly_cummins #RedHat some contract testing options • Pact • Microcks • Spring Cloud Contract
120. @holly_cummins #RedHat some contract testing options • Pact • Microcks • Spring Cloud Contract • OpenAPI + schema validation + mock generation • Prism • Schemathesis
121. Want to know/learn/watch more about Quarkus? • Battle of the frameworks : Quarkus vs SpringBoot (11:10-11:50, MC2) • Stream processing app. with Kafka and Quarkus (12:20-13:00, MC2) • Quarkus on AWS Lambda (14:10-14:25, MC3) • Why and when to write a Quarkus extension (14:35-14:50, MC3) • Quarkus Community BOF (15:05-16:05, MC3) ⇨ OpenId Connect and JWT Security in Quarkus (17:30-18:10, MC2)
122. @holly_cummins@hachyderm.io http://hollycummins.com/contract-testing-devoxx-greece/ slides thank you!
123. @holly_cummins #RedHat maturity model for quarkus extensions
124. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works
125. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works (a) works in JVM mode
126. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works (a) works in JVM mode (b)works in dev mode
127. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works (a) works in JVM mode (b)works in dev mode (c) works in native
128. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works (a) works in JVM mode (b)works in dev mode (c) works in native 2. developer joy
129. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works (a) works in JVM mode (b)works in dev mode (c) works in native 2. developer joy (a) use build-time application knowledge to remove boilerplate
130. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works (a) works in JVM mode (b)works in dev mode (c) works in native 2. developer joy (a) use build-time application knowledge to remove boilerplate 3. developer service (if there is an external service dependency)
131. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works (a) works in JVM mode (b)works in dev mode (c) works in native 2. developer joy (a) use build-time application knowledge to remove boilerplate 3. developer service (if there is an external service dependency) 4. dev UI
132. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works (a) works in JVM mode (b)works in dev mode (c) works in native 2. developer joy (a) use build-time application knowledge to remove boilerplate 3. developer service (if there is an external service dependency) 4. dev UI 5. quickstart code template
133. @holly_cummins #RedHat maturity model for quarkus extensions 1. actually works (a) works in JVM mode (b)works in dev mode (c) works in native 2. developer joy (a) use build-time application knowledge to remove boilerplate 3. developer service (if there is an external service dependency) 4. dev UI 5. quickstart code template quarkus-pact