Better APIs, faster tests and more resilient systems with Spring Cloud Contract

How can we better collaborate on creating useful APIs? How to reduce the testing time of your application? How can the team build API backwards compatibility verification into their deployment process?

In this talk, I will discuss how to leverage the Consumer-Driven Contracts approach to solve these and other issues and present one of CDC implementations - Spring Cloud Contract. We will discuss practical use-cases and look at code examples. I will also point out some of the possible issues and ways of handling them, as well as some good practices I have learnt while working both as a CDC tools end-user and their developer.

Transcript

1. Olga Maciaszek-Sharma June 2020 Better APIs, faster tests and more resilient systems with Spring Cloud Contract Copyright © 2020 VMware, Inc. or its affiliates.
2. About me Software Engineer in Spring Cloud Team • Spring Cloud LoadBalancer (Spring Cloud Commons) • Spring Cloud Contract • Spring Cloud OpenFeign • Spring Cloud Netflix https://github.com/OlgaMaciaszek https://twitter.com/olga_maciaszek
3. What will we try to improve?
4. Faster Tests
5. The Monolith Service 1 Service 2 Service 3
6. A LOT of E2E tests
7. Failing pipelines source: https://www.jenkins.io/blog/2016/05/26/introducing-blue-ocean/
8. Long build and test times source: https://ci.jenkins.io/
9. Hell automation @venkat_s
10. E2E tests Slow Unstable Rely on collaborators Reliable
11. Distributed architectures IoT API Gateway Mobile Browser Service 1 Service 3 Config Server Service Registry Spring Cloud Sleuth Service 2 Metrics Store Databases Message Brokers PLATFORM
12. Fast development, fast tests
13. Decoupling deployment
14. Agile teams, agile releases
15. Fail Safe Safe To Fail
16. Shift to the Left
17. Network Communication Stubs
18. A new endpoint { “request” : { “method”: “GET”, “url” : “/endpointX”, }, “response” : { “status” : 200 } }
19. A new endpoint { “request” : { “method”: “GET”, “url” : “/endpointX”, }, “response” : { “status” : 200 } }
20. A new endpoint
21. A new endpoint
22. A new endpoint { “request” : { “method”: “GET”, “url” : “/endpointX”, }, “response” : { “status” : 200 } } { “request” : { “method”: “GET”, “url” : “/endpointY”, }, “response” : { “status” : 200 } }
23. Messaging topic: yyy class Product { int productId; String productName; } topic: xxx class Product { int id; String productName; } topic: xxx class Product { int productId; String productName; }
24. Stubs alone don’t guarantee ANYTHING
25. Stubbed tests Unreliable Fast Stable Independent
26. Better APIs
27. The Monolith Service 1 Service 2 Service 3
28. A server with limited clients
29. Limited APIs
30. External APIs
31. Limited Network Communication for Internal Apps
32. Distributed architectures IoT API Gateway Mobile Browser Service 1 Service 3 Config Server Service Registry Spring Cloud Sleuth Service 2 Metrics Store Databases Message Brokers PLATFORM
33. Network communication as first-class citizen
34. Multiple instances of many services communicating with each other over the network
35. Numerous fast-growing and (even) changing internal APIs
36. Possibility of COLLABORATION between API producers and consumers
37. Some terms Producer Consumer
38. Why would we want to do that?
39. It’s the consumer that actually CONSUMES the API.
40. It makes sense for the consumer to actually DRIVE the creation of the APIs
42. Potential Answer: Consumer-Driven Contracts
43. Contracts Contract Consumer Input A Producer Output B
44. Consumer-Driven Contracts https://martinfowler.com/articles/consumerDrive nContracts.html https://martinfowler.com/bliki/ContractTest.html
45. Possible Implementation: Spring Cloud Contract
46. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
47. Producer Build Setup <build> <plugins> <plugin> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-contract-maven-plugin</artifactId> <version>2.2.3.RELEASE</version> <extensions>true</extensions> <configuration> <testFramework>JUNIT5</testFramework> <testMode>WEBTESTCLIENT</testMode> <baseClassForTests>FraudCheckTestsBaseClass</baseClassForTests> </configuration> </plugin> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build>
48. Contract Contract.make { request { method 'POST' url '/fraudcheck' body( clientId: anyUuid(), loanAmount: 99999 ) headers { header('Content-Type', 'application/vnd.fraud.v1+json;charset=UTF-8')} } response { status 200 body( fraudCheckStatus: "FRAUD", rejectionReason: "Amount too high") headers { header('Content-Type': 'application/vnd.fraud.v1+json;charset=UTF-8')} } }
49. Contract request: method: POST url: /fraudcheck body: clientId: 437ea752-7361-4d55-ad3d-2dcc40204acf loanAmount: 99999 headers: Content-Type: application/vnd.fraud.v1+json;charset=UTF-8 matchers: body: - path: $.['clientId'] type: by_regex predefined: uuid response: status: 200 body: fraudCheckStatus: "FRAUD" rejectionReason: "Amount too high" headers: Content-Type: application/vnd.fraud.v1+json;charset=UTF-8
50. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
51. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
52. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
53. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
54. Generated Contract Test @Test public void validate_shouldMarkClientAsFraud() throws Exception { // given: WebTestClientRequestSpecification request = given() .header("Content-Type", application/vnd.fraud.v1+json;charset=UTF-8") .body("{\"clientId\":\"437ea752-7361-4d55-ad3d-2dcc40204acf\",\"loanAmount\":99999}"); // when: WebTestClientResponse response = given().spec(request) .post("/fraudcheck"); // then: assertThat(response.statusCode()).isEqualTo(200); assertThat(response.header("Content-Type")) .isEqualTo("application/vnd.fraud.v1+json;charset=UTF-8"); // and: DocumentContext parsedJson = JsonPath.parse(response.getBody().asString()); assertThatJson(parsedJson).field("['fraudCheckStatus']").isEqualTo("FRAUD"); assertThatJson(parsedJson).field("['rejectionReason']").isEqualTo("Amount too high"); }
55. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
56. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
57. Generated WireMock Stub { "id" : "88dd475b-005d-4a1d-a6c5-3d40493c8d37", "request" : { "url" : "/fraudcheck", "method" : "POST", "headers" : { "Content-Type" : { "equalTo" : "application/vnd.fraud.v1+json;charset=UTF-8" }}, "bodyPatterns" : [ { "matchesJsonPath" : "$[?(@.['loanAmount'] == 99999)]" }, { "matchesJsonPath" : "$[?(@.['clientId'] =~ /([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})/)]" } ] }, "response" : { "status" : 200, "body" : "{\"fraudCheckStatus\":\"FRAUD\",\"rejectionReason\":\"Amount too high\"}", "headers" : { "Content-Type" : "application/vnd.fraud.v1+json;charset=UTF-8" }, "transformers" : [ "response-template" ]}, "uuid" : "88dd475b-005d-4a1d-a6c5-3d40493c8d37"}
58. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
59. Consumer Build Setup <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-contract-stub-runner</artifactId> <scope>test</scope> </dependency>
60. Consumer Tests @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE) @ExtendWith({SpringExtension.class}) @AutoConfigureStubRunner(ids = {"io.github.olgamaciaszek:scc-producer-demo:+:stubs:8080"}, stubsMode = StubRunnerProperties.StubsMode.REMOTE) class SccConsumerDemoApplicationTests { @Test void shouldBeRejectedDueToAbnormalLoanAmount() { LoanApplication application = new LoanApplication(new Client(UUID.randomUUID()), new BigDecimal("99999"), UUID.randomUUID()); LoanApplicationResult loanApplication = loanApplicationService .loanApplication(application).block(); assertThat(loanApplication.getLoanApplicationStatus()) .isEqualTo(LoanApplicationStatus.APPLICATION_REJECTED); assertThat(loanApplication.getRejectionReason()).isEqualTo("Amount too high"); }}
61. Consumer Tests @StubRunnerPort port; stubrunner: ids: - org.springframework.cloud.contract.verifier.stubs:loanIssuance - org.springframework.cloud.contract.verifier.stubs:fraudDetectionServer - org.springframework.cloud.contract.verifier.stubs:bootService
62. Consumer Tests @Rule public StubRunnerRule rule = new StubRunnerRule() .downloadStub("com.example","fraud-service") .stubsMode(StubRunnerProperties.StubsMode.LOCAL) .withStubPerConsumer(true) .withConsumerName("loan-service"); int fraudServicePort = rule.findStubUrl("fraud-service").getPort()
63. Code Demo
64. Messaging Contract Contract.make { label "payment_successful" input { triggeredBy "makeSuccessfulPayment()" } outputMessage { sentTo "payments" body( uuid: "d64c361b-29bb-43a9-8fe8-5e7e05493842", status: "SUCCESS" ) headers { header("contentType", applicationJson())}}}
65. Generated Contract Test @Test public void validate_should_payment_fail() throws Exception { // when: makeFailingPayment(); // then: ContractVerifierMessage response = contractVerifierMessaging.receive("payments"); assertThat(response).isNotNull(); // and: assertThat(response.getHeader("contentType")).isNotNull(); assertThat(response.getHeader("contentType").toString()).isEqualTo("application/json"); // and: DocumentContext parsedJson = JsonPath.parse(contractVerifierObjectMapper.writeValueAsString(response.getPayload())); assertThatJson(parsedJson).field("['uuid']").isEqualTo("d64c361b-29bb-43a9-8fe8-5e7e05493842"); assertThatJson(parsedJson).field("['status']").isEqualTo("FAILURE"); }
66. Consumer Test @SpringBootTest @AutoConfigureStubRunner(stubsMode = StubRunnerProperties.StubsMode.LOCAL, ids = "com.example:scc-payment") class PaymentProcessingSpec extends Specification { @Autowired StubTrigger stubTrigger def 'should handle successful payment'() { when: stubTrigger.trigger('payment_successful') then: accountingService.successfulPayments.size() == 1 }
67. Code Demo
68. Spring Cloud Contract A few highlighted features out of many: • Contracts in a separate repository • Spring Cloud Netflix and Spring Cloud LoadBalancer integration • Creating Spring Rest Docs from Contracts and Contracts from Spring Rest Docs • Polyglot support (using provided Docker image) • Community support for Swagger files
69. Architecture and Deployments
70. Architecture/ API TDD Refactor Red Green
71. Backward Compatibility Checks in Pipelines New API version Test with PROD contracts Deploy Test with new contracts
72. Deferred updates Deployment Contract fields Application fields 0 { "buildTime": 5 } { "buildTime": 5 } 1 { "buildTime": 5, "buildTimeMillis": 5000 } { "buildTime": 5, "buildTimeMillis": 5000 } 2 { "buildTimeMillis": 5000 } { "buildTime": 5, "buildTimeMillis": 5000 } 3 { "buildTimeMillis": 5000 } { "buildTimeMillis": 5000 }
73. Lessons Learnt and Good Practices
74. Usage Logic tests in lower layers
75. Resilience Monitoring Logging Alerting
76. Maintenance Skipping contracts for non-essential functionalities
77. Maintenance Skipping multiple values
78. Resources • Project page • Samples • https://github.com/OlgaMaciaszek/scc-producer-demo • https://github.com/OlgaMaciaszek/scc-consumer-demo • https://github.com/OlgaMaciaszek/scc-payment • https://github.com/OlgaMaciaszek/scc-shop • Spring Cloud Contract at Devskiller - case study • https://gitter.im/spring-cloud/spring-cloud-contract
79. Thank you Contact me at omaciaszeksh@vmware.com © 2020 Spring. A VMware-backed project.