Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Better APIs, faster tests and more resilient systems with Spring Cloud Contract

Better APIs, faster tests and more resilient systems with Spring Cloud Contract

How can we better collaborate on creating useful APIs? How to reduce the testing time of your application? How can the team build API backwards compatibility verification into their deployment process?

In this talk, I will discuss how to leverage the Consumer-Driven Contracts approach to solve these and other issues and present one of CDC implementations - Spring Cloud Contract. We will discuss practical use-cases and look at code examples. I will also point out some of the possible issues and ways of handling them, as well as some good practices I have learnt while working both as a CDC tools end-user and their developer.

2559c50bf24b65eaba52fd7c75260c93?s=128

Olga Maciaszek-Sharma

June 30, 2020
Tweet

More Decks by Olga Maciaszek-Sharma

Other Decks in Technology

Transcript

  1. Olga Maciaszek-Sharma June 2020 Better APIs, faster tests and more

    resilient systems with Spring Cloud Contract Copyright © 2020 VMware, Inc. or its affiliates.
  2. About me Software Engineer in Spring Cloud Team • Spring

    Cloud LoadBalancer (Spring Cloud Commons) • Spring Cloud Contract • Spring Cloud OpenFeign • Spring Cloud Netflix https://github.com/OlgaMaciaszek https://twitter.com/olga_maciaszek
  3. What will we try to improve?

  4. Faster Tests

  5. The Monolith Service 1 Service 2 Service 3

  6. A LOT of E2E tests

  7. Failing pipelines source: https://www.jenkins.io/blog/2016/05/26/introducing-blue-ocean/

  8. Long build and test times source: https://ci.jenkins.io/

  9. Hell automation @venkat_s

  10. E2E tests Slow Unstable Rely on collaborators Reliable

  11. Distributed architectures IoT API Gateway Mobile Browser Service 1 Service

    3 Config Server Service Registry Spring Cloud Sleuth Service 2 Metrics Store Databases Message Brokers PLATFORM
  12. Fast development, fast tests

  13. Decoupling deployment

  14. Agile teams, agile releases

  15. Fail Safe Safe To Fail

  16. Shift to the Left

  17. Network Communication Stubs

  18. A new endpoint { “request” : { “method”: “GET”, “url”

    : “/endpointX”, }, “response” : { “status” : 200 } }
  19. A new endpoint { “request” : { “method”: “GET”, “url”

    : “/endpointX”, }, “response” : { “status” : 200 } }
  20. A new endpoint

  21. A new endpoint

  22. A new endpoint { “request” : { “method”: “GET”, “url”

    : “/endpointX”, }, “response” : { “status” : 200 } } { “request” : { “method”: “GET”, “url” : “/endpointY”, }, “response” : { “status” : 200 } }
  23. Messaging topic: yyy class Product { int productId; String productName;

    } topic: xxx class Product { int id; String productName; } topic: xxx class Product { int productId; String productName; }
  24. Stubs alone don’t guarantee ANYTHING

  25. Stubbed tests Unreliable Fast Stable Independent

  26. Better APIs

  27. The Monolith Service 1 Service 2 Service 3

  28. A server with limited clients

  29. Limited APIs

  30. External APIs

  31. Limited Network Communication for Internal Apps

  32. Distributed architectures IoT API Gateway Mobile Browser Service 1 Service

    3 Config Server Service Registry Spring Cloud Sleuth Service 2 Metrics Store Databases Message Brokers PLATFORM
  33. Network communication as first-class citizen

  34. Multiple instances of many services communicating with each other over

    the network
  35. Numerous fast-growing and (even) changing internal APIs

  36. Possibility of COLLABORATION between API producers and consumers

  37. Some terms Producer Consumer

  38. Why would we want to do that?

  39. It’s the consumer that actually CONSUMES the API.

  40. It makes sense for the consumer to actually DRIVE the

    creation of the APIs
  41. None
  42. Potential Answer: Consumer-Driven Contracts

  43. Contracts Contract Consumer Input A Producer Output B

  44. Consumer-Driven Contracts https://martinfowler.com/articles/consumerDrive nContracts.html https://martinfowler.com/bliki/ContractTest.html

  45. Possible Implementation: Spring Cloud Contract

  46. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks

    out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
  47. Producer Build Setup <build> <plugins> <plugin> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-contract-maven-plugin</artifactId> <version>2.2.3.RELEASE</version> <extensions>true</extensions>

    <configuration> <testFramework>JUNIT5</testFramework> <testMode>WEBTESTCLIENT</testMode> <baseClassForTests>FraudCheckTestsBaseClass</baseClassForTests> </configuration> </plugin> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build>
  48. Contract Contract.make { request { method 'POST' url '/fraudcheck' body(

    clientId: anyUuid(), loanAmount: 99999 ) headers { header('Content-Type', 'application/vnd.fraud.v1+json;charset=UTF-8')} } response { status 200 body( fraudCheckStatus: "FRAUD", rejectionReason: "Amount too high") headers { header('Content-Type': 'application/vnd.fraud.v1+json;charset=UTF-8')} } }
  49. Contract request: method: POST url: /fraudcheck body: clientId: 437ea752-7361-4d55-ad3d-2dcc40204acf loanAmount:

    99999 headers: Content-Type: application/vnd.fraud.v1+json;charset=UTF-8 matchers: body: - path: $.['clientId'] type: by_regex predefined: uuid response: status: 200 body: fraudCheckStatus: "FRAUD" rejectionReason: "Amount too high" headers: Content-Type: application/vnd.fraud.v1+json;charset=UTF-8
  50. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks

    out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
  51. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks

    out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
  52. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks

    out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
  53. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks

    out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
  54. Generated Contract Test @Test public void validate_shouldMarkClientAsFraud() throws Exception {

    // given: WebTestClientRequestSpecification request = given() .header("Content-Type", application/vnd.fraud.v1+json;charset=UTF-8") .body("{\"clientId\":\"437ea752-7361-4d55-ad3d-2dcc40204acf\",\"loanAmount\":99999}"); // when: WebTestClientResponse response = given().spec(request) .post("/fraudcheck"); // then: assertThat(response.statusCode()).isEqualTo(200); assertThat(response.header("Content-Type")) .isEqualTo("application/vnd.fraud.v1+json;charset=UTF-8"); // and: DocumentContext parsedJson = JsonPath.parse(response.getBody().asString()); assertThatJson(parsedJson).field("['fraudCheckStatus']").isEqualTo("FRAUD"); assertThatJson(parsedJson).field("['rejectionReason']").isEqualTo("Amount too high"); }
  55. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks

    out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
  56. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks

    out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
  57. Generated WireMock Stub { "id" : "88dd475b-005d-4a1d-a6c5-3d40493c8d37", "request" : {

    "url" : "/fraudcheck", "method" : "POST", "headers" : { "Content-Type" : { "equalTo" : "application/vnd.fraud.v1+json;charset=UTF-8" }}, "bodyPatterns" : [ { "matchesJsonPath" : "$[?(@.['loanAmount'] == 99999)]" }, { "matchesJsonPath" : "$[?(@.['clientId'] =~ /([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})/)]" } ] }, "response" : { "status" : 200, "body" : "{\"fraudCheckStatus\":\"FRAUD\",\"rejectionReason\":\"Amount too high\"}", "headers" : { "Content-Type" : "application/vnd.fraud.v1+json;charset=UTF-8" }, "transformers" : [ "response-template" ]}, "uuid" : "88dd475b-005d-4a1d-a6c5-3d40493c8d37"}
  58. Spring Cloud Contract - Consumer-Driven Contract Flow Consumer • Checks

    out Producer code repo and works locally on a contract • Creates a pull request with the new Contract • Collaborates with the Producer on the Contract PR • Uses stubs published by the producer along the app for integration tests Producer • Reviews the Contract PR • Runs build on the PR branch (FAILURE) • Adds required endpoints • Runs build on the PR branch (PASSING) • Code Deployed along with generated stubs jar
  59. Consumer Build Setup <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-contract-stub-runner</artifactId> <scope>test</scope> </dependency>

  60. Consumer Tests @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE) @ExtendWith({SpringExtension.class}) @AutoConfigureStubRunner(ids = {"io.github.olgamaciaszek:scc-producer-demo:+:stubs:8080"}, stubsMode

    = StubRunnerProperties.StubsMode.REMOTE) class SccConsumerDemoApplicationTests { @Test void shouldBeRejectedDueToAbnormalLoanAmount() { LoanApplication application = new LoanApplication(new Client(UUID.randomUUID()), new BigDecimal("99999"), UUID.randomUUID()); LoanApplicationResult loanApplication = loanApplicationService .loanApplication(application).block(); assertThat(loanApplication.getLoanApplicationStatus()) .isEqualTo(LoanApplicationStatus.APPLICATION_REJECTED); assertThat(loanApplication.getRejectionReason()).isEqualTo("Amount too high"); }}
  61. Consumer Tests @StubRunnerPort port; stubrunner: ids: - org.springframework.cloud.contract.verifier.stubs:loanIssuance - org.springframework.cloud.contract.verifier.stubs:fraudDetectionServer

    - org.springframework.cloud.contract.verifier.stubs:bootService
  62. Consumer Tests @Rule public StubRunnerRule rule = new StubRunnerRule() .downloadStub("com.example","fraud-service")

    .stubsMode(StubRunnerProperties.StubsMode.LOCAL) .withStubPerConsumer(true) .withConsumerName("loan-service"); int fraudServicePort = rule.findStubUrl("fraud-service").getPort()
  63. Code Demo

  64. Messaging Contract Contract.make { label "payment_successful" input { triggeredBy "makeSuccessfulPayment()"

    } outputMessage { sentTo "payments" body( uuid: "d64c361b-29bb-43a9-8fe8-5e7e05493842", status: "SUCCESS" ) headers { header("contentType", applicationJson())}}}
  65. Generated Contract Test @Test public void validate_should_payment_fail() throws Exception {

    // when: makeFailingPayment(); // then: ContractVerifierMessage response = contractVerifierMessaging.receive("payments"); assertThat(response).isNotNull(); // and: assertThat(response.getHeader("contentType")).isNotNull(); assertThat(response.getHeader("contentType").toString()).isEqualTo("application/json"); // and: DocumentContext parsedJson = JsonPath.parse(contractVerifierObjectMapper.writeValueAsString(response.getPayload())); assertThatJson(parsedJson).field("['uuid']").isEqualTo("d64c361b-29bb-43a9-8fe8-5e7e05493842"); assertThatJson(parsedJson).field("['status']").isEqualTo("FAILURE"); }
  66. Consumer Test @SpringBootTest @AutoConfigureStubRunner(stubsMode = StubRunnerProperties.StubsMode.LOCAL, ids = "com.example:scc-payment") class

    PaymentProcessingSpec extends Specification { @Autowired StubTrigger stubTrigger def 'should handle successful payment'() { when: stubTrigger.trigger('payment_successful') then: accountingService.successfulPayments.size() == 1 }
  67. Code Demo

  68. Spring Cloud Contract A few highlighted features out of many:

    • Contracts in a separate repository • Spring Cloud Netflix and Spring Cloud LoadBalancer integration • Creating Spring Rest Docs from Contracts and Contracts from Spring Rest Docs • Polyglot support (using provided Docker image) • Community support for Swagger files
  69. Architecture and Deployments

  70. Architecture/ API TDD Refactor Red Green

  71. Backward Compatibility Checks in Pipelines New API version Test with

    PROD contracts Deploy Test with new contracts
  72. Deferred updates Deployment Contract fields Application fields 0 { "buildTime":

    5 } { "buildTime": 5 } 1 { "buildTime": 5, "buildTimeMillis": 5000 } { "buildTime": 5, "buildTimeMillis": 5000 } 2 { "buildTimeMillis": 5000 } { "buildTime": 5, "buildTimeMillis": 5000 } 3 { "buildTimeMillis": 5000 } { "buildTimeMillis": 5000 }
  73. Lessons Learnt and Good Practices

  74. Usage Logic tests in lower layers

  75. Resilience Monitoring Logging Alerting

  76. Maintenance Skipping contracts for non-essential functionalities

  77. Maintenance Skipping multiple values

  78. Resources • Project page • Samples • https://github.com/OlgaMaciaszek/scc-producer-demo • https://github.com/OlgaMaciaszek/scc-consumer-demo

    • https://github.com/OlgaMaciaszek/scc-payment • https://github.com/OlgaMaciaszek/scc-shop • Spring Cloud Contract at Devskiller - case study • https://gitter.im/spring-cloud/spring-cloud-contract
  79. Thank you Contact me at omaciaszeksh@vmware.com © 2020 Spring. A

    VMware-backed project.