Consumer-driven Contract Testing

Presented at the DevOps Gathering, Bochum, 2019-03-13

At Hermes, we are aiming to deliver value continuously to our customers. Following this path, we are transforming our applications using microservice based architecture. This brings many improvements but comes with the additional burden of having more and more systems that are interdependent.

We will demonstrate how we cope with these dependencies using consumer-based contract testing approaches and how this influences the roles and culture within our development.

For more infos, see here:
https://devops-gathering.io/talks/consumer-based-contract-testing/

Transcript

1. Consumer-driven Contract Testing Stephan Stapel, Hermes Germany Bochum, March 13, 2019
2. Stephan Stapel • 43 years old • Head of Development (Customer Solutions) • Areas of interest: Agility, applying Lean principles in IT (DevOps, Continuous Delivery)
3. Agenda 01 The case for Consumer-based Contract Testing (CDCT) 02 Using CDCT 03 Summary
4. 01 The case for CDCT
5. Hermes Germany is the second largest parcel delivery company in Germany. • Hermes Germany is Hamburg-based • Part of the Otto group • We deliver about 400 million parcels per year. • There are 4 competing parcel delivery companies in Germany. • IT is the key differentiator for Hermes.
6. Almost no standard software is available for our business. • Focus on inhouse development • Numerous medium-sized applications • Microservices to-the-rescue • Lots of concurrent development activity
7. Highly integrated application landscape • Numerous applications to be integrated • Verticalization (Microservices) • Horizontalization (SOA components providing base functionality for the entire application landscape)
8. Continuous Delivery helps us to shorten development cycles. Develop Test Deploy Build Develop Modern frameworks generate most boilerplate parts automatically. Build Jenkins makes it effortless to build, run and automatize our pipelines. Deploy Jenkins and K8s – both make deployment processes and infrastructure management easier. Test Unit tests and BDD can easily be automated.
9. Automatization reaches its limits when it comes to crossing team boundaries. 9 Consumer Provider Team 1 Team 2
10. With a close-to-perfect development process in place, collaboration between teams becomes the next bottleneck. Agreeing on the interface Implementing functionality Implementing functionality Implementing interface call Integration tests and End2End tests Team 1 Deployment Deployment Team 2 Synchronization of teams Synchronization of teams t t Synchronization leads to wait times.
11. When it comes to real-world scenarios, End2End Tests are often too complex and expensive. 11 Team 1 Team 2 Team 3 Team n … End2End Test Implementation Test Implementation Test Implementation Test Implementation Test
12. Skipping End2End Tests in favor of comprehensive Integration Tests reduces effort, assuring landscape quality remains complex and expensive. 12 Team 1 Team 2 Integration Test Team 3 Team n … Integration Test Integration Test … Implementation Test Implementation Test Implementation Test Implementation Test
13. Bottleneck: All teams need to finish their work before Integration Tests and End2End Tests can be kicked off today‘s focus E2E Tests Integration Tests Function Tests/ Module Tests
14. Comprehensive Trustworthy Fast Early Cheap Integration Tests are…
15. Agenda 15 How to solve?
16. Planning Architecture Development & Test We address dependencies on all levels. 16 Value stream roadmap Dependencies are identified early in the process, activities of teams can be aligned. myhermes Microservice Microservice Microservice Microservice CCM Microservice Microservice Microservice Microservice Architecture Mantra: As vertical as possible Effect: clear boundaries facilitate isolation of changes, allow more frequent deployments. Technology Consumer-driven Contract Tests support team interaction and interface negotiations. Consumer Provider
17. Development & Test Planning Architecture We address dependencies on all levels. 17 Value stream roadmap Dependencies are identified early in the process, activities of teams can be aligned. myhermes Microservice Microservice Microservice Microservice CCM Microservice Microservice Microservice Microservice Architecture Mantra: As vertical as possible Effect: clear boundaries facilitate isolation of changes, allow more frequent deployments. Technology Consumer-driven Contract Tests support team interaction and interface negotiations. Consumer Provider
18. Contracts describe customer‘s expectations from the provider. This structured approach speeds up interactions between teams. 18 Consumer Provider 1 Tests Tests Mock Provider 2 Mock Consumer 2 Contract
19. Consumer-driven Provider-driven Why consumer-driven instead of provider-driven? 19 • Fit for purpose • Detect unused functionality • Does not work with unknown consumers (public API) • Well-known concept (APIs) • Broad definition (schema, interface, policy, …) • Authoritative
20. Agenda 20 But: Contracts do not fix broken communication between teams.
21. Decoupling Integration Tests. Consumer Driven Contract Tests allow us to isolate Integration Tests. • Integration Tests locally within the teams • Each focusing on particular services • Stubbing and mocking of the partnering services E2E Tests Integration Tests Function Tests/ Module Tests
22. Provider-centric tests Consumer-centric tests Decoupling Integration Tests. Consumer Driven Contract Tests allow us to isolate Integration Tests. 22 Provider 3 Isolation against third-party services. Provider 2 Isolation against third-party services. Provider Consumer
23. With CDCT in place, wait times are minimized. Synchronization of teams Synchronization of teams Agreeing on the interface and contract design Implementing functionality and Integration Tests Implementing functionality and Integration Tests Team 1 Deployment Deployment Team 2 t t End2End Tests
24. With CDCT in place, wait times are minimized. Team 1 Team 2 t t Contract gets updated Interface is modified Test fails, deployment gets blocked. Deployment Deployment Synchronization of teams Synchronization of teams End2End-Tests Adoption
25. Comprehensive Indicate the overall quality Early Cheap Plus: allow late changes in interface design Consumer-driven Contract Tests are…
26. With a growing number of applications, interactions grow exponentially. Number of applications Interfaces • With a fast growing number of interfaces, Integration Testing becomes more important • Potential killer for lead times Traditional landscape
27. With a growing number of applications, interactions grow exponentially. Number of applications Interfaces • With a fast growing number of interfaces, Integration Testing becomes more important • Potential killer for lead times • Shrinking deployment units and increasing their amount (Microservice architecture), interactions grow even faster Microservice landscape Traditional landscape
28. Reshaping the test pyramid. Function Tests/ Module Tests Integrations Tests E2E Tests • CDCT makes Integration Testing cheaper • This allows more comprehensive test setups • Reduce End2End Tests in favor of automated CDCT Tests • Make sure that business logic tests are implemented as Function Tests (Unit Tests)! No holy grail • All Integration Testing is complex • This is also true CDCT  Don‘t let that stop you!
29. 02 Using CDCT
30. Two major frameworks for CDCT Both frameworks provide thorough support for CDCT. You can even use them together. PACT Spring Cloud Contract • Community based • PACT broker as the central component for managing contract lifecycle • Supports REST and AMQP • Polyglot • Spring (Pivotal) backing • Workflow more complicated • Focus on Java
31. Terms used with PACT 31 A pact between consumer and provider describes the consumer’s usage patterns of the interface. It consists of one or more provider states. A provider state can be understood as a fixture that the provider generates for a certain request.
32. Example 1 The contract can be modelled in either DSL or Java
33. Example 1 Using the mock provider to run tests on consumer side 33
34. Running the tests on provider side. Test cases are generated automatically based on the contract. 36
35. Test results on provider side: 37
36. Employing the PACT broker for centrally managing your contracts 39 Consumer .json file Option 1 PACT broker Contract Option 2 Git repository
37. Consumers and provider use the PACT broker to keep contracts and status up-to-date. 40 PACT broker Consumer 1 Consumer 2 Provider Retrieving the contract(s) Testing against the contract Writing the results Testing against the contract(s) The broker generated transparency for both consumers and provider. It also generates an overview about the dependencies of the application landscape.
38. The PACT broker makes the status of all contracts transparent 41
39. As soon as the contract is verified by the producer, the PACT broker indicates the validity 42
40. The PACT broker makes contract breaches by the provider transparent. 43
41. 04 Summary
42. CDCT bridges the gap for Integration Testing CDCT helps: • Supports collaboration between teams and discussion of interfaces • Implements essential tests of interfaces (structure, naming, typing, response codes) • Speeds up development processes for both consumer and provider CDCT does not help: • Replacement for End2End Tests  use End2End Tests only where really necessary • Replacement for business logic tests  use Unit Tests instead • Testing against non-functional requirements Adoption from Consumer-driven Contracts, Markus Knittig (https://entwicklertag.de/karlsruhe/2018/sites/entwicklertag.de.karlsruhe.2018/files/folien/4.%20Consumer-Driven%20Contracts.pdf)
43. Contact Stephan Stapel stephan.stapel@hermesworld.com Our presentation deck: www.speakerdeck.com/hermes CDCT sample applications: https://github.com/HermesGermany/jug-cdc