Investigating Contract Testing

Introductory talk about Contract testing with Spring Cloud Contract/Pact and probability of using Protocol Buffers to define contract.

Transcript

1. INVESTIGATING CONTRACT TESTING QA MEETUP NEXIGN 2018
2. VSEVOLOD BREKELOV Software developer @ Member of program @
 committee
 @brekelov @volekerb
3. КТО УЖЕ ДЕЛАЕТ КОНТРАКТНОЕ ТЕСТИРОВАНИЕ? 3
4. О ЧЕМ ЭТО МЫ CONTRACT TESTING: SPRING/PACT/PROTOBUF ▸ Что такое контрактное тестирование ▸ Spring Cloud Contract ▸ Pact ▸ Proto(или другие бинарные протоколы) ▸ Как жить, чтобы не было больно 4
5. ВОПРОС В ЗАЛ API TESTS
 OR
 UI
 OR … BACKEND GET /Users/Ivan {
 “firstName”: “Ivan”,
 “lastName”: “Petrushkin”
 “age”: 22,
 }
 5
6. TEXT 6
7. ВОПРОС В ЗАЛ API TESTS
 OR
 UI
 OR … BACKEND GET /Users/Ivan {
 “name” : {
 “firstName”: “Ivan”,
 “lastName”: “Petrushkin”
 }
 “age”: 22,
 }
 7
8. ИЗМЕНЕНИЯ {
 “name” : {
 “firstName”: “Ivan”,
 “lastName”: “Petrushkin”
 }
 “age”: 22
 }
 {
 “firstName”: “Ivan”,
 “lastName”: “Petrushkin”
 “age”: 22,
 }
 Было Стало 8
9. ЗАЧЕМ? 9
10. А МОЖЕТ ДОГОВОРИМСЯ? CONTRACT Consumer-Driven 10 Producer-Driven Consumer Producer
11. А МОЖЕТ ДОГОВОРИМСЯ? CONTRACT Consumer-Driven 11 Producer-Driven Consumer Producer API TESTS
 OR
 UI
 OR … BACKEND
12. PRODUCER DRIVEN CONTRACT ▸ Определить API -> ▸ Сгенерировать Документацию и Примеры -> ▸ Сгенерировать Клиентов Swagger 12
13. CONSUMER DRIVEN CONTRACT ▸ Spring Cloud Contract ▸ Pact 13
14. SPRING CLOUD CONTRACT ▸ Контракт описывается с помощью Groovy/YAML ▸ Генерирует тесты с помощью Spring Cloud Contract Verifier 14 Important Spring Cloud Contract Verifier’s purpose is NOT to start writing business features in the contracts. Assume that we have a business use case of fraud check. If a user can be a fraud for 100 different reasons, we would assume that you would create 2 contracts, one for the positive case and one for the negative case. Contract tests are used to test contracts between applications and not to simulate full behavior.
15. SPRING CLOUD CONTRACT 15 ADD CONTRACT IN
 
 /SRC/TEST/RESOURCES/ CONTRACTS ADD DEPENDENCIES/PLUGINS org.springframework.cloud.contract.spec.Contract.make { description(""" Sends a positive verification message when person is eligible to get the beer ``` given: client is old enough when: he applies for a beer then: we'll send a message with a positive verification ``` """) // Label by means of which the output message can be triggered label 'accepted_verification' // input to the contract input { // the contract will be triggered by a method triggeredBy('clientIsOldEnough()') } // output message of the contract outputMessage { // destination to which the output message will be sent sentTo 'verifications' // the body of the output message body([ eligible: true ]) } } <dependency> <groupId>org.springframework.cloud </groupId> <artifactId>spring-cloud- starter-contract-verifier
 </artifactId> <scope>test</scope> </dependency> <plugin> <groupId>org.springframework.cloud
 </groupId> <artifactId>spring-cloud- contract-maven-plugin</artifactId> <version>${spring-cloud- contract.version}</version> <extensions>true</extensions> </plugin> ./MVNW CLEAN INSTALL GENERATED TESTS PROVIDE IMPLEMENTATION
 TO FIX THE TESTS FOR PRODUCER
16. SPRING CLOUD CONTRACT 16 ADD STUBRUNNER 
 GET STUBS FROM LOCAL/ REMOTE REPO FOR CONSUMER <dependency> <groupId>org.springframework.cloud </groupId> <artifactId>spring-cloud -starter-contract-stub-runner </artifactId> <scope>test</scope> </dependency> @AutoConfigureStubRunner(
 stubsMode = StubRunnerProperties.StubsMode.LOCAL, 
 ids = "com.example:beer-api-producer") @AutoConfigureStubRunner(
 repositoryRoot="http://foo.bar" stubsMode = StubRunnerProperties.StubsMode.REMOTE, 
 ids = "com.example:beer-api-producer")
17. SPRING CLOUD CONTRACT GENERATED TEST 17 @Test public void validate_shouldSendAcceptedVerification() throws Exception { // when: clientIsOldEnough(); // then: ContractVerifierMessage response = contractVerifierMessaging.receive("verifications"); assertThat(response).isNotNull(); assertThat(response.getHeader("contentType")).isNotNull(); assertThat(response.getHeader("contentType").toString()).isEqualTo("application/json"); // and: DocumentContext parsedJson = JsonPath.parse(contractVerifierObjectMapper.writeValueAsString(response.getPayload())); assertThatJson(parsedJson).field("['eligible']").isEqualTo(true); }
18. SPRING CLOUD CONTRACT GENERATION SUPPORTS 18 ▸ Spock ▸ JUnit5 ▸ Reactive, Web-Flux ▸ RestAssured (from 2.1.0)
19. КОНТРАКТ КАК DEPENDENCY 19 CONTRACT-1.0.0.JAR SERVICE 1 SERVICE 2 OTHER TESTS SPRING CLOUD CONTRACT
20. REPO STRUCTURE (PRODUCER) 20 SPRING CLOUD CONTRACT !"" com # $"" example # $"" server # !"" client1 # # $"" expectation.groovy # !"" client2 # # $"" expectation.groovy # !"" client3 # # $"" expectation.groovy # $"" pom.xml !"" mvnw !"" mvnw.cmd !"" pom.xml $"" src $"" assembly $"" contracts.xml
22. PROS/CONS 22 SPRING CLOUD CONTRACT Contract может лежать отдельно как зависимость Генерирует тесты и стабы Изменения контракта приведут к падениям тестов Не понятно как быть с контрактом для UI (JavaScript) Много примеров Groovy DSL or YAML Поддерживает Pact Broker
23. PACT 23 PACT (NOUN): A FORMAL AGREEMENT BETWEEN INDIVIDUALS OR PARTIES. "THE COUNTRY NEGOTIATED A TRADE PACT WITH THE US" SYNONYMS: AGREEMENT, PROTOCOL, DEAL, CONTRACT ~ OXFORD DICTIONARIES
24. PACT 24 ADD DEPENDENCIES/PLUGINS <dependency>
 <groupId>au.com.dius
 </groupId>
 <artifactId>pact-jvm- consumer-junit_2.11</artifactId>
 <version>3.5.0</version>
 <scope>test</scope>
 </dependency> @Rule public PactProviderRule provider = 
 new PactProviderRule("test_provider",
 "localhost", 
 8081, this); ADD CODE FOR MOCK ADD CODE FOR TEST <plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-surefire-plugin</artifactId>
 <configuration>
 <systemPropertyVariables>
 <pact.rootDir>target/mypacts</pact.rootDir>
 </systemPropertyVariables>
 </configuration>
 …
 </plugin> GENERATE PACT FOR CONSUMER
25. PACT TEST 25 @Pact(state = "default", provider = "test_provider", consumer = "test_consumer") public PactFragment createFragment(PactDslWithProvider builder) { Map<String, String> headers = new HashMap<>(); headers.put("content-type", "application/json"); return builder .given("default") .uponReceiving("Test User Service") .path("/user/1") .method("GET") .willRespondWith() .status(200) .headers(headers) .body("{" + " \"userName\": \"Ivan\",\n" + " \"userId\": \"1\",\n" + " \"firstName\": Ivan,\n" + " \"lastName\": Petrushkin,\n" + " \"email\": null,\n" + " \"groups\": null\n" + "}") .toFragment(); }
26. PACT TEST 26 @Test @PactVerification("test_provider") public void runTest() throws IOException { final RestTemplate call = new RestTemplate(); final User expectedResponse = new User(); expectedResponse.setUserName("Ivan"); expectedResponse.setFirstName("Ivan"); expectedResponse.setUserId("1"); final User forEntity = call.getForObject(provider.getConfig().url() + "/user/1", User.class); assertThat(forEntity, sameBeanAs(expectedResponse)); }
27. PACT GENERATED CONTRACT 27 { "provider": { "name": "test_provider" }, "consumer": { "name": "other_client" }, "interactions": [ { "description": "Other Consumer Request", "request": { "method": "GET", "path": "/user/1" }, "response": { "status": 200, "headers": { "content-type": "application/json" }, "body": { "email": null, "firstName": "Ivan", "groups": null, "lastName": "Petrushkin", "userId": “1", "userName": “Ivan" } } } ], "metadata": { "pact-specification": { "version": "2.0.0" }, "pact-jvm": { "version": "3.3.6" } } }
28. PACT 28 ADD DEPENDENCIES/PLUGINS <dependency>
 <groupId>au.com.dius
 </groupId>
 <artifactId>pact-jvm- provider-junit_2.11</artifactId>
 <version>3.5.0</version>
 <scope>test</scope>
 </dependency> @RunWith(PactRunner.class) @Provider("test_provider" ) @PactBroker(host = "localhost", port = "80") @VerificationReports({"console", "markdown"}) public class TestPacts { ADD CODE FOR RUNNER ADD .JSON CONTRACT ADD TEST TARGET FOR PRODUCER private static ConfigurableApplicationContext application; @TestTarget public final Target target = new HttpTarget(8080); @BeforeClass public static void startSpring(){ application = SpringApplication.run(ProviderServiceApplication.class) } @State({"default", "extra"}) public void toDefaultState() { System.out.println("Now service in default state"); } ADD TEST WITH STATES WE WANT TO TEST
29. PACT PACT BROKER 29 ▸ It enables you to share your pacts and verification results between projects, and make them useful for people too ▸ Solves the problem of how to share contracts and verification results between consumer and provider projects ▸ Automatically versions your contracts ▸ Allows you to visualize the relationships between your services ▸ … https://docs.pact.io/getting_started/sharing_pacts
30. PACT PACT BROKER 30 ./pacts — Лежат все контракты docker pull pactfoundation/pact-stub-server docker run -t -p 8080:8080 -v "$(pwd)/pacts/:/app/pacts" pactfoundation/pact-stub-server -p 8080
 -d pacts # Test your stub endpoints curl -v localhost:8080/some/api
31. PACT PACT BROKER МОЖЕТ РАБОТАТЬ С SPRING CLOUD CONTRACT! 31
33. PACT ДЛЯ ЧЕГО НЕ НУЖНО ИСПОЛЬЗОВАТЬ ▸ Performance and load testing. ▸ Functional testing of the provider - that is what the provider's own tests should do. Pact is about checking the contents and format of requests and responses. ▸ … 33 https://docs.pact.io/getting_started/what_is_pact_good_for
34. PACT PROS/CONS 34 If you are writing tests on the Consumer side 
 with a different language on the Provider side,
 you must ensure you use a common 
 Pact Specification between them or
 you will be unable to validate the Pacts. Понятно как быть с контрактом для UI (JavaScript) Поддерживает много языков Pact Broker решает вопрос с версионированием и 
 где хранить контракты
35. PROTOCOL BUFFERS (PROTOBUF) Protocol Buffers (a.k.a., protobuf) are Google's language-neutral, 
 platform-neutral, extensible mechanism for serializing structured data 35
36. PROTOBUF message Person { required string firstName = 1; optional string lastName = 2; required int32 age = 3; } Person.proto Person ivan = Person.newBuilder() .setAge(22) .setFirstName(“Ivan") .setLastName(“Petrushkin") .build(); Some java class 36
37. PROTOBUF Person.proto JAVA JAVASCRIPT PYTHON … 37
38. PROTOBUF AS A CONTRACT PROTO UI TESTS MICROSERVICE 1 MICROSERVICE N UI BE TESTS Dependency 38
39. PROTOBUF AS A CONTRACT PROTO UI TESTS MICROSERVICE 1 MICROSERVICE N UI BE TESTS message Person { required string firstName = 1; optional string lastName = 2; required int32 age = 3; } message Person { required Name name = 1; required int32 age = 2; }
 
 message Name {
 required string firstName = 1; optional string lastName = 2;
 } 39
40. ОПЯТЬ ВСЕ ФИКСИТЬ 40
41. НЕДОСТАТКИ PROTOBUF AS A CONTRACT ▸ После изменений — необходимо идти и фиксить тесты/сервисы ▸ Или делать версионирование
 
 
 ▸ Изменение в Proto -> Ошибки при компиляции ПРЕИМУЩЕСТВО PROTOBUF AS A CONTRACT 41
42. ПОДВОДИМ ИТОГИ ▸ Контрактное тестирование : ▸ Быстрее дает фидбэк ▸ Быстрее e2e (выгоднее) ▸ Заставляет тех, кто делает изменения Думать ▸ Как использовать Spring Cloud Contract ▸ Как использовать Pact ▸ Protobuf можно использовать как контракт 42
43. HEISENBUG ДОКЛАД ПРО КОНТРАКТНОЕ ТЕСТИРОВАНИЕ С KOTLIN ▸ Гейзенбаг Москва 2018 6-7 декабря 43
44. THANK YOU! Q&A 44
45. LINKS ‣ Martin Fowler: Consumer Driven Contracts
 https://martinfowler.com/articles/consumerDrivenContracts.html 
 ‣ Pact: best practices
 https://docs.pact.io/best_practices/consumer/contract_tests_not_functional_tests 
 ‣ Martin Fowler: Contract Test
 https://martinfowler.com/bliki/ContractTest.html 
 ‣ InfoQ: Creating Spring Cloud Contract Testing
 https://www.infoq.com/articles/contract-testing-spring-cloud-contract 
 ‣ Pact with JS
 https://devblog.xero.com/trust-but-verify-using-pact-for-contract-testing-495a1e303a6 
 ‣ Protocol Buffers
 https://github.com/protocolbuffers/protobuf ‣ Spring Cloud Contract
 https://cloud.spring.io/spring-cloud-contract/ ‣ Spring Cloud Contract Samples
 https://github.com/spring-cloud-samples/spring-cloud-contract-samples 45