Expanding API test coverage through a pipeline

This presentation shows the importance to create the API tests following a pipeline strategy to give you the necessary confidence to deliver your software.

The project can be found at: https://github.com/eliasnogueira/restassured-complete-basic-example

Transcript

1. Expanding API test coverage through a pipeline Elias Nogueira Principal Software Engineer in Test eliasnogueira
2. Disclaimer This presentation will be done (almost) in a reverse order
3. Disclaimer The Test Pyramid focus will be on the API part, not related to Unit and UI layers
4. Concept
5. Original Test Pyramid UI Tests Service Tests Unit Tests more isolation more integration faster slower
6. Ideal Test Pyramid
7. Individual APIs API API API API API API API API
8. Individual APIs API API API API API API API API We’ll assume that each API is already tested Unit Test must be done
9. Individual APIs API API API API API API API API Functional testing in each API
10. Contract and E2E testing API API API API API API API API Contract & E2E Testing
11. Contract and E2E testing API API API API API API API API Contract & E2E Testing Guarantee that they are communicating without problem
12. Contract and E2E testing API API API API API API API API Contract & E2E Testing Assure that they can be used together (e2e)
13. Recall
14. Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based services import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest { @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } }
15. import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest { @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } } Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based services. import libraries
16. import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest { @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } } Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based services request pre-condition
17. import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest { @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } } Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based services action (request)
18. import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest { @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } } Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based services assert the response body
19. SUT – System Under Test
20. SUT – System Under Test | Front-end We should inform a CPF * Its a Brazilian social security number If a restriction is found, show a message
21. SUT – System Under Test | Front-end Fill in loan information
22. SUT – System Under Test | Front-end CRUD operations
23. Pipeline
24. API Pipeline Health Check Contract Functional Acceptance Verify if the endpoint is alive Assert that the specs haven’t changed Assert all the criteria from the requirement + happy/sad paths Assert that the most important user scenarios still works
25. Health Check Pipeline (pyramid view) Contract Acceptance Functional
26. Enabling the Pipeline Create a way to, easily, filter the tests by their focus/function/level… @Test(groups = "functional") public void testNgGroup() { // test goes here } @Test @Tag("functional") void junitTag() { // test goes here } XML file Suite Class Suite
27. [how to] Enabling the Pipeline In the example 1. Create a strategy to filter your tests ◦ Example of using groups in a test [1] 2. Create a strategy to enable the test be executed by command line ◦ JUnit 5 already do it through the maven-surefire-plugin ◦ We must pass the property –Dgroup=“group-name” in the command line 3. Create a pipeline as a code ◦ Example using GitHub Actions [2] [1] RestrictionsFunctionalTest.java [2] test-execution.yml
29. API health-check Health Check Contract Functional Acceptance Verify it the endpoint is alive Assert that the specs haven’t changed Assert all the criteria from the requirement + happy/sad paths Assert that the most important user scenarios still works
30. heath-check Verify if the API is available If there’s no way to verify by a monitoring strategy we can make a request and validate the status code @Test(groups = "health") public void healthCheckViaActuator() { basePath = "/actuator"; when(). get("/health"). then(). statusCode(200). body("status", is("UP")); } via monitoring @Test(groups = "health") public void healthCheckViaAPI() { given(). pathParam("cpf", "81016654049"). when(). get("/restrictions/{cpf}"). then(). statusCode(200); } via API
31. [how to] health-check 1. Identify if you have a health check endpoint ◦ If true find out, beyond the endpoint, any return status ◦ If false, make a request to the easiest endpoint (GET?)
32. [how to] health-check In the example • In the local project hit http://localhost:8088/actuator/health ◦ You’ll see the health status • See the implemented tests on: ◦ CreditHealthCheckTest [1] • Items to pay attention: ◦ It’s getting the health context from the file because it does not have the /api/v1 to hit the actuator endpoint [1] CreditHealthCheckTest.java
34. API contract Health Check Contract Functional Acceptance Verify it the endpoint is alive Assert that the specs haven’t changed Assert all the criteria from the requirement + happy/sad paths Assert that the most important user scenarios still works
35. • It’s the name given to the pact between producer and consumer • Ensures that API changes do not invalidate consumption: • path • parameters • sending data (request) • return data (response body) • json-schema is a contract that defines the expected data, types and formats of each field contract
36. { "name": "Elias", "age": 37 } { "$schema": "http://json-schema.org/draft-04/schema#", "type": "object", "properties": { "name": { "type": "string" }, "age": { "type": "integer" } }, "required": [ "name", "age" ], "additionalProperties": false } json-schema
37. { "name": "Elias", "age": 37 } { "$schema": "http://json-schema.org/draft-04/schema#", "type": "object", "properties": { "name": { "type": "string" }, "age": { "type": "integer" } }, "required": [ "name", "age" ], "additionalProperties": false } json-schema has the attribute name and data type json-schema
38. { "name": "Elias", "age": 37 } { "$schema": "http://json-schema.org/draft-04/schema#", "type": "object", "properties": { "name": { "type": "string" }, ”age": { "type": "integer" } }, "required": [ "name", "age" ], "additionalProperties": false } both attributes must be present json-schema
39. { "name": "Elias", "age": 37 } { "$schema": "http://json-schema.org/draft-04/schema#", "type": "object", "properties": { "name": { "type": "string" }, "age": { "type": "integer" } }, "required": [ ”name", ”age" ], "additionalProperties": false } json-schema no other attributes are allowed
40. [how to] contract 1. Create schema files ◦ We can use online tools to create if it’s not available 2. Create a test making a request and validating the json schema ◦ Add json-schema-validator library ◦ Statically import the matcher ◦ Use the matcher against json schema file
41. [how to] contract In the example • See the implemented tests on: ◦ RestrictionsContractTest.java [1] ◦ SimulationsContractTest.java [2] • Items to pay attention: ◦ The validation is done by the matcher matchesJsonSchemaInClasspath, checking the response body with the json schema file ◦ If you want to see different API enable the test contractOnV2 on RestrictionsContractTests [1] RestrictionsContractTest.java [2] SimulationsContractTest.java
43. API functional Health Check Contract Functional Acceptance Verify it the endpoint is alive Assert that the specs haven’t changed Assert all the criteria from the requirement + happy/sad paths Assert that the most important user scenarios still works
44. functional Validate positive and negative scenarios (happy and sad path) @Test(groups = {"functional"}) public void existentSocialSecirityNumber() { given(). pathParam("cpf", "66414919004"). when(). get("/simulations/{cpf}"). then(). statusCode(200). body( "id", equalTo(1), "name", equalTo("Oleksander"), "cpf", equalTo("66414919004"), "email", equalTo("oleksander@gmail.com"), "amount", equalTo(11000f), "installments", equalTo(3), "insurance", equalTo(true) ); } data validation
45. [how to] functional 1. Create the tests avoiding repetitions ◦ Use Test Data Factories and Data Driver in your advantage 2. Apply the strategy you like ◦ All tests in the same class ◦ One class per test
46. [how to] functional In the example • See the implemented tests on: ◦ RestrictionsFunctionalTest.java [1] ◦ SimulationsFunctionalTest.java [2] • Items to pay attention: ◦ The tests are using a soft assertion for the body validation ◦ MessageFormat class is used to properly concatenate String ◦ SimulationsFunctionalTests is using Test Data Factory, Builders for the model object and Data Driven [1] RestrictionsFunctionalTests.java [2] SimulationsFunctionalTests.java
48. Assert all the criteria from the requirement + happy/sad paths API acceptance Health Check Contract Functional Acceptance Verify it the endpoint is alive Assert that the specs haven’t changed Assert that the most important user scenarios still works
49. Testing from the user's perspective • Access the page and make the restriction search by the CPF • Insert a credit simulation
50. acceptance (e2e) @Test(groups = "e2e") public void completeSimulation() { new RestrictionsClient().queryRestrictionAndReturnNotFound(); new SimulationsClient().submitSuccessfulSimulation(); }
52. [how to] acceptance (e2e) 1. Create the tests avoiding repetitions ◦ Use Request and Response Specifications ◦ Try to break out in reusable methods common actions 2. Do the User Journey ◦ The few most important user journeys must run before the functional tests https://martinfowler.com/articles/practical-test-pyramid.html#RestApiEnd-to-endTest
53. [how to] functional In the example • See the implemented tests on: ◦ FullSimulationE2ETest.java [1] • Items to pay attention: ◦ There’s a utility class with the suffix Client in order to have an easy way to make request and response calls ◦ The request and response calls are Specification Re-use [2] created to reuse these actions ◦ See also the packages client [3] and specs [4] [1] FullSimulationE2ETest.java [2] https://github.com/rest-assured/rest-assured/wiki/usage#specification-re-use [3] client package [4] specs package
54. For free Work in progress in 2 languages Access Leanpub and subscribe
55. Thank you! @eliasnogueira github.com/eliasnogueira