Expanding API test coverage through a pipeline

Expanding API test coverage through a pipeline

This presentation shows the importance to create the API tests following a pipeline strategy to give you the necessary confidence to deliver your software.

The project can be found at: https://github.com/eliasnogueira/restassured-complete-basic-example

Baaa72bd2671a244ba9211e015f72d28?s=128

Elias Nogueira

October 05, 2020
Tweet

Transcript

  1. Expanding API test coverage through a pipeline Elias Nogueira Principal

    Software Engineer in Test eliasnogueira
  2. Disclaimer This presentation will be done (almost) in a reverse

    order
  3. Disclaimer The Test Pyramid focus will be on the API

    part, not related to Unit and UI layers
  4. Concept

  5. Original Test Pyramid UI Tests Service Tests Unit Tests more

    isolation more integration faster slower
  6. Ideal Test Pyramid

  7. Individual APIs API API API API API API API API

  8. Individual APIs API API API API API API API API

    We’ll assume that each API is already tested Unit Test must be done
  9. Individual APIs API API API API API API API API

    Functional testing in each API
  10. Contract and E2E testing API API API API API API

    API API Contract & E2E Testing
  11. Contract and E2E testing API API API API API API

    API API Contract & E2E Testing Guarantee that they are communicating without problem
  12. Contract and E2E testing API API API API API API

    API API Contract & E2E Testing Assure that they can be used together (e2e)
  13. Recall

  14. Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based

    services import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest { @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } }
  15. import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest {

    @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } } Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based services. import libraries
  16. import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest {

    @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } } Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based services request pre-condition
  17. import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest {

    @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } } Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based services action (request)
  18. import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; public class RestAssuredExampleTest {

    @Test public void welcome() { given(). param("name", "Elias"). when(). post("/register"). then(). body("message", is("Hello Elias")); } } Rest-Assured http://rest-assured.io Java DSL for simplifying testing of REST based services assert the response body
  19. SUT – System Under Test

  20. SUT – System Under Test | Front-end We should inform

    a CPF * Its a Brazilian social security number If a restriction is found, show a message
  21. SUT – System Under Test | Front-end Fill in loan

    information
  22. SUT – System Under Test | Front-end CRUD operations

  23. Pipeline

  24. API Pipeline Health Check Contract Functional Acceptance Verify if the

    endpoint is alive Assert that the specs haven’t changed Assert all the criteria from the requirement + happy/sad paths Assert that the most important user scenarios still works
  25. Health Check Pipeline (pyramid view) Contract Acceptance Functional

  26. Enabling the Pipeline Create a way to, easily, filter the

    tests by their focus/function/level… @Test(groups = "functional") public void testNgGroup() { // test goes here } @Test @Tag("functional") void junitTag() { // test goes here } XML file Suite Class Suite
  27. [how to] Enabling the Pipeline In the example 1. Create

    a strategy to filter your tests ◦ Example of using groups in a test [1] 2. Create a strategy to enable the test be executed by command line ◦ JUnit 5 already do it through the maven-surefire-plugin ◦ We must pass the property –Dgroup=“group-name” in the command line 3. Create a pipeline as a code ◦ Example using GitHub Actions [2] [1] RestrictionsFunctionalTest.java [2] test-execution.yml
  28. None
  29. API health-check Health Check Contract Functional Acceptance Verify it the

    endpoint is alive Assert that the specs haven’t changed Assert all the criteria from the requirement + happy/sad paths Assert that the most important user scenarios still works
  30. heath-check Verify if the API is available If there’s no

    way to verify by a monitoring strategy we can make a request and validate the status code @Test(groups = "health") public void healthCheckViaActuator() { basePath = "/actuator"; when(). get("/health"). then(). statusCode(200). body("status", is("UP")); } via monitoring @Test(groups = "health") public void healthCheckViaAPI() { given(). pathParam("cpf", "81016654049"). when(). get("/restrictions/{cpf}"). then(). statusCode(200); } via API
  31. [how to] health-check 1. Identify if you have a health

    check endpoint ◦ If true find out, beyond the endpoint, any return status ◦ If false, make a request to the easiest endpoint (GET?)
  32. [how to] health-check In the example • In the local

    project hit http://localhost:8088/actuator/health ◦ You’ll see the health status • See the implemented tests on: ◦ CreditHealthCheckTest [1] • Items to pay attention: ◦ It’s getting the health context from the file because it does not have the /api/v1 to hit the actuator endpoint [1] CreditHealthCheckTest.java
  33. None
  34. API contract Health Check Contract Functional Acceptance Verify it the

    endpoint is alive Assert that the specs haven’t changed Assert all the criteria from the requirement + happy/sad paths Assert that the most important user scenarios still works
  35. • It’s the name given to the pact between producer

    and consumer • Ensures that API changes do not invalidate consumption: • path • parameters • sending data (request) • return data (response body) • json-schema is a contract that defines the expected data, types and formats of each field contract
  36. { "name": "Elias", "age": 37 } { "$schema": "http://json-schema.org/draft-04/schema#", "type":

    "object", "properties": { "name": { "type": "string" }, "age": { "type": "integer" } }, "required": [ "name", "age" ], "additionalProperties": false } json-schema
  37. { "name": "Elias", "age": 37 } { "$schema": "http://json-schema.org/draft-04/schema#", "type":

    "object", "properties": { "name": { "type": "string" }, "age": { "type": "integer" } }, "required": [ "name", "age" ], "additionalProperties": false } json-schema has the attribute name and data type json-schema
  38. { "name": "Elias", "age": 37 } { "$schema": "http://json-schema.org/draft-04/schema#", "type":

    "object", "properties": { "name": { "type": "string" }, ”age": { "type": "integer" } }, "required": [ "name", "age" ], "additionalProperties": false } both attributes must be present json-schema
  39. { "name": "Elias", "age": 37 } { "$schema": "http://json-schema.org/draft-04/schema#", "type":

    "object", "properties": { "name": { "type": "string" }, "age": { "type": "integer" } }, "required": [ ”name", ”age" ], "additionalProperties": false } json-schema no other attributes are allowed
  40. [how to] contract 1. Create schema files ◦ We can

    use online tools to create if it’s not available 2. Create a test making a request and validating the json schema ◦ Add json-schema-validator library ◦ Statically import the matcher ◦ Use the matcher against json schema file
  41. [how to] contract In the example • See the implemented

    tests on: ◦ RestrictionsContractTest.java [1] ◦ SimulationsContractTest.java [2] • Items to pay attention: ◦ The validation is done by the matcher matchesJsonSchemaInClasspath, checking the response body with the json schema file ◦ If you want to see different API enable the test contractOnV2 on RestrictionsContractTests [1] RestrictionsContractTest.java [2] SimulationsContractTest.java
  42. None
  43. API functional Health Check Contract Functional Acceptance Verify it the

    endpoint is alive Assert that the specs haven’t changed Assert all the criteria from the requirement + happy/sad paths Assert that the most important user scenarios still works
  44. functional Validate positive and negative scenarios (happy and sad path)

    @Test(groups = {"functional"}) public void existentSocialSecirityNumber() { given(). pathParam("cpf", "66414919004"). when(). get("/simulations/{cpf}"). then(). statusCode(200). body( "id", equalTo(1), "name", equalTo("Oleksander"), "cpf", equalTo("66414919004"), "email", equalTo("oleksander@gmail.com"), "amount", equalTo(11000f), "installments", equalTo(3), "insurance", equalTo(true) ); } data validation
  45. [how to] functional 1. Create the tests avoiding repetitions ◦

    Use Test Data Factories and Data Driver in your advantage 2. Apply the strategy you like ◦ All tests in the same class ◦ One class per test
  46. [how to] functional In the example • See the implemented

    tests on: ◦ RestrictionsFunctionalTest.java [1] ◦ SimulationsFunctionalTest.java [2] • Items to pay attention: ◦ The tests are using a soft assertion for the body validation ◦ MessageFormat class is used to properly concatenate String ◦ SimulationsFunctionalTests is using Test Data Factory, Builders for the model object and Data Driven [1] RestrictionsFunctionalTests.java [2] SimulationsFunctionalTests.java
  47. None
  48. Assert all the criteria from the requirement + happy/sad paths

    API acceptance Health Check Contract Functional Acceptance Verify it the endpoint is alive Assert that the specs haven’t changed Assert that the most important user scenarios still works
  49. Testing from the user's perspective • Access the page and

    make the restriction search by the CPF • Insert a credit simulation
  50. acceptance (e2e) @Test(groups = "e2e") public void completeSimulation() { new

    RestrictionsClient().queryRestrictionAndReturnNotFound(); new SimulationsClient().submitSuccessfulSimulation(); }
  51. None
  52. [how to] acceptance (e2e) 1. Create the tests avoiding repetitions

    ◦ Use Request and Response Specifications ◦ Try to break out in reusable methods common actions 2. Do the User Journey ◦ The few most important user journeys must run before the functional tests https://martinfowler.com/articles/practical-test-pyramid.html#RestApiEnd-to-endTest
  53. [how to] functional In the example • See the implemented

    tests on: ◦ FullSimulationE2ETest.java [1] • Items to pay attention: ◦ There’s a utility class with the suffix Client in order to have an easy way to make request and response calls ◦ The request and response calls are Specification Re-use [2] created to reuse these actions ◦ See also the packages client [3] and specs [4] [1] FullSimulationE2ETest.java [2] https://github.com/rest-assured/rest-assured/wiki/usage#specification-re-use [3] client package [4] specs package
  54. For free Work in progress in 2 languages Access Leanpub

    and subscribe
  55. Thank you! @eliasnogueira github.com/eliasnogueira