Testing RESTful services: The What, The How and The Automated

Testing RESTful services: The What, The How and The Automated

Nowadays REST is a default architectural pattern for developing web services. But with great power comes great responsibility! Let's talk about what should be pinpointed and tested on a RESTful API level, what are the tips and tricks, where you could stumble upon common problems and what are the options on writing a beautifully crafted, maintainable test suit with Java or without. Furthermore let's talk about how to automate the whole beast to fit your delivery pipeline.

12d6ff93ca25d366161efccadd81bbb2?s=128

Alexey Buzdin

April 25, 2016
Tweet

Transcript

  1. 3.
  2. 4.
  3. 5.
  4. 7.
  5. 11.
  6. 17.

    Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  7. 18.

    public class SimpleServlet extends GenericServlet { public void service(ServletRequest request,

    ServletResponse response) throws ServletException, IOException { // do something in here } } Simple Servlet
  8. 19.

    Simple Servlet @Controller @RequestMapping("/") public class SimpleController { @RequestMapping(method =

    RequestMethod.GET) public String get() { return “Hello”; } } (Annotation Driven Development … he.. he..)
  9. 20.

    Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  10. 21.

    Simple Servlet @Controller @RequestMapping("/") public class SimpleController { @RequestMapping(method =

    POST) public String post(MyBean bean) { …. } } Annotation Driven Development
  11. 22.

    Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  12. 23.

    Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  13. 25.

    Hint: Decouple integrations from your app Design your integrations to

    be easily pluggable and mockable in advance, or feel the consequences
  14. 29.

    Moco https://github.com/dreamhead/moco java -jar moco-runner.jar http -p 12306 -c foo.json

    { "request": { "json": { "foo": “bar" } }, "response": { "text": “foo" } }
  15. 32.

    For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  16. 33.

    For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  17. 34.

    For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  18. 35.

    For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  19. 39.

    @RunWith(Arquillian.class) public class GreeterTest { @Deployment public static WebArchive.class createDeployment()

    { return ShrinkWrap.create(WebArchive.class) .addClasses(Greeter.class) .setWebXML("WEB-INF/web.xml"); } …. } https://github.com/arquillian
  20. 40.

    3. Write a HealthCheck test @Test public void applicationIsUp() {

    given().port(8888). when().get("/healthcheck"). then().statusCode(200); } https://github.com/jayway/rest-assured
  21. 41.

    https://github.com/jayway/rest-assured RequestSpecification - build the request ResponseSpecification - contains parsed

    response given(). param("name", "Johan"). cookie("cookie", "is mine"). body("{}"). header("Accept", "application/json") then(). statusCode(200). header("Content-Type", "application/json")
  22. 42.

    @Test public void applicationIsUp() { given().port(8888). when().get("/healthcheck"). then().statusCode(200); } Scenario:

    Application is up Given baseUri is http://localhost:8888/ When GET request on /healthcheck Then status code is 200 https://github.com/ctco/cukes-rest
  23. 43.

    Hint: Always make url/port configurable At some point you would

    like to run the tests against your hotfixed api or even generate some traffic on your staging environment
  24. 45.
  25. 46.
  26. 47.
  27. 48.
  28. 50.
  29. 52.

    Scenario: Should accept a cute RoboGirl Given request body: """

    name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ When POST request on /robots Then status code is 201 HJson / HumanJSON
  30. 53.

    Scenario: Should accept a cute RoboGirl Given request body: """

    name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ When POST request on /robots Then status code is 201 And header Location ends with "/robots/Tay.ai"
  31. 54.

    Accept tested • Application is started • Resource collection exists

    • Provided JSON is parsed successfully • No validation errors
  32. 56.

    Scenario: Should create a new Robot … When POST request

    on /robots Then status code is 201 And header Location ends with “/robots/Tay.ai" When GET request on {(header.Location)} Then status code is 200
  33. 57.

    Created tested • Application is started • Resource collection exists

    • Provided JSON is parsed successfully • No validation errors • Resource was stored on the server
  34. 58.

    What did we miss? Scenario: ….. Given request body: """

    name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ When POST request on /robots
  35. 59.

    Test Actual Data …. When GET request on /robots/Tay.ai Then

    status code is 200 And response equals to: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """
  36. 60.

    Unicode is not working java.lang.AssertionError: 1 expectation failed. Response body

    doesn't match expectation. Expected: "{\"name\" : \"Tay.ai\", \"personality\" : { \"name\": \"Tãy\",\"race\" : \"Hispanic\", \"age\" : 20}}" Actual: {"name" : "Tay.ai", "personality" : { "name" : "T y","race" : "Hispanic","age" : 20}}
  37. 61.

    Test Actual Data …. When GET request on /robots/Tay.ai Then

    status code is 200 And response equals to: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """
  38. 62.

    Hint: Never use equals to compare server response Usually you

    have tons of stuff generated automatically on the backend which value can not be predicted in advance*: createdOn, updatedOn, auto generated IDs, etc
  39. 63.

    Hint: Use JSONPath and XPath to assert equality When GET

    request on /robots/Tay.ai Then status code is 200 And response contains properties: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ name = Tay.ai personality.name = Tãy personality.race = Hispanic personality.age = 20
  40. 64.

    JSONPath and Arrays name : Tay.ai personality : { name

    : Tãy race : Hispanic age : 20 } skills: [ "driving", "hacking", "translating" ] name = Tay.ai personality.name = Tãy personality.race = Hispanic personality.age = 20 skills[0] = driving skills[1] = hacking skills[2] = translating
  41. 65.

    Hint: Arrays are unordered if fetched straight from DB Either

    implement order by or find the correct index programmatically
  42. 67.
  43. 68.
  44. 69.

    Hint: Always prepare test data yourself You don’t want to

    have race conditions or any other unexpected trouble born from shared state.
  45. 70.

    Hint: Run your tests in parallel • Configure failsafe or

    gradle config to have maxParallelForks > 1 • Organise your test scenarios into Test Suites with atomic sense
  46. 71.

    More POST on a resource • Created successfully -> Created

    (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
  47. 72.

    More POST on a resource • Created successfully -> Created

    (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
  48. 73.

    More POST on a resource • Created successfully -> Created

    (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
  49. 74.

    • Created successfully -> Created (201) • Missing a mandatory

    property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
  50. 75.

    • Created successfully -> Created (201) • Missing a mandatory

    property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
  51. 76.

    Hint: Expose all errors and conditions to the client in

    RESTful way! REST is statless; be statless with errors as well!
  52. 79.

    More GET on a resource • Get existing resource ->

    OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
  53. 80.

    More GET on a resource • Get existing resource ->

    OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
  54. 81.

    More GET on a resource • Get existing resource ->

    OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
  55. 83.

    DELETE on a resource • Delete non existing resource ->

    Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
  56. 84.

    DELETE on a resource • Delete non existing resource ->

    Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
  57. 85.

    DELETE on a resource • Delete non existing resource ->

    Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
  58. 87.

    PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  59. 88.

    PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  60. 89.

    PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  61. 90.

    PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  62. 91.

    PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  63. 93.

    Hint: Invest into describing your API Either it be a

    API Doc or better an API representation in some data format
  64. 95.

    swagger: 2.0 host: petstore.swagger.io schemes: [ “http" ] consumes: [

    “application/json" ] produces: [ “application/json" ] paths: { /pets: { post: { summary: Create a pet responses: { 201: { description: "Null response” } default: { description: unexpected error, schema: { $ref: “#/definitions/Error" } } …. }}}} Error: { required: [ “code", “message" ], properties: { code: { type: integer format: int32 }, message: { type: string } } }
  65. 96.
  66. 98.

    Asynchronous Scenario: ….. …. When POST request on /robots Then

    status code is 202 And header Location contains "/robots/Tay.ai" ….
  67. 101.

    Other REST-assured features • form • basic • certificate •

    oauth • oauth2 Easy authentication mechanism
  68. 102.

    Other REST-assured features • proxy support • JSON-Shema validation •

    Multi-value headers and cookies • Multi-part form data support • Filters
  69. 104.

    Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  70. 105.

    Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  71. 106.

    Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  72. 107.

    Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  73. 108.

    Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
  74. 109.

    Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
  75. 110.

    Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
  76. 111.

    Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath/XPath to assert the results
  77. 112.

    Tools to look at • REST-assured or cukes-rest for testing

    RESTful api • Awaitility to deal with asynchronous jobs • Swagger, RAML, API Blueprint for API description • Arquillian to start up your Java EE based projects • Moco to startup a mock web service • HJSON is awesome!