Testing RESTful services: The What, The How and The Automated

Nowadays REST is a default architectural pattern for developing web services. But with great power comes great responsibility! Let's talk about what should be pinpointed and tested on a RESTful API level, what are the tips and tricks, where you could stumble upon common problems and what are the options on writing a beautifully crafted, maintainable test suit with Java or without. Furthermore let's talk about how to automate the whole beast to fit your delivery pipeline.

Transcript

1. Testing RESTful services
   The What, The How and The Automated
   @AlexeyBuzdin
   

   View Slide

2. Lets start
   with a tale…
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. Layered
   Architecture
   

   View Slide

7. View Slide

8. UI
   Service
   Unit
   

   View Slide

9. UI
   Service
   Unit
   Effort
   

   View Slide

10. UI
    Service
    Unit
    Effort Cost
    

    View Slide

11. View Slide

12. BFF
    BFF
    DB
    DB
    DB
    AMQP
    

    View Slide

13. BFF
    BFF
    DB
    DB
    DB
    AMQP
    

    View Slide

14. API is the new UI!
    

    View Slide

15. To test a microservice
    test it’s API
    

    View Slide

16. Why API tests?
    

    View Slide

17. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code
    

    View Slide

18. public class SimpleServlet extends GenericServlet {
    public void service(ServletRequest request, ServletResponse response)
    throws ServletException, IOException {
    // do something in here
    }
    }
    Simple Servlet
    

    View Slide

19. Simple Servlet
    @Controller
    @RequestMapping("/")
    public class SimpleController {
    @RequestMapping(method = RequestMethod.GET)
    public String get() {
    return “Hello”;
    }
    }
    (Annotation Driven Development … he.. he..)
    

    View Slide

20. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code
    

    View Slide

21. Simple Servlet
    @Controller
    @RequestMapping("/")
    public class SimpleController {
    @RequestMapping(method = POST)
    public String post(MyBean bean) {
    ….
    }
    }
    Annotation Driven Development
    

    View Slide

22. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code
    

    View Slide

23. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code
    

    View Slide

24. DB
    DB
    AMQP
    1. Isolate the Subject
    

    View Slide

25. Hint: Decouple integrations from
    your app
    Design your integrations to be easily pluggable
    and mockable in advance, or feel the
    consequences
    

    View Slide

26. http://projects.spring.io/spring-integration/
    

    View Slide

27. Spring Integration
    service-interface=“com.my.RequestGateway”
    default-request-channel="requestChannel"/>
    
    url="http://localhost:8080/http/receiveGateway"
    http-method="POST"
    expected-response-type="java.lang.String"/>
    

    View Slide

28. Spring Integration
    public interface RequestGateway {
    String echo(String request);
    }
    

    View Slide

29. Moco
    https://github.com/dreamhead/moco
    java -jar moco-runner.jar http -p 12306 -c foo.json
    {
    "request": {
    "json": { "foo": “bar" }
    },
    "response": { "text": “foo" }
    }
    

    View Slide

30. Moco
    https://github.com/dreamhead/moco
    server.request(by("foo")).response("bar");
    MocoHttpServer server = new MocoHttpServer(ActualHttpServer.createLogServer(of(port())));
    server.start();
    

    View Slide

31. For DB Mocking
    

    View Slide

32. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory
    

    View Slide

33. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory
    

    View Slide

34. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory
    

    View Slide

35. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory
    

    View Slide

36. DB
    DB
    AMQP
    2. Automate service startup
    

    View Slide

37. Application Server?
    

    View Slide

38. @RunWith(SpringJUnit4ClassRunner.class)
    @SpringApplicationConfiguration(classes = Application.class)
    @WebAppConfiguration
    @IntegrationTest("server.port:0")
    public class CharacterControllerTest {
    @Value("${local.server.port}")
    int port;
    ….
    }
    

    View Slide

39. @RunWith(Arquillian.class)
    public class GreeterTest {
    @Deployment
    public static WebArchive.class createDeployment() {
    return ShrinkWrap.create(WebArchive.class)
    .addClasses(Greeter.class)
    .setWebXML("WEB-INF/web.xml");
    }
    ….
    }
    https://github.com/arquillian
    

    View Slide

40. 3. Write a HealthCheck test
    @Test
    public void applicationIsUp() {
    given().port(8888).
    when().get("/healthcheck").
    then().statusCode(200);
    }
    https://github.com/jayway/rest-assured
    

    View Slide

41. https://github.com/jayway/rest-assured
    RequestSpecification - build the request
    ResponseSpecification - contains parsed response
    given().
    param("name", "Johan").
    cookie("cookie", "is mine").
    body("{}").
    header("Accept", "application/json")
    then().
    statusCode(200).
    header("Content-Type", "application/json")
    

    View Slide

42. @Test
    public void applicationIsUp() {
    given().port(8888).
    when().get("/healthcheck").
    then().statusCode(200);
    }
    Scenario: Application is up
    Given baseUri is http://localhost:8888/
    When GET request on /healthcheck
    Then status code is 200
    https://github.com/ctco/cukes-rest
    

    View Slide

43. Hint: Always make url/port
    configurable
    At some point you would like to run the tests
    against your hotfixed api or even generate some
    traffic on your staging environment
    

    View Slide

44. Lets test a Sample Server
    

    View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. REST is all about the resources
    /robots
    

    View Slide

50. C
    R
    U
    D
    

    View Slide

51. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE
    

    View Slide

52. Scenario: Should accept a cute RoboGirl
    Given request body:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    When POST request on /robots
    Then status code is 201
    HJson / HumanJSON
    

    View Slide

53. Scenario: Should accept a cute RoboGirl
    Given request body:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    When POST request on /robots
    Then status code is 201
    And header Location ends with "/robots/Tay.ai"
    

    View Slide

54. Accept tested
    • Application is started
    • Resource collection exists
    • Provided JSON is parsed successfully
    • No validation errors
    

    View Slide

55. But we haven’t tested if the
    resource was created on the server!
    

    View Slide

56. Scenario: Should create a new Robot
    …
    When POST request on /robots
    Then status code is 201
    And header Location ends with “/robots/Tay.ai"
    When GET request on {(header.Location)}
    Then status code is 200
    

    View Slide

57. Created tested
    • Application is started
    • Resource collection exists
    • Provided JSON is parsed successfully
    • No validation errors
    • Resource was stored on the server
    

    View Slide

58. What did we miss?
    Scenario: …..
    Given request body:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    When POST request on /robots
    

    View Slide

59. Test Actual Data
    ….
    When GET request on /robots/Tay.ai
    Then status code is 200
    And response equals to:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    

    View Slide

60. Unicode is not working
    java.lang.AssertionError: 1 expectation failed.
    Response body doesn't match expectation.
    Expected: "{\"name\" : \"Tay.ai\", \"personality\" : { \"name\": \"Tãy\",\"race\" : \"Hispanic\",
    \"age\" : 20}}"
    Actual: {"name" : "Tay.ai", "personality" : { "name" : "T y","race" : "Hispanic","age" : 20}}
    

    View Slide

61. Test Actual Data
    ….
    When GET request on /robots/Tay.ai
    Then status code is 200
    And response equals to:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    

    View Slide

62. Hint: Never use equals to compare
    server response
    Usually you have tons of stuff generated
    automatically on the backend which value can not
    be predicted in advance*:
    createdOn, updatedOn, auto generated IDs, etc
    

    View Slide

63. Hint: Use JSONPath and XPath to
    assert equality
    When GET request on /robots/Tay.ai
    Then status code is 200
    And response contains properties:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    name = Tay.ai
    personality.name = Tãy
    personality.race = Hispanic
    personality.age = 20
    

    View Slide

64. JSONPath and Arrays
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    skills: [
    "driving",
    "hacking",
    "translating"
    ]
    name = Tay.ai
    personality.name = Tãy
    personality.race = Hispanic
    personality.age = 20
    skills[0] = driving
    skills[1] = hacking
    skills[2] = translating
    

    View Slide

65. Hint: Arrays are unordered if
    fetched straight from DB
    Either implement order by or find the correct
    index programmatically
    

    View Slide

66. JSONPath and the World
    

    View Slide

67. To Sum Up
    

    View Slide

68. View Slide

69. Hint: Always prepare test data
    yourself
    You don’t want to have race conditions or any
    other unexpected trouble born from shared state.
    

    View Slide

70. Hint: Run your tests in parallel
    • Configure failsafe or gradle config to have
    maxParallelForks > 1
    • Organise your test scenarios into Test Suites
    with atomic sense
    

    View Slide

71. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    

    View Slide

72. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    

    View Slide

73. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    

    View Slide

74. • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    More POST on a resource
    

    View Slide

75. • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    More POST on a resource
    

    View Slide

76. Hint: Expose all errors and conditions
    to the client in RESTful way!
    REST is statless; be statless with errors as well!
    

    View Slide

77. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE
    

    View Slide

78. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE
    

    View Slide

79. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)
    

    View Slide

80. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)
    

    View Slide

81. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)
    

    View Slide

82. DELETE on a resource
    

    View Slide

83. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)
    

    View Slide

84. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)
    

    View Slide

85. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)
    

    View Slide

86. PUT on a resource
    

    View Slide

87. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

88. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

89. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

90. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

91. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

92. One CRUD but a lot of effort
    /robots
    

    View Slide

93. Hint: Invest into describing your API
    Either it be a API Doc or better an API
    representation in some data format
    

    View Slide

94. Open APIs
    https://openapis.org/
    

    View Slide

95. swagger: 2.0
    host: petstore.swagger.io
    schemes: [ “http" ]
    consumes: [ “application/json" ]
    produces: [ “application/json" ]
    paths: {
    /pets: {
    post: {
    summary: Create a pet
    responses: {
    201: { description: "Null response” }
    default: {
    description: unexpected error,
    schema: { $ref: “#/definitions/Error" }
    }
    ….
    }}}}
    Error: {
    required: [
    “code", “message"
    ],
    properties: {
    code: {
    type: integer
    format: int32
    },
    message: {
    type: string
    }
    }
    }
    

    View Slide

96. View Slide

97. Alternatives
    http://nordicapis.com/top-specification-formats-for-rest-apis/
    • http://raml.org/
    • https://github.com/apiaryio/api-blueprint
    

    View Slide

98. Asynchronous
    Scenario: …..
    ….
    When POST request on /robots
    Then status code is 202
    And header Location contains "/robots/Tay.ai"
    ….
    

    View Slide

99. Asynchronous
    await().until(newRobotIsCreated());
    await().until( getRobot(“Tay.ai”), isNotNull() );
    https://github.com/jayway/awaitility
    

    View Slide

100. Asynchronous
     https://github.com/jayway/awaitility
     with().pollInterval(ONE_HUNDERED_MILLISECONDS)
     .and().with().pollDelay(20, MILLISECONDS)
     .await(“creating a robot”)
     .until(getRobot(“Tay.ai”), hasProperty(“status”, is(“live”));
     

     View Slide

101. Other REST-assured features
     • form
     • basic
     • certificate
     • oauth
     • oauth2
     Easy authentication mechanism
     

     View Slide

102. Other REST-assured features
     • proxy support
     • JSON-Shema validation
     • Multi-value headers and cookies
     • Multi-part form data support
     • Filters
     

     View Slide

103. Points Made
     

     View Slide

104. Points Made
     • In addition to unit, integration and ui tests, cover at
     least core functionality and cases with API tests
     • Design your app integrations to be easily pluggable
     and mockable
     • Decouple connection settings with the test suite itself
     (uri, proxy settings, authentication methods, etc)
     • Never use equals to compare server response and try
     to identify the most significant properties for a the
     comparison, over making an all-to-all comparison
     

     View Slide

105. Points Made
     • In addition to unit, integration and ui tests, cover at
     least core functionality and cases with API tests
     • Design your app integrations to be easily pluggable
     and mockable
     • Decouple connection settings with the test suite itself
     (uri, proxy settings, authentication methods, etc)
     • Never use equals to compare server response and try
     to identify the most significant properties for a the
     comparison, over making an all-to-all comparison
     

     View Slide

106. Points Made
     • In addition to unit, integration and ui tests, cover at
     least core functionality and cases with API tests
     • Design your app integrations to be easily pluggable
     and mockable
     • Decouple connection settings with the test suite itself
     (uri, proxy settings, authentication methods, etc)
     • Never use equals to compare server response and try
     to identify the most significant properties for a the
     comparison, over making an all-to-all comparison
     

     View Slide

107. Points Made
     • In addition to unit, integration and ui tests, cover at
     least core functionality and cases with API tests
     • Design your app integrations to be easily pluggable
     and mockable
     • Decouple connection settings with the test suite itself
     (uri, proxy settings, authentication methods, etc)
     • Never use equals to compare server response and try
     to identify the most significant properties for a the
     comparison, over making an all-to-all comparison
     

     View Slide

108. Points Made
     • Don’t relay on existing data and prepare ones for
     testing before test starts
     • Design your tests to be easily parallel and sort
     them in a way that they won’t affect each other
     • Invest in having a descriptive API
     representation like Swagger, RAML, or API
     Blueprints
     • Use JSONPath to assert the results
     

     View Slide

109. Points Made
     • Don’t relay on existing data and prepare ones for
     testing before test starts
     • Design your tests to be easily parallel and sort
     them in a way that they won’t affect each other
     • Invest in having a descriptive API
     representation like Swagger, RAML, or API
     Blueprints
     • Use JSONPath to assert the results
     

     View Slide

110. Points Made
     • Don’t relay on existing data and prepare ones for
     testing before test starts
     • Design your tests to be easily parallel and sort
     them in a way that they won’t affect each other
     • Invest in having a descriptive API
     representation like Swagger, RAML, or API
     Blueprints
     • Use JSONPath to assert the results
     

     View Slide

111. Points Made
     • Don’t relay on existing data and prepare ones for
     testing before test starts
     • Design your tests to be easily parallel and sort
     them in a way that they won’t affect each other
     • Invest in having a descriptive API
     representation like Swagger, RAML, or API
     Blueprints
     • Use JSONPath/XPath to assert the results
     

     View Slide

112. Tools to look at
     • REST-assured or cukes-rest for testing RESTful api
     • Awaitility to deal with asynchronous jobs
     • Swagger, RAML, API Blueprint for API description
     • Arquillian to start up your Java EE based projects
     • Moco to startup a mock web service
     • HJSON is awesome!
     

     View Slide

113. TESTS ARE MADE
     TO MAKE YOU FEEL SECURE!
     

     View Slide

114. LIKE A LOVELY HUG ♥
     

     View Slide

115. Q&A
     Thank You!
     

     View Slide