Testing RESTful services: GoT JavaDay Kiev Edition

Nowadays REST is a default architectural pattern for developing web services. But with great power comes great responsibility! Let's talk about what should be pinpointed and tested on a RESTful API level, what are the tips and tricks, where you could stumble upon common problems and what are the options on writing a beautifully crafted, maintainable test suit with Java or without. Furthermore let's talk about how to automate the whole beast to fit your delivery pipeline.

Transcript

1. We came here to kick ass and test your
   services, and you’re all out of services
   @AlexeyBuzdin
   

   View Slide

2. @AlexeyBuzdin
   Testing RESTful services
   

   View Slide

3. @AlexeyBuzdin
   Developer / Trainer at
   GDGRiga.lv RigaDevDay.lv
   

   View Slide

4. Lets start
   with a tale…
   

   View Slide

5. View Slide

6. View Slide

7. View Slide

8. Layered
   Architecture
   

   View Slide

9. View Slide

10. UI
    Service
    Unit
    

    View Slide

11. UI
    Service
    Unit
    Effort
    

    View Slide

12. UI
    Service
    Unit
    Effort Cost
    

    View Slide

13. View Slide

14. BFF
    BFF
    DB
    DB
    DB
    AMQP
    

    View Slide

15. BFF
    BFF
    DB
    DB
    DB
    AMQP
    

    View Slide

16. API is the new UI!
    

    View Slide

17. test it’s API
    To test a microservice
    

    View Slide

18. Why API tests?
    

    View Slide

19. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code
    

    View Slide

20. public class SimpleServlet extends GenericServlet {

    

    public void service(ServletRequest req, ServletResponse res)

    throws ServletException, IOException {

    // do something in here

    }

    }
    Simple Servlet
    

    View Slide

21. Simple Servlet
    (Annotation Driven Development … he.. he..)
    @Controller

    @RequestMapping("/")

    public class SimpleController {

    

    @RequestMapping(method = RequestMethod.GET)

    public String get() {

    return "Hello";

    }

    }
    

    View Slide

22. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code
    

    View Slide

23. Simple Servlet
    @Controller

    @RequestMapping("/")

    public class SimpleController {

    

    @RequestMapping(method = POST)

    public String post(MyBean bean) {

    ...

    }

    }
    (Annotation Driven Development … he.. he..)
    

    View Slide

24. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code
    

    View Slide

25. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code
    

    View Slide

26. DB
    DB
    AMQP
    1. Isolate the Subject
    

    View Slide

27. Hint: Decouple integrations from
    your app
    Design your integrations to be easily pluggable
    and mockable in advance, or feel the
    consequences
    

    View Slide

28. http://projects.spring.io/spring-integration/
    

    View Slide

29. Spring Integration
    service-interface="com.my.RequestGateway"

    default-request-channel="requestChannel"/>

    

    

    

    url="http://localhost:8080/http/receiveGateway"

    http-method="POST"

    expected-response-type="java.lang.String"/>
    

    View Slide

30. Spring Integration
    public interface RequestGateway {

    String echo(String request);

    }
    

    View Slide

31. Moco
    https://github.com/dreamhead/moco
    java -jar moco-runner.jar http -p 12306 -c foo.json
    {

    "request": {

    "json": { "foo": "bar" }

    },

    "response": { "text": "foo" }

    }
    

    View Slide

32. Moco
    MocoHttpServer server = new MocoHttpServer(
    ActualHttpServer.createLogServer(of(port())));
    server.start();
    server.request(by("foo")).response("bar");
    https://github.com/dreamhead/moco
    

    View Slide

33. WireMock
    http://wiremock.org/
    @Rule
    public WireMockRule wireMockRule = new WireMockRule(8089);
    stubFor(get(urlEqualTo("/my/resource"))
    .withHeader("Accept", equalTo("text/xml"))
    .willReturn(aResponse()
    .withStatus(200)
    .withHeader("Content-Type", "text/xml")
    .withBody("Some content")));
    

    View Slide

34. For DB Mocking
    

    View Slide

35. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory
    

    View Slide

36. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory
    

    View Slide

37. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory
    

    View Slide

38. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory
    

    View Slide

39. DB
    DB
    AMQP
    2. Automate service startup
    

    View Slide

40. Application Server?
    

    View Slide

41. @RunWith(SpringJUnit4ClassRunner.class)

    @SpringApplicationConfiguration(classes = Application.class)

    @WebAppConfiguration

    @IntegrationTest("server.port:0")

    public class CharacterControllerTest {

    

    @Value("${local.server.port}")

    int port;
    ...

    }
    

    View Slide

42. https://github.com/arquillian
    @RunWith(Arquillian.class)

    public class GreeterTest {

    

    @Deployment

    public static WebArchive createDeployment() {

    return ShrinkWrap.create(WebArchive.class)

    .addClasses(Greeter.class)

    .setWebXML("WEB-INF/web.xml");

    }
    ...

    }
    

    View Slide

43. DB
    DB
    AMQP
    2. Automate service startup
    

    View Slide

44. 3. Write a HealthCheck test
    https://github.com/jayway/rest-assured
    @Test

    public void applicationIsUp() {

    given().port(8888).

    when().get("/healthcheck").

    then().statusCode(200);

    }
    

    View Slide

45. https://github.com/jayway/rest-assured
    RequestSpecification - build the request
    ResponseSpecification - contains parsed response
    given().

    param("name", "Johan").

    cookie("cookie", "is mine").

    body("{}").

    header("Accept", “application/json").
    then().

    statusCode(200).

    header("Content-Type", "application/json");

    when().post(“/resource").
    

    View Slide

46. Hint: Always make url/port
    configurable
    At some point you would like to run the tests
    against your hotfixed api or even generate some
    traffic on your staging environment
    

    View Slide

47. Lets test a Sample Server
    

    View Slide

48. View Slide

49. REST is all about the resources
    /houses
    

    View Slide

50. C
    R
    U
    D
    

    View Slide

51. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE
    

    View Slide

52. C
    R
    U
    D
    - GET
    

    View Slide

53. HJson / HumanJSON
    https://hjson.org/
    given().
    body(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString()).
    when().post("/houses").
    then().statusCode(201);
    .and().header("Location", "Iron Throne");
    

    View Slide

54. given().
    body(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString()).
    when().post("/houses").
    then().statusCode(201)
    .and().header("Location", "Iron Throne");
    

    View Slide

55. Accept tested
    • Application is started
    • Resource collection exists
    • Provided JSON is parsed successfully
    • No validation errors
    

    View Slide

56. But we haven’t tested if the
    resource was created on the server!
    

    View Slide

57. String location =
    …
    when().post("/houses").
    then().statusCode(201)
    .extract().header(“Location”);
    assertEquals("Iron Throne", location);
    get(location).then().statusCode(200);
    

    View Slide

58. Created tested
    • Application is started
    • Resource collection exists
    • Provided JSON is parsed successfully
    • No validation errors
    • Resource was stored on the server
    

    View Slide

59. given().
    body(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString()).
    when().post("/houses").
    then().statusCode(201);
    What did we miss?
    

    View Slide

60. assertEquals("Iron Throne", location);
    get(location).then().statusCode(200)
    .and().body(".", equalTo(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString());
    Test Actual Data
    

    View Slide

61. Unicode is not working
    java.lang.AssertionError: 1 expectation failed.
    Response body doesn't match expectation.
    Expected: “\”words\” : \”Hear Me Röar!\””
    Actual: ”words” : ”Hear Me R䡦ar!”
    

    View Slide

62. assertEquals("Iron Throne", location);
    get(location).then().statusCode(200)
    .and().body(".", equalTo(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString());
    Test Actual Data
    

    View Slide

63. Hint: Never use equals to compare
    server response
    Usually you have tons of stuff generated
    automatically on the backend which value can not
    be predicted in advance*:
    createdOn, updatedOn, auto generated IDs, etc
    

    View Slide

64. Hint: Use JSONPath and XPath to
    assert equality
    name = Lannister
    lore.sigil = …
    lore.words = …
    lore.shouldBe = …
    "name : Lannister" +
    "lore : {" +
    "sigil : …” +
    "words : ….” +
    "shouldBe : ….” +
    "}"
    

    View Slide

65. Hint: Use JSONPath and XPath to
    assert equality
    name = Lannister
    lore.sigil = …
    lore.words = …
    lore.shouldBe = …
    members[0] = Cercei
    members[1] = Tyrion
    members[2] = Jamie
    "name : Lannister" +
    "lore : {" +
    …
    “}"
    “members : [“ +
    “Cercei”
    “Tyrion”
    “Jamie”
    “]”
    

    View Slide

66. Hint: Implement order for your
    arrays or use contains matchers
    Arrays are unordered if fetched straight from DB
    

    View Slide

67. To Sum Up
    

    View Slide

68. View Slide

69. Hint: Always prepare test data
    yourself
    You don’t want to have race conditions or any
    other unexpected trouble born from shared state.
    

    View Slide

70. Hint: Run your tests in parallel
    • Configure failsafe or gradle config to have
    maxParallelForks > 1
    • Organise your test scenarios into Test Suites
    with atomic sense
    

    View Slide

71. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    

    View Slide

72. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    

    View Slide

73. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    

    View Slide

74. • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    More POST on a resource
    

    View Slide

75. • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    More POST on a resource
    

    View Slide

76. Hint: Expose all errors and conditions
    to the client in RESTful way!
    REST is stateless; pass errors to the client!
    

    View Slide

77. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE
    

    View Slide

78. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE
    

    View Slide

79. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)
    

    View Slide

80. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)
    

    View Slide

81. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)
    

    View Slide

82. DELETE on a resource
    

    View Slide

83. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)
    

    View Slide

84. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)
    

    View Slide

85. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)
    

    View Slide

86. PUT on a resource
    

    View Slide

87. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

88. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

89. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

90. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

91. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)
    

    View Slide

92. One Throne but a lot of deaths
    CRUD effort
    

    View Slide

93. Hint: Invest into describing your API
    Either it be a API Doc or better an API
    representation in some data format
    

    View Slide

94. Open APIs
    https://openapis.org/
    

    View Slide

95. swagger: 2.0
    host: petstore.swagger.io
    schemes: [ “http" ]
    consumes: [ “application/json" ]
    produces: [ “application/json" ]
    paths: {
    /pets: {
    post: {
    summary: Create a pet
    responses: {
    201: { description: "Null response” }
    default: {
    description: unexpected error,
    schema: { $ref: “#/definitions/Error" }
    }
    ….
    }}}}
    Error: {
    required: [
    “code", “message"
    ],
    properties: {
    code: {
    type: integer
    format: int32
    },
    message: {
    type: string
    }
    }
    }
    

    View Slide

96. View Slide

97. SpringFox
    http://springfox.github.io/springfox/
    @EnableSwagger2
    public class SwaggerConfiguration {
    @Override
    public void addResourceHandlers(ResourceHandlerRegistry registry) {
    registry.addResourceHandler("swagger-ui.html")
    .addResourceLocations("classpath:/META-INF/resources/");
    registry.addResourceHandler("/webjars/**")
    .addResourceLocations("classpath:/META-INF/resources/webjars/");
    }
    }
    

    View Slide

98. JAXRS-Analyzer
    https://github.com/sdaschner/jaxrs-analyzer
    Creates REST documentation for
    JAX-RS projects
    

    View Slide

99. Alternatives
    http://nordicapis.com/top-specification-formats-for-rest-apis/
    • http://raml.org/
    • https://github.com/apiaryio/api-blueprint
    

    View Slide

100. Asynchronous
     …
     when().post("/houses").
     then().statusCode(202)
     .and().header("Location", "Iron Throne”);
     …
     

     View Slide

101. Asynchronous
     await().until(newHouseEmerges());
     await().until( getHouse(“Lannister”), isNotNull() );
     https://github.com/jayway/awaitility
     

     View Slide

102. Asynchronous
     https://github.com/jayway/awaitility
     with().pollInterval(ONE_HUNDERED_MILLISECONDS)

     .and().with().pollDelay(20, MILLISECONDS)

     .await("creating a house")

     .until(getHouse("Lannister"), hasProperty("status", is("live")));
     

     View Slide

103. Other REST-assured features
     • form
     • basic
     • certificate
     • oauth
     • oauth2
     Easy authentication mechanism
     

     View Slide

104. Other REST-assured features
     • proxy support
     • JSON-Shema validation
     • Multi-value headers and cookies
     • Multi-part form data support
     • Filters
     

     View Slide

105. Points Made
     

     View Slide

106. Points Made
     • In addition to unit, integration and ui tests, cover at
     least core functionality and cases with API tests
     • Design your app integrations to be easily pluggable
     and mockable
     • Decouple connection settings with the test suite itself
     (uri, proxy settings, authentication methods, etc)
     • Never use equals to compare server response and try
     to identify the most significant properties for a the
     comparison, over making an all-to-all comparison
     

     View Slide

107. Points Made
     • In addition to unit, integration and ui tests, cover at
     least core functionality and cases with API tests
     • Design your app integrations to be easily pluggable
     and mockable
     • Decouple connection settings with the test suite itself
     (uri, proxy settings, authentication methods, etc)
     • Never use equals to compare server response and try
     to identify the most significant properties for a the
     comparison, over making an all-to-all comparison
     

     View Slide

108. Points Made
     • In addition to unit, integration and ui tests, cover at
     least core functionality and cases with API tests
     • Design your app integrations to be easily pluggable
     and mockable
     • Decouple connection settings with the test suite itself
     (uri, proxy settings, authentication methods, etc)
     • Never use equals to compare server response and try
     to identify the most significant properties for a the
     comparison, over making an all-to-all comparison
     

     View Slide

109. Points Made
     • In addition to unit, integration and ui tests, cover at
     least core functionality and cases with API tests
     • Design your app integrations to be easily pluggable
     and mockable
     • Decouple connection settings with the test suite itself
     (uri, proxy settings, authentication methods, etc)
     • Never use equals to compare server response and try
     to identify the most significant properties for a the
     comparison, over making an all-to-all comparison
     

     View Slide

110. Points Made
     • Don’t relay on existing data and prepare ones for
     testing before test starts
     • Design your tests to be easily parallel and sort
     them in a way that they won’t affect each other
     • Invest in having a descriptive API
     representation like Swagger, RAML, or API
     Blueprints
     • Use JSONPath to assert the results
     

     View Slide

111. Points Made
     • Don’t relay on existing data and prepare ones for
     testing before test starts
     • Design your tests to be easily parallel and sort
     them in a way that they won’t affect each other
     • Invest in having a descriptive API
     representation like Swagger, RAML, or API
     Blueprints
     • Use JSONPath to assert the results
     

     View Slide

112. Points Made
     • Don’t relay on existing data and prepare ones for
     testing before test starts
     • Design your tests to be easily parallel and sort
     them in a way that they won’t affect each other
     • Invest in having a descriptive API
     representation like Swagger, RAML, or API
     Blueprints
     • Use JSONPath to assert the results
     

     View Slide

113. Points Made
     • Don’t relay on existing data and prepare ones for
     testing before test starts
     • Design your tests to be easily parallel and sort
     them in a way that they won’t affect each other
     • Invest in having a descriptive API
     representation like Swagger, RAML, or API
     Blueprints
     • Use JSONPath/XPath to assert the results
     

     View Slide

114. Tools to look at
     • REST-assured for testing RESTful api
     • Awaitility to deal with asynchronous jobs
     • Swagger, RAML, API Blueprint for API description
     • Arquillian to start up your Java EE based projects
     • Moco or WireShark to startup a mock web service
     • HJSON is awesome!
     

     View Slide

115. TESTS ARE MADE
     TO MAKE YOU FEEL SECURE!
     

     View Slide

116. LIKE A LOVELY HUG ♥
     

     View Slide

117. Q&A
     Thank You!
     @AlexeyBuzdin
     Follow me at
     

     View Slide