Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing RESTful services: GoT JavaDay Kiev Edition

Testing RESTful services: GoT JavaDay Kiev Edition

Nowadays REST is a default architectural pattern for developing web services. But with great power comes great responsibility! Let's talk about what should be pinpointed and tested on a RESTful API level, what are the tips and tricks, where you could stumble upon common problems and what are the options on writing a beautifully crafted, maintainable test suit with Java or without. Furthermore let's talk about how to automate the whole beast to fit your delivery pipeline.

Alexey Buzdin

October 15, 2016
Tweet

More Decks by Alexey Buzdin

Other Decks in Programming

Transcript

  1. We came here to kick ass and test your
    services, and you’re all out of services
    @AlexeyBuzdin

    View Slide

  2. @AlexeyBuzdin
    Testing RESTful services

    View Slide

  3. @AlexeyBuzdin
    Developer / Trainer at
    GDGRiga.lv RigaDevDay.lv

    View Slide

  4. Lets start
    with a tale…

    View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. Layered
    Architecture

    View Slide

  9. View Slide

  10. UI
    Service
    Unit

    View Slide

  11. UI
    Service
    Unit
    Effort

    View Slide

  12. UI
    Service
    Unit
    Effort Cost

    View Slide

  13. View Slide

  14. BFF
    BFF
    DB
    DB
    DB
    AMQP

    View Slide

  15. BFF
    BFF
    DB
    DB
    DB
    AMQP

    View Slide

  16. API is the new UI!

    View Slide

  17. test it’s API
    To test a microservice

    View Slide

  18. Why API tests?

    View Slide

  19. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code

    View Slide

  20. public class SimpleServlet extends GenericServlet {


    public void service(ServletRequest req, ServletResponse res)

    throws ServletException, IOException {

    // do something in here

    }

    }
    Simple Servlet

    View Slide

  21. Simple Servlet
    (Annotation Driven Development … he.. he..)
    @Controller

    @RequestMapping("/")

    public class SimpleController {


    @RequestMapping(method = RequestMethod.GET)

    public String get() {

    return "Hello";

    }

    }

    View Slide

  22. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code

    View Slide

  23. Simple Servlet
    @Controller

    @RequestMapping("/")

    public class SimpleController {


    @RequestMapping(method = POST)

    public String post(MyBean bean) {

    ...

    }

    }
    (Annotation Driven Development … he.. he..)

    View Slide

  24. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code

    View Slide

  25. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code

    View Slide

  26. DB
    DB
    AMQP
    1. Isolate the Subject

    View Slide

  27. Hint: Decouple integrations from
    your app
    Design your integrations to be easily pluggable
    and mockable in advance, or feel the
    consequences

    View Slide

  28. http://projects.spring.io/spring-integration/

    View Slide

  29. Spring Integration
    service-interface="com.my.RequestGateway"

    default-request-channel="requestChannel"/>




    url="http://localhost:8080/http/receiveGateway"

    http-method="POST"

    expected-response-type="java.lang.String"/>

    View Slide

  30. Spring Integration
    public interface RequestGateway {

    String echo(String request);

    }

    View Slide

  31. Moco
    https://github.com/dreamhead/moco
    java -jar moco-runner.jar http -p 12306 -c foo.json
    {

    "request": {

    "json": { "foo": "bar" }

    },

    "response": { "text": "foo" }

    }

    View Slide

  32. Moco
    MocoHttpServer server = new MocoHttpServer(
    ActualHttpServer.createLogServer(of(port())));
    server.start();
    server.request(by("foo")).response("bar");
    https://github.com/dreamhead/moco

    View Slide

  33. WireMock
    http://wiremock.org/
    @Rule
    public WireMockRule wireMockRule = new WireMockRule(8089);
    stubFor(get(urlEqualTo("/my/resource"))
    .withHeader("Accept", equalTo("text/xml"))
    .willReturn(aResponse()
    .withStatus(200)
    .withHeader("Content-Type", "text/xml")
    .withBody("Some content")));

    View Slide

  34. For DB Mocking

    View Slide

  35. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory

    View Slide

  36. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory

    View Slide

  37. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory

    View Slide

  38. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory

    View Slide

  39. DB
    DB
    AMQP
    2. Automate service startup

    View Slide

  40. Application Server?

    View Slide

  41. @RunWith(SpringJUnit4ClassRunner.class)

    @SpringApplicationConfiguration(classes = Application.class)

    @WebAppConfiguration

    @IntegrationTest("server.port:0")

    public class CharacterControllerTest {


    @Value("${local.server.port}")

    int port;
    ...

    }

    View Slide

  42. https://github.com/arquillian
    @RunWith(Arquillian.class)

    public class GreeterTest {


    @Deployment

    public static WebArchive createDeployment() {

    return ShrinkWrap.create(WebArchive.class)

    .addClasses(Greeter.class)

    .setWebXML("WEB-INF/web.xml");

    }
    ...

    }

    View Slide

  43. DB
    DB
    AMQP
    2. Automate service startup

    View Slide

  44. 3. Write a HealthCheck test
    https://github.com/jayway/rest-assured
    @Test

    public void applicationIsUp() {

    given().port(8888).

    when().get("/healthcheck").

    then().statusCode(200);

    }

    View Slide

  45. https://github.com/jayway/rest-assured
    RequestSpecification - build the request
    ResponseSpecification - contains parsed response
    given().

    param("name", "Johan").

    cookie("cookie", "is mine").

    body("{}").

    header("Accept", “application/json").
    then().

    statusCode(200).

    header("Content-Type", "application/json");

    when().post(“/resource").

    View Slide

  46. Hint: Always make url/port
    configurable
    At some point you would like to run the tests
    against your hotfixed api or even generate some
    traffic on your staging environment

    View Slide

  47. Lets test a Sample Server

    View Slide

  48. View Slide

  49. REST is all about the resources
    /houses

    View Slide

  50. C
    R
    U
    D

    View Slide

  51. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE

    View Slide

  52. C
    R
    U
    D
    - GET

    View Slide

  53. HJson / HumanJSON
    https://hjson.org/
    given().
    body(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString()).
    when().post("/houses").
    then().statusCode(201);
    .and().header("Location", "Iron Throne");

    View Slide

  54. given().
    body(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString()).
    when().post("/houses").
    then().statusCode(201)
    .and().header("Location", "Iron Throne");

    View Slide

  55. Accept tested
    • Application is started
    • Resource collection exists
    • Provided JSON is parsed successfully
    • No validation errors

    View Slide

  56. But we haven’t tested if the
    resource was created on the server!

    View Slide

  57. String location =

    when().post("/houses").
    then().statusCode(201)
    .extract().header(“Location”);
    assertEquals("Iron Throne", location);
    get(location).then().statusCode(200);

    View Slide

  58. Created tested
    • Application is started
    • Resource collection exists
    • Provided JSON is parsed successfully
    • No validation errors
    • Resource was stored on the server

    View Slide

  59. given().
    body(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString()).
    when().post("/houses").
    then().statusCode(201);
    What did we miss?

    View Slide

  60. assertEquals("Iron Throne", location);
    get(location).then().statusCode(200)
    .and().body(".", equalTo(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString());
    Test Actual Data

    View Slide

  61. Unicode is not working
    java.lang.AssertionError: 1 expectation failed.
    Response body doesn't match expectation.
    Expected: “\”words\” : \”Hear Me Röar!\””
    Actual: ”words” : ”Hear Me R䡦ar!”

    View Slide

  62. assertEquals("Iron Throne", location);
    get(location).then().statusCode(200)
    .and().body(".", equalTo(readHjson(
    "name : Lannister" +
    "lore : {" +
    "sigil : A golden lion rampant on a crimson field." +
    "words : Hear Me Röar!” +
    "shouldBe : Twincest is Wincest!” +
    "}"
    ).toString());
    Test Actual Data

    View Slide

  63. Hint: Never use equals to compare
    server response
    Usually you have tons of stuff generated
    automatically on the backend which value can not
    be predicted in advance*:
    createdOn, updatedOn, auto generated IDs, etc

    View Slide

  64. Hint: Use JSONPath and XPath to
    assert equality
    name = Lannister
    lore.sigil = …
    lore.words = …
    lore.shouldBe = …
    "name : Lannister" +
    "lore : {" +
    "sigil : …” +
    "words : ….” +
    "shouldBe : ….” +
    "}"

    View Slide

  65. Hint: Use JSONPath and XPath to
    assert equality
    name = Lannister
    lore.sigil = …
    lore.words = …
    lore.shouldBe = …
    members[0] = Cercei
    members[1] = Tyrion
    members[2] = Jamie
    "name : Lannister" +
    "lore : {" +

    “}"
    “members : [“ +
    “Cercei”
    “Tyrion”
    “Jamie”
    “]”

    View Slide

  66. Hint: Implement order for your
    arrays or use contains matchers
    Arrays are unordered if fetched straight from DB

    View Slide

  67. To Sum Up

    View Slide

  68. View Slide

  69. Hint: Always prepare test data
    yourself
    You don’t want to have race conditions or any
    other unexpected trouble born from shared state.

    View Slide

  70. Hint: Run your tests in parallel
    • Configure failsafe or gradle config to have
    maxParallelForks > 1
    • Organise your test scenarios into Test Suites
    with atomic sense

    View Slide

  71. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)

    View Slide

  72. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)

    View Slide

  73. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)

    View Slide

  74. • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    More POST on a resource

    View Slide

  75. • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    More POST on a resource

    View Slide

  76. Hint: Expose all errors and conditions
    to the client in RESTful way!
    REST is stateless; pass errors to the client!

    View Slide

  77. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE

    View Slide

  78. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE

    View Slide

  79. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)

    View Slide

  80. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)

    View Slide

  81. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)

    View Slide

  82. DELETE on a resource

    View Slide

  83. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)

    View Slide

  84. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)

    View Slide

  85. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)

    View Slide

  86. PUT on a resource

    View Slide

  87. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  88. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  89. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  90. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  91. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  92. One Throne but a lot of deaths
    CRUD effort

    View Slide

  93. Hint: Invest into describing your API
    Either it be a API Doc or better an API
    representation in some data format

    View Slide

  94. Open APIs
    https://openapis.org/

    View Slide

  95. swagger: 2.0
    host: petstore.swagger.io
    schemes: [ “http" ]
    consumes: [ “application/json" ]
    produces: [ “application/json" ]
    paths: {
    /pets: {
    post: {
    summary: Create a pet
    responses: {
    201: { description: "Null response” }
    default: {
    description: unexpected error,
    schema: { $ref: “#/definitions/Error" }
    }
    ….
    }}}}
    Error: {
    required: [
    “code", “message"
    ],
    properties: {
    code: {
    type: integer
    format: int32
    },
    message: {
    type: string
    }
    }
    }

    View Slide

  96. View Slide

  97. SpringFox
    http://springfox.github.io/springfox/
    @EnableSwagger2
    public class SwaggerConfiguration {
    @Override
    public void addResourceHandlers(ResourceHandlerRegistry registry) {
    registry.addResourceHandler("swagger-ui.html")
    .addResourceLocations("classpath:/META-INF/resources/");
    registry.addResourceHandler("/webjars/**")
    .addResourceLocations("classpath:/META-INF/resources/webjars/");
    }
    }

    View Slide

  98. JAXRS-Analyzer
    https://github.com/sdaschner/jaxrs-analyzer
    Creates REST documentation for
    JAX-RS projects

    View Slide

  99. Alternatives
    http://nordicapis.com/top-specification-formats-for-rest-apis/
    • http://raml.org/
    • https://github.com/apiaryio/api-blueprint

    View Slide

  100. Asynchronous

    when().post("/houses").
    then().statusCode(202)
    .and().header("Location", "Iron Throne”);

    View Slide

  101. Asynchronous
    await().until(newHouseEmerges());
    await().until( getHouse(“Lannister”), isNotNull() );
    https://github.com/jayway/awaitility

    View Slide

  102. Asynchronous
    https://github.com/jayway/awaitility
    with().pollInterval(ONE_HUNDERED_MILLISECONDS)

    .and().with().pollDelay(20, MILLISECONDS)

    .await("creating a house")

    .until(getHouse("Lannister"), hasProperty("status", is("live")));

    View Slide

  103. Other REST-assured features
    • form
    • basic
    • certificate
    • oauth
    • oauth2
    Easy authentication mechanism

    View Slide

  104. Other REST-assured features
    • proxy support
    • JSON-Shema validation
    • Multi-value headers and cookies
    • Multi-part form data support
    • Filters

    View Slide

  105. Points Made

    View Slide

  106. Points Made
    • In addition to unit, integration and ui tests, cover at
    least core functionality and cases with API tests
    • Design your app integrations to be easily pluggable
    and mockable
    • Decouple connection settings with the test suite itself
    (uri, proxy settings, authentication methods, etc)
    • Never use equals to compare server response and try
    to identify the most significant properties for a the
    comparison, over making an all-to-all comparison

    View Slide

  107. Points Made
    • In addition to unit, integration and ui tests, cover at
    least core functionality and cases with API tests
    • Design your app integrations to be easily pluggable
    and mockable
    • Decouple connection settings with the test suite itself
    (uri, proxy settings, authentication methods, etc)
    • Never use equals to compare server response and try
    to identify the most significant properties for a the
    comparison, over making an all-to-all comparison

    View Slide

  108. Points Made
    • In addition to unit, integration and ui tests, cover at
    least core functionality and cases with API tests
    • Design your app integrations to be easily pluggable
    and mockable
    • Decouple connection settings with the test suite itself
    (uri, proxy settings, authentication methods, etc)
    • Never use equals to compare server response and try
    to identify the most significant properties for a the
    comparison, over making an all-to-all comparison

    View Slide

  109. Points Made
    • In addition to unit, integration and ui tests, cover at
    least core functionality and cases with API tests
    • Design your app integrations to be easily pluggable
    and mockable
    • Decouple connection settings with the test suite itself
    (uri, proxy settings, authentication methods, etc)
    • Never use equals to compare server response and try
    to identify the most significant properties for a the
    comparison, over making an all-to-all comparison

    View Slide

  110. Points Made
    • Don’t relay on existing data and prepare ones for
    testing before test starts
    • Design your tests to be easily parallel and sort
    them in a way that they won’t affect each other
    • Invest in having a descriptive API
    representation like Swagger, RAML, or API
    Blueprints
    • Use JSONPath to assert the results

    View Slide

  111. Points Made
    • Don’t relay on existing data and prepare ones for
    testing before test starts
    • Design your tests to be easily parallel and sort
    them in a way that they won’t affect each other
    • Invest in having a descriptive API
    representation like Swagger, RAML, or API
    Blueprints
    • Use JSONPath to assert the results

    View Slide

  112. Points Made
    • Don’t relay on existing data and prepare ones for
    testing before test starts
    • Design your tests to be easily parallel and sort
    them in a way that they won’t affect each other
    • Invest in having a descriptive API
    representation like Swagger, RAML, or API
    Blueprints
    • Use JSONPath to assert the results

    View Slide

  113. Points Made
    • Don’t relay on existing data and prepare ones for
    testing before test starts
    • Design your tests to be easily parallel and sort
    them in a way that they won’t affect each other
    • Invest in having a descriptive API
    representation like Swagger, RAML, or API
    Blueprints
    • Use JSONPath/XPath to assert the results

    View Slide

  114. Tools to look at
    • REST-assured for testing RESTful api
    • Awaitility to deal with asynchronous jobs
    • Swagger, RAML, API Blueprint for API description
    • Arquillian to start up your Java EE based projects
    • Moco or WireShark to startup a mock web service
    • HJSON is awesome!

    View Slide

  115. TESTS ARE MADE
    TO MAKE YOU FEEL SECURE!

    View Slide

  116. LIKE A LOVELY HUG ♥

    View Slide

  117. Q&A
    Thank You!
    @AlexeyBuzdin
    Follow me at

    View Slide