Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing RESTful services: The What, The How and The Automated

Testing RESTful services: The What, The How and The Automated

Nowadays REST is a default architectural pattern for developing web services. But with great power comes great responsibility! Let's talk about what should be pinpointed and tested on a RESTful API level, what are the tips and tricks, where you could stumble upon common problems and what are the options on writing a beautifully crafted, maintainable test suit with Java or without. Furthermore let's talk about how to automate the whole beast to fit your delivery pipeline.

Alexey Buzdin

April 25, 2016
Tweet

More Decks by Alexey Buzdin

Other Decks in Programming

Transcript

  1. Testing RESTful services
    The What, The How and The Automated
    @AlexeyBuzdin

    View Slide

  2. Lets start
    with a tale…

    View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. Layered
    Architecture

    View Slide

  7. View Slide

  8. UI
    Service
    Unit

    View Slide

  9. UI
    Service
    Unit
    Effort

    View Slide

  10. UI
    Service
    Unit
    Effort Cost

    View Slide

  11. View Slide

  12. BFF
    BFF
    DB
    DB
    DB
    AMQP

    View Slide

  13. BFF
    BFF
    DB
    DB
    DB
    AMQP

    View Slide

  14. API is the new UI!

    View Slide

  15. To test a microservice
    test it’s API

    View Slide

  16. Why API tests?

    View Slide

  17. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code

    View Slide

  18. public class SimpleServlet extends GenericServlet {
    public void service(ServletRequest request, ServletResponse response)
    throws ServletException, IOException {
    // do something in here
    }
    }
    Simple Servlet

    View Slide

  19. Simple Servlet
    @Controller
    @RequestMapping("/")
    public class SimpleController {
    @RequestMapping(method = RequestMethod.GET)
    public String get() {
    return “Hello”;
    }
    }
    (Annotation Driven Development … he.. he..)

    View Slide

  20. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code

    View Slide

  21. Simple Servlet
    @Controller
    @RequestMapping("/")
    public class SimpleController {
    @RequestMapping(method = POST)
    public String post(MyBean bean) {
    ….
    }
    }
    Annotation Driven Development

    View Slide

  22. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code

    View Slide

  23. Why API tests?
    • Tests routing
    • Tests serialization/deserialization
    • Tests server response
    • Can be written by a QA who does not knows the
    code

    View Slide

  24. DB
    DB
    AMQP
    1. Isolate the Subject

    View Slide

  25. Hint: Decouple integrations from
    your app
    Design your integrations to be easily pluggable
    and mockable in advance, or feel the
    consequences

    View Slide

  26. http://projects.spring.io/spring-integration/

    View Slide

  27. Spring Integration
    service-interface=“com.my.RequestGateway”
    default-request-channel="requestChannel"/>

    url="http://localhost:8080/http/receiveGateway"
    http-method="POST"
    expected-response-type="java.lang.String"/>

    View Slide

  28. Spring Integration
    public interface RequestGateway {
    String echo(String request);
    }

    View Slide

  29. Moco
    https://github.com/dreamhead/moco
    java -jar moco-runner.jar http -p 12306 -c foo.json
    {
    "request": {
    "json": { "foo": “bar" }
    },
    "response": { "text": “foo" }
    }

    View Slide

  30. Moco
    https://github.com/dreamhead/moco
    server.request(by("foo")).response("bar");
    MocoHttpServer server = new MocoHttpServer(ActualHttpServer.createLogServer(of(port())));
    server.start();

    View Slide

  31. For DB Mocking

    View Slide

  32. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory

    View Slide

  33. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory

    View Slide

  34. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory

    View Slide

  35. For DB Mocking
    • Substitute to an in-memory DB (Spring Data and
    JPA/ Hibernate will help here a lot)
    • Start up a clean instance of a DB for testing
    • Reuse developers instance for testing purposes
    • Mock data to be fetched from file system or
    memory

    View Slide

  36. DB
    DB
    AMQP
    2. Automate service startup

    View Slide

  37. Application Server?

    View Slide

  38. @RunWith(SpringJUnit4ClassRunner.class)
    @SpringApplicationConfiguration(classes = Application.class)
    @WebAppConfiguration
    @IntegrationTest("server.port:0")
    public class CharacterControllerTest {
    @Value("${local.server.port}")
    int port;
    ….
    }

    View Slide

  39. @RunWith(Arquillian.class)
    public class GreeterTest {
    @Deployment
    public static WebArchive.class createDeployment() {
    return ShrinkWrap.create(WebArchive.class)
    .addClasses(Greeter.class)
    .setWebXML("WEB-INF/web.xml");
    }
    ….
    }
    https://github.com/arquillian

    View Slide

  40. 3. Write a HealthCheck test
    @Test
    public void applicationIsUp() {
    given().port(8888).
    when().get("/healthcheck").
    then().statusCode(200);
    }
    https://github.com/jayway/rest-assured

    View Slide

  41. https://github.com/jayway/rest-assured
    RequestSpecification - build the request
    ResponseSpecification - contains parsed response
    given().
    param("name", "Johan").
    cookie("cookie", "is mine").
    body("{}").
    header("Accept", "application/json")
    then().
    statusCode(200).
    header("Content-Type", "application/json")

    View Slide

  42. @Test
    public void applicationIsUp() {
    given().port(8888).
    when().get("/healthcheck").
    then().statusCode(200);
    }
    Scenario: Application is up
    Given baseUri is http://localhost:8888/
    When GET request on /healthcheck
    Then status code is 200
    https://github.com/ctco/cukes-rest

    View Slide

  43. Hint: Always make url/port
    configurable
    At some point you would like to run the tests
    against your hotfixed api or even generate some
    traffic on your staging environment

    View Slide

  44. Lets test a Sample Server

    View Slide

  45. View Slide

  46. View Slide

  47. View Slide

  48. View Slide

  49. REST is all about the resources
    /robots

    View Slide

  50. C
    R
    U
    D

    View Slide

  51. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE

    View Slide

  52. Scenario: Should accept a cute RoboGirl
    Given request body:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    When POST request on /robots
    Then status code is 201
    HJson / HumanJSON

    View Slide

  53. Scenario: Should accept a cute RoboGirl
    Given request body:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    When POST request on /robots
    Then status code is 201
    And header Location ends with "/robots/Tay.ai"

    View Slide

  54. Accept tested
    • Application is started
    • Resource collection exists
    • Provided JSON is parsed successfully
    • No validation errors

    View Slide

  55. But we haven’t tested if the
    resource was created on the server!

    View Slide

  56. Scenario: Should create a new Robot

    When POST request on /robots
    Then status code is 201
    And header Location ends with “/robots/Tay.ai"
    When GET request on {(header.Location)}
    Then status code is 200

    View Slide

  57. Created tested
    • Application is started
    • Resource collection exists
    • Provided JSON is parsed successfully
    • No validation errors
    • Resource was stored on the server

    View Slide

  58. What did we miss?
    Scenario: …..
    Given request body:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    When POST request on /robots

    View Slide

  59. Test Actual Data
    ….
    When GET request on /robots/Tay.ai
    Then status code is 200
    And response equals to:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """

    View Slide

  60. Unicode is not working
    java.lang.AssertionError: 1 expectation failed.
    Response body doesn't match expectation.
    Expected: "{\"name\" : \"Tay.ai\", \"personality\" : { \"name\": \"Tãy\",\"race\" : \"Hispanic\",
    \"age\" : 20}}"
    Actual: {"name" : "Tay.ai", "personality" : { "name" : "T y","race" : "Hispanic","age" : 20}}

    View Slide

  61. Test Actual Data
    ….
    When GET request on /robots/Tay.ai
    Then status code is 200
    And response equals to:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """

    View Slide

  62. Hint: Never use equals to compare
    server response
    Usually you have tons of stuff generated
    automatically on the backend which value can not
    be predicted in advance*:
    createdOn, updatedOn, auto generated IDs, etc

    View Slide

  63. Hint: Use JSONPath and XPath to
    assert equality
    When GET request on /robots/Tay.ai
    Then status code is 200
    And response contains properties:
    """
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    """
    name = Tay.ai
    personality.name = Tãy
    personality.race = Hispanic
    personality.age = 20

    View Slide

  64. JSONPath and Arrays
    name : Tay.ai
    personality : {
    name : Tãy
    race : Hispanic
    age : 20
    }
    skills: [
    "driving",
    "hacking",
    "translating"
    ]
    name = Tay.ai
    personality.name = Tãy
    personality.race = Hispanic
    personality.age = 20
    skills[0] = driving
    skills[1] = hacking
    skills[2] = translating

    View Slide

  65. Hint: Arrays are unordered if
    fetched straight from DB
    Either implement order by or find the correct
    index programmatically

    View Slide

  66. JSONPath and the World

    View Slide

  67. To Sum Up

    View Slide

  68. View Slide

  69. Hint: Always prepare test data
    yourself
    You don’t want to have race conditions or any
    other unexpected trouble born from shared state.

    View Slide

  70. Hint: Run your tests in parallel
    • Configure failsafe or gradle config to have
    maxParallelForks > 1
    • Organise your test scenarios into Test Suites
    with atomic sense

    View Slide

  71. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)

    View Slide

  72. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)

    View Slide

  73. More POST on a resource
    • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)

    View Slide

  74. • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    More POST on a resource

    View Slide

  75. • Created successfully -> Created (201)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource already exists -> Conflict (409)
    • Unsupported Media Type (415)
    More POST on a resource

    View Slide

  76. Hint: Expose all errors and conditions
    to the client in RESTful way!
    REST is statless; be statless with errors as well!

    View Slide

  77. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE

    View Slide

  78. C
    R
    U
    D
    - POST
    - GET
    - PUT
    - DELETE

    View Slide

  79. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)

    View Slide

  80. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)

    View Slide

  81. More GET on a resource
    • Get existing resource -> OK (200)
    • Get non existing resource -> Not Found (404)
    • Unsupported Media Type (415)

    View Slide

  82. DELETE on a resource

    View Slide

  83. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)

    View Slide

  84. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)

    View Slide

  85. DELETE on a resource
    • Delete non existing resource -> Not Found (404)
    • Delete existing resource -> OK (200)
    • Resource is in use -> Forbidden (403)

    View Slide

  86. PUT on a resource

    View Slide

  87. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  88. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  89. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  90. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  91. PUT on a resource
    • Update non existing resource -> Not Found (404)
    • Update existing resource -> OK (200)
    • Missing a mandatory property -> BadRequest (400)
    • Validation failed -> BadRequest (400)
    • Resource is in use -> Forbidden (403)
    • Unsupported Media Type (415)

    View Slide

  92. One CRUD but a lot of effort
    /robots

    View Slide

  93. Hint: Invest into describing your API
    Either it be a API Doc or better an API
    representation in some data format

    View Slide

  94. Open APIs
    https://openapis.org/

    View Slide

  95. swagger: 2.0
    host: petstore.swagger.io
    schemes: [ “http" ]
    consumes: [ “application/json" ]
    produces: [ “application/json" ]
    paths: {
    /pets: {
    post: {
    summary: Create a pet
    responses: {
    201: { description: "Null response” }
    default: {
    description: unexpected error,
    schema: { $ref: “#/definitions/Error" }
    }
    ….
    }}}}
    Error: {
    required: [
    “code", “message"
    ],
    properties: {
    code: {
    type: integer
    format: int32
    },
    message: {
    type: string
    }
    }
    }

    View Slide

  96. View Slide

  97. Alternatives
    http://nordicapis.com/top-specification-formats-for-rest-apis/
    • http://raml.org/
    • https://github.com/apiaryio/api-blueprint

    View Slide

  98. Asynchronous
    Scenario: …..
    ….
    When POST request on /robots
    Then status code is 202
    And header Location contains "/robots/Tay.ai"
    ….

    View Slide

  99. Asynchronous
    await().until(newRobotIsCreated());
    await().until( getRobot(“Tay.ai”), isNotNull() );
    https://github.com/jayway/awaitility

    View Slide

  100. Asynchronous
    https://github.com/jayway/awaitility
    with().pollInterval(ONE_HUNDERED_MILLISECONDS)
    .and().with().pollDelay(20, MILLISECONDS)
    .await(“creating a robot”)
    .until(getRobot(“Tay.ai”), hasProperty(“status”, is(“live”));

    View Slide

  101. Other REST-assured features
    • form
    • basic
    • certificate
    • oauth
    • oauth2
    Easy authentication mechanism

    View Slide

  102. Other REST-assured features
    • proxy support
    • JSON-Shema validation
    • Multi-value headers and cookies
    • Multi-part form data support
    • Filters

    View Slide

  103. Points Made

    View Slide

  104. Points Made
    • In addition to unit, integration and ui tests, cover at
    least core functionality and cases with API tests
    • Design your app integrations to be easily pluggable
    and mockable
    • Decouple connection settings with the test suite itself
    (uri, proxy settings, authentication methods, etc)
    • Never use equals to compare server response and try
    to identify the most significant properties for a the
    comparison, over making an all-to-all comparison

    View Slide

  105. Points Made
    • In addition to unit, integration and ui tests, cover at
    least core functionality and cases with API tests
    • Design your app integrations to be easily pluggable
    and mockable
    • Decouple connection settings with the test suite itself
    (uri, proxy settings, authentication methods, etc)
    • Never use equals to compare server response and try
    to identify the most significant properties for a the
    comparison, over making an all-to-all comparison

    View Slide

  106. Points Made
    • In addition to unit, integration and ui tests, cover at
    least core functionality and cases with API tests
    • Design your app integrations to be easily pluggable
    and mockable
    • Decouple connection settings with the test suite itself
    (uri, proxy settings, authentication methods, etc)
    • Never use equals to compare server response and try
    to identify the most significant properties for a the
    comparison, over making an all-to-all comparison

    View Slide

  107. Points Made
    • In addition to unit, integration and ui tests, cover at
    least core functionality and cases with API tests
    • Design your app integrations to be easily pluggable
    and mockable
    • Decouple connection settings with the test suite itself
    (uri, proxy settings, authentication methods, etc)
    • Never use equals to compare server response and try
    to identify the most significant properties for a the
    comparison, over making an all-to-all comparison

    View Slide

  108. Points Made
    • Don’t relay on existing data and prepare ones for
    testing before test starts
    • Design your tests to be easily parallel and sort
    them in a way that they won’t affect each other
    • Invest in having a descriptive API
    representation like Swagger, RAML, or API
    Blueprints
    • Use JSONPath to assert the results

    View Slide

  109. Points Made
    • Don’t relay on existing data and prepare ones for
    testing before test starts
    • Design your tests to be easily parallel and sort
    them in a way that they won’t affect each other
    • Invest in having a descriptive API
    representation like Swagger, RAML, or API
    Blueprints
    • Use JSONPath to assert the results

    View Slide

  110. Points Made
    • Don’t relay on existing data and prepare ones for
    testing before test starts
    • Design your tests to be easily parallel and sort
    them in a way that they won’t affect each other
    • Invest in having a descriptive API
    representation like Swagger, RAML, or API
    Blueprints
    • Use JSONPath to assert the results

    View Slide

  111. Points Made
    • Don’t relay on existing data and prepare ones for
    testing before test starts
    • Design your tests to be easily parallel and sort
    them in a way that they won’t affect each other
    • Invest in having a descriptive API
    representation like Swagger, RAML, or API
    Blueprints
    • Use JSONPath/XPath to assert the results

    View Slide

  112. Tools to look at
    • REST-assured or cukes-rest for testing RESTful api
    • Awaitility to deal with asynchronous jobs
    • Swagger, RAML, API Blueprint for API description
    • Arquillian to start up your Java EE based projects
    • Moco to startup a mock web service
    • HJSON is awesome!

    View Slide

  113. TESTS ARE MADE
    TO MAKE YOU FEEL SECURE!

    View Slide

  114. LIKE A LOVELY HUG ♥

    View Slide

  115. Q&A
    Thank You!

    View Slide