Pro Yearly is on sale from $80 to $50! »

Testing RESTful services: The What, The How and The Automated

Testing RESTful services: The What, The How and The Automated

Nowadays REST is a default architectural pattern for developing web services. But with great power comes great responsibility! Let's talk about what should be pinpointed and tested on a RESTful API level, what are the tips and tricks, where you could stumble upon common problems and what are the options on writing a beautifully crafted, maintainable test suit with Java or without. Furthermore let's talk about how to automate the whole beast to fit your delivery pipeline.

12d6ff93ca25d366161efccadd81bbb2?s=128

Alexey Buzdin

April 25, 2016
Tweet

Transcript

  1. Testing RESTful services The What, The How and The Automated

    @AlexeyBuzdin
  2. Lets start with a tale…

  3. None
  4. None
  5. None
  6. Layered Architecture

  7. None
  8. UI Service Unit

  9. UI Service Unit Effort

  10. UI Service Unit Effort Cost

  11. None
  12. BFF BFF DB DB DB AMQP

  13. BFF BFF DB DB DB AMQP

  14. API is the new UI!

  15. To test a microservice test it’s API

  16. Why API tests?

  17. Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  18. public class SimpleServlet extends GenericServlet { public void service(ServletRequest request,

    ServletResponse response) throws ServletException, IOException { // do something in here } } Simple Servlet
  19. Simple Servlet @Controller @RequestMapping("/") public class SimpleController { @RequestMapping(method =

    RequestMethod.GET) public String get() { return “Hello”; } } (Annotation Driven Development … he.. he..)
  20. Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  21. Simple Servlet @Controller @RequestMapping("/") public class SimpleController { @RequestMapping(method =

    POST) public String post(MyBean bean) { …. } } Annotation Driven Development
  22. Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  23. Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  24. DB DB AMQP 1. Isolate the Subject

  25. Hint: Decouple integrations from your app Design your integrations to

    be easily pluggable and mockable in advance, or feel the consequences
  26. http://projects.spring.io/spring-integration/

  27. Spring Integration <int:gateway id="requestGateway" service-interface=“com.my.RequestGateway” default-request-channel="requestChannel"/> <int:channel id="requestChannel"/> <int-http:outbound-gateway request-channel="requestChannel"

    url="http://localhost:8080/http/receiveGateway" http-method="POST" expected-response-type="java.lang.String"/>
  28. Spring Integration public interface RequestGateway { String echo(String request); }

  29. Moco https://github.com/dreamhead/moco java -jar moco-runner.jar http -p 12306 -c foo.json

    { "request": { "json": { "foo": “bar" } }, "response": { "text": “foo" } }
  30. Moco https://github.com/dreamhead/moco server.request(by("foo")).response("bar"); MocoHttpServer server = new MocoHttpServer(ActualHttpServer.createLogServer(of(port()))); server.start();

  31. For DB Mocking

  32. For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  33. For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  34. For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  35. For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  36. DB DB AMQP 2. Automate service startup

  37. Application Server?

  38. @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = Application.class) @WebAppConfiguration @IntegrationTest("server.port:0") public class CharacterControllerTest {

    @Value("${local.server.port}") int port; …. }
  39. @RunWith(Arquillian.class) public class GreeterTest { @Deployment public static WebArchive.class createDeployment()

    { return ShrinkWrap.create(WebArchive.class) .addClasses(Greeter.class) .setWebXML("WEB-INF/web.xml"); } …. } https://github.com/arquillian
  40. 3. Write a HealthCheck test @Test public void applicationIsUp() {

    given().port(8888). when().get("/healthcheck"). then().statusCode(200); } https://github.com/jayway/rest-assured
  41. https://github.com/jayway/rest-assured RequestSpecification - build the request ResponseSpecification - contains parsed

    response given(). param("name", "Johan"). cookie("cookie", "is mine"). body("{}"). header("Accept", "application/json") then(). statusCode(200). header("Content-Type", "application/json")
  42. @Test public void applicationIsUp() { given().port(8888). when().get("/healthcheck"). then().statusCode(200); } Scenario:

    Application is up Given baseUri is http://localhost:8888/ When GET request on /healthcheck Then status code is 200 https://github.com/ctco/cukes-rest
  43. Hint: Always make url/port configurable At some point you would

    like to run the tests against your hotfixed api or even generate some traffic on your staging environment
  44. Lets test a Sample Server

  45. None
  46. None
  47. None
  48. None
  49. REST is all about the resources /robots

  50. C R U D

  51. C R U D - POST - GET - PUT

    - DELETE
  52. Scenario: Should accept a cute RoboGirl Given request body: """

    name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ When POST request on /robots Then status code is 201 HJson / HumanJSON
  53. Scenario: Should accept a cute RoboGirl Given request body: """

    name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ When POST request on /robots Then status code is 201 And header Location ends with "/robots/Tay.ai"
  54. Accept tested • Application is started • Resource collection exists

    • Provided JSON is parsed successfully • No validation errors
  55. But we haven’t tested if the resource was created on

    the server!
  56. Scenario: Should create a new Robot … When POST request

    on /robots Then status code is 201 And header Location ends with “/robots/Tay.ai" When GET request on {(header.Location)} Then status code is 200
  57. Created tested • Application is started • Resource collection exists

    • Provided JSON is parsed successfully • No validation errors • Resource was stored on the server
  58. What did we miss? Scenario: ….. Given request body: """

    name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ When POST request on /robots
  59. Test Actual Data …. When GET request on /robots/Tay.ai Then

    status code is 200 And response equals to: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """
  60. Unicode is not working java.lang.AssertionError: 1 expectation failed. Response body

    doesn't match expectation. Expected: "{\"name\" : \"Tay.ai\", \"personality\" : { \"name\": \"Tãy\",\"race\" : \"Hispanic\", \"age\" : 20}}" Actual: {"name" : "Tay.ai", "personality" : { "name" : "T y","race" : "Hispanic","age" : 20}}
  61. Test Actual Data …. When GET request on /robots/Tay.ai Then

    status code is 200 And response equals to: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """
  62. Hint: Never use equals to compare server response Usually you

    have tons of stuff generated automatically on the backend which value can not be predicted in advance*: createdOn, updatedOn, auto generated IDs, etc
  63. Hint: Use JSONPath and XPath to assert equality When GET

    request on /robots/Tay.ai Then status code is 200 And response contains properties: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ name = Tay.ai personality.name = Tãy personality.race = Hispanic personality.age = 20
  64. JSONPath and Arrays name : Tay.ai personality : { name

    : Tãy race : Hispanic age : 20 } skills: [ "driving", "hacking", "translating" ] name = Tay.ai personality.name = Tãy personality.race = Hispanic personality.age = 20 skills[0] = driving skills[1] = hacking skills[2] = translating
  65. Hint: Arrays are unordered if fetched straight from DB Either

    implement order by or find the correct index programmatically
  66. JSONPath and the World

  67. To Sum Up

  68. None
  69. Hint: Always prepare test data yourself You don’t want to

    have race conditions or any other unexpected trouble born from shared state.
  70. Hint: Run your tests in parallel • Configure failsafe or

    gradle config to have maxParallelForks > 1 • Organise your test scenarios into Test Suites with atomic sense
  71. More POST on a resource • Created successfully -> Created

    (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
  72. More POST on a resource • Created successfully -> Created

    (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
  73. More POST on a resource • Created successfully -> Created

    (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
  74. • Created successfully -> Created (201) • Missing a mandatory

    property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
  75. • Created successfully -> Created (201) • Missing a mandatory

    property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
  76. Hint: Expose all errors and conditions to the client in

    RESTful way! REST is statless; be statless with errors as well!
  77. C R U D - POST - GET - PUT

    - DELETE
  78. C R U D - POST - GET - PUT

    - DELETE
  79. More GET on a resource • Get existing resource ->

    OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
  80. More GET on a resource • Get existing resource ->

    OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
  81. More GET on a resource • Get existing resource ->

    OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
  82. DELETE on a resource

  83. DELETE on a resource • Delete non existing resource ->

    Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
  84. DELETE on a resource • Delete non existing resource ->

    Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
  85. DELETE on a resource • Delete non existing resource ->

    Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
  86. PUT on a resource

  87. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  88. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  89. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  90. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  91. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  92. One CRUD but a lot of effort /robots

  93. Hint: Invest into describing your API Either it be a

    API Doc or better an API representation in some data format
  94. Open APIs https://openapis.org/

  95. swagger: 2.0 host: petstore.swagger.io schemes: [ “http" ] consumes: [

    “application/json" ] produces: [ “application/json" ] paths: { /pets: { post: { summary: Create a pet responses: { 201: { description: "Null response” } default: { description: unexpected error, schema: { $ref: “#/definitions/Error" } } …. }}}} Error: { required: [ “code", “message" ], properties: { code: { type: integer format: int32 }, message: { type: string } } }
  96. None
  97. Alternatives http://nordicapis.com/top-specification-formats-for-rest-apis/ • http://raml.org/ • https://github.com/apiaryio/api-blueprint

  98. Asynchronous Scenario: ….. …. When POST request on /robots Then

    status code is 202 And header Location contains "/robots/Tay.ai" ….
  99. Asynchronous await().until(newRobotIsCreated()); await().until( getRobot(“Tay.ai”), isNotNull() ); https://github.com/jayway/awaitility

  100. Asynchronous https://github.com/jayway/awaitility with().pollInterval(ONE_HUNDERED_MILLISECONDS) .and().with().pollDelay(20, MILLISECONDS) .await(“creating a robot”) .until(getRobot(“Tay.ai”), hasProperty(“status”,

    is(“live”));
  101. Other REST-assured features • form • basic • certificate •

    oauth • oauth2 Easy authentication mechanism
  102. Other REST-assured features • proxy support • JSON-Shema validation •

    Multi-value headers and cookies • Multi-part form data support • Filters
  103. Points Made

  104. Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  105. Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  106. Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  107. Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  108. Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
  109. Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
  110. Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
  111. Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath/XPath to assert the results
  112. Tools to look at • REST-assured or cukes-rest for testing

    RESTful api • Awaitility to deal with asynchronous jobs • Swagger, RAML, API Blueprint for API description • Arquillian to start up your Java EE based projects • Moco to startup a mock web service • HJSON is awesome!
  113. TESTS ARE MADE TO MAKE YOU FEEL SECURE!

  114. LIKE A LOVELY HUG ♥

  115. Q&A Thank You!