Testing RESTful services: The What, The How and The Automated

Nowadays REST is a default architectural pattern for developing web services. But with great power comes great responsibility! Let's talk about what should be pinpointed and tested on a RESTful API level, what are the tips and tricks, where you could stumble upon common problems and what are the options on writing a beautifully crafted, maintainable test suit with Java or without. Furthermore let's talk about how to automate the whole beast to fit your delivery pipeline.

Transcript

1. Testing RESTful services The What, The How and The Automated @AlexeyBuzdin
2. Lets start with a tale…
6. Layered Architecture
8. UI Service Unit
9. UI Service Unit Effort
10. UI Service Unit Effort Cost
12. BFF BFF DB DB DB AMQP
13. BFF BFF DB DB DB AMQP
14. API is the new UI!
15. To test a microservice test it’s API
16. Why API tests?
17. Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code
18. public class SimpleServlet extends GenericServlet { public void service(ServletRequest request, ServletResponse response) throws ServletException, IOException { // do something in here } } Simple Servlet
19. Simple Servlet @Controller @RequestMapping("/") public class SimpleController { @RequestMapping(method = RequestMethod.GET) public String get() { return “Hello”; } } (Annotation Driven Development … he.. he..)
20. Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code
21. Simple Servlet @Controller @RequestMapping("/") public class SimpleController { @RequestMapping(method = POST) public String post(MyBean bean) { …. } } Annotation Driven Development
22. Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code
23. Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code
24. DB DB AMQP 1. Isolate the Subject
25. Hint: Decouple integrations from your app Design your integrations to be easily pluggable and mockable in advance, or feel the consequences
26. http://projects.spring.io/spring-integration/
27. Spring Integration <int:gateway id="requestGateway" service-interface=“com.my.RequestGateway” default-request-channel="requestChannel"/> <int:channel id="requestChannel"/> <int-http:outbound-gateway request-channel="requestChannel" url="http://localhost:8080/http/receiveGateway" http-method="POST" expected-response-type="java.lang.String"/>
28. Spring Integration public interface RequestGateway { String echo(String request); }
29. Moco https://github.com/dreamhead/moco java -jar moco-runner.jar http -p 12306 -c foo.json { "request": { "json": { "foo": “bar" } }, "response": { "text": “foo" } }
30. Moco https://github.com/dreamhead/moco server.request(by("foo")).response("bar"); MocoHttpServer server = new MocoHttpServer(ActualHttpServer.createLogServer(of(port()))); server.start();
31. For DB Mocking
32. For DB Mocking • Substitute to an in-memory DB (Spring Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
33. For DB Mocking • Substitute to an in-memory DB (Spring Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
34. For DB Mocking • Substitute to an in-memory DB (Spring Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
35. For DB Mocking • Substitute to an in-memory DB (Spring Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
36. DB DB AMQP 2. Automate service startup
37. Application Server?
38. @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = Application.class) @WebAppConfiguration @IntegrationTest("server.port:0") public class CharacterControllerTest { @Value("${local.server.port}") int port; …. }
39. @RunWith(Arquillian.class) public class GreeterTest { @Deployment public static WebArchive.class createDeployment() { return ShrinkWrap.create(WebArchive.class) .addClasses(Greeter.class) .setWebXML("WEB-INF/web.xml"); } …. } https://github.com/arquillian
40. 3. Write a HealthCheck test @Test public void applicationIsUp() { given().port(8888). when().get("/healthcheck"). then().statusCode(200); } https://github.com/jayway/rest-assured
41. https://github.com/jayway/rest-assured RequestSpecification - build the request ResponseSpecification - contains parsed response given(). param("name", "Johan"). cookie("cookie", "is mine"). body("{}"). header("Accept", "application/json") then(). statusCode(200). header("Content-Type", "application/json")
42. @Test public void applicationIsUp() { given().port(8888). when().get("/healthcheck"). then().statusCode(200); } Scenario: Application is up Given baseUri is http://localhost:8888/ When GET request on /healthcheck Then status code is 200 https://github.com/ctco/cukes-rest
43. Hint: Always make url/port configurable At some point you would like to run the tests against your hotfixed api or even generate some traffic on your staging environment
44. Lets test a Sample Server
49. REST is all about the resources /robots
50. C R U D
51. C R U D - POST - GET - PUT - DELETE
52. Scenario: Should accept a cute RoboGirl Given request body: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ When POST request on /robots Then status code is 201 HJson / HumanJSON
53. Scenario: Should accept a cute RoboGirl Given request body: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ When POST request on /robots Then status code is 201 And header Location ends with "/robots/Tay.ai"
54. Accept tested • Application is started • Resource collection exists • Provided JSON is parsed successfully • No validation errors
55. But we haven’t tested if the resource was created on the server!
56. Scenario: Should create a new Robot … When POST request on /robots Then status code is 201 And header Location ends with “/robots/Tay.ai" When GET request on {(header.Location)} Then status code is 200
57. Created tested • Application is started • Resource collection exists • Provided JSON is parsed successfully • No validation errors • Resource was stored on the server
58. What did we miss? Scenario: ….. Given request body: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ When POST request on /robots
59. Test Actual Data …. When GET request on /robots/Tay.ai Then status code is 200 And response equals to: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """
60. Unicode is not working java.lang.AssertionError: 1 expectation failed. Response body doesn't match expectation. Expected: "{\"name\" : \"Tay.ai\", \"personality\" : { \"name\": \"Tãy\",\"race\" : \"Hispanic\", \"age\" : 20}}" Actual: {"name" : "Tay.ai", "personality" : { "name" : "T y","race" : "Hispanic","age" : 20}}
61. Test Actual Data …. When GET request on /robots/Tay.ai Then status code is 200 And response equals to: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """
62. Hint: Never use equals to compare server response Usually you have tons of stuff generated automatically on the backend which value can not be predicted in advance*: createdOn, updatedOn, auto generated IDs, etc
63. Hint: Use JSONPath and XPath to assert equality When GET request on /robots/Tay.ai Then status code is 200 And response contains properties: """ name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } """ name = Tay.ai personality.name = Tãy personality.race = Hispanic personality.age = 20
64. JSONPath and Arrays name : Tay.ai personality : { name : Tãy race : Hispanic age : 20 } skills: [ "driving", "hacking", "translating" ] name = Tay.ai personality.name = Tãy personality.race = Hispanic personality.age = 20 skills[0] = driving skills[1] = hacking skills[2] = translating
65. Hint: Arrays are unordered if fetched straight from DB Either implement order by or find the correct index programmatically
66. JSONPath and the World
67. To Sum Up
69. Hint: Always prepare test data yourself You don’t want to have race conditions or any other unexpected trouble born from shared state.
70. Hint: Run your tests in parallel • Configure failsafe or gradle config to have maxParallelForks > 1 • Organise your test scenarios into Test Suites with atomic sense
71. More POST on a resource • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
72. More POST on a resource • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
73. More POST on a resource • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
74. • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
75. • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
76. Hint: Expose all errors and conditions to the client in RESTful way! REST is statless; be statless with errors as well!
77. C R U D - POST - GET - PUT - DELETE
78. C R U D - POST - GET - PUT - DELETE
79. More GET on a resource • Get existing resource -> OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
80. More GET on a resource • Get existing resource -> OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
81. More GET on a resource • Get existing resource -> OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
82. DELETE on a resource
83. DELETE on a resource • Delete non existing resource -> Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
84. DELETE on a resource • Delete non existing resource -> Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
85. DELETE on a resource • Delete non existing resource -> Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
86. PUT on a resource
87. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
88. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
89. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
90. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
91. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
92. One CRUD but a lot of effort /robots
93. Hint: Invest into describing your API Either it be a API Doc or better an API representation in some data format
94. Open APIs https://openapis.org/
95. swagger: 2.0 host: petstore.swagger.io schemes: [ “http" ] consumes: [ “application/json" ] produces: [ “application/json" ] paths: { /pets: { post: { summary: Create a pet responses: { 201: { description: "Null response” } default: { description: unexpected error, schema: { $ref: “#/definitions/Error" } } …. }}}} Error: { required: [ “code", “message" ], properties: { code: { type: integer format: int32 }, message: { type: string } } }
97. Alternatives http://nordicapis.com/top-specification-formats-for-rest-apis/ • http://raml.org/ • https://github.com/apiaryio/api-blueprint
98. Asynchronous Scenario: ….. …. When POST request on /robots Then status code is 202 And header Location contains "/robots/Tay.ai" ….
99. Asynchronous await().until(newRobotIsCreated()); await().until( getRobot(“Tay.ai”), isNotNull() ); https://github.com/jayway/awaitility
100. Asynchronous https://github.com/jayway/awaitility with().pollInterval(ONE_HUNDERED_MILLISECONDS) .and().with().pollDelay(20, MILLISECONDS) .await(“creating a robot”) .until(getRobot(“Tay.ai”), hasProperty(“status”, is(“live”));
101. Other REST-assured features • form • basic • certificate • oauth • oauth2 Easy authentication mechanism
102. Other REST-assured features • proxy support • JSON-Shema validation • Multi-value headers and cookies • Multi-part form data support • Filters
103. Points Made
104. Points Made • In addition to unit, integration and ui tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
105. Points Made • In addition to unit, integration and ui tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
106. Points Made • In addition to unit, integration and ui tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
107. Points Made • In addition to unit, integration and ui tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
108. Points Made • Don’t relay on existing data and prepare ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
109. Points Made • Don’t relay on existing data and prepare ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
110. Points Made • Don’t relay on existing data and prepare ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
111. Points Made • Don’t relay on existing data and prepare ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath/XPath to assert the results
112. Tools to look at • REST-assured or cukes-rest for testing RESTful api • Awaitility to deal with asynchronous jobs • Swagger, RAML, API Blueprint for API description • Arquillian to start up your Java EE based projects • Moco to startup a mock web service • HJSON is awesome!
113. TESTS ARE MADE TO MAKE YOU FEEL SECURE!
114. LIKE A LOVELY HUG ♥
115. Q&A Thank You!