Testing RESTful services: GoT JavaDay Kiev Edition

Nowadays REST is a default architectural pattern for developing web services. But with great power comes great responsibility! Let's talk about what should be pinpointed and tested on a RESTful API level, what are the tips and tricks, where you could stumble upon common problems and what are the options on writing a beautifully crafted, maintainable test suit with Java or without. Furthermore let's talk about how to automate the whole beast to fit your delivery pipeline.

Transcript

1. We came here to kick ass and test your services, and you’re all out of services @AlexeyBuzdin
2. @AlexeyBuzdin Testing RESTful services
3. @AlexeyBuzdin Developer / Trainer at GDGRiga.lv RigaDevDay.lv
4. Lets start with a tale…
8. Layered Architecture
10. UI Service Unit
11. UI Service Unit Effort
12. UI Service Unit Effort Cost
14. BFF BFF DB DB DB AMQP
15. BFF BFF DB DB DB AMQP
16. API is the new UI!
17. test it’s API To test a microservice
18. Why API tests?
19. Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code
20. public class SimpleServlet extends GenericServlet {
 
 public void service(ServletRequest req, ServletResponse res)
 throws ServletException, IOException {
 // do something in here
 }
 } Simple Servlet
21. Simple Servlet (Annotation Driven Development … he.. he..) @Controller
 @RequestMapping("/")
 public class SimpleController {
 
 @RequestMapping(method = RequestMethod.GET)
 public String get() {
 return "Hello";
 }
 }
22. Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code
23. Simple Servlet @Controller
 @RequestMapping("/")
 public class SimpleController {
 
 @RequestMapping(method = POST)
 public String post(MyBean bean) {
 ...
 }
 } (Annotation Driven Development … he.. he..)
24. Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code
25. Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code
26. DB DB AMQP 1. Isolate the Subject
27. Hint: Decouple integrations from your app Design your integrations to be easily pluggable and mockable in advance, or feel the consequences
28. http://projects.spring.io/spring-integration/
29. Spring Integration <int:gateway id="requestGateway"
 service-interface="com.my.RequestGateway"
 default-request-channel="requestChannel"/>
 
 <int:channel id="requestChannel"/>
 
 <int-http:outbound-gateway request-channel="requestChannel"
 url="http://localhost:8080/http/receiveGateway"
 http-method="POST"
 expected-response-type="java.lang.String"/>
30. Spring Integration public interface RequestGateway {
 String echo(String request);
 }
31. Moco https://github.com/dreamhead/moco java -jar moco-runner.jar http -p 12306 -c foo.json {
 "request": {
 "json": { "foo": "bar" }
 },
 "response": { "text": "foo" }
 }
32. Moco MocoHttpServer server = new MocoHttpServer( ActualHttpServer.createLogServer(of(port()))); server.start(); server.request(by("foo")).response("bar"); https://github.com/dreamhead/moco
33. WireMock http://wiremock.org/ @Rule public WireMockRule wireMockRule = new WireMockRule(8089); stubFor(get(urlEqualTo("/my/resource")) .withHeader("Accept", equalTo("text/xml")) .willReturn(aResponse() .withStatus(200) .withHeader("Content-Type", "text/xml") .withBody("<response>Some content</response>")));
34. For DB Mocking
35. For DB Mocking • Substitute to an in-memory DB (Spring Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
36. For DB Mocking • Substitute to an in-memory DB (Spring Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
37. For DB Mocking • Substitute to an in-memory DB (Spring Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
38. For DB Mocking • Substitute to an in-memory DB (Spring Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
39. DB DB AMQP 2. Automate service startup
40. Application Server?
41. @RunWith(SpringJUnit4ClassRunner.class)
 @SpringApplicationConfiguration(classes = Application.class)
 @WebAppConfiguration
 @IntegrationTest("server.port:0")
 public class CharacterControllerTest {
 
 @Value("${local.server.port}")
 int port; ...
 }
42. https://github.com/arquillian @RunWith(Arquillian.class)
 public class GreeterTest {
 
 @Deployment
 public static WebArchive createDeployment() {
 return ShrinkWrap.create(WebArchive.class)
 .addClasses(Greeter.class)
 .setWebXML("WEB-INF/web.xml");
 } ...
 }
43. DB DB AMQP 2. Automate service startup
44. 3. Write a HealthCheck test https://github.com/jayway/rest-assured @Test
 public void applicationIsUp() {
 given().port(8888).
 when().get("/healthcheck").
 then().statusCode(200);
 }
45. https://github.com/jayway/rest-assured RequestSpecification - build the request ResponseSpecification - contains parsed response given().
 param("name", "Johan").
 cookie("cookie", "is mine").
 body("{}").
 header("Accept", “application/json"). then().
 statusCode(200).
 header("Content-Type", "application/json");
 when().post(“/resource").
46. Hint: Always make url/port configurable At some point you would like to run the tests against your hotfixed api or even generate some traffic on your staging environment
47. Lets test a Sample Server
49. REST is all about the resources /houses
50. C R U D
51. C R U D - POST - GET - PUT - DELETE
52. C R U D - GET
53. HJson / HumanJSON https://hjson.org/ given(). body(readHjson( "name : Lannister" + "lore : {" + "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()). when().post("/houses"). then().statusCode(201); .and().header("Location", "Iron Throne");
54. given(). body(readHjson( "name : Lannister" + "lore : {" + "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()). when().post("/houses"). then().statusCode(201) .and().header("Location", "Iron Throne");
55. Accept tested • Application is started • Resource collection exists • Provided JSON is parsed successfully • No validation errors
56. But we haven’t tested if the resource was created on the server!
57. String location = … when().post("/houses"). then().statusCode(201) .extract().header(“Location”); assertEquals("Iron Throne", location); get(location).then().statusCode(200);
58. Created tested • Application is started • Resource collection exists • Provided JSON is parsed successfully • No validation errors • Resource was stored on the server
59. given(). body(readHjson( "name : Lannister" + "lore : {" + "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()). when().post("/houses"). then().statusCode(201); What did we miss?
60. assertEquals("Iron Throne", location); get(location).then().statusCode(200) .and().body(".", equalTo(readHjson( "name : Lannister" + "lore : {" + "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()); Test Actual Data
61. Unicode is not working java.lang.AssertionError: 1 expectation failed. Response body doesn't match expectation. Expected: “\”words\” : \”Hear Me Röar!\”” Actual: ”words” : ”Hear Me R䡦ar!”
62. assertEquals("Iron Throne", location); get(location).then().statusCode(200) .and().body(".", equalTo(readHjson( "name : Lannister" + "lore : {" + "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()); Test Actual Data
63. Hint: Never use equals to compare server response Usually you have tons of stuff generated automatically on the backend which value can not be predicted in advance*: createdOn, updatedOn, auto generated IDs, etc
64. Hint: Use JSONPath and XPath to assert equality name = Lannister lore.sigil = … lore.words = … lore.shouldBe = … "name : Lannister" + "lore : {" + "sigil : …” + "words : ….” + "shouldBe : ….” + "}"
65. Hint: Use JSONPath and XPath to assert equality name = Lannister lore.sigil = … lore.words = … lore.shouldBe = … members[0] = Cercei members[1] = Tyrion members[2] = Jamie "name : Lannister" + "lore : {" + … “}" “members : [“ + “Cercei” “Tyrion” “Jamie” “]”
66. Hint: Implement order for your arrays or use contains matchers Arrays are unordered if fetched straight from DB
67. To Sum Up
69. Hint: Always prepare test data yourself You don’t want to have race conditions or any other unexpected trouble born from shared state.
70. Hint: Run your tests in parallel • Configure failsafe or gradle config to have maxParallelForks > 1 • Organise your test scenarios into Test Suites with atomic sense
71. More POST on a resource • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
72. More POST on a resource • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
73. More POST on a resource • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
74. • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
75. • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
76. Hint: Expose all errors and conditions to the client in RESTful way! REST is stateless; pass errors to the client!
77. C R U D - POST - GET - PUT - DELETE
78. C R U D - POST - GET - PUT - DELETE
79. More GET on a resource • Get existing resource -> OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
80. More GET on a resource • Get existing resource -> OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
81. More GET on a resource • Get existing resource -> OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
82. DELETE on a resource
83. DELETE on a resource • Delete non existing resource -> Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
84. DELETE on a resource • Delete non existing resource -> Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
85. DELETE on a resource • Delete non existing resource -> Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
86. PUT on a resource
87. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
88. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
89. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
90. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
91. PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
92. One Throne but a lot of deaths CRUD effort
93. Hint: Invest into describing your API Either it be a API Doc or better an API representation in some data format
94. Open APIs https://openapis.org/
95. swagger: 2.0 host: petstore.swagger.io schemes: [ “http" ] consumes: [ “application/json" ] produces: [ “application/json" ] paths: { /pets: { post: { summary: Create a pet responses: { 201: { description: "Null response” } default: { description: unexpected error, schema: { $ref: “#/definitions/Error" } } …. }}}} Error: { required: [ “code", “message" ], properties: { code: { type: integer format: int32 }, message: { type: string } } }
97. SpringFox http://springfox.github.io/springfox/ @EnableSwagger2 public class SwaggerConfiguration { @Override public void addResourceHandlers(ResourceHandlerRegistry registry) { registry.addResourceHandler("swagger-ui.html") .addResourceLocations("classpath:/META-INF/resources/"); registry.addResourceHandler("/webjars/**") .addResourceLocations("classpath:/META-INF/resources/webjars/"); } }
98. JAXRS-Analyzer https://github.com/sdaschner/jaxrs-analyzer Creates REST documentation for JAX-RS projects
99. Alternatives http://nordicapis.com/top-specification-formats-for-rest-apis/ • http://raml.org/ • https://github.com/apiaryio/api-blueprint
100. Asynchronous … when().post("/houses"). then().statusCode(202) .and().header("Location", "Iron Throne”); …
101. Asynchronous await().until(newHouseEmerges()); await().until( getHouse(“Lannister”), isNotNull() ); https://github.com/jayway/awaitility
102. Asynchronous https://github.com/jayway/awaitility with().pollInterval(ONE_HUNDERED_MILLISECONDS)
 .and().with().pollDelay(20, MILLISECONDS)
 .await("creating a house")
 .until(getHouse("Lannister"), hasProperty("status", is("live")));
103. Other REST-assured features • form • basic • certificate • oauth • oauth2 Easy authentication mechanism
104. Other REST-assured features • proxy support • JSON-Shema validation • Multi-value headers and cookies • Multi-part form data support • Filters
105. Points Made
106. Points Made • In addition to unit, integration and ui tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
107. Points Made • In addition to unit, integration and ui tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
108. Points Made • In addition to unit, integration and ui tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
109. Points Made • In addition to unit, integration and ui tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
110. Points Made • Don’t relay on existing data and prepare ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
111. Points Made • Don’t relay on existing data and prepare ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
112. Points Made • Don’t relay on existing data and prepare ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
113. Points Made • Don’t relay on existing data and prepare ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath/XPath to assert the results
114. Tools to look at • REST-assured for testing RESTful api • Awaitility to deal with asynchronous jobs • Swagger, RAML, API Blueprint for API description • Arquillian to start up your Java EE based projects • Moco or WireShark to startup a mock web service • HJSON is awesome!
115. TESTS ARE MADE TO MAKE YOU FEEL SECURE!
116. LIKE A LOVELY HUG ♥
117. Q&A Thank You! @AlexeyBuzdin Follow me at