Testing RESTful services: GoT JavaDay Kiev Edition

Testing RESTful services: GoT JavaDay Kiev Edition

Nowadays REST is a default architectural pattern for developing web services. But with great power comes great responsibility! Let's talk about what should be pinpointed and tested on a RESTful API level, what are the tips and tricks, where you could stumble upon common problems and what are the options on writing a beautifully crafted, maintainable test suit with Java or without. Furthermore let's talk about how to automate the whole beast to fit your delivery pipeline.

12d6ff93ca25d366161efccadd81bbb2?s=128

Alexey Buzdin

October 15, 2016
Tweet

Transcript

  1. We came here to kick ass and test your services,

    and you’re all out of services @AlexeyBuzdin
  2. @AlexeyBuzdin Testing RESTful services

  3. @AlexeyBuzdin Developer / Trainer at GDGRiga.lv RigaDevDay.lv

  4. Lets start with a tale…

  5. None
  6. None
  7. None
  8. Layered Architecture

  9. None
  10. UI Service Unit

  11. UI Service Unit Effort

  12. UI Service Unit Effort Cost

  13. None
  14. BFF BFF DB DB DB AMQP

  15. BFF BFF DB DB DB AMQP

  16. API is the new UI!

  17. test it’s API To test a microservice

  18. Why API tests?

  19. Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  20. public class SimpleServlet extends GenericServlet {
 
 public void service(ServletRequest

    req, ServletResponse res)
 throws ServletException, IOException {
 // do something in here
 }
 } Simple Servlet
  21. Simple Servlet (Annotation Driven Development … he.. he..) @Controller
 @RequestMapping("/")


    public class SimpleController {
 
 @RequestMapping(method = RequestMethod.GET)
 public String get() {
 return "Hello";
 }
 }
  22. Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  23. Simple Servlet @Controller
 @RequestMapping("/")
 public class SimpleController {
 
 @RequestMapping(method

    = POST)
 public String post(MyBean bean) {
 ...
 }
 } (Annotation Driven Development … he.. he..)
  24. Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  25. Why API tests? • Tests routing • Tests serialization/deserialization •

    Tests server response • Can be written by a QA who does not knows the code
  26. DB DB AMQP 1. Isolate the Subject

  27. Hint: Decouple integrations from your app Design your integrations to

    be easily pluggable and mockable in advance, or feel the consequences
  28. http://projects.spring.io/spring-integration/

  29. Spring Integration <int:gateway id="requestGateway"
 service-interface="com.my.RequestGateway"
 default-request-channel="requestChannel"/>
 
 <int:channel id="requestChannel"/>
 


    <int-http:outbound-gateway request-channel="requestChannel"
 url="http://localhost:8080/http/receiveGateway"
 http-method="POST"
 expected-response-type="java.lang.String"/>
  30. Spring Integration public interface RequestGateway {
 String echo(String request);
 }

  31. Moco https://github.com/dreamhead/moco java -jar moco-runner.jar http -p 12306 -c foo.json

    {
 "request": {
 "json": { "foo": "bar" }
 },
 "response": { "text": "foo" }
 }
  32. Moco MocoHttpServer server = new MocoHttpServer( ActualHttpServer.createLogServer(of(port()))); server.start(); server.request(by("foo")).response("bar"); https://github.com/dreamhead/moco

  33. WireMock http://wiremock.org/ @Rule public WireMockRule wireMockRule = new WireMockRule(8089); stubFor(get(urlEqualTo("/my/resource"))

    .withHeader("Accept", equalTo("text/xml")) .willReturn(aResponse() .withStatus(200) .withHeader("Content-Type", "text/xml") .withBody("<response>Some content</response>")));
  34. For DB Mocking

  35. For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  36. For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  37. For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  38. For DB Mocking • Substitute to an in-memory DB (Spring

    Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from file system or memory
  39. DB DB AMQP 2. Automate service startup

  40. Application Server?

  41. @RunWith(SpringJUnit4ClassRunner.class)
 @SpringApplicationConfiguration(classes = Application.class)
 @WebAppConfiguration
 @IntegrationTest("server.port:0")
 public class CharacterControllerTest {


    
 @Value("${local.server.port}")
 int port; ...
 }
  42. https://github.com/arquillian @RunWith(Arquillian.class)
 public class GreeterTest {
 
 @Deployment
 public static

    WebArchive createDeployment() {
 return ShrinkWrap.create(WebArchive.class)
 .addClasses(Greeter.class)
 .setWebXML("WEB-INF/web.xml");
 } ...
 }
  43. DB DB AMQP 2. Automate service startup

  44. 3. Write a HealthCheck test https://github.com/jayway/rest-assured @Test
 public void applicationIsUp()

    {
 given().port(8888).
 when().get("/healthcheck").
 then().statusCode(200);
 }
  45. https://github.com/jayway/rest-assured RequestSpecification - build the request ResponseSpecification - contains parsed

    response given().
 param("name", "Johan").
 cookie("cookie", "is mine").
 body("{}").
 header("Accept", “application/json"). then().
 statusCode(200).
 header("Content-Type", "application/json");
 when().post(“/resource").
  46. Hint: Always make url/port configurable At some point you would

    like to run the tests against your hotfixed api or even generate some traffic on your staging environment
  47. Lets test a Sample Server

  48. None
  49. REST is all about the resources /houses

  50. C R U D

  51. C R U D - POST - GET - PUT

    - DELETE
  52. C R U D - GET

  53. HJson / HumanJSON https://hjson.org/ given(). body(readHjson( "name : Lannister" +

    "lore : {" + "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()). when().post("/houses"). then().statusCode(201); .and().header("Location", "Iron Throne");
  54. given(). body(readHjson( "name : Lannister" + "lore : {" +

    "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()). when().post("/houses"). then().statusCode(201) .and().header("Location", "Iron Throne");
  55. Accept tested • Application is started • Resource collection exists

    • Provided JSON is parsed successfully • No validation errors
  56. But we haven’t tested if the resource was created on

    the server!
  57. String location = … when().post("/houses"). then().statusCode(201) .extract().header(“Location”); assertEquals("Iron Throne", location);

    get(location).then().statusCode(200);
  58. Created tested • Application is started • Resource collection exists

    • Provided JSON is parsed successfully • No validation errors • Resource was stored on the server
  59. given(). body(readHjson( "name : Lannister" + "lore : {" +

    "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()). when().post("/houses"). then().statusCode(201); What did we miss?
  60. assertEquals("Iron Throne", location); get(location).then().statusCode(200) .and().body(".", equalTo(readHjson( "name : Lannister" +

    "lore : {" + "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()); Test Actual Data
  61. Unicode is not working java.lang.AssertionError: 1 expectation failed. Response body

    doesn't match expectation. Expected: “\”words\” : \”Hear Me Röar!\”” Actual: ”words” : ”Hear Me R䡦ar!”
  62. assertEquals("Iron Throne", location); get(location).then().statusCode(200) .and().body(".", equalTo(readHjson( "name : Lannister" +

    "lore : {" + "sigil : A golden lion rampant on a crimson field." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()); Test Actual Data
  63. Hint: Never use equals to compare server response Usually you

    have tons of stuff generated automatically on the backend which value can not be predicted in advance*: createdOn, updatedOn, auto generated IDs, etc
  64. Hint: Use JSONPath and XPath to assert equality name =

    Lannister lore.sigil = … lore.words = … lore.shouldBe = … "name : Lannister" + "lore : {" + "sigil : …” + "words : ….” + "shouldBe : ….” + "}"
  65. Hint: Use JSONPath and XPath to assert equality name =

    Lannister lore.sigil = … lore.words = … lore.shouldBe = … members[0] = Cercei members[1] = Tyrion members[2] = Jamie "name : Lannister" + "lore : {" + … “}" “members : [“ + “Cercei” “Tyrion” “Jamie” “]”
  66. Hint: Implement order for your arrays or use contains matchers

    Arrays are unordered if fetched straight from DB
  67. To Sum Up

  68. None
  69. Hint: Always prepare test data yourself You don’t want to

    have race conditions or any other unexpected trouble born from shared state.
  70. Hint: Run your tests in parallel • Configure failsafe or

    gradle config to have maxParallelForks > 1 • Organise your test scenarios into Test Suites with atomic sense
  71. More POST on a resource • Created successfully -> Created

    (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
  72. More POST on a resource • Created successfully -> Created

    (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
  73. More POST on a resource • Created successfully -> Created

    (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415)
  74. • Created successfully -> Created (201) • Missing a mandatory

    property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
  75. • Created successfully -> Created (201) • Missing a mandatory

    property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conflict (409) • Unsupported Media Type (415) More POST on a resource
  76. Hint: Expose all errors and conditions to the client in

    RESTful way! REST is stateless; pass errors to the client!
  77. C R U D - POST - GET - PUT

    - DELETE
  78. C R U D - POST - GET - PUT

    - DELETE
  79. More GET on a resource • Get existing resource ->

    OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
  80. More GET on a resource • Get existing resource ->

    OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
  81. More GET on a resource • Get existing resource ->

    OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)
  82. DELETE on a resource

  83. DELETE on a resource • Delete non existing resource ->

    Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
  84. DELETE on a resource • Delete non existing resource ->

    Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
  85. DELETE on a resource • Delete non existing resource ->

    Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)
  86. PUT on a resource

  87. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  88. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  89. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  90. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  91. PUT on a resource • Update non existing resource ->

    Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)
  92. One Throne but a lot of deaths CRUD effort

  93. Hint: Invest into describing your API Either it be a

    API Doc or better an API representation in some data format
  94. Open APIs https://openapis.org/

  95. swagger: 2.0 host: petstore.swagger.io schemes: [ “http" ] consumes: [

    “application/json" ] produces: [ “application/json" ] paths: { /pets: { post: { summary: Create a pet responses: { 201: { description: "Null response” } default: { description: unexpected error, schema: { $ref: “#/definitions/Error" } } …. }}}} Error: { required: [ “code", “message" ], properties: { code: { type: integer format: int32 }, message: { type: string } } }
  96. None
  97. SpringFox http://springfox.github.io/springfox/ @EnableSwagger2 public class SwaggerConfiguration { @Override public void

    addResourceHandlers(ResourceHandlerRegistry registry) { registry.addResourceHandler("swagger-ui.html") .addResourceLocations("classpath:/META-INF/resources/"); registry.addResourceHandler("/webjars/**") .addResourceLocations("classpath:/META-INF/resources/webjars/"); } }
  98. JAXRS-Analyzer https://github.com/sdaschner/jaxrs-analyzer Creates REST documentation for JAX-RS projects

  99. Alternatives http://nordicapis.com/top-specification-formats-for-rest-apis/ • http://raml.org/ • https://github.com/apiaryio/api-blueprint

  100. Asynchronous … when().post("/houses"). then().statusCode(202) .and().header("Location", "Iron Throne”); …

  101. Asynchronous await().until(newHouseEmerges()); await().until( getHouse(“Lannister”), isNotNull() ); https://github.com/jayway/awaitility

  102. Asynchronous https://github.com/jayway/awaitility with().pollInterval(ONE_HUNDERED_MILLISECONDS)
 .and().with().pollDelay(20, MILLISECONDS)
 .await("creating a house")
 .until(getHouse("Lannister"), hasProperty("status",

    is("live")));
  103. Other REST-assured features • form • basic • certificate •

    oauth • oauth2 Easy authentication mechanism
  104. Other REST-assured features • proxy support • JSON-Shema validation •

    Multi-value headers and cookies • Multi-part form data support • Filters
  105. Points Made

  106. Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  107. Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  108. Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  109. Points Made • In addition to unit, integration and ui

    tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most significant properties for a the comparison, over making an all-to-all comparison
  110. Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
  111. Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
  112. Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results
  113. Points Made • Don’t relay on existing data and prepare

    ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath/XPath to assert the results
  114. Tools to look at • REST-assured for testing RESTful api

    • Awaitility to deal with asynchronous jobs • Swagger, RAML, API Blueprint for API description • Arquillian to start up your Java EE based projects • Moco or WireShark to startup a mock web service • HJSON is awesome!
  115. TESTS ARE MADE TO MAKE YOU FEEL SECURE!

  116. LIKE A LOVELY HUG ♥

  117. Q&A Thank You! @AlexeyBuzdin Follow me at