Testing RESTful services: GoT JavaDay Kiev Edition

Slide 1

Slide 1 text

We came here to kick ass and test your services, and you’re all out of services @AlexeyBuzdin

Slide 2

Slide 2 text

@AlexeyBuzdin Testing RESTful services

Slide 3

Slide 3 text

@AlexeyBuzdin Developer / Trainer at GDGRiga.lv RigaDevDay.lv

Slide 4

Slide 4 text

Lets start with a tale…

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Layered Architecture

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

UI Service Unit

Slide 11

Slide 11 text

UI Service Unit Effort

Slide 12

Slide 12 text

UI Service Unit Effort Cost

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

BFF BFF DB DB DB AMQP

Slide 15

Slide 15 text

BFF BFF DB DB DB AMQP

Slide 16

Slide 16 text

API is the new UI!

Slide 17

Slide 17 text

test it’s API To test a microservice

Slide 18

Slide 18 text

Why API tests?

Slide 19

Slide 19 text

Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code

Slide 20

Slide 20 text

public class SimpleServlet extends GenericServlet {    public void service(ServletRequest req, ServletResponse res)  throws ServletException, IOException {  // do something in here  }  } Simple Servlet

Slide 21

Slide 21 text

Simple Servlet (Annotation Driven Development … he.. he..) @Controller  @RequestMapping("/")  public class SimpleController {    @RequestMapping(method = RequestMethod.GET)  public String get() {  return "Hello";  }  }

Slide 22

Slide 22 text

Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code

Slide 23

Slide 23 text

Simple Servlet @Controller  @RequestMapping("/")  public class SimpleController {    @RequestMapping(method = POST)  public String post(MyBean bean) {  ...  }  } (Annotation Driven Development … he.. he..)

Slide 24

Slide 24 text

Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code

Slide 25

Slide 25 text

Why API tests? • Tests routing • Tests serialization/deserialization • Tests server response • Can be written by a QA who does not knows the code

Slide 26

Slide 26 text

DB DB AMQP 1. Isolate the Subject

Slide 27

Slide 27 text

Hint: Decouple integrations from your app Design your integrations to be easily pluggable and mockable in advance, or feel the consequences

Slide 28

Slide 28 text

http://projects.spring.io/spring-integration/

Slide 29

Slide 29 text

Spring Integration        

Slide 30

Slide 30 text

Spring Integration public interface RequestGateway {  String echo(String request);  }

Slide 31

Slide 31 text

Moco https://github.com/dreamhead/moco java -jar moco-runner.jar http -p 12306 -c foo.json {  "request": {  "json": { "foo": "bar" }  },  "response": { "text": "foo" }  }

Slide 32

Slide 32 text

Moco MocoHttpServer server = new MocoHttpServer( ActualHttpServer.createLogServer(of(port()))); server.start(); server.request(by("foo")).response("bar"); https://github.com/dreamhead/moco

Slide 33

Slide 33 text

WireMock http://wiremock.org/ @Rule public WireMockRule wireMockRule = new WireMockRule(8089); stubFor(get(urlEqualTo("/my/resource")) .withHeader("Accept", equalTo("text/xml")) .willReturn(aResponse() .withStatus(200) .withHeader("Content-Type", "text/xml") .withBody("Some content")));

Slide 34

Slide 34 text

For DB Mocking

Slide 35

Slide 35 text

For DB Mocking • Substitute to an in-memory DB (Spring Data and JPA/ Hibernate will help here a lot) • Start up a clean instance of a DB for testing • Reuse developers instance for testing purposes • Mock data to be fetched from ﬁle system or memory

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

DB DB AMQP 2. Automate service startup

Slide 40

Slide 40 text

Application Server?

Slide 41

Slide 41 text

@RunWith(SpringJUnit4ClassRunner.class)  @SpringApplicationConfiguration(classes = Application.class)  @WebAppConfiguration  @IntegrationTest("server.port:0")  public class CharacterControllerTest {    @Value("${local.server.port}")  int port; ...  }

Slide 42

Slide 42 text

https://github.com/arquillian @RunWith(Arquillian.class)  public class GreeterTest {    @Deployment  public static WebArchive createDeployment() {  return ShrinkWrap.create(WebArchive.class)  .addClasses(Greeter.class)  .setWebXML("WEB-INF/web.xml");  } ...  }

Slide 43

Slide 43 text

DB DB AMQP 2. Automate service startup

Slide 44

Slide 44 text

3. Write a HealthCheck test https://github.com/jayway/rest-assured @Test  public void applicationIsUp() {  given().port(8888).  when().get("/healthcheck").  then().statusCode(200);  }

Slide 45

Slide 45 text

https://github.com/jayway/rest-assured RequestSpeciﬁcation - build the request ResponseSpecification - contains parsed response given().  param("name", "Johan").  cookie("cookie", "is mine").  body("{}").  header("Accept", “application/json"). then().  statusCode(200).  header("Content-Type", "application/json");  when().post(“/resource").

Slide 46

Slide 46 text

Hint: Always make url/port configurable At some point you would like to run the tests against your hotfixed api or even generate some traffic on your staging environment

Slide 47

Slide 47 text

Lets test a Sample Server

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

REST is all about the resources /houses

Slide 50

Slide 50 text

C R U D

Slide 51

Slide 51 text

C R U D - POST - GET - PUT - DELETE

Slide 52

Slide 52 text

C R U D - GET

Slide 53

Slide 53 text

HJson / HumanJSON https://hjson.org/ given(). body(readHjson( "name : Lannister" + "lore : {" + "sigil : A golden lion rampant on a crimson ﬁeld." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()). when().post("/houses"). then().statusCode(201); .and().header("Location", "Iron Throne");

Slide 54

Slide 54 text

given(). body(readHjson( "name : Lannister" + "lore : {" + "sigil : A golden lion rampant on a crimson ﬁeld." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()). when().post("/houses"). then().statusCode(201) .and().header("Location", "Iron Throne");

Slide 55

Slide 55 text

Accept tested • Application is started • Resource collection exists • Provided JSON is parsed successfully • No validation errors

Slide 56

Slide 56 text

But we haven’t tested if the resource was created on the server!

Slide 57

Slide 57 text

String location = … when().post("/houses"). then().statusCode(201) .extract().header(“Location”); assertEquals("Iron Throne", location); get(location).then().statusCode(200);

Slide 58

Slide 58 text

Created tested • Application is started • Resource collection exists • Provided JSON is parsed successfully • No validation errors • Resource was stored on the server

Slide 59

Slide 59 text

Slide 60

Slide 60 text

assertEquals("Iron Throne", location); get(location).then().statusCode(200) .and().body(".", equalTo(readHjson( "name : Lannister" + "lore : {" + "sigil : A golden lion rampant on a crimson ﬁeld." + "words : Hear Me Röar!” + "shouldBe : Twincest is Wincest!” + "}" ).toString()); Test Actual Data

Slide 61

Slide 61 text

Unicode is not working java.lang.AssertionError: 1 expectation failed. Response body doesn't match expectation. Expected: “\”words\” : \”Hear Me Röar!\”” Actual: ”words” : ”Hear Me R䡦ar!”

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Hint: Never use equals to compare server response Usually you have tons of stuff generated automatically on the backend which value can not be predicted in advance*: createdOn, updatedOn, auto generated IDs, etc

Slide 64

Slide 64 text

Hint: Use JSONPath and XPath to assert equality name = Lannister lore.sigil = … lore.words = … lore.shouldBe = … "name : Lannister" + "lore : {" + "sigil : …” + "words : ….” + "shouldBe : ….” + "}"

Slide 65

Slide 65 text

Hint: Use JSONPath and XPath to assert equality name = Lannister lore.sigil = … lore.words = … lore.shouldBe = … members[0] = Cercei members[1] = Tyrion members[2] = Jamie "name : Lannister" + "lore : {" + … “}" “members : [“ + “Cercei” “Tyrion” “Jamie” “]”

Slide 66

Slide 66 text

Hint: Implement order for your arrays or use contains matchers Arrays are unordered if fetched straight from DB

Slide 67

Slide 67 text

To Sum Up

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

Hint: Always prepare test data yourself You don’t want to have race conditions or any other unexpected trouble born from shared state.

Slide 70

Slide 70 text

Hint: Run your tests in parallel • Conﬁgure failsafe or gradle conﬁg to have maxParallelForks > 1 • Organise your test scenarios into Test Suites with atomic sense

Slide 71

Slide 71 text

More POST on a resource • Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conﬂict (409) • Unsupported Media Type (415)

Slide 72

Slide 72 text

Slide 73

Slide 73 text

Slide 74

Slide 74 text

• Created successfully -> Created (201) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource already exists -> Conﬂict (409) • Unsupported Media Type (415) More POST on a resource

Slide 75

Slide 75 text

Slide 76

Slide 76 text

Hint: Expose all errors and conditions to the client in RESTful way! REST is stateless; pass errors to the client!

Slide 77

Slide 77 text

C R U D - POST - GET - PUT - DELETE

Slide 78

Slide 78 text

C R U D - POST - GET - PUT - DELETE

Slide 79

Slide 79 text

More GET on a resource • Get existing resource -> OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)

Slide 80

Slide 80 text

More GET on a resource • Get existing resource -> OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)

Slide 81

Slide 81 text

More GET on a resource • Get existing resource -> OK (200) • Get non existing resource -> Not Found (404) • Unsupported Media Type (415)

Slide 82

Slide 82 text

DELETE on a resource

Slide 83

Slide 83 text

DELETE on a resource • Delete non existing resource -> Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)

Slide 84

Slide 84 text

DELETE on a resource • Delete non existing resource -> Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)

Slide 85

Slide 85 text

DELETE on a resource • Delete non existing resource -> Not Found (404) • Delete existing resource -> OK (200) • Resource is in use -> Forbidden (403)

Slide 86

Slide 86 text

PUT on a resource

Slide 87

Slide 87 text

PUT on a resource • Update non existing resource -> Not Found (404) • Update existing resource -> OK (200) • Missing a mandatory property -> BadRequest (400) • Validation failed -> BadRequest (400) • Resource is in use -> Forbidden (403) • Unsupported Media Type (415)

Slide 88

Slide 88 text

Slide 89

Slide 89 text

Slide 90

Slide 90 text

Slide 91

Slide 91 text

Slide 92

Slide 92 text

One Throne but a lot of deaths CRUD effort

Slide 93

Slide 93 text

Hint: Invest into describing your API Either it be a API Doc or better an API representation in some data format

Slide 94

Slide 94 text

Open APIs https://openapis.org/

Slide 95

Slide 95 text

swagger: 2.0 host: petstore.swagger.io schemes: [ “http" ] consumes: [ “application/json" ] produces: [ “application/json" ] paths: { /pets: { post: { summary: Create a pet responses: { 201: { description: "Null response” } default: { description: unexpected error, schema: { $ref: “#/deﬁnitions/Error" } } …. }}}} Error: { required: [ “code", “message" ], properties: { code: { type: integer format: int32 }, message: { type: string } } }

Slide 96

Slide 96 text

No content

Slide 97

Slide 97 text

SpringFox http://springfox.github.io/springfox/ @EnableSwagger2 public class SwaggerConﬁguration { @Override public void addResourceHandlers(ResourceHandlerRegistry registry) { registry.addResourceHandler("swagger-ui.html") .addResourceLocations("classpath:/META-INF/resources/"); registry.addResourceHandler("/webjars/**") .addResourceLocations("classpath:/META-INF/resources/webjars/"); } }

Slide 98

Slide 98 text

JAXRS-Analyzer https://github.com/sdaschner/jaxrs-analyzer Creates REST documentation for JAX-RS projects

Slide 99

Slide 99 text

Alternatives http://nordicapis.com/top-speciﬁcation-formats-for-rest-apis/ • http://raml.org/ • https://github.com/apiaryio/api-blueprint

Slide 100

Slide 100 text

Asynchronous … when().post("/houses"). then().statusCode(202) .and().header("Location", "Iron Throne”); …

Slide 101

Slide 101 text

Asynchronous await().until(newHouseEmerges()); await().until( getHouse(“Lannister”), isNotNull() ); https://github.com/jayway/awaitility

Slide 102

Slide 102 text

Asynchronous https://github.com/jayway/awaitility with().pollInterval(ONE_HUNDERED_MILLISECONDS)  .and().with().pollDelay(20, MILLISECONDS)  .await("creating a house")  .until(getHouse("Lannister"), hasProperty("status", is("live")));

Slide 103

Slide 103 text

Other REST-assured features • form • basic • certiﬁcate • oauth • oauth2 Easy authentication mechanism

Slide 104

Slide 104 text

Other REST-assured features • proxy support • JSON-Shema validation • Multi-value headers and cookies • Multi-part form data support • Filters

Slide 105

Slide 105 text

Points Made

Slide 106

Slide 106 text

Points Made • In addition to unit, integration and ui tests, cover at least core functionality and cases with API tests • Design your app integrations to be easily pluggable and mockable • Decouple connection settings with the test suite itself (uri, proxy settings, authentication methods, etc) • Never use equals to compare server response and try to identify the most signiﬁcant properties for a the comparison, over making an all-to-all comparison

Slide 107

Slide 107 text

Slide 108

Slide 108 text

Slide 109

Slide 109 text

Slide 110

Slide 110 text

Points Made • Don’t relay on existing data and prepare ones for testing before test starts • Design your tests to be easily parallel and sort them in a way that they won’t affect each other • Invest in having a descriptive API representation like Swagger, RAML, or API Blueprints • Use JSONPath to assert the results

Slide 111

Slide 111 text

Slide 112

Slide 112 text

Slide 113

Slide 113 text

Slide 114

Slide 114 text

Tools to look at • REST-assured for testing RESTful api • Awaitility to deal with asynchronous jobs • Swagger, RAML, API Blueprint for API description • Arquillian to start up your Java EE based projects • Moco or WireShark to startup a mock web service • HJSON is awesome!