Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Web Development with OpenAPI Spec & Fast Testing

Web Development with OpenAPI Spec & Fast Testing

Modern web applications are mostly built on API and client side apps. Centralizing the business logic in API can prevent clients duplicating features on their sides.

However, in certain situations, we might face a tight schedule, work with clients who use waterfall workflow, or work with clients who are very sensitive about any unexpected behaviors.

This talk will cover how we utilize OpenAPI Spec for cross-team communication, keep the development speed by using slow but steady integration tests, and how we speed up those tests.

This talk was originally given at WebHack Meetup, a software engineering meetup in Tokyo. https://webhack.connpass.com/event/121030/

I gave this talk as a web engineer at Moneytree KK, which provides financial data platform for general customers, financial institutions and businesses. Check Moneytree's careers page for current open positions. https://www.moneytree.jp/careers

Yu-Cheng Chuang

February 27, 2019
Tweet

More Decks by Yu-Cheng Chuang

Other Decks in Technology

Transcript

  1. 2019 © Moneytree KK. Web Development with OpenAPI Spec &

    Fast Testing Yu-Cheng Chuang Feb 27, 2019 Slides will be available online
  2. @yorkxin @chitsaou Yu-Cheng Chuang
 Core Systems Engineer 
 at Moneytree

    From Taiwan BSD / Linux User since 2005 Web Developer since 2009 #typography #railway #cooking About me
  3. Moneytree is a financial data portability platform, comprised of a

    personal financial management app, and an API (Application Programming Interface) feed of customer data for financial institutions. Moneytree in Brief "
  4. • Backend server built on Ruby on Rails • Server

    is running on AWS ECS • User management on AWS Cognito • CircleCI / AWS Code Pipeline • Web App built on React • iOS, Android Apps The Stack Project Background
  5. # Customer™ $ Backend Developers % $ % Web Developers

    $ % $ % Mobile Developers $ % $ % & Project Managers '
  6. • Allows you to design / define HTTP API in

    YAML or JSON • Request & Responses can be defined as Schema • Using JSON Schema-like DSL Open API Spec (f.k.a. Swagger) Open API Spec
  7. paths: /pets: get: summary: List all pets operationId: listPets tags:

    - pets parameters: - name: limit in: query description: How many items to return at one time (max 100) required: false schema: type: integer format: int32 responses: '200': description: A paged array of pets headers: x-next: description: A link to the next page of responses schema: type: string content: application/json: schema: $ref: "#/components/schemas/Pets" default: description: unexpected error content: application/json: schema: $ref: "#/components/schemas/Error" components: schemas: Pet: required: - id - name properties: id: type: integer format: int64 name: type: string tag: type: string nullable: true Pets: type: array items: $ref: "#/components/schemas/Pet" Error: required: - code - message properties: code: type: integer format: int32 message: type: string - - - ! ! !
  8. { "$schema": "http://json-schema.org/draft-07/schema#", "$id": "http://example.com/product.schema.json", "title": "Product", "description": "A product

    from Acme's catalog", "type": "object", "properties": { "productId": { "description": "The unique identifier for a product", "type": "integer" }, "productName": { "description": "Name of the product", "type": "string" }, "price": { "description": "The price of the product", "type": "number", "exclusiveMinimum": 0 }, "tags": { "description": "Tags for the product", "type": "array", "items": { "type": "string" }, "minItems": 1, "uniqueItems": true } }, "required": [ "productId", "productName", "price" ] } { "productId": 1, "productName": "A green door", "price": 12.50, "tags": [ "home", "green" ] } https:/ /json-schema.org/learn/getting-started-step-by-step.html
  9. { "$schema": "http://json-schema.org/draft-07/schema#", "$id": "http://example.com/product.schema.json", "title": "Product", "description": "A product

    from Acme's catalog", "type": "object", "properties": { "productId": { "description": "The unique identifier for a product", "type": "integer" }, "productName": { "description": "Name of the product", "type": "string" }, "price": { "description": "The price of the product", "type": "number", "exclusiveMinimum": 0 }, "tags": { "description": "Tags for the product", "type": "array", "items": { "type": "string" }, "minItems": 1, "uniqueItems": true } }, "required": [ "productId", "productName", "price" ] } schemas: Pet: required: - id - name properties: id: type: integer format: int64 name: type: string tag: type: string nullable: true ᶃ ᶃ ᶄ ᶄ OpenAPI Schema vs. JSON Schema
  10. ⋊> ~/D/pet-store openapi2schema -i pet-store.openapi.yml | jq { "/pets": {

    "get": { ...}, "post": {…} }, "/pets/{petId}": { "get": { "responses": { "200": { "type": "array", "items": { "required": [“id", "name"], "properties": { "id": { "type": "integer", "format": "int64", "minimum": -9223372036854776000, "maximum": 9223372036854776000 }, "name": { "type": "string" }, "tag": { "type": [ "string", "null" ] } } }, "$schema": "http://json-schema.org/draft-04/schema#" }, "default": {...} } } } } components: schemas: Pet: required: - id - name properties: id: type: integer format: int64 name: type: string tag: type: string nullable: true
  11. • YAML / JSON = Machine-friendly • Schema is defined

    using JSON Schema (*partial compatible) Pros Open API Spec
  12. • Most libraries only support version 2 • See https:/

    /openapi.tools/ • To validate data structure, you need to
 convert Schema definitions to JSON Schema manually Cons Open API Spec
  13. def load_schema_file(path) # Converts to JSON Schema with a Node

    CLI tool JSON.parse(`npx openapi2schema -i #{path}`) end ※ Don’t do this in production (only in tests) Since we’re using Ruby…
  14. 3) GET /pets/{id} Should be successful matches response schema (success)

    Failure/Error: expect(response.parsed_body).to match_json_schema(schema) The property '#/pet/name' of type integer did not match the following type: string in schema eb4470d8-8767-56a6-83b7-789839be31e0 Shared Example Group: "Should conform OAS spec" called from ./spec/requests/ shared_examples/response_examples.rb:2
  15. • Seed data using test factories or database dumps •

    Test output with response schema from API Spec • Client developers can start working 
 with fake data and real response structure • Backend developers can start working on 
 JSON serializers Stub Server Extra Benefits
  16. ⋊> ~/D/pet-store find . -name *.yml -print0 | xargs -n

    1 -0 swagger-cli validate Swagger schema validation failed. Additional properties not allowed: responses at #/paths//pets JSON_OBJECT_VALIDATION_FAILED
  17. • Test scenarios are close to business cases • Think

    from Client Developer’s and User’s perspective • Useful for Regression Tests • Ensuring all responses to match API Spec • So that Client developers will be happy every day Pros Request Tests
  18. ⋊> ~/D/rocketscience on develop ◦ bundle exec rake stats +----------------------+--------+--------+---------+---------+-----+-------+

    | Name | Lines | LOC | Classes | Methods | M/C | LOC/M | +----------------------+--------+--------+---------+---------+-----+-------+ | Controllers | 1489 | 1151 | 42 | 145 | 3 | 5 | | Helpers | 11 | 10 | 0 | 1 | 0 | 8 | | Jobs | 2 | 2 | 1 | 0 | 0 | 0 | | Models | 2822 | 1781 | 33 | 155 | 4 | 9 | | Mailers | 135 | 95 | 4 | 5 | 1 | 17 | | Channels | 8 | 8 | 2 | 0 | 0 | 0 | | JavaScripts | 1 | 0 | 0 | 0 | 0 | 0 | | Libraries | 155 | 119 | 4 | 8 | 2 | 12 | | Controller tests | 42 | 36 | 1 | 5 | 5 | 5 | | Model tests | 690 | 481 | 4 | 38 | 9 | 10 | | Mailer tests | 0 | 0 | 0 | 0 | 0 | 0 | | Integration tests | 0 | 0 | 0 | 0 | 0 | 0 | | Model specs | 1040 | 834 | 0 | 0 | 0 | 0 | | Presenter specs | 637 | 528 | 4 | 7 | 1 | 73 | | Request specs | 11377 | 9330 | 0 | 78 | 0 | 117 | | Searcher specs | 361 | 298 | 0 | 0 | 0 | 0 | | Lib specs | 1197 | 994 | 0 | 0 | 0 | 0 | | Route specs | 43 | 38 | 0 | 0 | 0 | 0 | +----------------------+--------+--------+---------+---------+-----+-------+ | Total | 20010 | 15705 | 95 | 442 | 4 | 33 | +----------------------+--------+--------+---------+---------+-----+-------+ Code LOC: 3166 Test LOC: 12539 Code to Test Ratio: 1:4.0
  19. • Hard to debug due to deep stacks • Long

    Roundtrip → Slow • Very Slow ./0 Cons Request Tests
  20. • It assumes each job has same load • You

    need to write your own script to 
 distribute test files evenly by execution time or size Using GNU Parallel Run Test in Parallel
  21. Time 0 sec 500 sec 1,000 sec 1,500 sec 2,000

    sec 2,500 sec ~1600s (70%) ~650s (30%) Bottleneck ~250s
  22. • One endpoint per test file • It’s also easier

    for human to read • Avoid Race Condition — Reset Database for each case • Use as less requests for one scenario as possible • Data preparations (db insertions) are usually fast… • …Unless you’re inserting hundreds of rows Load Balancing Test Files Run Test in Parallel
  23. Took 441 seconds (7:21) real 7m21.858s user 26m6.616s sys 0m43.592s

    Took 278 seconds (4:38) real 4m39.159s user 27m46.976s sys 2m3.372s Took 439 seconds (7:19) real 7m19.914s user 28m23.356s sys 5m35.944s ⚙ ⚙ ⚙ × 4 × 8 × 16 ✅
  24. 8 processes might be faster than 16 • /proc/info may

    != # of CPU Cores • Be aware of startup time overhead (Rails is slow) • Find best number by measuring time • One CI machine might be enough Deciding # of Processes Speed up total test runtime
  25. 0 3000 6000 9000 12000 0 ms 120,000 ms 240,000

    ms 360,000 ms 480,000 ms 600,000 ms 720,000 ms Past ← Time → Now Total Runtime (ms) Number of Cases Time / Cases Ratio No junit.xml Reduced requests & Split into 2 machines Start using GNU Parallel 4 processes on 1 machine Found some files not tested Switched to Ruby parallel_tests Started 8 Processes Started testing JWT
  26. • Find an existing parallel testing solution 
 before you

    rollout your own one • Store test reports from day 1, if your CI supports it Lessons Parallel Testing
  27. “Amazon Cognito User Pools provide a secure user directory that

    scales to hundreds of millions of users. As a fully managed service, User Pools are easy to set up without any worries about server infrastructure. User Pools provide user profiles and authentication tokens for users who sign up directly and for federated users who sign in with social and enterprise identity providers.” AWS Cognito User Pools AWS Cognito https:/ /aws.amazon.com/cognito/details/
  28. • HMAC + SHA256 (Shared Secrets) • RSA / ECDSA

    (Asymmetric Signing) Algorithms JSON Web Token
  29. 4 4 JWT Issuer 5 6 API Server Signed by

    Private Key Verified by Public Key
  30. { "sub": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", // Unique user 'sub' in Cognito User

    Pool "aud": "xxxxxxxxxxxxexample", // OAuth Client ID "email_verified": true, "token_use": "id", "auth_time": 1500009400, "iss": "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_example", // Token Issuer "cognito:username": "janedoe", "exp": 1500013000, // Token Valid Until "given_name": "Jane", "iat": 1500009400, "email": "[email protected]" } A typical JWT Claim from AWS Cognito
  31. • The Customer™ trusts on AWS’s Security • Real Stateless

    Authentication • No (hashed) password stored in our database • Client App pinning (checking aud) • Easy to find user in database by inspecting the token Pros JSON Web Token
  32. • Token Revocation, obviously • Need to store JWKs (Public

    Keys) in every server daemon Cons JSON Web Token
  33. Just do what Cognito is doing • Generate an RSA

    Key Pair for Test Environment • Store the JWK of Public Key • Sign the JWT Token with Private Key • Verify the JWT with JWK from Public Key Mocking JWT Verification in Tests JSON Web Token
  34. ⋊> ~/ openssl genrsa 2048 > test.pem Generating RSA private

    key, 2048 bit long modulus ................................................................+++ ................................................+++ e is 65537 (0x10001) ⋊> ~/ npx pem-jwk test.pem {“kty”:”RSA”,”n”:”m3Knu8N5…”,”e”:”AQAB”,”d”:”X69q6lwo…”,"p":"zrFBKps h...","q":"wIfj...","dp":"f9WHPWe...","qi":"o_O3bEU..."}
  35. • Writing tests in the same framework may have blind

    spot • Incorrectly assuming Accept: application/json • Sending payload without serializing into JSON • Passing parameters in wrong data type • To make sure your service runs in Docker Why Black Box Integration Tests? Black Box Integration Tests
  36. • Docker Build • Docker Run • Database Migrations •

    PoC of an API Client using OpenAPI Spec • Typical scenarios (sign-up, create data, etc.) • Edge case scenarios that involve multiple actions • Job Workers What to test? Black Box Integration Tests
  37. SQL Database 1 Redis Cache 1 API Server 7 Job

    Worker ⚙ Fake Clients (Node.js) 89 Docker Compose
  38. 4 4 My Test Script 5 6 API Server Sign

    Token by my Private Key Verify Token by my Public Key
  39. import OpenAPIClientAxios from 'openapi-client-axios'; import * as fs from 'fs';

    import * as jwt from 'jsonwebtoken'; import * as jsYaml from 'js-yaml'; export class MyAppClient extends OpenAPIClientAxios { constructor(host = 'localhost:3000') { const url = `http://${host}/`; const definition = jsYaml.safeLoad(fs.readFileSync('/path/to/api/spec.openapi.yml')); definition.servers = [{ url }]; super({ definition }); } async showPetById(id) { return await this.client.query('showPetById', id); } // Sign a JWT Token loginAs(sub, email = null) { const options = { algorithm: 'RS256', issuer: 'test-issuer', audience: 'test-audience' }; const cert = fs.readFileSync('/path/to/jwk.pem'); this.token = jwt.sign({ email, sub }, cert, options); } }
  40. async init() { await super.init(); // NOTE: workaround https://github.com/axios/axios/pull/1395 this.client.interceptors.request.use((request)

    => { request.headers.Authorization = `Token ${this.token}`; return request; }); } Workaround Axios per-instance header issue
  41. version: '3' services: app: build: . ports: - "3000:3000" env_file:

    - __tests__/rails.env - __tests__/shared/jwt/jwks.env depends_on: ['db', 'redis'] db: image: "postgres:9" ports: - "15432:5432" environment: POSTGRES_PASSWORD: "password" redis: image: "redis:5" ports: - "16379:6379" • Expose ports for dockerize heath checking
  42. build: machine: true steps: - checkout - run: name: Install

    Docker Compose command: | curl -L https://github.com/docker/compose/releases/download/1.19.0/docker-compose-`uname -s`-`una chmod +x ~/docker-compose sudo mv ~/docker-compose /usr/local/bin/docker-compose • Run CircleCI task outside a container so that we can mount local files • Install docker-compose if necessary
  43. - run: name: Build image and pull dependencies command: docker-compose

    up --no-start - run: name: Run Containers background: true command: docker-compose up - run: name: Wait for containers command: | dockerize -wait tcp://localhost:15432 \ -wait tcp://localhost:16379 \ -wait tcp://localhost:3000 • Run Docker build & pull in an individual task • Start containers in background after build & pull were done • Wait until containers ready before running anything else (some RDB takes few seconds to initialize)
  44. - run: name: Test - Database preparation command: | docker-compose

    exec app rake \ db:create \ db:migrate \ db:seed • Prepare database by migration instead of loading schema
  45. github.com/wolfcw/libfaketime • A dynamic linking library to modify date() system

    call • Download a binary from Debian APT Repository • Or build it on a Linux you use for server container • A macOS dylib won’t work libfaketime.so Mocking System Time
  46. SQL Database 1 Redis Cache 1 API Server 7 Job

    Worker ⚙ Fake Clients (Node.js) 89 Docker Compose API Server + 2d 7
  47. # An API server that shifts machine time by +2d

    app-future: build: . ports: - “3001:3000” environment: LD_PRELOAD: /mnt/libfaketime/libfaketime.so.1 FAKETIME: +2d volumes: - ./path/to/libfaketime:/mnt/libfaketime env_file: - __tests__/rails.env - __tests__/shared/jwt/jwks.env depends_on: ['db', 'redis'] 1) Export to another host port 2) Load libfaketime.so for API server 3) Set Shifts 4) Mount faketime binaries into container ! ᶃ " ᶄ : ᶅ : ᶆ
  48. SQL Database 1 Redis Cache 1 API Server 7 Job

    Worker ⚙ Fake Clients (Node.js) 89 Docker Compose API Server + 2d 7 MailHog ; Scrape Emails through its API
  49. • Write as detailed test cases as possible • Use

    them as regression tests • Run tests in parallel to reduce total execution time • Test Docker Images to prevent shipping broken image Your Time is < < < Summary