Automatic Code Generation for SPA

Slide 1

Slide 1 text

Automatic Code Generation for SPA Mercari x Merpay Frontend Tech Talk vol.2 2019/07/03 @_tanakaworld

Slide 2

Slide 2 text

About Me ● Twitter: @_tanakaworld ● GitHub: tanakaworld ● Merpay ○ Software Engineer (Frontend) ○ CSTool / MSTool

Slide 3

Slide 3 text

1. What is Code Generation? 2. Tell about the Real World Rearchtecture project 3. Tips of Generator Agenda

Slide 4

Slide 4 text

What is Code Generation?

Slide 5

Slide 5 text

Automatically generate SDK from schema Spec Source Code SDK Schema Source Code SDK Generator

Slide 6

Slide 6 text

Schema Driven Development ● Make Development flow more eﬀiciently ● Automatic ○ Source Code Generation ○ Document Generation ○ Testing

Slide 7

Slide 7 text

Microservices in Merpay Protocol Buﬀers ● gRPC ● Write Schema by Protocol Buﬀers ● Generate Client and Server

Slide 8

Slide 8 text

Protocol Buﬀers service SampleAPI { rpc GetUser(GetUsersRequest) returns (GetUsersResponse) { option (google.api.http) = { get : "/users" body: "*" }; } } message GetUsersRequest {} message GetUsersResponse { repeated User user = 1; } .proto

Slide 9

Slide 9 text

Code Generation in Merpay Frontend Proxy HTTP gRPC

Slide 10

Slide 10 text

.proto to TypeScript export namespace SampleAPI { export class GetUser implements APIRequest { _response?: GetUsersResponse; path = "/users"; method: HTTPMethod = "get"; constructor(public parameter: GetUsersRequest) {} } } export interface GetUsersRequest {} export interface GetUsersResponse { user?: User[]; }

Slide 11

Slide 11 text

REAL WORLD Rearchitecture Project

Slide 12

Slide 12 text

REAL WORLD Rearchitecture Project 1. Overview 2. Backend Implementation 3. API schema 4. Generator 5. Frontend Implementation

Slide 13

Slide 13 text

1. Overview

Slide 14

Slide 14 text

https://proﬀ.io/

Slide 15

Slide 15 text

Traditional Rails App ● Render every time in Server ● Bad UX

Slide 16

Slide 16 text

Rearchitecture ● Re-implement 70% features ● Backend RESTful API 56 Endpoints ● Frontend SPA 24 pages

Slide 17

Slide 17 text

2. Backend Implementation

Slide 18

Slide 18 text

Well known JSON serializers ● jbuilder ● active_model_serializers ● ••• ● fast_jsonapi

Slide 19

Slide 19 text

fast_jsonapi https://medium.com/netflix-techblog/fast-json-api-serialization-with-ruby-on-rails-7c06578ad17f ● A lightning fast JSON:API serializer for Ruby Objects ● Around 25 times faster than AMS ● is used in several APIs at Netflix

Slide 20

Slide 20 text

JSON:API ● https://jsonapi.org

Slide 21

Slide 21 text

Why fast_jsonapi was developed? ● Netflix ○ always use JSON:API ● AMS is too flexible ○ AMS is designed to serialize JSON in several diﬀerent formats, not just JSON:API

Slide 22

Slide 22 text

Serializer class BookSerializer include FastJsonapi::ObjectSerializer attributes :id, :title, :description, :created_at, :updated_at end

Slide 23

Slide 23 text

{ "data":[ { "id":"1", "type":"book", "attributes":{ "id":1, "title":"AAA", "description":"Test", "created_at":"2019-06-23T13:28:32.550Z", "updated_at":"2019-06-23T13:28:32.573Z" } } ] }

Slide 24

Slide 24 text

3. API schema

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

OpenAPI Speciﬁcation ● https://swagger.io/specification/v2/ ● fka “Swagger” ● help you design, build, document and consume REST APIs

Slide 27

Slide 27 text

TL;DR; API ● Resource ● Request ● Response ● Method ● URL ● Params RSpec Schema /app/books

Slide 28

Slide 28 text

● https://github.com/fotinakis/swagger-blocks ● Works with all Ruby web frameworks ● 100% support for all features of the Swagger 2.0 spec ● Can use Ruby DSL to write OpenAPI schema swagger-blocks

Slide 29

Slide 29 text

# app/models/concerns/swagger/book_schema.rb module Swagger::BookSchema extend ActiveSupport::Concern include Swagger::Blocks included do swagger_schema :Book, required: [:title, :description, :image_url] do property :id, type: :integer property :title, type: :string property :description, type: :string property :created_at, type: :string property :updated_at, type: :string end end end

Slide 30

Slide 30 text

Wrote Extension for JSON:API ● https://github.com/tanakaworld/swagger-blocks-fastjson-api/blob/master/config /initializers/swagger_blocks.rb

Slide 31

Slide 31 text

4. Generator

Slide 32

Slide 32 text

Generate API Client API Client Schema

Slide 33

Slide 33 text

openapi-generator ● https://github.com/OpenAPITools/openapi-generator ● It allows generation from OpenAPI Spec ○ API client libraries ○ Server stubs ○ Documentation ○ Configuration

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

Install npm i -D @openapitools/openapi-generator-cli

Slide 36

Slide 36 text

Generate openapi-generator generate \ -i ./tmp/swagger.json \ -g typescript-axios \ -o ./app/javascript/src/gen/ --additional-properties="modelPropertyNaming=snake_ case" https://github.com/OpenAPITools/openapi-generator/blob/master/docs/generators/typescript-axios.md

Slide 37

Slide 37 text

export interface Book { id?: number; title: string; description: string; image_url: string; created_at?: string; updated_at?: string; } export interface BookResponse { id: string; type: string; attributes: Book; } export interface CreateBookRequest { title?: string; description?: string; image?: object; }

Slide 38

Slide 38 text

export class SampleAppApi extends BaseAPI { public createBook(title?: string, description?: string, image?: object, options?: any) { ••• } public deleteBook(id: number, options?: any) { ••• } public getBook(id: number, options?: any) { ••• } public getBooks(options?: any) { ••• } public updateBook(id: number, title?: string, description?: string, image?: object, options?: any) { ••• } }

Slide 39

Slide 39 text

$ npm run api:gen

Slide 40

Slide 40 text

5. Frontend Implementation

Slide 41

Slide 41 text

Overview ● Render SPA bundle via Rails View ● with Turbolinks ● SPA ○ CSR ○ Vue ○ Vue Router ○ Vuex API Client /app/books SPA

Slide 42

Slide 42 text

Rails Routing Rails.application.routes.draw do // path for SPA // app/books // app/books/new // app/books/1 // app/books/1/edit // ••• match 'app/(*spa_path)' => 'spa_app#index', via: :get end

Slide 43

Slide 43 text

Response Empty DOM with ‘data-vue’ // app/views/spa_app/index.html.erb

Slide 44

Slide 44 text

How it works with Turbolinks ● Make Vue instance on 'turbolinks:load' ● Destroy Vue instance on 'turbolinks:visit' ● ref: https://qiita.com/midnightSuyama/items/efc5441a577f3d3abe74

Slide 45

Slide 45 text

Make/Destroy Vue instances document.addEventListener('turbolinks:load', () => { const templates = document.querySelectorAll('[data-vue]'); templates.forEach((el: HTMLElement) => { const option = options[el.dataset.vue] const vm = new Vue(Object.assign(option, { el })); vms.push(vm); }); }); document.addEventListener('turbolinks:visit', () => { for (let vm of vms) { vm.$destroy(); } vms = []; });

Slide 46

Slide 46 text

Use require.context of webpack // app/javascript/packs/vue-turbolinks.ts const options = {}; const requireContext = require.context('../options', false, /\.ts$/); requireContext.keys().forEach(key => { const name = key .split('/') .pop() .split('.') .shift(); options[name] = requireContext(key).default; });

Slide 47

Slide 47 text

Option // app/javascript/options/spa-app.ts import router from '@spa-app/router'; import store from '@spa-app/store'; export default { router, store };

Slide 48

Slide 48 text

Vuex Store Implementation ● Use generated API Client in Store ● FSA = Flux Standard Action ○ https://github.com/redux-utilities/flux-standard-action ○ https://github.com/sue71/vuex-typescript-fsa

Slide 49

Slide 49 text

Vuex + FSA store.dispatch('increment') // or store.dispatch({ type: 'increment', meta: null, error: null, payload: { ••• } }) https://vuex.vuejs.org/guide/actions.html#dispatching-actions

Slide 50

Slide 50 text

import { actionCreatorFactory } from 'vuex-typescript-fsa/lib'; import { UpdateBookRequest } from '@gen'; export const namespace = 'books'; const actionCreator = actionCreatorFactory(namespace); export const GetBook = actionCreator<{ id: number; }>( 'GET_BOOK' ); Deﬁne FSAs

Slide 51

Slide 51 text

Dispatch import * as BooksModule from '@spa-app/store/modules/books'; BooksModule.GetBook.namespaced({ id: 1 });

Slide 52

Slide 52 text

Actions / Mutations / Getters export const module: Module = { namespaced: true, actions: combineAction( action(GetBook, async function(context, action) { const { data } = await this.sampleAppAPI .getBook(action.payload.id); context.commit(BookReceived(data.data)); }) ) };

Slide 53

Slide 53 text

Tips of OpenAPI Generator

Slide 54

Slide 54 text

Error: URLSearchParams is not deﬁned

Slide 55

Slide 55 text

Error: URLSearchParams is not deﬁned import * as url from 'url'; // Error const params = url.URLSearchParams(•••);

Slide 56

Slide 56 text

Workaround ● https://github.com/swagger-api/swagger-codegen/issues/6403 ● I don’t want to change generated code

Slide 57

Slide 57 text

Override ‘url’ package (Monkey patch) // webpacker config resolve: { alias: { '@': path.resolve(__dirname, '../../app/javascript/src'), urlOriginal: path.resolve(__dirname, '../../node_modules/url'), url: path.resolve(__dirname, './extensions/url') } } // config/webpack/extensions/url.js import { parse, resolve, resolveObject, format, Url } from 'urlOriginal'; import URLSearchParams from '@ungap/url-search-params'; export { parse, resolve, resolveObject, format, Url, URLSearchParams };

Slide 58

Slide 58 text

multipart/form-data

Slide 59

Slide 59 text

was ﬁxed ● https://github.com/OpenAPITools/op enapi-generator/commit/550774a6e2 e597d5f4460ec6f8a80951f3ada37e

Slide 60

Slide 60 text

Heavy Base64 Image ● Easy to upload image as Base64 String in Rails ○ https://github.com/y9v/carrierwave-base64 ○ Able to send image via ‘application/json’ ○ size tooooooo big ● Use multipart/form-data instead

Slide 61

Slide 61 text

Attribute Overlapping

Slide 62

Slide 62 text

e.g. ‘url’ // ‍♂ This overlaps with import * url from 'url' if (url !== undefined) { localVarFormParams.append('url', url as any); } // ‍♂ rename to '_url' if (_url !== undefined) { localVarFormParams.append('_url', _url as any); } // convert in Rails params.tap {|p| p[:url] = p[:_url]}.permit( :title, :description, :url, :image )

Slide 63

Slide 63 text

ﬁxed now ● [TS][Axios] To fix conflict params name 'url' ○ https://github.com/OpenAPITools/openapi-generator/pull/2921

Slide 64

Slide 64 text

Try BETA Version

Slide 65

Slide 65 text

Put .jar into node_modules/ // @openapitools/openapi-generator-cli/bin/openapi-generator const binPath = resolve(__dirname, 'openapi-generator.jar'); const JAVA_OPTS = process.env['JAVA_OPTS'] || ''; let command = `java ${JAVA_OPTS} -jar "${binPath}"`;

Slide 66

Slide 66 text

Generator is changed every day. ● You can download and use snapshot ● https://oss.sonatype.org/content/repositories/ snapshots/org/openapitools

Slide 67

Slide 67 text

Summary

Slide 68

Slide 68 text

Overview Schema OpenAPI Spec API Client SPA

Slide 69

Slide 69 text

Pro/Con ● Pro ○ Able to detect unexpected changes ○ Reduce change by hand ○ Able to reuse API Client for another platform ○ Automatically detect diﬀ ● Con ○ Took time to setup ○ Too many things to do ○ Too much for single person project

Slide 70

Slide 70 text

Sample ● https://github.com/tanakaworld/swagger-blocks-fastjson-api

Slide 71

Slide 71 text

Thanks

Slide 72

Slide 72 text

Appendix

Slide 73

Slide 73 text

OpenAPI + JSON:API is Bad approach? ● https://apisyouwonthate.com/blog/json-api-openapi-and-json-schema-working-i n-harmony ● “At WeWork, we use JSON Schema to describe the data models, OpenAPI to describe everything else, then the message being described is usually JSON API.”

Slide 74

Slide 74 text

※ Fast JSON API is not a replacement for AMS ● AMS does many things and is very flexible ○ This is why fast_jsonapi is more faster ● Netflix still use AMS for non JSON:API serialization and deserialization ● ref: Performance methodology