Automatic Code Generation for SPA

https://mercari.connpass.com/event/134185/

Transcript

1. Automatic Code Generation for SPA Mercari x Merpay Frontend Tech Talk vol.2 2019/07/03 @_tanakaworld
2. About Me • Twitter: @_tanakaworld • GitHub: tanakaworld • Merpay ◦ Software Engineer (Frontend) ◦ CSTool / MSTool
3. 1. What is Code Generation? 2. Tell about the Real World Rearchtecture project 3. Tips of Generator Agenda
4. What is Code Generation?
5. Automatically generate SDK from schema Spec Source Code SDK Schema Source Code SDK Generator
6. Schema Driven Development • Make Development flow more efficiently • Automatic ◦ Source Code Generation ◦ Document Generation ◦ Testing
7. Microservices in Merpay Protocol Buffers • gRPC • Write Schema by Protocol Buffers • Generate Client and Server
8. Protocol Buffers service SampleAPI { rpc GetUser(GetUsersRequest) returns (GetUsersResponse) { option (google.api.http) = { get : "/users" body: "*" }; } } message GetUsersRequest {} message GetUsersResponse { repeated User user = 1; } .proto
9. Code Generation in Merpay Frontend Proxy HTTP gRPC
10. .proto to TypeScript export namespace SampleAPI { export class GetUser implements APIRequest<GetUsersRequest, GetUsersResponse> { _response?: GetUsersResponse; path = "/users"; method: HTTPMethod = "get"; constructor(public parameter: GetUsersRequest) {} } } export interface GetUsersRequest {} export interface GetUsersResponse { user?: User[]; }
11. REAL WORLD Rearchitecture Project
12. REAL WORLD Rearchitecture Project 1. Overview 2. Backend Implementation 3. API schema 4. Generator 5. Frontend Implementation
13. 1. Overview
14. https://proff.io/
15. Traditional Rails App • Render every time in Server • Bad UX
16. Rearchitecture • Re-implement 70% features • Backend RESTful API 56 Endpoints • Frontend SPA 24 pages
17. 2. Backend Implementation
18. Well known JSON serializers • jbuilder • active_model_serializers • ••• • fast_jsonapi
19. fast_jsonapi https://medium.com/netflix-techblog/fast-json-api-serialization-with-ruby-on-rails-7c06578ad17f • A lightning fast JSON:API serializer for Ruby Objects • Around 25 times faster than AMS • is used in several APIs at Netflix
20. JSON:API • https://jsonapi.org
21. Why fast_jsonapi was developed? • Netflix ◦ always use JSON:API • AMS is too flexible ◦ AMS is designed to serialize JSON in several different formats, not just JSON:API
22. Serializer class BookSerializer include FastJsonapi::ObjectSerializer attributes :id, :title, :description, :created_at, :updated_at end
23. { "data":[ { "id":"1", "type":"book", "attributes":{ "id":1, "title":"AAA", "description":"Test", "created_at":"2019-06-23T13:28:32.550Z", "updated_at":"2019-06-23T13:28:32.573Z" } } ] }
24. 3. API schema
26. OpenAPI Specification • https://swagger.io/specification/v2/ • fka “Swagger” • help you design, build, document and consume REST APIs
27. TL;DR; API • Resource • Request • Response • Method • URL • Params RSpec Schema /app/books
28. • https://github.com/fotinakis/swagger-blocks • Works with all Ruby web frameworks • 100% support for all features of the Swagger 2.0 spec • Can use Ruby DSL to write OpenAPI schema swagger-blocks
29. # app/models/concerns/swagger/book_schema.rb module Swagger::BookSchema extend ActiveSupport::Concern include Swagger::Blocks included do swagger_schema :Book, required: [:title, :description, :image_url] do property :id, type: :integer property :title, type: :string property :description, type: :string property :created_at, type: :string property :updated_at, type: :string end end end
30. Wrote Extension for JSON:API • https://github.com/tanakaworld/swagger-blocks-fastjson-api/blob/master/config /initializers/swagger_blocks.rb
31. 4. Generator
32. Generate API Client API Client Schema
33. openapi-generator • https://github.com/OpenAPITools/openapi-generator • It allows generation from OpenAPI Spec ◦ API client libraries ◦ Server stubs ◦ Documentation ◦ Configuration
35. Install npm i -D @openapitools/openapi-generator-cli
36. Generate openapi-generator generate \ -i ./tmp/swagger.json \ -g typescript-axios \ -o ./app/javascript/src/gen/ --additional-properties="modelPropertyNaming=snake_ case" https://github.com/OpenAPITools/openapi-generator/blob/master/docs/generators/typescript-axios.md
37. export interface Book { id?: number; title: string; description: string; image_url: string; created_at?: string; updated_at?: string; } export interface BookResponse { id: string; type: string; attributes: Book; } export interface CreateBookRequest { title?: string; description?: string; image?: object; }
38. export class SampleAppApi extends BaseAPI { public createBook(title?: string, description?: string, image?: object, options?: any) { ••• } public deleteBook(id: number, options?: any) { ••• } public getBook(id: number, options?: any) { ••• } public getBooks(options?: any) { ••• } public updateBook(id: number, title?: string, description?: string, image?: object, options?: any) { ••• } }
39. $ npm run api:gen
40. 5. Frontend Implementation
41. Overview • Render SPA bundle via Rails View • with Turbolinks • SPA ◦ CSR ◦ Vue ◦ Vue Router ◦ Vuex API Client /app/books SPA
42. Rails Routing Rails.application.routes.draw do // path for SPA // app/books // app/books/new // app/books/1 // app/books/1/edit // ••• match 'app/(*spa_path)' => 'spa_app#index', via: :get end
43. Response Empty DOM with ‘data-vue’ // app/views/spa_app/index.html.erb <div data-vue="spa-app"> <router-view/> </div>
44. How it works with Turbolinks • Make Vue instance on 'turbolinks:load' • Destroy Vue instance on 'turbolinks:visit' • ref: https://qiita.com/midnightSuyama/items/efc5441a577f3d3abe74
45. Make/Destroy Vue instances document.addEventListener('turbolinks:load', () => { const templates = document.querySelectorAll('[data-vue]'); templates.forEach((el: HTMLElement) => { const option = options[el.dataset.vue] const vm = new Vue(Object.assign(option, { el })); vms.push(vm); }); }); document.addEventListener('turbolinks:visit', () => { for (let vm of vms) { vm.$destroy(); } vms = []; });
46. Use require.context of webpack // app/javascript/packs/vue-turbolinks.ts const options = {}; const requireContext = require.context('../options', false, /\.ts$/); requireContext.keys().forEach(key => { const name = key .split('/') .pop() .split('.') .shift(); options[name] = requireContext(key).default; });
47. Option // app/javascript/options/spa-app.ts import router from '@spa-app/router'; import store from '@spa-app/store'; export default { router, store };
48. Vuex Store Implementation • Use generated API Client in Store • FSA = Flux Standard Action ◦ https://github.com/redux-utilities/flux-standard-action ◦ https://github.com/sue71/vuex-typescript-fsa
49. Vuex + FSA store.dispatch('increment') // or store.dispatch({ type: 'increment', meta: null, error: null, payload: { ••• } }) https://vuex.vuejs.org/guide/actions.html#dispatching-actions
50. import { actionCreatorFactory } from 'vuex-typescript-fsa/lib'; import { UpdateBookRequest } from '@gen'; export const namespace = 'books'; const actionCreator = actionCreatorFactory(namespace); export const GetBook = actionCreator<{ id: number; }>( 'GET_BOOK' ); Define FSAs
51. Dispatch import * as BooksModule from '@spa-app/store/modules/books'; BooksModule.GetBook.namespaced({ id: 1 });
52. Actions / Mutations / Getters export const module: Module<State, RootState> = { namespaced: true, actions: combineAction( action(GetBook, async function(context, action) { const { data } = await this.sampleAppAPI .getBook(action.payload.id); context.commit(BookReceived(data.data)); }) ) };
53. Tips of OpenAPI Generator
54. Error: URLSearchParams is not defined
55. Error: URLSearchParams is not defined import * as url from 'url'; // Error const params = url.URLSearchParams(•••);
56. Workaround • https://github.com/swagger-api/swagger-codegen/issues/6403 • I don’t want to change generated code
57. Override ‘url’ package (Monkey patch) // webpacker config resolve: { alias: { '@': path.resolve(__dirname, '../../app/javascript/src'), urlOriginal: path.resolve(__dirname, '../../node_modules/url'), url: path.resolve(__dirname, './extensions/url') } } // config/webpack/extensions/url.js import { parse, resolve, resolveObject, format, Url } from 'urlOriginal'; import URLSearchParams from '@ungap/url-search-params'; export { parse, resolve, resolveObject, format, Url, URLSearchParams };
58. multipart/form-data
59. was fixed • https://github.com/OpenAPITools/op enapi-generator/commit/550774a6e2 e597d5f4460ec6f8a80951f3ada37e
60. Heavy Base64 Image • Easy to upload image as Base64 String in Rails ◦ https://github.com/y9v/carrierwave-base64 ◦ Able to send image via ‘application/json’ ◦ size tooooooo big • Use multipart/form-data instead
61. Attribute Overlapping
62. e.g. ‘url’ // ‍♂ This overlaps with import * url from 'url' if (url !== undefined) { localVarFormParams.append('url', url as any); } // ‍♂ rename to '_url' if (_url !== undefined) { localVarFormParams.append('_url', _url as any); } // convert in Rails params.tap {|p| p[:url] = p[:_url]}.permit( :title, :description, :url, :image )
63. fixed now • [TS][Axios] To fix conflict params name 'url' ◦ https://github.com/OpenAPITools/openapi-generator/pull/2921
64. Try BETA Version
65. Put .jar into node_modules/ // @openapitools/openapi-generator-cli/bin/openapi-generator const binPath = resolve(__dirname, 'openapi-generator.jar'); const JAVA_OPTS = process.env['JAVA_OPTS'] || ''; let command = `java ${JAVA_OPTS} -jar "${binPath}"`;
66. Generator is changed every day. • You can download and use snapshot • https://oss.sonatype.org/content/repositories/ snapshots/org/openapitools
67. Summary
68. Overview Schema OpenAPI Spec API Client SPA
69. Pro/Con • Pro ◦ Able to detect unexpected changes ◦ Reduce change by hand ◦ Able to reuse API Client for another platform ◦ Automatically detect diff • Con ◦ Took time to setup ◦ Too many things to do ◦ Too much for single person project
70. Sample • https://github.com/tanakaworld/swagger-blocks-fastjson-api
71. Thanks
72. Appendix
73. OpenAPI + JSON:API is Bad approach? • https://apisyouwonthate.com/blog/json-api-openapi-and-json-schema-working-i n-harmony • “At WeWork, we use JSON Schema to describe the data models, OpenAPI to describe everything else, then the message being described is usually JSON API.”
74. ※ Fast JSON API is not a replacement for AMS • AMS does many things and is very flexible ◦ This is why fast_jsonapi is more faster • Netflix still use AMS for non JSON:API serialization and deserialization • ref: Performance methodology