Automatic Code Generation for SPA

Transcript

1. 1.

   Automatic Code Generation for SPA Mercari x Merpay Frontend Tech

   Talk vol.2 2019/07/03 @_tanakaworld
2. 2.

   About Me • Twitter: @_tanakaworld • GitHub: tanakaworld • Merpay

   ◦ Software Engineer (Frontend) ◦ CSTool / MSTool
3. 3.

   1. What is Code Generation? 2. Tell about the Real

   World Rearchtecture project 3. Tips of Generator Agenda
4. 4.

   What is Code Generation?

5. 5.

   Automatically generate SDK from schema Spec Source Code SDK Schema

   Source Code SDK Generator
6. 6.

   Schema Driven Development • Make Development flow more efficiently •

   Automatic ◦ Source Code Generation ◦ Document Generation ◦ Testing
7. 7.

   Microservices in Merpay Protocol Buffers • gRPC • Write Schema

   by Protocol Buffers • Generate Client and Server
8. 8.

   Protocol Buffers service SampleAPI { rpc GetUser(GetUsersRequest) returns (GetUsersResponse) {

   option (google.api.http) = { get : "/users" body: "*" }; } } message GetUsersRequest {} message GetUsersResponse { repeated User user = 1; } .proto
9. 9.

   Code Generation in Merpay Frontend Proxy HTTP gRPC

10. 10.

    .proto to TypeScript export namespace SampleAPI { export class GetUser

    implements APIRequest<GetUsersRequest, GetUsersResponse> { _response?: GetUsersResponse; path = "/users"; method: HTTPMethod = "get"; constructor(public parameter: GetUsersRequest) {} } } export interface GetUsersRequest {} export interface GetUsersResponse { user?: User[]; }
11. 11.

    REAL WORLD Rearchitecture Project

12. 12.

    REAL WORLD Rearchitecture Project 1. Overview 2. Backend Implementation 3.

    API schema 4. Generator 5. Frontend Implementation
13. 13.

    1. Overview

14. 14.

    https://proff.io/

15. 15.

    Traditional Rails App • Render every time in Server •

    Bad UX
16. 16.

    Rearchitecture • Re-implement 70% features • Backend RESTful API 56

    Endpoints • Frontend SPA 24 pages
17. 17.

    2. Backend Implementation

18. 18.

    Well known JSON serializers • jbuilder • active_model_serializers • •••

    • fast_jsonapi
19. 19.

    fast_jsonapi https://medium.com/netflix-techblog/fast-json-api-serialization-with-ruby-on-rails-7c06578ad17f • A lightning fast JSON:API serializer for Ruby

    Objects • Around 25 times faster than AMS • is used in several APIs at Netflix
20. 20.

    JSON:API • https://jsonapi.org

21. 21.

    Why fast_jsonapi was developed? • Netflix ◦ always use JSON:API

    • AMS is too flexible ◦ AMS is designed to serialize JSON in several different formats, not just JSON:API
22. 22.

    Serializer class BookSerializer include FastJsonapi::ObjectSerializer attributes :id, :title, :description, :created_at,

    :updated_at end
23. 23.

    { "data":[ { "id":"1", "type":"book", "attributes":{ "id":1, "title":"AAA", "description":"Test", "created_at":"2019-06-23T13:28:32.550Z",

    "updated_at":"2019-06-23T13:28:32.573Z" } } ] }
24. 24.

    3. API schema

25. 25.

    None
26. 26.

    OpenAPI Specification • https://swagger.io/specification/v2/ • fka “Swagger” • help you

    design, build, document and consume REST APIs
27. 27.

    TL;DR; API • Resource • Request • Response • Method

    • URL • Params RSpec Schema /app/books
28. 28.

    • https://github.com/fotinakis/swagger-blocks • Works with all Ruby web frameworks •

    100% support for all features of the Swagger 2.0 spec • Can use Ruby DSL to write OpenAPI schema swagger-blocks
29. 29.

    # app/models/concerns/swagger/book_schema.rb module Swagger::BookSchema extend ActiveSupport::Concern include Swagger::Blocks included do

    swagger_schema :Book, required: [:title, :description, :image_url] do property :id, type: :integer property :title, type: :string property :description, type: :string property :created_at, type: :string property :updated_at, type: :string end end end
30. 30.

    Wrote Extension for JSON:API • https://github.com/tanakaworld/swagger-blocks-fastjson-api/blob/master/config /initializers/swagger_blocks.rb

31. 31.

    4. Generator

32. 32.

    Generate API Client API Client Schema

33. 33.

    openapi-generator • https://github.com/OpenAPITools/openapi-generator • It allows generation from OpenAPI Spec

    ◦ API client libraries ◦ Server stubs ◦ Documentation ◦ Configuration
34. 34.

    None
35. 35.

    Install npm i -D @openapitools/openapi-generator-cli

36. 36.

    Generate openapi-generator generate \ -i ./tmp/swagger.json \ -g typescript-axios \

    -o ./app/javascript/src/gen/ --additional-properties="modelPropertyNaming=snake_ case" https://github.com/OpenAPITools/openapi-generator/blob/master/docs/generators/typescript-axios.md
37. 37.

    export interface Book { id?: number; title: string; description: string;

    image_url: string; created_at?: string; updated_at?: string; } export interface BookResponse { id: string; type: string; attributes: Book; } export interface CreateBookRequest { title?: string; description?: string; image?: object; }
38. 38.

    export class SampleAppApi extends BaseAPI { public createBook(title?: string, description?:

    string, image?: object, options?: any) { ••• } public deleteBook(id: number, options?: any) { ••• } public getBook(id: number, options?: any) { ••• } public getBooks(options?: any) { ••• } public updateBook(id: number, title?: string, description?: string, image?: object, options?: any) { ••• } }
39. 39.

    $ npm run api:gen

40. 40.

    5. Frontend Implementation

41. 41.

    Overview • Render SPA bundle via Rails View • with

    Turbolinks • SPA ◦ CSR ◦ Vue ◦ Vue Router ◦ Vuex API Client /app/books SPA
42. 42.

    Rails Routing Rails.application.routes.draw do // path for SPA // app/books

    // app/books/new // app/books/1 // app/books/1/edit // ••• match 'app/(*spa_path)' => 'spa_app#index', via: :get end
43. 43.

    Response Empty DOM with ‘data-vue’ // app/views/spa_app/index.html.erb <div data-vue="spa-app"> <router-view/>

    </div>
44. 44.

    How it works with Turbolinks • Make Vue instance on

    'turbolinks:load' • Destroy Vue instance on 'turbolinks:visit' • ref: https://qiita.com/midnightSuyama/items/efc5441a577f3d3abe74
45. 45.

    Make/Destroy Vue instances document.addEventListener('turbolinks:load', () => { const templates =

    document.querySelectorAll('[data-vue]'); templates.forEach((el: HTMLElement) => { const option = options[el.dataset.vue] const vm = new Vue(Object.assign(option, { el })); vms.push(vm); }); }); document.addEventListener('turbolinks:visit', () => { for (let vm of vms) { vm.$destroy(); } vms = []; });
46. 46.

    Use require.context of webpack // app/javascript/packs/vue-turbolinks.ts const options = {};

    const requireContext = require.context('../options', false, /\.ts$/); requireContext.keys().forEach(key => { const name = key .split('/') .pop() .split('.') .shift(); options[name] = requireContext(key).default; });
47. 47.

    Option // app/javascript/options/spa-app.ts import router from '@spa-app/router'; import store from

    '@spa-app/store'; export default { router, store };
48. 48.

    Vuex Store Implementation • Use generated API Client in Store

    • FSA = Flux Standard Action ◦ https://github.com/redux-utilities/flux-standard-action ◦ https://github.com/sue71/vuex-typescript-fsa
49. 49.

    Vuex + FSA store.dispatch('increment') // or store.dispatch({ type: 'increment', meta:

    null, error: null, payload: { ••• } }) https://vuex.vuejs.org/guide/actions.html#dispatching-actions
50. 50.

    import { actionCreatorFactory } from 'vuex-typescript-fsa/lib'; import { UpdateBookRequest }

    from '@gen'; export const namespace = 'books'; const actionCreator = actionCreatorFactory(namespace); export const GetBook = actionCreator<{ id: number; }>( 'GET_BOOK' ); Define FSAs
51. 51.

    Dispatch import * as BooksModule from '@spa-app/store/modules/books'; BooksModule.GetBook.namespaced({ id: 1

    });
52. 52.

    Actions / Mutations / Getters export const module: Module<State, RootState>

    = { namespaced: true, actions: combineAction( action(GetBook, async function(context, action) { const { data } = await this.sampleAppAPI .getBook(action.payload.id); context.commit(BookReceived(data.data)); }) ) };
53. 53.

    Tips of OpenAPI Generator

54. 54.

    Error: URLSearchParams is not defined

55. 55.

    Error: URLSearchParams is not defined import * as url from

    'url'; // Error const params = url.URLSearchParams(•••);
56. 56.

    Workaround • https://github.com/swagger-api/swagger-codegen/issues/6403 • I don’t want to change generated

    code
57. 57.

    Override ‘url’ package (Monkey patch) // webpacker config resolve: {

    alias: { '@': path.resolve(__dirname, '../../app/javascript/src'), urlOriginal: path.resolve(__dirname, '../../node_modules/url'), url: path.resolve(__dirname, './extensions/url') } } // config/webpack/extensions/url.js import { parse, resolve, resolveObject, format, Url } from 'urlOriginal'; import URLSearchParams from '@ungap/url-search-params'; export { parse, resolve, resolveObject, format, Url, URLSearchParams };
58. 58.

    multipart/form-data

59. 59.

    was fixed • https://github.com/OpenAPITools/op enapi-generator/commit/550774a6e2 e597d5f4460ec6f8a80951f3ada37e

60. 60.

    Heavy Base64 Image • Easy to upload image as Base64

    String in Rails ◦ https://github.com/y9v/carrierwave-base64 ◦ Able to send image via ‘application/json’ ◦ size tooooooo big • Use multipart/form-data instead
61. 61.

    Attribute Overlapping

62. 62.

    e.g. ‘url’ // ‍♂ This overlaps with import * url

    from 'url' if (url !== undefined) { localVarFormParams.append('url', url as any); } // ‍♂ rename to '_url' if (_url !== undefined) { localVarFormParams.append('_url', _url as any); } // convert in Rails params.tap {|p| p[:url] = p[:_url]}.permit( :title, :description, :url, :image )
63. 63.

    fixed now • [TS][Axios] To fix conflict params name 'url'

    ◦ https://github.com/OpenAPITools/openapi-generator/pull/2921
64. 64.

    Try BETA Version

65. 65.

    Put .jar into node_modules/ // @openapitools/openapi-generator-cli/bin/openapi-generator const binPath = resolve(__dirname,

    'openapi-generator.jar'); const JAVA_OPTS = process.env['JAVA_OPTS'] || ''; let command = `java ${JAVA_OPTS} -jar "${binPath}"`;
66. 66.

    Generator is changed every day. • You can download and

    use snapshot • https://oss.sonatype.org/content/repositories/ snapshots/org/openapitools
67. 67.

    Summary

68. 68.

    Overview Schema OpenAPI Spec API Client SPA

69. 69.

    Pro/Con • Pro ◦ Able to detect unexpected changes ◦

    Reduce change by hand ◦ Able to reuse API Client for another platform ◦ Automatically detect diff • Con ◦ Took time to setup ◦ Too many things to do ◦ Too much for single person project
70. 70.

    Sample • https://github.com/tanakaworld/swagger-blocks-fastjson-api

71. 71.

    Thanks

72. 72.

    Appendix

73. 73.

    OpenAPI + JSON:API is Bad approach? • https://apisyouwonthate.com/blog/json-api-openapi-and-json-schema-working-i n-harmony •

    “At WeWork, we use JSON Schema to describe the data models, OpenAPI to describe everything else, then the message being described is usually JSON API.”
74. 74.

    ※ Fast JSON API is not a replacement for AMS

    • AMS does many things and is very flexible ◦ This is why fast_jsonapi is more faster • Netflix still use AMS for non JSON:API serialization and deserialization • ref: Performance methodology