Frontend Re-architecture of a decade-old Rails App

Keywords

- Refactoring
- BFF (Backend-For-Frontend)
- GraphQL
- React.js
- Apollo Client
- Snapshot Testing with Jest
- React Storybook
- ESLint
- Atomic Designs

Transcript

1. Frontend Re-architecture of a decade-old Rails App w/ BFF/GraphQL/React Cookpad Inc. Kenju Wagatsuma 2018/11/13 @ Rails Tokyo Meetup
2. Who? Kenju Wagatsuma • Backend Engineer (Ruby/Go/AWS) • Streaming Pipeline ❤ • Recruit HD ~ Bit Journey ~ Cookpad https://github.com/kenju https://twitter.com/itiskj
3. Find me on https://speakerdeck.com
4. Cookpad Ads
5. Cookpad is a Platform $PNQBOZ 6TFST
6. Category Page Hijack
7. Sponsored Tie-up Content
8. One-day Site Hijack
9. Targeting • Extreme Audience Targeting (EAT) • default user segments • “Professional-oriented”, “healthy”, “kids”, … etc. • Customized EAT • optimized user segments • Shopping Time Hijack • 17:00 - 19:00 • SOV (= Share of Voice) 100%
10. Architecture
11. Mobile client Client admin MySQL Operation Team batch cookpad MySQL mobile write Old
12. Mobile client Client delivery admin batch MySQL MySQL Operation Team batch cookpad MySQL mobile write write New
13. database.yml 
    EBUBCBTFZNM
    
    BENJO
    
    QSPEVDUJPO@BE
    
    
    
    QSPEVDUJPO@CBTF
    
    "E
    %FMJWFSZ
    %#
    
    QSPEVDUJPO@BE
    
    
    
    QSPEVDUJPO@CBTF
    
    .BJO
    DPPLQBE
    
    QSPEVDUJPO
    
    
    
    QSPEVDUJPO@CBTF
    
    %8)
    3FE4IJGU
    
    QSPEVDUJPO@EXI
    
    
    
    EXI@CBTF
    3 + 1 database config One for the admin app One for the Cookpad app One for the ad delivery app The last one for Redshift
14. How old are you, admin app? > git show $(git rev-list --max-parents=0 HEAD)
15. This is “Legacy” • First git commit is in 2008 • Don’t make fun of me. The first commit was “imported from SVN to Git” • The fortunate is… • The former team members did a lot of great refactoring • The codebase is not as large as the Cookpad application
16. batch MySQL Operation Team batch MySQL write write admin Campaign Product Creative Week Frame DeliveryPlan DeliveryPayload TargetingRule Site Goal AdFormat CampaignPeriod CreativeTarget Delivery Booking Old New
17. It’s SO HARD to maintain • Which model is which, old or for new one? • Incoherent Business Logic Implementation for Frontend • Controller, Services, ViewHelper, ViewModel… • Old libraries • jQuery 1.x (latest = 3.x) • Bootstrap 3.x (latest = 4.x) • Not used package.json dependencies (…!!??) • Migration on the way (e.g. CoffeeScript w/ ES6, many TODOs) • No JavaScript Unit Testing
18. Hello, BFF
19. BFF = Backend For Frontend • “Pattern: Backends For Frontends” by Sam Newman, the author of “Building Microservices” • Definition: “Single-purpose Edge Services for UIs and external parties”
20. From General to Frontend-Oriented 'JHVSF
    IUUQTTBNOFXNBOJPQBUUFSOTBSDIJUFDUVSBMC⒎
21. Why BFF: 3 Use Cases by @yosuke_furukawa, Node.js collaborator 1. Accelerate Development for Frontend/Backend Developers 2. Special Use-cases (e.g. Server-Side Rendering for SEO, WebSocket for Real-time app) 3. Rearchitect Legacy System
22. BFF Implementation
23. B meets G
24. Boy meets Girl?
25. Boy BFF meets Girl GraphQL
26. What is GraphQL? GraphQL … - is an open source query language created by Facebook - as an alternative to the common REST architecture - gives clients more control over what information is sent - is NOT opinionated about the network layer & payload format - which is often HTTP & JSON - is NOT opinionated about the application architecture - “GraphQL is only a query language”
27. Hybrid approach w/ DB & integration of existing system 'JHVSF
    IUUQTXXXIPXUPHSBQIRMDPNCBTJDTCJHQJDUVSF
28. batch MySQL Operation Team batch MySQL write write admin Campaign Product Creative Week Frame DeliveryPlan DeliveryPayload TargetingRule Site Goal AdFormat CampaignPeriod CreativeTarget Delivery Booking Old New
29. Operation Team admin Campaign Product Creative DeliveryPlan DeliveryPayload Site Goal batch write Week Frame TargetingRule Delivery batch write AdFormat CampaignPeriod CreativeTarget Booking Old New
30. Operation Team admin Campaign Product Creative DeliveryPlan DeliveryPayload Site Goal batch write AdFormat CampaignPeriod CreativeTarget Booking
31. GraphQL Implementation on Rails
32. Query query { campaign(id: 19000) { id status creatives { id campaignId } } } { "data": { "campaign": { "id": "19000", "status": 0, "creatives": [ { "id": "61031", "campaignId": 19000 } ] } } } Query Example Response in JSON
33. Schema type Campaign { creatives: [Creative!] id: ID! status: Int! } type Creative { campaignId: Int id: ID! text: String } module Types class BaseObject < GraphQL::Schema::Object end end class Types::CampaignType < Types::BaseObject field :id, ID, null: false field :status, Integer, null: false field :creatives, [Types::CreativeType], null: true end class Types::CreativeType < Types::BaseObject field :id, ID, null: false field :campaign_id, Integer, null: true field :text, String, null: true end schema.graphql schema in Ruby (graphql gem)
34. Query Implementation with “graphql” gem class Types::QueryType < Types::BaseObject description "The query root of this schema” field :campaign, Types::CampaignType, null: true do description "Campaign" argument :id, ID, required: true end def campaign(id:) Campaign.find(id) end end query_type.rb
35. Setup class MySchem < GraphQL::Schema query(Types::QueryType) # NOTE: # add mutation, middle-wares, # complexity and other config here end class GraphqlController < ApplicationController def execute result = MySchema.execute( params[:query], variables: params[:variables], context: { login_user: current_user, }, operation_name: params[:operationName]) render json: result end end my_schema.rb graphql_controller.rb
36. Our GraphQL Design Principles • ✅ GraphQL server should be as thin as possible • write minimum tests for GraphQL • “Graph is a thin layer” by Lee Byron, the co- creator of GraphQL & ex-Facebook engineer • ✅ Implement gradually • do NOT write all schemas at once • write schemas ONLY needed for client IUUQTXXXHSBQIRMDPNBSUJDMFTZFBSTPGHSBQIRMMFFCZSPO
37. GraphQL Tools Why Tool Interactive Debugging graphql/graphiql rmosolgo/graphiql-rails Generate Documentation gjtorikian/graphql-docs Visualize Schema APIs-guru/graphql-voyager
38. graphql/graphiql & rmosolgo/graphiql-rails
39. gjtorikian/graphql-docs
40. APIs-guru/graphql-voyager
41. “Thinking in Graphs” 'JHVSF
    IUUQTHSBQIRMPSHMFBSOUIJOLJOHJOHSBQIT
42. JavaScript on Rails
43. JavaScript on Our Admin Rails App Before After No JavaScript Unit Test Snapshot Testing w/ Jest CoffeeScript & some ES6 Pure ES6 (ES6-Oriented ESLint Rules) jQuery React.js & Apollo Client (still some jQuery) - React Storybook
44. Snapshot Testing w/ Jest import React from 'react'; import Loading from '../../../app/javascript/ components/atoms/Loading'; import renderer from 'react-test-renderer'; test('Loading', () => { const component = renderer.create( <Loading></Loading> ); const tree = component.toJSON(); expect(tree).toMatchSnapshot(); }); // Jest Snapshot v1, https://goo.gl/fbAQLP exports[`Loading 1`] = ` <div> ... Loading <img alt="indicator" src="/images/lib/indicator.gif" /> </div> `; __tests__/components/Loading.test.js __snapshots__/Loading.test.js.snap
45. Snapshot Testing w/ Jest 'JHVSF
    IUUQTKFTUKTJPEPDTFOTOBQTIPUUFTUJOH
46. ES6-oriented ESLint Rules eslint/recommended + plugin:react/recommended + Custom ES6-oriented rules
47. React Storybook
48. Design Guidelines
49. Our Design Guidelines (Beta) • Improve DX (Developer Experience) • “Developing the internal admin app” should be a valuable experience for their career • Avoid the Bleeding Edge • no dedicated frontend engineers -> maintenance costs • Lay the rails for junior & future developers • “Once you see the code, you will know how you write” • follow well-known best practices
50. Design Guidelines in Practice • Bootstrap • well-known technology, easy to develop • Systematic Styling Architecture • styled-components? Still under consideration • “Atomic Design” • to achieve reusability
51. Atomic Design
52. Our Atomic Design Guideline (Beta) Component Definition Example Atoms minimum components button/input/toast Molecules handle single event composed of atoms toast/pager Organisms handle multiple events use GraphQL provider charts/form Templates wireframe level - Pages per #index or #show actions -
53. vs. Legacy
54. How much “Legacy”? • Original members already left the team • become friends with git blame / git log • broken URL to deprecated SaaS (e.g. Firebase) • Multiple Versioning • versioning for delivery, versioning for targeting, versioning for Web/iOS/Android SDKs • & they are independent from each other • No refactoring expert • 20 % Rule for almost 1 year
55. Is Legacy System “evil”? • Harder to maintain • “What is this module in lib/ ???” • Harder to add new features • “OMG, somehow feature spec failed” • Harder to explain architecture to newcomers • “What is the difference between ‘CampaignController’ & ‘CampaignsController’?”
56. Mindset of Refactoring • Being Respectful to the current codebase • this is why you are here and receive salary • Re-architect from “the ideal”, not “the codebase” • making decision from … current business requirements, member’s skill level, impact range & risk of changes • Don’t confuse means and ends • Why refactoring? Because to drive business growth • Type System/Static Analysis/Microservices/CI … they all means, not ends
57. How to “Fight” with Legacy First of all, “Understanding” the codebase by “Exploratory Refactoring” “Re-Engineering Legacy Software” by Chris Birchall 6OEFSTUBOEJOH 7JTVBMJ[F
    
    2VBMJUZ
    X
    5FTU 3FGBDUPS
58. How to “Fight” with Legacy Secondly, visualize quality with tests • Write regression tests to protect you weekend calls • Before refactoring drastically, write test first to “understand” the codebase more 6OEFSTUBOEJOH 7JTVBMJ[F
    
    2VBMJUZ
    X
    5FTUT 3FGBDUPS
59. How to “Fight” with Legacy 6OEFSTUBOEJOH 7JTVBMJ[F
    
    2VBMJUZ
    X
    5FTU 3FGBDUPS Finally, do refactor. No silver bullet. Rome was not built in a day. • Readability • Design Patterns • OOP • Encapsulation
60. Make “Right” Decision “When the future cost of doing nothing is the same as the current cost, postpone the decision.”
61. Resources for FAQ
62. Diagrams
63. Mobile client Client delivery admin batch admin MySQL MySQL Operation Team batch cookpad MySQL delivery cookpad mobile write write Old New
64. Pagination
65. “Relay Cursor Connections Specification” IUUQGBDFCPPLHJUIVCJPSFMBZHSBQIRMDPOOFDUJPOTIUN
66. Domain Models
67. Operation $BNQBJHO
    1FSJPE #PPLJOH 1MBDFNFOU $MJFOU $BNQBJHO 1SPEVDU 0SHBOJ[BUJPO $SFBUJWF 0SEFS
68. Targeting $SFBUJWF $SFBUJWF
    5BSHFU ,FZXPSE
    3VMF 4DIFEVMF
    3VMF "SFB
    3VMF "VEJFODF
    3VMF
69. Delivery - DeliveryPlan { "version": 1, "site_id": 3, "product_id": 537, "product_capacity": 1, "campaigns": [ { "id": 18585, "goal_type": "percentage", "creatives": [ { "id": 60057, "formats": [ "category_hijack_spw_with_image" ], "targets": [ { "keywords": [ "͘Ζ·Ί", "ΫϩϚϝ" ] } ] } ], "rate_numerator": 10000, "rate_denominator": 10000 },
70. Delivery - DeliveryPayload [ { "key": "cookpad-android-search-master", "creatives": [ [ { "id": 111, "template": "image", "click_url": "https://...", "media": { "mime_type": "image/jpeg", "original": "https://example.com/d6663ab44a0c48a1996debc690893493.jpg?1510564210", "width": 300, "height": 250 } } ] ] }
71. ຖ೔ͷྉཧΛָ͠Έʹ͢Δ Thank you!