GraphQL Asia 2019 "Re-architecture of a decade-old app with BFF/GraphQL"

https://www.graphql-asia.org/speaker/kenju-wagatsuma

Transcript

1. Re-architecture of A Decade Old App with BFF/GraphQL Ken (Kenju Wagatsuma) @GraphQL Asia 2019 April 13th, 2019
2. Hello, it’s Ken from Japan Kenju Wagatsuma • #GraphQL #Go #TypeScript • Group Lead of Marketing Service Development Team • Twitter … @itiskj • GitHub … @kenju
3. How old are you precisely, A Decade Old App ?
6. Today’s Message
7. GraphQL is not ??? ,
8. GraphQL is not a silver bullet,
9. But, If you put GraphQL into the ??? ,
10. But, If you put GraphQL into the appropriate problems,
11. Then, you can achieve...
12. Then, you can achieve... Great Developer Experiences
13. And, you can also achieve...
14. And, you can also achieve... Rapid Development with High Quality
15. Agenda
16. 6 Problems we had,
17. 6 Solutions we achieved by GraphQL, 6 Problems we had,
18. 6 Solutions we achieved by GraphQL, 6 Problems we had, 4 Tradeoffs we suffered from.
19. Business Model
20. Cookpad Inc. • World largest recipe sharing site ◦ cookpad.com ◦ cookpad.com/in (for India) • Mission: “Make everyday cooking fun!” • Over 26 languages, 71 countries • 94 Million Monthly Average Users
21. Cookpad is a Platform User Ad Clients Advertisement Premium Users
22. My Team Marketing Service Development Team develops... • Ad Campaign Ordering Service • Ad Delivery Service • Logging (Streaming + Batching) • Targeting Service • Ad Creative Optimization Service ...
23. The high-level Marketing Technology flow Operation Team Data Warehouse Ad Creative Optimization Ad Campaign Ordering Service Ad Delivery Service Targeting Service Delivery Metadata Mobile Client Ad Creatives log
24. The high-level Marketing Technology flow Operation Team Data Warehouse Ad Creative Optimization Ad Campaign Ordering Service Ad Delivery Service Targeting Service Delivery Metadata Mobile Client Ad Creatives log
25. The high-level Marketing Technology flow Operation Team Data Warehouse Ad Creative Optimization Ad Campaign Ordering Service Ad Delivery Service Targeting Service Delivery Metadata Mobile Client Ad Creatives log
26. The high-level Marketing Technology flow Operation Team Data Warehouse Ad Creative Optimization Ad Campaign Ordering Service Ad Delivery Service Targeting Service Delivery Metadata Mobile Client Ad Creatives log
27. The high-level Marketing Technology flow Operation Team Data Warehouse Ad Creative Optimization Ad Campaign Ordering Service Ad Delivery Service Targeting Service Delivery Metadata Mobile Client Ad Creatives log
28. + Targeting Options • Default User Segments ◦ e.g. “Healthy”, “Family”, “30s” • Customized Segments ◦ optimized user segments per ad clients • Shopping Time Jack ◦ 17:00 - 19:00 for peak time jacking • Area Targeting
29. Linux OS Docker, Terraform, Jenkins DevOps Nginx, Fluentd, ELB Infra Ruby Rails TypeScript React.js GraphQL Apollo Ruby Rails Go AWS ElastiCache AWS DynamoDB Service Mesh Envoy Python AWS APIGateway AWS Lambda Tableau Application SQL AWS Redshift AWS Athena Ruby Python AWS Kinesis AWS DynamoDB AWS Lambda Batch Layer Streaming Layer Ad campaign ordering service Ad delivery service Logging Machine Learning
30. Re-architect Timeline
31. Re-architect Timeline October, 2017 … I joined the team November, 2017 … Step 1: Understanding by “Exploratory Refactoring”
32. January, 2018 … Step 0 “Exploratory Refactoring” to understand the codebase “Re-Engineering Legacy Software” by Chris Birchall • Rename method/class/const • Extract/Split/Merge method • Add comment/documentation • Inline/Move/Encapsulate, etc.
33. Re-architect Timeline October, 2017 … I joined the team November, 2017 … Step 1: Understanding by “Exploratory Refactoring” March, 2018 … Step 2: Muddy “Refactoring” + Prepare for the Big Bang!
34. Muddy “Refactoring” No Silver bullet. • Goodbye CoffeeScript, Hello ES6 • Introduce ESLint • Introduce JSDoc (and gonna be TypeScript later) • Introduce React.js, and reduce jQuery dependencies • Introduce Testing with Jest • Drop dead codes day by day….
35. Re-architect Timeline October, 2017 … I joined the team November, 2017 … Step 1: Understanding by “Exploratory Refactoring” March, 2018 … Step 2: Muddy “Refactoring” + Prepare for the Big Bang! October, 2018 … Step 3: Big Bang! Re-architecture with BFF/GraphQL
36. Our Goal Improve Developer Experiences and... • Accelerate the growth of our advertisement system • So that to make companies can continue investment in new services
37. 6 Solutions we achieved by GraphQL, 6 Problems we had, 4 Tradeoffs we suffered from.
38. Use-cases & Resources are tightly coupled Problem #1
39. Legacy New BFR Backend for Resource Frontend Resource Resource Legacy Usecase New Usecase Legacy New
40. GraphQL as a BFF Solution #1
41. What’s BFF? • Pattern: Backends For Frontends by Sam Newman, the author of “Building Microservices” • Definition: “Single-purpose Edge Services for UIs and external parties” https://samnewman.io/patterns/architectural/bff/
42. Why BFF? - 3 Use Cases @yosuke_furukawa, Node.js collaborator • Accelerate Development ◦ for Frontend/Backend Developers • Special Use-cases ◦ (e.g. Server-Side Rendering for SEO, WebSocket for Real-time app) • Re-architect Legacy System ◦ this is exactly our cases
43. Legacy New GraphQL Schema BFR Backend for Resource BFF Backend for Frontend Frontend Resource Resource Legacy Usecase New Usecase
44. TOO MUCH Domain Knowledge is required Problem #2
45. New ad delivery service Legacy ad delivery service Data Warehouse Machine Learning services Redshift API Gateway Ad Campaign Ordering Service Tracking service DynamoDB batch job
46. GraphQL as a BFF Solution #2
47. New ad delivery service Legacy ad delivery service Data Warehouse Machine Learning services Redshift API Gateway Ad Campaign Ordering Service Tracking service DynamoDB batch job
48. Flexibility of backend is limited, due to its complexity Problem #3
49. New ad delivery service Legacy ad delivery service Data Warehouse Machine Learning services Redshift API Gateway Ad Campaign Ordering Service Tracking service DynamoDB batch job
50. GraphQL as a BFF Solution #3
51. Hybrid approach w/ DB & existing systems Encapsulation of... • Own database • The existing legacy system • Third-party API https://www.howtographql.com/basics/3-big-picture/
52. No Frontend Specialist Problem #4
53. Problems #4 for example… • CoffeeScript • jQuery 1.x (major was 3.x then) • NO JavaScript tests at all • NO error tracking for the clientside • NO Static Analysis tools • Unused libraries in package.json • A bunch of “TODO” and “FIXME”
54. GraphQL x Component Oriented Solution #4
55. what we have achieved Resiliency x Reusability Solution #4
56. Resiliency by SFD-Component Reusability by Atomic Design Solution #4
57. Style-Function-Data Component Data Function Style Component Bootstrap React Hooks or HoC GraphQL & Apollo Client React.js
58. Style-Function-Data Component is... Framework agnostic • “Component” can be … ◦ React, Vue, or even pure JS • “Style” can be … ◦ Styled Components, Bootstrap, CSSNext, Material Design, or anything • “Function” can be … ◦ pure JS, class, HoC, or anything • “Data” can be … ◦ hard-coded JSON, GraphQL query, or anything
59. Atomic Design http://bradfrost.com/blog/post/atomic-web-design/
60. Atomic Design Guideline Component Definition Example Atoms minimum components button / toast / label / panel Molecules handle single event (useState / useEffect / useContext) input / pagination / selector / popup Organisms handle multiple events / use GraphQL chart / form / list / table Templates wireframe level header / footer / body Pages per pages top / dashboard / settings / campaigns
61. Atomic Design can… • boost the reusability of components • pave the road for developers to architect class in a proper way
62. I love Dynamic Typing, but it’s not for a Decade Old App, is it? Problem #5
63. Problems #5 for example… • Meta programming makes it hard to find dead codes • Runtime Error often happens • No Static Analysis leads to no coherency in the codebase
64. GraphQL x TypeScript Solution #5
65. Discover errors as soon as possible The later errors discovered, the more it takes to fix the errors. “Software Engineering Economics”, Barry W. Boehm, 1983 Error Discovered Time(t) Fixing cost
66. Ad Campaign Ordering Service Resolvers Plain Object / Data Entity Classes GraphQL Schema Clients Apollo 3rd-Party Library Clients React Adapter Classes graphql-code-generator TypeScript Service Classes
67. Ad Campaign Ordering Service Type-Safety Resolvers Plain Object / Data Entity Classes GraphQL Schema Clients Apollo 3rd-Party Library Clients React Adapter Classes graphql-code-generator TypeScript Service Classes Generate typings.d.ts
68. Gradual Typing Introduce TypeScript Compiler into your build pipelines Introduce 3rd Party typings with DefinitelyTyped Write your own typing definitions Set all strict option to true Enjoy type safety life! Introduction Early Stage Gradual Typing Final Stage Stable Stage
69. Is our “New Bleeding Edge GraphQL/React/TS App” gonna be “legacy” soon? Problem #6
70. GraphQL makes your API Versionless Solution #6
71. Client should not & must not know about model versioning in the backend
72. 6 Solutions we achieved by GraphQL, 6 Problems we had, 4 Tradeoffs we suffered from.
73. Cache Strategy is in the early stage, compared to RESTful Tradeoff #1
74. We do not need it (yet) Tradeoff #1
75. Monitoring practices is still in the early stage Tradeoff #2
76. We bet on the GraphQL Ecosystems Tradeoff #2
78. jQuery & CoffeeScript is widely spread. So Why New Technologies? Tradeoff #3
79. Technology Selection takes many perspectives, not only members’ motivation but also career path. Tradeoff #3
80. 3 key factors for Technology Selection Tech Company Team • Pros & Cons of the technology • Ecosystem • Tooling • Trend • Community • Maturity • Skillset • Career path • Scale • Motivation • Mission • Business success • Technology stacks • Culture • Recruitment
81. Type-Safety v.s. Rapid Development Tradeoff #4
83. Type Safety wins in the long term Tradeoff #4
84. 6 Solutions we achieved by GraphQL, 6 Problems we had, 4 Tradeoffs we suffered from.
85. Thank you! https://www.cookpadteam.com
86. References Title URL Atomic Design http://bradfrost.com/blog/post/atomic-web-design/ Hybrid approach with DB & integration of existing system https://www.howtographql.com/basics/3-big-picture/ BFF https://samnewman.io/patterns/architectural/bff/ BFF 3 Use Cases https://speakerdeck.com/yosuke_furukawa/backends-for-frontends-aws-dev-day (ja) BFU / BFR https://lacolaco.hatenablog.com/entry/2018/07/13/001820 (ja) Micro-Frontends https://micro-frontends.org/ Apollo Engine https://www.apollographql.com/docs/engine/