GraphQL Asia 2019 "Re-architecture of a decade-old app with BFF/GraphQL"

Slide 1

Slide 1 text

Re-architecture of A Decade Old App with BFF/GraphQL Ken (Kenju Wagatsuma) @GraphQL Asia 2019 April 13th, 2019

Slide 2

Slide 2 text

Hello, it’s Ken from Japan Kenju Wagatsuma ● #GraphQL #Go #TypeScript ● Group Lead of Marketing Service Development Team ● Twitter … @itiskj ● GitHub … @kenju

Slide 3

Slide 3 text

How old are you precisely, A Decade Old App ?

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Today’s Message

Slide 7

Slide 7 text

GraphQL is not ??? ,

Slide 8

Slide 8 text

GraphQL is not a silver bullet,

Slide 9

Slide 9 text

But, If you put GraphQL into the ??? ,

Slide 10

Slide 10 text

But, If you put GraphQL into the appropriate problems,

Slide 11

Slide 11 text

Then, you can achieve...

Slide 12

Slide 12 text

Then, you can achieve... Great Developer Experiences

Slide 13

Slide 13 text

And, you can also achieve...

Slide 14

Slide 14 text

And, you can also achieve... Rapid Development with High Quality

Slide 15

Slide 15 text

Agenda

Slide 16

Slide 16 text

6 Problems we had,

Slide 17

Slide 17 text

6 Solutions we achieved by GraphQL, 6 Problems we had,

Slide 18

Slide 18 text

6 Solutions we achieved by GraphQL, 6 Problems we had, 4 Tradeoffs we suffered from.

Slide 19

Slide 19 text

Business Model

Slide 20

Slide 20 text

Cookpad Inc. ● World largest recipe sharing site ○ cookpad.com ○ cookpad.com/in (for India) ● Mission: “Make everyday cooking fun!” ● Over 26 languages, 71 countries ● 94 Million Monthly Average Users

Slide 21

Slide 21 text

Cookpad is a Platform User Ad Clients Advertisement Premium Users

Slide 22

Slide 22 text

My Team Marketing Service Development Team develops... ● Ad Campaign Ordering Service ● Ad Delivery Service ● Logging (Streaming + Batching) ● Targeting Service ● Ad Creative Optimization Service ...

Slide 23

Slide 23 text

The high-level Marketing Technology flow Operation Team Data Warehouse Ad Creative Optimization Ad Campaign Ordering Service Ad Delivery Service Targeting Service Delivery Metadata Mobile Client Ad Creatives log

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

+ Targeting Options ● Default User Segments ○ e.g. “Healthy”, “Family”, “30s” ● Customized Segments ○ optimized user segments per ad clients ● Shopping Time Jack ○ 17:00 - 19:00 for peak time jacking ● Area Targeting

Slide 29

Slide 29 text

Linux OS Docker, Terraform, Jenkins DevOps Nginx, Fluentd, ELB Infra Ruby Rails TypeScript React.js GraphQL Apollo Ruby Rails Go AWS ElastiCache AWS DynamoDB Service Mesh Envoy Python AWS APIGateway AWS Lambda Tableau Application SQL AWS Redshift AWS Athena Ruby Python AWS Kinesis AWS DynamoDB AWS Lambda Batch Layer Streaming Layer Ad campaign ordering service Ad delivery service Logging Machine Learning

Slide 30

Slide 30 text

Re-architect Timeline

Slide 31

Slide 31 text

Re-architect Timeline October, 2017 … I joined the team November, 2017 … Step 1: Understanding by “Exploratory Refactoring”

Slide 32

Slide 32 text

January, 2018 … Step 0 “Exploratory Refactoring” to understand the codebase “Re-Engineering Legacy Software” by Chris Birchall ● Rename method/class/const ● Extract/Split/Merge method ● Add comment/documentation ● Inline/Move/Encapsulate, etc.

Slide 33

Slide 33 text

Re-architect Timeline October, 2017 … I joined the team November, 2017 … Step 1: Understanding by “Exploratory Refactoring” March, 2018 … Step 2: Muddy “Refactoring” + Prepare for the Big Bang!

Slide 34

Slide 34 text

Muddy “Refactoring” No Silver bullet. ● Goodbye CoffeeScript, Hello ES6 ● Introduce ESLint ● Introduce JSDoc (and gonna be TypeScript later) ● Introduce React.js, and reduce jQuery dependencies ● Introduce Testing with Jest ● Drop dead codes day by day….

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Our Goal Improve Developer Experiences and... ● Accelerate the growth of our advertisement system ● So that to make companies can continue investment in new services

Slide 37

Slide 37 text

6 Solutions we achieved by GraphQL, 6 Problems we had, 4 Tradeoffs we suffered from.

Slide 38

Slide 38 text

Use-cases & Resources are tightly coupled Problem #1

Slide 39

Slide 39 text

Legacy New BFR Backend for Resource Frontend Resource Resource Legacy Usecase New Usecase Legacy New

Slide 40

Slide 40 text

GraphQL as a BFF Solution #1

Slide 41

Slide 41 text

What’s BFF? ● Pattern: Backends For Frontends by Sam Newman, the author of “Building Microservices” ● Definition: “Single-purpose Edge Services for UIs and external parties” https://samnewman.io/patterns/architectural/bff/

Slide 42

Slide 42 text

Why BFF? - 3 Use Cases @yosuke_furukawa, Node.js collaborator ● Accelerate Development ○ for Frontend/Backend Developers ● Special Use-cases ○ (e.g. Server-Side Rendering for SEO, WebSocket for Real-time app) ● Re-architect Legacy System ○ this is exactly our cases

Slide 43

Slide 43 text

Legacy New GraphQL Schema BFR Backend for Resource BFF Backend for Frontend Frontend Resource Resource Legacy Usecase New Usecase

Slide 44

Slide 44 text

TOO MUCH Domain Knowledge is required Problem #2

Slide 45

Slide 45 text

New ad delivery service Legacy ad delivery service Data Warehouse Machine Learning services Redshift API Gateway Ad Campaign Ordering Service Tracking service DynamoDB batch job

Slide 46

Slide 46 text

GraphQL as a BFF Solution #2

Slide 47

Slide 47 text

New ad delivery service Legacy ad delivery service Data Warehouse Machine Learning services Redshift API Gateway Ad Campaign Ordering Service Tracking service DynamoDB batch job

Slide 48

Slide 48 text

Flexibility of backend is limited, due to its complexity Problem #3

Slide 49

Slide 49 text

New ad delivery service Legacy ad delivery service Data Warehouse Machine Learning services Redshift API Gateway Ad Campaign Ordering Service Tracking service DynamoDB batch job

Slide 50

Slide 50 text

GraphQL as a BFF Solution #3

Slide 51

Slide 51 text

Hybrid approach w/ DB & existing systems Encapsulation of... ● Own database ● The existing legacy system ● Third-party API https://www.howtographql.com/basics/3-big-picture/

Slide 52

Slide 52 text

No Frontend Specialist Problem #4

Slide 53

Slide 53 text

Problems #4 for example… ● CoffeeScript ● jQuery 1.x (major was 3.x then) ● NO JavaScript tests at all ● NO error tracking for the clientside ● NO Static Analysis tools ● Unused libraries in package.json ● A bunch of “TODO” and “FIXME”

Slide 54

Slide 54 text

GraphQL x Component Oriented Solution #4

Slide 55

Slide 55 text

what we have achieved Resiliency x Reusability Solution #4

Slide 56

Slide 56 text

Resiliency by SFD-Component Reusability by Atomic Design Solution #4

Slide 57

Slide 57 text

Style-Function-Data Component Data Function Style Component Bootstrap React Hooks or HoC GraphQL & Apollo Client React.js

Slide 58

Slide 58 text

Style-Function-Data Component is... Framework agnostic ● “Component” can be … ○ React, Vue, or even pure JS ● “Style” can be … ○ Styled Components, Bootstrap, CSSNext, Material Design, or anything ● “Function” can be … ○ pure JS, class, HoC, or anything ● “Data” can be … ○ hard-coded JSON, GraphQL query, or anything

Slide 59

Slide 59 text

Atomic Design http://bradfrost.com/blog/post/atomic-web-design/

Slide 60

Slide 60 text

Atomic Design Guideline Component Definition Example Atoms minimum components button / toast / label / panel Molecules handle single event (useState / useEffect / useContext) input / pagination / selector / popup Organisms handle multiple events / use GraphQL chart / form / list / table Templates wireframe level header / footer / body Pages per pages top / dashboard / settings / campaigns

Slide 61

Slide 61 text

Atomic Design can… ● boost the reusability of components ● pave the road for developers to architect class in a proper way

Slide 62

Slide 62 text

I love Dynamic Typing, but it’s not for a Decade Old App, is it? Problem #5

Slide 63

Slide 63 text

Problems #5 for example… ● Meta programming makes it hard to find dead codes ● Runtime Error often happens ● No Static Analysis leads to no coherency in the codebase

Slide 64

Slide 64 text

GraphQL x TypeScript Solution #5

Slide 65

Slide 65 text

Discover errors as soon as possible The later errors discovered, the more it takes to fix the errors. “Software Engineering Economics”, Barry W. Boehm, 1983 Error Discovered Time(t) Fixing cost

Slide 66

Slide 66 text

Ad Campaign Ordering Service Resolvers Plain Object / Data Entity Classes GraphQL Schema Clients Apollo 3rd-Party Library Clients React Adapter Classes graphql-code-generator TypeScript Service Classes

Slide 67

Slide 67 text

Ad Campaign Ordering Service Type-Safety Resolvers Plain Object / Data Entity Classes GraphQL Schema Clients Apollo 3rd-Party Library Clients React Adapter Classes graphql-code-generator TypeScript Service Classes Generate typings.d.ts

Slide 68

Slide 68 text

Gradual Typing Introduce TypeScript Compiler into your build pipelines Introduce 3rd Party typings with DefinitelyTyped Write your own typing definitions Set all strict option to true Enjoy type safety life! Introduction Early Stage Gradual Typing Final Stage Stable Stage

Slide 69

Slide 69 text

Is our “New Bleeding Edge GraphQL/React/TS App” gonna be “legacy” soon? Problem #6

Slide 70

Slide 70 text

GraphQL makes your API Versionless Solution #6

Slide 71

Slide 71 text

Client should not & must not know about model versioning in the backend

Slide 72

Slide 72 text

6 Solutions we achieved by GraphQL, 6 Problems we had, 4 Tradeoffs we suffered from.

Slide 73

Slide 73 text

Cache Strategy is in the early stage, compared to RESTful Tradeoff #1

Slide 74

Slide 74 text

We do not need it (yet) Tradeoff #1

Slide 75

Slide 75 text

Monitoring practices is still in the early stage Tradeoff #2

Slide 76

Slide 76 text

We bet on the GraphQL Ecosystems Tradeoff #2

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

jQuery & CoffeeScript is widely spread. So Why New Technologies? Tradeoff #3

Slide 79

Slide 79 text

Technology Selection takes many perspectives, not only members’ motivation but also career path. Tradeoff #3

Slide 80

Slide 80 text

3 key factors for Technology Selection Tech Company Team ● Pros & Cons of the technology ● Ecosystem ● Tooling ● Trend ● Community ● Maturity ● Skillset ● Career path ● Scale ● Motivation ● Mission ● Business success ● Technology stacks ● Culture ● Recruitment

Slide 81

Slide 81 text

Type-Safety v.s. Rapid Development Tradeoff #4

Slide 82

Slide 82 text

No content

Slide 83

Slide 83 text

Type Safety wins in the long term Tradeoff #4

Slide 84

Slide 84 text

6 Solutions we achieved by GraphQL, 6 Problems we had, 4 Tradeoffs we suffered from.

Slide 85

Slide 85 text

Thank you! https://www.cookpadteam.com

Slide 86

Slide 86 text

References Title URL Atomic Design http://bradfrost.com/blog/post/atomic-web-design/ Hybrid approach with DB & integration of existing system https://www.howtographql.com/basics/3-big-picture/ BFF https://samnewman.io/patterns/architectural/bff/ BFF 3 Use Cases https://speakerdeck.com/yosuke_furukawa/backends-for-frontends-aws-dev-day (ja) BFU / BFR https://lacolaco.hatenablog.com/entry/2018/07/13/001820 (ja) Micro-Frontends https://micro-frontends.org/ Apollo Engine https://www.apollographql.com/docs/engine/