Slide 1

Slide 1 text

Gradually migrating a 7-year-old app to GraphQL exAspArk Evgeny Li

Slide 2

Slide 2 text

My long journey to __ !

Slide 3

Slide 3 text

February 5, 2018

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

GraphQL Summit

Slide 6

Slide 6 text

Say Hi and grab some stickers!

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

My first GraphQL application – fight FOMO https://sociallimiter.herokuapp.com August 2016

Slide 9

Slide 9 text

• GraphQL Ruby gem, Sinatra web framework • React, Relay Classic, Material Design, PWA • Docker, Ansible, and other buzzwords • No DB: Twitter OAuth + User session + API calls • Simple GraphQL queries My first GraphQL application – still using it every day August 2016

Slide 10

Slide 10 text

• Ethereum smart contracts • PWA with Service Worker • Containers, machine learning • Chatbots, bluetooth mesh networking • GraphQL, Elixir Hack Day – monthly hackathons at Universe August 2016

Slide 11

Slide 11 text

I hacked our website during my first Hack Day _ https://giphy.com/gifs/hacker-fg4EQNbMABa6c #

Slide 12

Slide 12 text

1 server-side and 1 client-side developer spent a few weeks to research, bootstrap, and try to use GraphQL. After that they gave a presentation to one the most experienced Ruby developers I’ve ever worked with. First attempt to use it at my previous work August 2016 March 2017

Slide 13

Slide 13 text

• GraphQL spec • DSL • Types • Interfaces • Fields • Resolvers • N+1 queries • Relay Classic • Nodes • Connections • Edges • Cursors • Mutations • Error handling First attempt to use it at my previous work August 2016 March 2017

Slide 14

Slide 14 text

https://giphy.com/gifs/silicon-valley-xT0Gqr8S3sT5DQzHOM First attempt to use it at my previous work August 2016 March 2017

Slide 15

Slide 15 text

https://giphy.com/gifs/l3q2Ph0I1osaagoQE First attempt to use it at my previous work August 2016 March 2017

Slide 16

Slide 16 text

• We were overwhelmed by the information overload • There were no best practices for server-side with Ruby • Relay Classic was the most popular option for client-side. But too opinionated, one size doesn’t fit all • Nobody understood the most popular graphql-batch gem with promises to avoid N+1 queries, sorry Shopify :) • GraphQL required to write a lot of code and duplications, something similar to the existing serializers • Almost no comprehensive examples and documentation First attempt to use it at my previous work August 2016 March 2017

Slide 17

Slide 17 text

Gradual migration to GraphQL at Universe OK, let’s try GraphQL… But this time not everything at once August 2016 March 2017 July 2017

Slide 18

Slide 18 text

Gradual migration to GraphQL at Universe • Agile – able to move quickly and easily • Agile software development – advocates adaptive planning, evolutionary development, early delivery, and continuous improvement, and it encourages rapid and flexible response to change August 2016 March 2017 July 2017

Slide 19

Slide 19 text

Gradual migration to GraphQL at Universe https://www.youtube.com/watch?v=Y7XW-mewUm8

Slide 20

Slide 20 text

But why? https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779

Slide 21

Slide 21 text

Strong client contracts https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • Strong guarantees to clients about the data structures of our server responses. E.g. enums – a particular set of allowed values • Allows to validate client-side queries against the GraphQL schema during build time! • We can generate type declarations for our client apps (e.g. Flow types for JavaScript with apollo-codegen)

Slide 22

Slide 22 text

Strong client contracts

Slide 23

Slide 23 text

Domain layer abstraction https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • A way to abstractly layer new interfaces which make sense • Our existing data structures can stay the same, e.g. ActiveRecord models • We can maintain and keep our existing RESTful APIs indefinitely

Slide 24

Slide 24 text

We love DDD https://twitter.com/l_pan_/status/901270996308692992

Slide 25

Slide 25 text

Typed request / response interfaces https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • GraphQL specification, no need to think about JSON payload structure, root key names (data and errors) • Response format matches client’s request

Slide 26

Slide 26 text

Expensive fields https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • Clients can ask for only the data they need in a single query – better performance, especially for mobile • Can control maximum query depth or maximum query complexity per field • Easies to instrument performance metrics per field

Slide 27

Slide 27 text

Documentation https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • Description field designated for use by implementers to describe the functionality of the field • We can automatically generate developer-friendly docs. For example, GraphiQL

Slide 28

Slide 28 text

Documentation https://developers.universe.com/page/graphql-explorer

Slide 29

Slide 29 text

• First GraphQL schema at /graphql/alpha • Query types for publicly available information • Field descriptions for documentation • Schema, integration, and unit tests GraphQL alpha version August 2016 March 2017 July 2017

Slide 30

Slide 30 text

There are only 2 hard problems in Computer Science. Naming things, cache invalidation and off-by-one errors. – Phil Haack

Slide 31

Slide 31 text

1. Naming One language that reflects our domain language UI / API Customers Development Team

Slide 32

Slide 32 text

2. Caching For now, we don’t use extra caching mechanisms for GraphQL: • Works fast enough • It’s quite complicated • We are trying to defer any decisions about it

Slide 33

Slide 33 text

A good architecture is the architecture that allows critical decisions to be deferred. Why? Longer you can defer, the more information you have to make them. – Rober C. Martin (aka Uncle Bob)

Slide 34

Slide 34 text

Possible solution: Apollo Cache Control { "data": ..., "errors": ..., "extensions": { "cacheControl": { "version": 1, "hints: [ { "path": [...], "maxAge": , "scope": }, ... ] } } } https://github.com/apollographql/apollo-cache-control

Slide 35

Slide 35 text

• We mostly use sequential pages (1, 2, 3, …) instead of infinite scrolls like Facebook does (so and Relay) • We don’t have real-time item count changes on those pages very often • Much simpler to implement and understand (connection, pageInfo, cursor, first / after arguments, edge, node) Limit / offset pagination instead of cursor-based https://dev-blog.apollodata.com/understanding-pagination-rest-graphql-and-relay-b10f835549e7

Slide 36

Slide 36 text

Field-level GraphQL authorization http://graphql.pro/

Slide 37

Slide 37 text

https://github.com/exAspArk/graphql-guard GraphQL-Guard

Slide 38

Slide 38 text

QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[!PostType] do argument :user_id, !types.ID resolve ->(obj, args, ctx) { Post.where(user_id: args[:user_id]) } end end Schema = GraphQL::Schema.define do query QueryType end query = %{ query MyPosts($user_id: ID) { posts(user_id: $user_id) { id } } } variables = { user_id: 1 } context = { current_user: current_user } Schema.execute(query, variables: variables, context: context) GraphQL-Guard https://github.com/exAspArk/graphql-guard

Slide 39

Slide 39 text

QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[!PostType] do argument :user_id, !types.ID resolve ->(obj, args, ctx) { Post.where(user_id: args[:user_id]) } end end Schema = GraphQL::Schema.define do query QueryType end query = %{ query MyPosts($user_id: ID) { posts(user_id: $user_id) { id } } } variables = { user_id: 1 } context = { current_user: current_user } Schema.execute(query, variables: variables, context: context) GraphQL-Guard # authorization logic can also be extracted outside schema definition guard ->(obj, args, ctx) { args[:user_id] == ctx[:current_user].id } https://github.com/exAspArk/graphql-guard

Slide 40

Slide 40 text

August 2016 March 2017 July 2017 • Publicly available GraphQL at /graphql/beta • More query types, interfaces, enums, connections • Maximum query depth, arg validations, tiny mutation • Viewer (aka current_user) • First contributions by people from other teams in the company. The barrier of entry lower for new devs, IMO GraphQL beta version August 2017

Slide 41

Slide 41 text

https://engineering.universe.com/batching-a-powerful-way-to-solve-n-1-queries-every-rubyist-should-know-24e20c6e7b94 How to kill N+1 queries?

Slide 42

Slide 42 text

https://github.com/exaspark/batch-loader Batch-Loader • Generic utility to avoid N+1 DB queries, HTTP requests, etc. • Batching is isolated and lazy, load data in batch where and when it's needed • Automatically caches previous queries (identity map) • Thread-safe • No need to share batching through variables or custom defined classes • No dependencies, no monkey-patches, no extra primitives such as Promises

Slide 43

Slide 43 text

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15370/diffs#f34dbf3de4996b6652de2cbafe00a9a79e516365_79_79 Batch-Loader • We’re using it for both RESTful API and GraphQL • GitLab uses it with their git data storage Gitaly

Slide 44

Slide 44 text

https://github.com/exaspark/batch-loader Batch-Loader QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[PostType] do resolve: ->(obj, args, ctx) { Post.all } end end PostType = GraphQL::ObjectType.define do name "Post" field :user, !UserType do resolve: ->(post, args, ctx) do # N + 1 post.user # queries end end end UserType = GraphQL::ObjectType.define do name "User" field :name, !types.String end

Slide 45

Slide 45 text

https://github.com/exaspark/batch-loader Batch-Loader QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[PostType] do resolve: ->(obj, args, ctx) { Post.all } end end PostType = GraphQL::ObjectType.define do name "Post" field :user, !UserType do resolve: ->(post, args, ctx) do BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end end end end UserType = GraphQL::ObjectType.define do name "User" field :name, !types.String end BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end

Slide 46

Slide 46 text

https://github.com/exaspark/batch-loader Batch-Loader QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[PostType] do resolve: ->(obj, args, ctx) { Post.all } end end PostType = GraphQL::ObjectType.define do name "Post" field :user, !UserType do resolve: ->(post, args, ctx) do BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end end end end UserType = GraphQL::ObjectType.define do name "User" field :name, !types.String end BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end post.user_id u.id

Slide 47

Slide 47 text

August 2016 March 2017 July 2017 First REST → GraphQL production client August 2017 + +

Slide 48

Slide 48 text

August 2016 March 2017 July 2017 August 2017 September 2017 More GraphQL on client-side

Slide 49

Slide 49 text

https://github.com/exaspark/graphql-errors Error handling

Slide 50

Slide 50 text

https://dev-blog.apollodata.com/graphql-schema-stitching-8af23354ac37 First beta testers _

Slide 51

Slide 51 text

https://www.okgrow.com/training One day GraphQL training in the office • JS ES6, ES7, ES8, etc. • React • Apollo v2 • GraphQL.js • Express (Node.js) • Mongo

Slide 52

Slide 52 text

August 2016 March 2017 July 2017 August 2017 September 2017 Fully powered by GraphQL page

Slide 53

Slide 53 text

Managing state on the client-side • We use Redux with Saga / NuclearJS with Immutable for REST API requests. For example, authorization. • We use Apollo Client for GraphQL requests.

Slide 54

Slide 54 text

We’re not sure that Apollo Client can solve all our problems https://dev-blog.apollodata.com/apollo-client-2-0-beyond-graphql-apis-888807b53afe

Slide 55

Slide 55 text

August 2016 March 2017 July 2017 More experiments August 2017 September 2017 October 2017

Slide 56

Slide 56 text

August 2016 March 2017 July 2017 A lot more! August 2017 September 2017 October 2017

Slide 57

Slide 57 text

Vodka – possible gradual migration to Elixir? • Elixir + Absinthe – accept GraphQL requests • Makes gRPC (protobuf, HTTP2) call to existing Rails app • Rails app is responsible for business logic

Slide 58

Slide 58 text

August 2016 March 2017 July 2017 More GraphQL August 2017 September 2017 October 2017 November 2017 • Second GraphQL schema for internal use • Monitoring

Slide 59

Slide 59 text

Better monitoring with Apollo Engine https://www.apollographql.com/engine

Slide 60

Slide 60 text

https://github.com/apollographql/apollo-tracing DIY

Slide 61

Slide 61 text

https://github.com/uniiverse/apollo-tracing-ruby Apollo Tracing Ruby

Slide 62

Slide 62 text

https://github.com/apollographql/optics-agent-ruby Deprecated Optics Agent Ruby

Slide 63

Slide 63 text

August 2016 March 2017 July 2017 First GraphQL page with mutation on production! _ August 2017 September 2017 October 2017 November 2017 December 2017

Slide 64

Slide 64 text

https://dev-blog.apollodata.com/designing-graphql-mutations-e09de826ed97 Designing GraphQL mutations

Slide 65

Slide 65 text

What’s next? • Apollo GraphQL 2.x • Caching • More mutations • Persistent queries • AWS AppSync • Subscriptions and live queries • More experiments!

Slide 66

Slide 66 text

client.mutate({ mutation: TodoCreateMutation, variables: { text }, update: (proxy, { data: { createTodo } }) => { // Read the data from our cache for this query. const data = proxy.readQuery({ query: TodoAppQuery }); // Add our todo from the mutation to the end. data.todos.push(createTodo); // Write our data back to the cache. proxy.writeQuery({ query: TodoAppQuery, data }); }, }); Apollo Client 2: direct cache access https://www.apollographql.com/docs/react/features/caching.html // Read the data from our cache for this query. const data = proxy.readQuery({ query: TodoAppQuery }); // Write our data back to the cache. proxy.writeQuery({ query: TodoAppQuery, data });

Slide 67

Slide 67 text

import gql from 'graphql-tag'; import { Query } from 'react-apollo'; const query = gql` query SomeQuery { foo { bar } } `; function MyComponent() { return ( {(result) => { if (result.loading) return ; if (result.error) return ; return

Hello {result.data.foo.bar}!

; }) ); } React Apollo 2.1: Query component instead of HOC https://dev-blog.apollodata.com/whats-next-for-react-apollo-4d41ba12c2cb

Slide 68

Slide 68 text

AWS AppSync: apps with real-time and offline capabilities https://aws.amazon.com/blogs/aws/introducing-amazon-appsync/

Slide 69

Slide 69 text

https://www.youtube.com/watch?v=PsjiL6Yl1ag Gradually adding GraphQL subscriptions

Slide 70

Slide 70 text

Takeaways Never try to solve all the problems at once – make them line up for you one-by-one. – Richard Sloma