Gradually migrating a 7-year-old app to GraphQL

The talk was given at GraphQL Meetup in Toronto, 2018

Transcript

1. Gradually migrating a 7-year-old app to GraphQL exAspArk Evgeny Li
2. My long journey to __ !
3. February 5, 2018
5. GraphQL Summit
6. Say Hi and grab some stickers!
8. My first GraphQL application – fight FOMO https://sociallimiter.herokuapp.com August 2016
9. • GraphQL Ruby gem, Sinatra web framework • React, Relay Classic, Material Design, PWA • Docker, Ansible, and other buzzwords • No DB: Twitter OAuth + User session + API calls • Simple GraphQL queries My first GraphQL application – still using it every day August 2016
10. • Ethereum smart contracts • PWA with Service Worker • Containers, machine learning • Chatbots, bluetooth mesh networking • GraphQL, Elixir Hack Day – monthly hackathons at Universe August 2016
11. I hacked our website during my first Hack Day _ https://giphy.com/gifs/hacker-fg4EQNbMABa6c #
12. 1 server-side and 1 client-side developer spent a few weeks to research, bootstrap, and try to use GraphQL. After that they gave a presentation to one the most experienced Ruby developers I’ve ever worked with. First attempt to use it at my previous work August 2016 March 2017
13. • GraphQL spec • DSL • Types • Interfaces • Fields • Resolvers • N+1 queries • Relay Classic • Nodes • Connections • Edges • Cursors • Mutations • Error handling First attempt to use it at my previous work August 2016 March 2017
14. https://giphy.com/gifs/silicon-valley-xT0Gqr8S3sT5DQzHOM First attempt to use it at my previous work August 2016 March 2017
15. https://giphy.com/gifs/l3q2Ph0I1osaagoQE First attempt to use it at my previous work August 2016 March 2017
16. • We were overwhelmed by the information overload • There were no best practices for server-side with Ruby • Relay Classic was the most popular option for client-side. But too opinionated, one size doesn’t fit all • Nobody understood the most popular graphql-batch gem with promises to avoid N+1 queries, sorry Shopify :) • GraphQL required to write a lot of code and duplications, something similar to the existing serializers • Almost no comprehensive examples and documentation First attempt to use it at my previous work August 2016 March 2017
17. Gradual migration to GraphQL at Universe OK, let’s try GraphQL… But this time not everything at once August 2016 March 2017 July 2017
18. Gradual migration to GraphQL at Universe • Agile – able to move quickly and easily • Agile software development – advocates adaptive planning, evolutionary development, early delivery, and continuous improvement, and it encourages rapid and flexible response to change August 2016 March 2017 July 2017
19. Gradual migration to GraphQL at Universe https://www.youtube.com/watch?v=Y7XW-mewUm8
20. But why? https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779
21. Strong client contracts https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • Strong guarantees to clients about the data structures of our server responses. E.g. enums – a particular set of allowed values • Allows to validate client-side queries against the GraphQL schema during build time! • We can generate type declarations for our client apps (e.g. Flow types for JavaScript with apollo-codegen)
22. Strong client contracts
23. Domain layer abstraction https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • A way to abstractly layer new interfaces which make sense • Our existing data structures can stay the same, e.g. ActiveRecord models • We can maintain and keep our existing RESTful APIs indefinitely
24. We love DDD https://twitter.com/l_pan_/status/901270996308692992
25. Typed request / response interfaces https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • GraphQL specification, no need to think about JSON payload structure, root key names (data and errors) • Response format matches client’s request
26. Expensive fields https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • Clients can ask for only the data they need in a single query – better performance, especially for mobile • Can control maximum query depth or maximum query complexity per field • Easies to instrument performance metrics per field
27. Documentation https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • Description field designated for use by implementers to describe the functionality of the field • We can automatically generate developer-friendly docs. For example, GraphiQL
28. Documentation https://developers.universe.com/page/graphql-explorer
29. • First GraphQL schema at /graphql/alpha • Query types for publicly available information • Field descriptions for documentation • Schema, integration, and unit tests GraphQL alpha version August 2016 March 2017 July 2017
30. There are only 2 hard problems in Computer Science. Naming things, cache invalidation and off-by-one errors. – Phil Haack
31. 1. Naming One language that reflects our domain language UI / API Customers Development Team
32. 2. Caching For now, we don’t use extra caching mechanisms for GraphQL: • Works fast enough • It’s quite complicated • We are trying to defer any decisions about it
33. A good architecture is the architecture that allows critical decisions to be deferred. Why? Longer you can defer, the more information you have to make them. – Rober C. Martin (aka Uncle Bob)
34. Possible solution: Apollo Cache Control { "data": ..., "errors": ..., "extensions": { "cacheControl": { "version": 1, "hints: [ { "path": [...], "maxAge": <seconds>, "scope": <PUBLIC or PRIVATE> }, ... ] } } } https://github.com/apollographql/apollo-cache-control
35. • We mostly use sequential pages (1, 2, 3, …) instead of infinite scrolls like Facebook does (so and Relay) • We don’t have real-time item count changes on those pages very often • Much simpler to implement and understand (connection, pageInfo, cursor, first / after arguments, edge, node) Limit / offset pagination instead of cursor-based https://dev-blog.apollodata.com/understanding-pagination-rest-graphql-and-relay-b10f835549e7
36. Field-level GraphQL authorization http://graphql.pro/
37. https://github.com/exAspArk/graphql-guard GraphQL-Guard
38. QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[!PostType] do argument :user_id, !types.ID resolve ->(obj, args, ctx) { Post.where(user_id: args[:user_id]) } end end Schema = GraphQL::Schema.define do query QueryType end query = %{ query MyPosts($user_id: ID) { posts(user_id: $user_id) { id } } } variables = { user_id: 1 } context = { current_user: current_user } Schema.execute(query, variables: variables, context: context) GraphQL-Guard https://github.com/exAspArk/graphql-guard
39. QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[!PostType] do argument :user_id, !types.ID resolve ->(obj, args, ctx) { Post.where(user_id: args[:user_id]) } end end Schema = GraphQL::Schema.define do query QueryType end query = %{ query MyPosts($user_id: ID) { posts(user_id: $user_id) { id } } } variables = { user_id: 1 } context = { current_user: current_user } Schema.execute(query, variables: variables, context: context) GraphQL-Guard # authorization logic can also be extracted outside schema definition guard ->(obj, args, ctx) { args[:user_id] == ctx[:current_user].id } https://github.com/exAspArk/graphql-guard
40. August 2016 March 2017 July 2017 • Publicly available GraphQL at /graphql/beta • More query types, interfaces, enums, connections • Maximum query depth, arg validations, tiny mutation • Viewer (aka current_user) • First contributions by people from other teams in the company. The barrier of entry lower for new devs, IMO GraphQL beta version August 2017
41. https://engineering.universe.com/batching-a-powerful-way-to-solve-n-1-queries-every-rubyist-should-know-24e20c6e7b94 How to kill N+1 queries?
42. https://github.com/exaspark/batch-loader Batch-Loader • Generic utility to avoid N+1 DB queries, HTTP requests, etc. • Batching is isolated and lazy, load data in batch where and when it's needed • Automatically caches previous queries (identity map) • Thread-safe • No need to share batching through variables or custom defined classes • No dependencies, no monkey-patches, no extra primitives such as Promises
43. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15370/diffs#f34dbf3de4996b6652de2cbafe00a9a79e516365_79_79 Batch-Loader • We’re using it for both RESTful API and GraphQL • GitLab uses it with their git data storage Gitaly
44. https://github.com/exaspark/batch-loader Batch-Loader QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[PostType] do resolve: ->(obj, args, ctx) { Post.all } end end PostType = GraphQL::ObjectType.define do name "Post" field :user, !UserType do resolve: ->(post, args, ctx) do # N + 1 post.user # queries end end end UserType = GraphQL::ObjectType.define do name "User" field :name, !types.String end
45. https://github.com/exaspark/batch-loader Batch-Loader QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[PostType] do resolve: ->(obj, args, ctx) { Post.all } end end PostType = GraphQL::ObjectType.define do name "Post" field :user, !UserType do resolve: ->(post, args, ctx) do BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end end end end UserType = GraphQL::ObjectType.define do name "User" field :name, !types.String end BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end
46. https://github.com/exaspark/batch-loader Batch-Loader QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[PostType] do resolve: ->(obj, args, ctx) { Post.all } end end PostType = GraphQL::ObjectType.define do name "Post" field :user, !UserType do resolve: ->(post, args, ctx) do BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end end end end UserType = GraphQL::ObjectType.define do name "User" field :name, !types.String end BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end post.user_id u.id
47. August 2016 March 2017 July 2017 First REST → GraphQL production client August 2017 + +
48. August 2016 March 2017 July 2017 August 2017 September 2017 More GraphQL on client-side
49. https://github.com/exaspark/graphql-errors Error handling
50. https://dev-blog.apollodata.com/graphql-schema-stitching-8af23354ac37 First beta testers _
51. https://www.okgrow.com/training One day GraphQL training in the office • JS ES6, ES7, ES8, etc. • React • Apollo v2 • GraphQL.js • Express (Node.js) • Mongo
52. August 2016 March 2017 July 2017 August 2017 September 2017 Fully powered by GraphQL page
53. Managing state on the client-side • We use Redux with Saga / NuclearJS with Immutable for REST API requests. For example, authorization. • We use Apollo Client for GraphQL requests.
54. We’re not sure that Apollo Client can solve all our problems https://dev-blog.apollodata.com/apollo-client-2-0-beyond-graphql-apis-888807b53afe
55. August 2016 March 2017 July 2017 More experiments August 2017 September 2017 October 2017
56. August 2016 March 2017 July 2017 A lot more! August 2017 September 2017 October 2017
57. Vodka – possible gradual migration to Elixir? • Elixir + Absinthe – accept GraphQL requests • Makes gRPC (protobuf, HTTP2) call to existing Rails app • Rails app is responsible for business logic
58. August 2016 March 2017 July 2017 More GraphQL August 2017 September 2017 October 2017 November 2017 • Second GraphQL schema for internal use • Monitoring
59. Better monitoring with Apollo Engine https://www.apollographql.com/engine
60. https://github.com/apollographql/apollo-tracing DIY
61. https://github.com/uniiverse/apollo-tracing-ruby Apollo Tracing Ruby
62. https://github.com/apollographql/optics-agent-ruby Deprecated Optics Agent Ruby
63. August 2016 March 2017 July 2017 First GraphQL page with mutation on production! _ August 2017 September 2017 October 2017 November 2017 December 2017
64. https://dev-blog.apollodata.com/designing-graphql-mutations-e09de826ed97 Designing GraphQL mutations
65. What’s next? • Apollo GraphQL 2.x • Caching • More mutations • Persistent queries • AWS AppSync • Subscriptions and live queries • More experiments!
66. client.mutate({ mutation: TodoCreateMutation, variables: { text }, update: (proxy, { data: { createTodo } }) => { // Read the data from our cache for this query. const data = proxy.readQuery({ query: TodoAppQuery }); // Add our todo from the mutation to the end. data.todos.push(createTodo); // Write our data back to the cache. proxy.writeQuery({ query: TodoAppQuery, data }); }, }); Apollo Client 2: direct cache access https://www.apollographql.com/docs/react/features/caching.html // Read the data from our cache for this query. const data = proxy.readQuery({ query: TodoAppQuery }); // Write our data back to the cache. proxy.writeQuery({ query: TodoAppQuery, data });
67. import gql from 'graphql-tag'; import { Query } from 'react-apollo'; const query = gql` query SomeQuery { foo { bar } } `; function MyComponent() { return ( <Query query={query}> {(result) => { if (result.loading) return <Loading />; if (result.error) return <Error error={error} />; return <h1>Hello {result.data.foo.bar}!</h1>; }) </Query> ); } React Apollo 2.1: Query component instead of HOC https://dev-blog.apollodata.com/whats-next-for-react-apollo-4d41ba12c2cb <Query query={query}> </Query>
68. AWS AppSync: apps with real-time and offline capabilities https://aws.amazon.com/blogs/aws/introducing-amazon-appsync/
69. https://www.youtube.com/watch?v=PsjiL6Yl1ag Gradually adding GraphQL subscriptions
70. Takeaways Never try to solve all the problems at once – make them line up for you one-by-one. – Richard Sloma