Save 37% off PRO during our Black Friday Sale! »

Gradually migrating a 7-year-old app to GraphQL

Cb09696b034cce3cc79cab80a4bba4a3?s=47 exAspArk
February 05, 2018

Gradually migrating a 7-year-old app to GraphQL

The talk was given at GraphQL Meetup in Toronto, 2018

Cb09696b034cce3cc79cab80a4bba4a3?s=128

exAspArk

February 05, 2018
Tweet

Transcript

  1. Gradually migrating a 7-year-old app to GraphQL exAspArk Evgeny Li

  2. My long journey to __ !

  3. February 5, 2018

  4. None
  5. GraphQL Summit

  6. Say Hi and grab some stickers!

  7. None
  8. My first GraphQL application – fight FOMO https://sociallimiter.herokuapp.com August 2016

  9. • GraphQL Ruby gem, Sinatra web framework • React, Relay

    Classic, Material Design, PWA • Docker, Ansible, and other buzzwords • No DB: Twitter OAuth + User session + API calls • Simple GraphQL queries My first GraphQL application – still using it every day August 2016
  10. • Ethereum smart contracts • PWA with Service Worker •

    Containers, machine learning • Chatbots, bluetooth mesh networking • GraphQL, Elixir Hack Day – monthly hackathons at Universe August 2016
  11. I hacked our website during my first Hack Day _

    https://giphy.com/gifs/hacker-fg4EQNbMABa6c #
  12. 1 server-side and 1 client-side developer spent a few weeks

    to research, bootstrap, and try to use GraphQL. After that they gave a presentation to one the most experienced Ruby developers I’ve ever worked with. First attempt to use it at my previous work August 2016 March 2017
  13. • GraphQL spec • DSL • Types • Interfaces •

    Fields • Resolvers • N+1 queries • Relay Classic • Nodes • Connections • Edges • Cursors • Mutations • Error handling First attempt to use it at my previous work August 2016 March 2017
  14. https://giphy.com/gifs/silicon-valley-xT0Gqr8S3sT5DQzHOM First attempt to use it at my previous work

    August 2016 March 2017
  15. https://giphy.com/gifs/l3q2Ph0I1osaagoQE First attempt to use it at my previous work

    August 2016 March 2017
  16. • We were overwhelmed by the information overload • There

    were no best practices for server-side with Ruby • Relay Classic was the most popular option for client-side. But too opinionated, one size doesn’t fit all • Nobody understood the most popular graphql-batch gem with promises to avoid N+1 queries, sorry Shopify :) • GraphQL required to write a lot of code and duplications, something similar to the existing serializers • Almost no comprehensive examples and documentation First attempt to use it at my previous work August 2016 March 2017
  17. Gradual migration to GraphQL at Universe OK, let’s try GraphQL…

    But this time not everything at once August 2016 March 2017 July 2017
  18. Gradual migration to GraphQL at Universe • Agile – able

    to move quickly and easily • Agile software development – advocates adaptive planning, evolutionary development, early delivery, and continuous improvement, and it encourages rapid and flexible response to change August 2016 March 2017 July 2017
  19. Gradual migration to GraphQL at Universe https://www.youtube.com/watch?v=Y7XW-mewUm8

  20. But why? https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779

  21. Strong client contracts https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • Strong guarantees to clients about

    the data structures of our server responses. E.g. enums – a particular set of allowed values • Allows to validate client-side queries against the GraphQL schema during build time! • We can generate type declarations for our client apps (e.g. Flow types for JavaScript with apollo-codegen)
  22. Strong client contracts

  23. Domain layer abstraction https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • A way to abstractly layer

    new interfaces which make sense • Our existing data structures can stay the same, e.g. ActiveRecord models • We can maintain and keep our existing RESTful APIs indefinitely
  24. We love DDD https://twitter.com/l_pan_/status/901270996308692992

  25. Typed request / response interfaces https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • GraphQL specification, no

    need to think about JSON payload structure, root key names (data and errors) • Response format matches client’s request
  26. Expensive fields https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • Clients can ask for only the

    data they need in a single query – better performance, especially for mobile • Can control maximum query depth or maximum query complexity per field • Easies to instrument performance metrics per field
  27. Documentation https://engineering.universe.com/why-were-betting-on-graphql-233ddf1a0779 • Description field designated for use by implementers

    to describe the functionality of the field • We can automatically generate developer-friendly docs. For example, GraphiQL
  28. Documentation https://developers.universe.com/page/graphql-explorer

  29. • First GraphQL schema at /graphql/alpha • Query types for

    publicly available information • Field descriptions for documentation • Schema, integration, and unit tests GraphQL alpha version August 2016 March 2017 July 2017
  30. There are only 2 hard problems in Computer Science. Naming

    things, cache invalidation and off-by-one errors. – Phil Haack
  31. 1. Naming One language that reflects our domain language UI

    / API Customers Development Team
  32. 2. Caching For now, we don’t use extra caching mechanisms

    for GraphQL: • Works fast enough • It’s quite complicated • We are trying to defer any decisions about it
  33. A good architecture is the architecture that allows critical decisions

    to be deferred. Why? Longer you can defer, the more information you have to make them. – Rober C. Martin (aka Uncle Bob)
  34. Possible solution: Apollo Cache Control { "data": ..., "errors": ...,

    "extensions": { "cacheControl": { "version": 1, "hints: [ { "path": [...], "maxAge": <seconds>, "scope": <PUBLIC or PRIVATE> }, ... ] } } } https://github.com/apollographql/apollo-cache-control
  35. • We mostly use sequential pages (1, 2, 3, …)

    instead of infinite scrolls like Facebook does (so and Relay) • We don’t have real-time item count changes on those pages very often • Much simpler to implement and understand (connection, pageInfo, cursor, first / after arguments, edge, node) Limit / offset pagination instead of cursor-based https://dev-blog.apollodata.com/understanding-pagination-rest-graphql-and-relay-b10f835549e7
  36. Field-level GraphQL authorization http://graphql.pro/

  37. https://github.com/exAspArk/graphql-guard GraphQL-Guard

  38. QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[!PostType] do

    argument :user_id, !types.ID resolve ->(obj, args, ctx) { Post.where(user_id: args[:user_id]) } end end Schema = GraphQL::Schema.define do query QueryType end query = %{ query MyPosts($user_id: ID) { posts(user_id: $user_id) { id } } } variables = { user_id: 1 } context = { current_user: current_user } Schema.execute(query, variables: variables, context: context) GraphQL-Guard https://github.com/exAspArk/graphql-guard
  39. QueryType = GraphQL::ObjectType.define do name "Query" field :posts, !types[!PostType] do

    argument :user_id, !types.ID resolve ->(obj, args, ctx) { Post.where(user_id: args[:user_id]) } end end Schema = GraphQL::Schema.define do query QueryType end query = %{ query MyPosts($user_id: ID) { posts(user_id: $user_id) { id } } } variables = { user_id: 1 } context = { current_user: current_user } Schema.execute(query, variables: variables, context: context) GraphQL-Guard # authorization logic can also be extracted outside schema definition guard ->(obj, args, ctx) { args[:user_id] == ctx[:current_user].id } https://github.com/exAspArk/graphql-guard
  40. August 2016 March 2017 July 2017 • Publicly available GraphQL

    at /graphql/beta • More query types, interfaces, enums, connections • Maximum query depth, arg validations, tiny mutation • Viewer (aka current_user) • First contributions by people from other teams in the company. The barrier of entry lower for new devs, IMO GraphQL beta version August 2017
  41. https://engineering.universe.com/batching-a-powerful-way-to-solve-n-1-queries-every-rubyist-should-know-24e20c6e7b94 How to kill N+1 queries?

  42. https://github.com/exaspark/batch-loader Batch-Loader • Generic utility to avoid N+1 DB queries,

    HTTP requests, etc. • Batching is isolated and lazy, load data in batch where and when it's needed • Automatically caches previous queries (identity map) • Thread-safe • No need to share batching through variables or custom defined classes • No dependencies, no monkey-patches, no extra primitives such as Promises
  43. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15370/diffs#f34dbf3de4996b6652de2cbafe00a9a79e516365_79_79 Batch-Loader • We’re using it for both RESTful API

    and GraphQL • GitLab uses it with their git data storage Gitaly
  44. https://github.com/exaspark/batch-loader Batch-Loader QueryType = GraphQL::ObjectType.define do name "Query" field :posts,

    !types[PostType] do resolve: ->(obj, args, ctx) { Post.all } end end PostType = GraphQL::ObjectType.define do name "Post" field :user, !UserType do resolve: ->(post, args, ctx) do # N + 1 post.user # queries end end end UserType = GraphQL::ObjectType.define do name "User" field :name, !types.String end
  45. https://github.com/exaspark/batch-loader Batch-Loader QueryType = GraphQL::ObjectType.define do name "Query" field :posts,

    !types[PostType] do resolve: ->(obj, args, ctx) { Post.all } end end PostType = GraphQL::ObjectType.define do name "Post" field :user, !UserType do resolve: ->(post, args, ctx) do BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end end end end UserType = GraphQL::ObjectType.define do name "User" field :name, !types.String end BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end
  46. https://github.com/exaspark/batch-loader Batch-Loader QueryType = GraphQL::ObjectType.define do name "Query" field :posts,

    !types[PostType] do resolve: ->(obj, args, ctx) { Post.all } end end PostType = GraphQL::ObjectType.define do name "Post" field :user, !UserType do resolve: ->(post, args, ctx) do BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end end end end UserType = GraphQL::ObjectType.define do name "User" field :name, !types.String end BatchLoader.for(post.user_id).batch do |ids, loader| User.where(id: ids).each { |u| loader.call(u.id, u) } end post.user_id u.id
  47. August 2016 March 2017 July 2017 First REST → GraphQL

    production client August 2017 + +
  48. August 2016 March 2017 July 2017 August 2017 September 2017

    More GraphQL on client-side
  49. https://github.com/exaspark/graphql-errors Error handling

  50. https://dev-blog.apollodata.com/graphql-schema-stitching-8af23354ac37 First beta testers _

  51. https://www.okgrow.com/training One day GraphQL training in the office • JS

    ES6, ES7, ES8, etc. • React • Apollo v2 • GraphQL.js • Express (Node.js) • Mongo
  52. August 2016 March 2017 July 2017 August 2017 September 2017

    Fully powered by GraphQL page
  53. Managing state on the client-side • We use Redux with

    Saga / NuclearJS with Immutable for REST API requests. For example, authorization. • We use Apollo Client for GraphQL requests.
  54. We’re not sure that Apollo Client can solve all our

    problems https://dev-blog.apollodata.com/apollo-client-2-0-beyond-graphql-apis-888807b53afe
  55. August 2016 March 2017 July 2017 More experiments August 2017

    September 2017 October 2017
  56. August 2016 March 2017 July 2017 A lot more! August

    2017 September 2017 October 2017
  57. Vodka – possible gradual migration to Elixir? • Elixir +

    Absinthe – accept GraphQL requests • Makes gRPC (protobuf, HTTP2) call to existing Rails app • Rails app is responsible for business logic
  58. August 2016 March 2017 July 2017 More GraphQL August 2017

    September 2017 October 2017 November 2017 • Second GraphQL schema for internal use • Monitoring
  59. Better monitoring with Apollo Engine https://www.apollographql.com/engine

  60. https://github.com/apollographql/apollo-tracing DIY

  61. https://github.com/uniiverse/apollo-tracing-ruby Apollo Tracing Ruby

  62. https://github.com/apollographql/optics-agent-ruby Deprecated Optics Agent Ruby

  63. August 2016 March 2017 July 2017 First GraphQL page with

    mutation on production! _ August 2017 September 2017 October 2017 November 2017 December 2017
  64. https://dev-blog.apollodata.com/designing-graphql-mutations-e09de826ed97 Designing GraphQL mutations

  65. What’s next? • Apollo GraphQL 2.x • Caching • More

    mutations • Persistent queries • AWS AppSync • Subscriptions and live queries • More experiments!
  66. client.mutate({ mutation: TodoCreateMutation, variables: { text }, update: (proxy, {

    data: { createTodo } }) => { // Read the data from our cache for this query. const data = proxy.readQuery({ query: TodoAppQuery }); // Add our todo from the mutation to the end. data.todos.push(createTodo); // Write our data back to the cache. proxy.writeQuery({ query: TodoAppQuery, data }); }, }); Apollo Client 2: direct cache access https://www.apollographql.com/docs/react/features/caching.html // Read the data from our cache for this query. const data = proxy.readQuery({ query: TodoAppQuery }); // Write our data back to the cache. proxy.writeQuery({ query: TodoAppQuery, data });
  67. import gql from 'graphql-tag'; import { Query } from 'react-apollo';

    const query = gql` query SomeQuery { foo { bar } } `; function MyComponent() { return ( <Query query={query}> {(result) => { if (result.loading) return <Loading />; if (result.error) return <Error error={error} />; return <h1>Hello {result.data.foo.bar}!</h1>; }) </Query> ); } React Apollo 2.1: Query component instead of HOC https://dev-blog.apollodata.com/whats-next-for-react-apollo-4d41ba12c2cb <Query query={query}> </Query>
  68. AWS AppSync: apps with real-time and offline capabilities https://aws.amazon.com/blogs/aws/introducing-amazon-appsync/

  69. https://www.youtube.com/watch?v=PsjiL6Yl1ag Gradually adding GraphQL subscriptions

  70. Takeaways Never try to solve all the problems at once

    – make them line up for you one-by-one. – Richard Sloma