Rescuing legacy codebases with GraphQL

Rescuing legacy codebases with GraphQL

GraphQL is one of the hottest technologies of the past year or two. Still, very little is talked about GraphQL outside of the realm of front-end applications.

We used a different approach at IFTTT and applied GraphQL as an integration layer for different backends and client apps.

This talk goes beyond the basic configuration of a GraphQL endpoint with Rails. I’ll cover topics such ActiveRecord Query optimization, performance monitoring, batching and share some of the challenges we ran into while building a GraphQL API that serves over 10 thousand queries per minute.

D3247e8f4b25f6041ab71da426f1c86e?s=128

Ivayr Farah Netto

May 22, 2017
Tweet

Transcript

  1. Rescuing legacy codebases with GraphQL @nettofarah

  2. Netto Farah @nettofarah Eng. Manager at

  3. None
  4. None
  5. Context • Millions of users • Billions of API calls

    every day • Website, iOS app, Android app
  6. Tech Stack (at the time) • Seasoned Rails 3 monolith

    app • APIs v1, v2, v3, dev_api… • Challenging to deploy/iterate/run tests • sole web dev
  7. Challenge: Build an entirely new product

  8. With a 9 months deadline

  9. We knew we needed to make some changes

  10. Majestic Monoliths vs Micro-services

  11. Not a binary decision

  12. A hybrid approach: Rich API + specific clients

  13. How can we make our frontend and backend apps communicate?

  14. Through the database?

  15. Why are database-driven integrations tempting?

  16. Why are database-driven integrations challenging?

  17. What about APIs?

  18. Challenges with Traditional APIs

  19. Multiple use cases

  20. Different access pattern

  21. Ambiguity

  22. Solved with documentation or conventions

  23. [',(,)..*] => ✅ ,

  24. , [-] => ask me later…

  25. How can we solve a few of these challenges with

    APIs ?
  26. Types

  27. Ability to load just what we need

  28. Always get predictable results

  29. You know where I’m going with this, right?

  30. TYPES + PREDICTABLE RESULTS + COMPOSABLE QUERIES

  31. = GraphQL ❤

  32. We built a GraphQL API on top of our monolith

  33. GraphQL API as an integration layer for multiple (not so

    micro) services
  34. GraphQL API ——— MonoRail ☁ A P I 
 G


    A
 T E
 W
 A Y Service A Service B Service C
  35. GraphQL API ——— MonoRail ⛈ A P I 
 G


    A
 T E
 W
 A Y Service A Service B Service C
  36. GraphQL (and Rails) in production

  37. Challenge #1

  38. N+1 queries

  39. None
  40. query { recipes { title ingredients { name vendor {

    name } } } }
  41. SELECT "recipes".* FROM “recipes" SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id"

    = 1 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 1 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 2 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 3 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 4 SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 2 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 5 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 6 SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 3 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 7 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 8 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 9 SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 4 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 10 SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 11
  42. SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 1 SELECT *

    FROM "vendors" WHERE "id" = 1 SELECT * FROM "vendors" WHERE "id" = 2 SELECT * FROM "vendors" WHERE "id" = 3 SELECT * FROM "vendors" WHERE "id" = 4
  43. How do people usually solve this problem?

  44. DataLoader but that’s a javascript only tool

  45. GraphQL-Batch resolve -> (obj, args, context) do Loader.for(Product).load(args["id"]).then do |p|

    Loader.for(Image).load(p.image_id) end end
  46. Let’s take a second look at our data models

  47. None
  48. Recipe.all.includes({ ingredients: 'vendor' })

  49. query { recipes { title ingredients { name vendor }

    } } Recipe.all.includes({ ingredients: 'vendor' })
  50. SELECT "recipes".* FROM "recipes"
 SELECT "ingredients".* FROM “ingredients” WHERE “ingredients"."recipe_id"

    IN (1, 2, 3, 4) 
 SELECT "vendors".* FROM “vendors" WHERE “vendors"."id" IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
  51. github.com/nettofarah/graphql-query-resolver

  52. ~60% reduction in database IOPS =

  53. Selectively choosing our Database environment

  54. if includes_mutation?(query) DatabaseSelection.use_main_database do GraphQL.execute_query(query) end else DatabaseSelection.use_readonly_replica do GraphQL.execute_query(query)

    end end
  55. ☠ Eliminated contention locks

  56. Lessons

  57. #1 Figure out batching as early as you can

  58. #2 Leverage GraphQL types

  59. Challenge #2

  60. Monitoring and Errors

  61. This is not really useful

  62. What’s up with my
 errors?

  63. #2 Leverage GraphQL types (again) Lesson

  64. # At the field level new_resolver = -> (obj, args,

    ctx) { name = [“GraphQL/field/#{type.name}.#{field.name}"] NewRelic.trace_execution_scoped(name) do old_resolver.call(obj, args, ctx) end } # At the query level NewRelic::Agent.set_transaction_name(query_name)
  65. None
  66. None
  67. None
  68. http://bit.ly/gql-rb-nr

  69. #3 Proper monitoring is as important as good performance

  70. #4 GraphQL is awesome

  71. @nettofarah nettofarah@gmail.com