Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Rescuing legacy codebases with GraphQL

Rescuing legacy codebases with GraphQL

GraphQL is one of the hottest technologies of the past year or two. Still, very little is talked about GraphQL outside of the realm of front-end applications.

We used a different approach at IFTTT and applied GraphQL as an integration layer for different backends and client apps.

This talk goes beyond the basic configuration of a GraphQL endpoint with Rails. I’ll cover topics such ActiveRecord Query optimization, performance monitoring, batching and share some of the challenges we ran into while building a GraphQL API that serves over 10 thousand queries per minute.

Ivayr Farah Netto

May 22, 2017
Tweet

More Decks by Ivayr Farah Netto

Other Decks in Programming

Transcript

  1. Rescuing legacy
    codebases with
    GraphQL
    @nettofarah

    View Slide

  2. Netto Farah
    @nettofarah
    Eng. Manager at

    View Slide

  3. View Slide

  4. View Slide

  5. Context
    • Millions of users
    • Billions of API calls every day
    • Website, iOS app, Android app

    View Slide

  6. Tech Stack (at the time)
    • Seasoned Rails 3 monolith app
    • APIs v1, v2, v3, dev_api…
    • Challenging to deploy/iterate/run tests
    • sole web dev

    View Slide

  7. Challenge:
    Build an entirely new product

    View Slide

  8. With a 9 months
    deadline

    View Slide

  9. We knew we needed
    to make some changes

    View Slide

  10. Majestic Monoliths
    vs
    Micro-services

    View Slide

  11. Not a binary
    decision

    View Slide

  12. A hybrid approach:
    Rich API + specific clients

    View Slide

  13. How can we make our
    frontend and backend
    apps communicate?

    View Slide

  14. Through the
    database?

    View Slide

  15. Why are database-driven
    integrations tempting?

    View Slide

  16. Why are database-driven
    integrations challenging?

    View Slide

  17. What about APIs?

    View Slide

  18. Challenges with
    Traditional APIs

    View Slide

  19. Multiple use cases

    View Slide

  20. Different access pattern

    View Slide

  21. Ambiguity

    View Slide

  22. Solved with documentation
    or conventions

    View Slide

  23. [',(,)..*] => ✅
    ,

    View Slide

  24. ,
    [-] =>
    ask me later…

    View Slide

  25. How can we
    solve a few of these
    challenges with APIs ?

    View Slide

  26. Types

    View Slide

  27. Ability to load just what
    we need

    View Slide

  28. Always get predictable
    results

    View Slide

  29. You know where
    I’m going with this, right?

    View Slide

  30. TYPES +
    PREDICTABLE RESULTS +
    COMPOSABLE QUERIES

    View Slide

  31. = GraphQL ❤

    View Slide

  32. We built a GraphQL API
    on top of our monolith

    View Slide

  33. GraphQL API
    as an integration layer
    for multiple (not so micro) services

    View Slide

  34. GraphQL API
    ———
    MonoRail



    A
    P
    I

    G

    A

    T
    E

    W

    A
    Y
    Service A
    Service B
    Service C

    View Slide

  35. GraphQL API
    ———
    MonoRail



    A
    P
    I

    G

    A

    T
    E

    W

    A
    Y
    Service A
    Service B
    Service C



    View Slide

  36. GraphQL (and Rails) in
    production

    View Slide

  37. Challenge #1

    View Slide

  38. N+1 queries

    View Slide

  39. View Slide

  40. query {
    recipes {
    title
    ingredients {
    name
    vendor { name }
    }
    }
    }

    View Slide

  41. SELECT "recipes".* FROM “recipes"
    SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 1
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 1
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 2
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 3
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 4
    SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 2
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 5
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 6
    SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 3
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 7
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 8
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 9
    SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 4
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 10
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 11

    View Slide

  42. SELECT "ingredients".* FROM "ingredients" WHERE
    "ingredients"."recipe_id" = 1
    SELECT * FROM "vendors" WHERE "id" = 1
    SELECT * FROM "vendors" WHERE "id" = 2
    SELECT * FROM "vendors" WHERE "id" = 3
    SELECT * FROM "vendors" WHERE "id" = 4

    View Slide

  43. How do people
    usually solve this problem?

    View Slide

  44. DataLoader
    but that’s a javascript only tool

    View Slide

  45. GraphQL-Batch
    resolve -> (obj, args, context) do
    Loader.for(Product).load(args["id"]).then do |p|
    Loader.for(Image).load(p.image_id)
    end
    end

    View Slide

  46. Let’s take a second
    look at our data models

    View Slide

  47. View Slide

  48. Recipe.all.includes({
    ingredients: 'vendor'
    })

    View Slide

  49. query {
    recipes {
    title
    ingredients {
    name
    vendor
    }
    }
    }
    Recipe.all.includes({
    ingredients: 'vendor'
    })

    View Slide

  50. SELECT "recipes".* FROM "recipes"

    SELECT "ingredients".* FROM “ingredients”
    WHERE “ingredients"."recipe_id"
    IN (1, 2, 3, 4)

    SELECT "vendors".* FROM “vendors"
    WHERE “vendors"."id"
    IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)

    View Slide

  51. github.com/nettofarah/graphql-query-resolver

    View Slide

  52. ~60% reduction
    in database IOPS
    =

    View Slide

  53. Selectively choosing
    our Database environment

    View Slide

  54. if includes_mutation?(query)
    DatabaseSelection.use_main_database do
    GraphQL.execute_query(query)
    end
    else
    DatabaseSelection.use_readonly_replica do
    GraphQL.execute_query(query)
    end
    end

    View Slide

  55. ☠ Eliminated
    contention locks

    View Slide

  56. Lessons

    View Slide

  57. #1 Figure out
    batching as early
    as you can

    View Slide

  58. #2 Leverage
    GraphQL types

    View Slide

  59. Challenge #2

    View Slide

  60. Monitoring and
    Errors

    View Slide

  61. This is not really useful

    View Slide

  62. What’s up with my

    errors?

    View Slide

  63. #2 Leverage
    GraphQL types
    (again)
    Lesson

    View Slide

  64. # At the field level
    new_resolver = -> (obj, args, ctx) {
    name = [“GraphQL/field/#{type.name}.#{field.name}"]
    NewRelic.trace_execution_scoped(name) do
    old_resolver.call(obj, args, ctx)
    end
    }
    # At the query level
    NewRelic::Agent.set_transaction_name(query_name)

    View Slide

  65. View Slide

  66. View Slide

  67. View Slide

  68. http://bit.ly/gql-rb-nr

    View Slide

  69. #3 Proper monitoring
    is as important as
    good performance

    View Slide

  70. #4 GraphQL is
    awesome

    View Slide

  71. @nettofarah
    [email protected]

    View Slide