Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Rescuing legacy codebases with GraphQL

Rescuing legacy codebases with GraphQL

GraphQL is one of the hottest technologies of the past year or two. Still, very little is talked about GraphQL outside of the realm of front-end applications.

We used a different approach at IFTTT and applied GraphQL as an integration layer for different backends and client apps.

This talk goes beyond the basic configuration of a GraphQL endpoint with Rails. I’ll cover topics such ActiveRecord Query optimization, performance monitoring, batching and share some of the challenges we ran into while building a GraphQL API that serves over 10 thousand queries per minute.

Ivayr Farah Netto

May 22, 2017
Tweet

More Decks by Ivayr Farah Netto

Other Decks in Programming

Transcript

  1. Rescuing legacy
    codebases with
    GraphQL
    @nettofarah

    View full-size slide

  2. Netto Farah
    @nettofarah
    Eng. Manager at

    View full-size slide

  3. Context
    • Millions of users
    • Billions of API calls every day
    • Website, iOS app, Android app

    View full-size slide

  4. Tech Stack (at the time)
    • Seasoned Rails 3 monolith app
    • APIs v1, v2, v3, dev_api…
    • Challenging to deploy/iterate/run tests
    • sole web dev

    View full-size slide

  5. Challenge:
    Build an entirely new product

    View full-size slide

  6. With a 9 months
    deadline

    View full-size slide

  7. We knew we needed
    to make some changes

    View full-size slide

  8. Majestic Monoliths
    vs
    Micro-services

    View full-size slide

  9. Not a binary
    decision

    View full-size slide

  10. A hybrid approach:
    Rich API + specific clients

    View full-size slide

  11. How can we make our
    frontend and backend
    apps communicate?

    View full-size slide

  12. Through the
    database?

    View full-size slide

  13. Why are database-driven
    integrations tempting?

    View full-size slide

  14. Why are database-driven
    integrations challenging?

    View full-size slide

  15. What about APIs?

    View full-size slide

  16. Challenges with
    Traditional APIs

    View full-size slide

  17. Multiple use cases

    View full-size slide

  18. Different access pattern

    View full-size slide

  19. Solved with documentation
    or conventions

    View full-size slide

  20. [',(,)..*] => ✅
    ,

    View full-size slide

  21. ,
    [-] =>
    ask me later…

    View full-size slide

  22. How can we
    solve a few of these
    challenges with APIs ?

    View full-size slide

  23. Ability to load just what
    we need

    View full-size slide

  24. Always get predictable
    results

    View full-size slide

  25. You know where
    I’m going with this, right?

    View full-size slide

  26. TYPES +
    PREDICTABLE RESULTS +
    COMPOSABLE QUERIES

    View full-size slide

  27. = GraphQL ❤

    View full-size slide

  28. We built a GraphQL API
    on top of our monolith

    View full-size slide

  29. GraphQL API
    as an integration layer
    for multiple (not so micro) services

    View full-size slide

  30. GraphQL API
    ———
    MonoRail



    A
    P
    I

    G

    A

    T
    E

    W

    A
    Y
    Service A
    Service B
    Service C

    View full-size slide

  31. GraphQL API
    ———
    MonoRail



    A
    P
    I

    G

    A

    T
    E

    W

    A
    Y
    Service A
    Service B
    Service C



    View full-size slide

  32. GraphQL (and Rails) in
    production

    View full-size slide

  33. Challenge #1

    View full-size slide

  34. query {
    recipes {
    title
    ingredients {
    name
    vendor { name }
    }
    }
    }

    View full-size slide

  35. SELECT "recipes".* FROM “recipes"
    SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 1
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 1
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 2
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 3
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 4
    SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 2
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 5
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 6
    SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 3
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 7
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 8
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 9
    SELECT "ingredients".* FROM "ingredients" WHERE "ingredients"."recipe_id" = 4
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 10
    SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = 11

    View full-size slide

  36. SELECT "ingredients".* FROM "ingredients" WHERE
    "ingredients"."recipe_id" = 1
    SELECT * FROM "vendors" WHERE "id" = 1
    SELECT * FROM "vendors" WHERE "id" = 2
    SELECT * FROM "vendors" WHERE "id" = 3
    SELECT * FROM "vendors" WHERE "id" = 4

    View full-size slide

  37. How do people
    usually solve this problem?

    View full-size slide

  38. DataLoader
    but that’s a javascript only tool

    View full-size slide

  39. GraphQL-Batch
    resolve -> (obj, args, context) do
    Loader.for(Product).load(args["id"]).then do |p|
    Loader.for(Image).load(p.image_id)
    end
    end

    View full-size slide

  40. Let’s take a second
    look at our data models

    View full-size slide

  41. Recipe.all.includes({
    ingredients: 'vendor'
    })

    View full-size slide

  42. query {
    recipes {
    title
    ingredients {
    name
    vendor
    }
    }
    }
    Recipe.all.includes({
    ingredients: 'vendor'
    })

    View full-size slide

  43. SELECT "recipes".* FROM "recipes"

    SELECT "ingredients".* FROM “ingredients”
    WHERE “ingredients"."recipe_id"
    IN (1, 2, 3, 4)

    SELECT "vendors".* FROM “vendors"
    WHERE “vendors"."id"
    IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)

    View full-size slide

  44. github.com/nettofarah/graphql-query-resolver

    View full-size slide

  45. ~60% reduction
    in database IOPS
    =

    View full-size slide

  46. Selectively choosing
    our Database environment

    View full-size slide

  47. if includes_mutation?(query)
    DatabaseSelection.use_main_database do
    GraphQL.execute_query(query)
    end
    else
    DatabaseSelection.use_readonly_replica do
    GraphQL.execute_query(query)
    end
    end

    View full-size slide

  48. ☠ Eliminated
    contention locks

    View full-size slide

  49. #1 Figure out
    batching as early
    as you can

    View full-size slide

  50. #2 Leverage
    GraphQL types

    View full-size slide

  51. Challenge #2

    View full-size slide

  52. Monitoring and
    Errors

    View full-size slide

  53. This is not really useful

    View full-size slide

  54. What’s up with my

    errors?

    View full-size slide

  55. #2 Leverage
    GraphQL types
    (again)
    Lesson

    View full-size slide

  56. # At the field level
    new_resolver = -> (obj, args, ctx) {
    name = [“GraphQL/field/#{type.name}.#{field.name}"]
    NewRelic.trace_execution_scoped(name) do
    old_resolver.call(obj, args, ctx)
    end
    }
    # At the query level
    NewRelic::Agent.set_transaction_name(query_name)

    View full-size slide

  57. http://bit.ly/gql-rb-nr

    View full-size slide

  58. #3 Proper monitoring
    is as important as
    good performance

    View full-size slide

  59. #4 GraphQL is
    awesome

    View full-size slide