Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Scalable APIs with GraphQL

Building Scalable APIs with GraphQL

In this presentation we will do an overview into the main paradigm for developing APIs (REST), to then learn what we can improve and researching into GraphQL and how it solves this issues. We will also learn how to make a simple API with Graphene and what are the things that we can adapt for special use cases.

Syrus Akbary

May 02, 2017
Tweet

More Decks by Syrus Akbary

Other Decks in Programming

Transcript

  1. @syrusakbary github.com/syrusakbary
    GRAPHENE
    Building Scalable APIs with GraphQL
    Syrus Akbary Nieto
    @syrusakbary

    View Slide

  2. @syrusakbary github.com/syrusakbary graphene-python.org 2
    • CTO @ Try.com
    • Author of:

    pyjade, validate_email, interpy, jsjinja, promise, gdom…
    github.com/syrusakbary

    View Slide

  3. TA L K O V E RV I E W

    View Slide

  4. T Y P I C A L P Y T H O N
    P R O J E C T S T R U C T U R E

    View Slide

  5. @syrusakbary github.com/syrusakbary graphene-python.org 5
    Python
    BACKEND
    Server Side Templates

    Jinja / Django / Mako / …
    FRONTEND

    View Slide

  6. @syrusakbary github.com/syrusakbary graphene-python.org 6

    View Slide

  7. @syrusakbary github.com/syrusakbary graphene-python.org 7
    Python
    BACKEND
    Server Side Templates
    Destktop webapps (React.js, Angular…)
    Mobile apps
    FRONTEND
    API

    View Slide

  8. D O M I N A N T D ATA S Y N C H R O N I Z AT I O N : R E S T

    View Slide

  9. @syrusakbary github.com/syrusakbary graphene-python.org 9
    REST Example
    Let’s create a Conference App!

    View Slide

  10. @syrusakbary github.com/syrusakbary graphene-python.org 10
    User
    Talk
    Speaker
    • Title
    • Time schedule
    • Name
    • Avatar
    Talks

    View Slide

  11. @syrusakbary github.com/syrusakbary graphene-python.org 11
    • Get all the talks
    • What is the title for each talk?
    • What is the name and avatar of the speaker in the talk?
    • …
    Create

    View Slide

  12. @syrusakbary github.com/syrusakbary graphene-python.org 12
    • /schedule
    • /talk/1
    • /talk/2
    • /talk/3
    • /user/1
    • /user/2
    • /user/3
    HTTP Requests

    View Slide

  13. @syrusakbary github.com/syrusakbary graphene-python.org 13
    • API Versioning
    • Input Validation
    • Output Validation
    • Data Underfetching / Overfetching
    • Network Errors
    • Network Latency
    Dealing with REST

    View Slide

  14. @syrusakbary github.com/syrusakbary graphene-python.org 14
    • API Versioning
    • Input Validation
    • Output Validation
    • Data Underfetching / Overfetching
    • Network Errors
    • Network Latency
    Dealing with REST
    Partially solved by serializers and
    views in some API Frameworks

    View Slide

  15. @syrusakbary github.com/syrusakbary graphene-python.org 15
    • API Versioning
    • Input Validation
    • Output Validation
    • Data Underfetching / Overfetching
    • Network Errors
    • Network Latency
    Dealing with REST
    Glue everything together in one RPC
    endpoint.

    View Slide

  16. @syrusakbary github.com/syrusakbary graphene-python.org 16
    • /all_talks_with_user_name_and_avatar
    “Could you please add an option to
    get the data back without this field
    but with this extra field for that new
    view that I’m making?”
    • Logic of fetching is moved
    to the server
    • Each time the client
    changes any data
    fetching requirements the
    server has to be updated
    (even if the schema/
    models remain constant)
    • Difficult to scale

    View Slide

  17. D E C L A R AT I V E D ATA F E T C H I N G

    View Slide

  18. @syrusakbary github.com/syrusakbary graphene-python.org 18
    GraphQL
    A new look into APIs

    View Slide

  19. @syrusakbary github.com/syrusakbary graphene-python.org 19
    {
    me {
    name
    }
    }
    GraphQL Query

    View Slide

  20. @syrusakbary github.com/syrusakbary graphene-python.org 20
    {
    "me": {
    "name": "Syrus Akbary"
    }
    }
    {
    me {
    name
    }
    }
    GraphQL Query GraphQL Response

    View Slide

  21. @syrusakbary github.com/syrusakbary graphene-python.org 21
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query

    View Slide

  22. @syrusakbary github.com/syrusakbary graphene-python.org 22
    {
    "me": {
    "name": "Syrus Akbary"
    "talks": [{
    "title": "Graphene: A new…"
    "time": "2016-07-18T17:10"
    }]
    }
    }
    GraphQL Response
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query

    View Slide

  23. @syrusakbary github.com/syrusakbary graphene-python.org 23
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    type Query {
    me: User
    }

    View Slide

  24. @syrusakbary github.com/syrusakbary graphene-python.org 24
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    type Query {
    me: User
    }
    type User {
    name: String
    talks: [Talk]
    }

    View Slide

  25. @syrusakbary github.com/syrusakbary graphene-python.org 25
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    type Query {
    me: User
    }
    type User {
    name: String
    talks: [Talk]
    }
    type Talk {
    title: String
    time: DateTime
    }

    View Slide

  26. @syrusakbary github.com/syrusakbary graphene-python.org 26
    • Query validation
    • Strictly typed: input and output
    • No data Underfetching / Overfetching
    • Introspection
    • Resolver Context
    • One roundtrip for data fetching
    • Backend agnostic
    Advantages of GraphQL

    View Slide

  27. @syrusakbary github.com/syrusakbary graphene-python.org 27
    GraphQL implementations

    View Slide

  28. @syrusakbary github.com/syrusakbary graphene-python.org 28
    Graphene
    The Pythonic way to GraphQL

    View Slide

  29. @syrusakbary github.com/syrusakbary graphene-python.org 29
    • Most starred GraphQL framework (excluding FB
    implementation) - 1600⭐… much like!
    • Used by 100+ companies in production (Try.com
    included)
    • About 25k installs/month (growing 2x / 4 months)
    • Large community
    • Supports Python 2.7+ and 3.2+ (asyncio included!)
    Graphene Stats

    View Slide

  30. @syrusakbary github.com/syrusakbary graphene-python.org 30
    Who’s using it?

    View Slide

  31. C O D E S A M P L E S

    View Slide

  32. @syrusakbary github.com/syrusakbary graphene-python.org 32
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    class Query(graphene.ObjectType):
    me = graphene.Field(User)
    GraphQL Query Implementation

    View Slide

  33. @syrusakbary github.com/syrusakbary graphene-python.org 33
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    class Query(graphene.ObjectType):
    me = graphene.Field(User)
    class User(graphene.ObjectType):
    name = graphene.String()
    talks = graphene.List(Talk)
    GraphQL Query Implementation

    View Slide

  34. @syrusakbary github.com/syrusakbary graphene-python.org 34
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    class Query(graphene.ObjectType):
    me = graphene.Field(User)
    class User(graphene.ObjectType):
    name = graphene.String()
    talks = graphene.List(Talk)
    class Talk(graphene.ObjectType):
    title = graphene.String()
    time = DateTime()
    GraphQL Query Implementation

    View Slide

  35. @syrusakbary github.com/syrusakbary graphene-python.org 35
    class Query(graphene.ObjectType):
    me = graphene.Field(User)
    class User(graphene.ObjectType):
    name = graphene.String()
    talks = graphene.List(Talk)
    class Talk(graphene.ObjectType):
    title = graphene.String()
    time = DateTime()

    View Slide

  36. P L AY G R O U N D

    View Slide

  37. @syrusakbary github.com/syrusakbary graphene-python.org 37
    graphene-python.org/playground

    View Slide

  38. @syrusakbary github.com/syrusakbary graphene-python.org 38
    • Django
    • Flask
    • WebOb
    • Pyramid
    • Pylons
    • Webapp2
    • Sanic
    Server Integrations

    View Slide

  39. A U TO M AT I C M A P P I N G F R O M
    O R M M O D E L S ❤

    View Slide

  40. @syrusakbary github.com/syrusakbary graphene-python.org 40
    • Django
    • SQLAlchemy
    • Google App Engine (NDB)
    • Pewee
    • PynamoDB
    ORM Integrations

    View Slide

  41. B E S T P R A C T I C E S

    View Slide

  42. @syrusakbary github.com/syrusakbary graphene-python.org 42
    • No versioning in GraphQL
    • Append-only schema
    • No breaking changes introduced
    • Some fields can be marked as deprecated, throwing
    warnings in the client if used.

    (as long as the deprecated fields are being consumed,
    you will need to maintain them)
    • Delete fields/types only when they have 0 hits
    GraphQL versioning

    View Slide

  43. @syrusakbary github.com/syrusakbary graphene-python.org 43
    • An addition to the GraphQL spec
    • Node
    • Mechanism to refetch a object from the root given the
    object id.
    • Enforces a unique global id per object
    • Connection
    • Structured types for paginate resources easily
    Relay

    View Slide

  44. @syrusakbary github.com/syrusakbary graphene-python.org 44
    • Simplified & consistent API over remote data sources via
    batching and caching.
    • Initial implementation inspired by FB one, but there are
    more for python
    • github.com/facebook/dataloader
    • github.com/quora/asynq
    Dataloader

    View Slide

  45. @syrusakbary github.com/syrusakbary graphene-python.org 45
    Dataloader
    class UserLoader(DataLoader):
    def batch_load_fn(self, ids):
    # Here we return a promise that will result on the
    # corresponding user for each key in keys
    return get_all_users_given_its_ids(ids)
    user_loader = UserLoader()
    user_loader.load(1) # Return as Promise
    user_loader.load(2) # Return as Promise
    get_all_users_given_its_ids([1, 2])
    Dataloader
    will batch the
    calls for you

    View Slide

  46. G R A P H E N E Q U I R K S

    View Slide

  47. @syrusakbary github.com/syrusakbary graphene-python.org 47
    • Field resolution Sync by default
    Executors
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query
    1
    2
    3
    4
    5
    Sync
    Execution
    schema.execute(
    query,
    executor=SyncExecutor()
    )

    View Slide

  48. @syrusakbary github.com/syrusakbary graphene-python.org 48
    • Field resolution Sync by default
    • But can be changed!
    • Parallel field resolution… for
    free!
    • One line of code changed
    Executors
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query
    1
    2
    2
    3
    3
    Parallel
    Execution
    schema.execute(
    query,
    executor=ThreadExecutor()
    )

    View Slide

  49. @syrusakbary github.com/syrusakbary graphene-python.org 49
    • Affect the evaluation of fields in the query in
    runtime
    Middleware
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query
    context = {
    'counters': defaultdict(int)
    }
    class CounterResolutionMiddleware(object):
    def resolve(self, next, root, args, context, info):
    key = '{}:{}'.format(
    info.parent_type.name,
    info.field_name
    )
    context['counters'][key] += 1
    return next(root, args, context, info)
    1
    1
    1
    5
    5
    Query:me
    User:name
    User:talks
    Talk:title
    Talk:time

    View Slide

  50. T E S T I N G

    View Slide

  51. @syrusakbary github.com/syrusakbary graphene-python.org 51
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }

    View Slide

  52. @syrusakbary github.com/syrusakbary graphene-python.org 52
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }

    View Slide

  53. @syrusakbary github.com/syrusakbary graphene-python.org 53
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }

    View Slide

  54. @syrusakbary github.com/syrusakbary graphene-python.org 54
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }

    View Slide

  55. @syrusakbary github.com/syrusakbary graphene-python.org 55
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }
    Tedious to write

    View Slide

  56. @syrusakbary github.com/syrusakbary graphene-python.org 56
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer(snapshot):
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    snapshot.assert_match(executed)

    View Slide

  57. @syrusakbary github.com/syrusakbary graphene-python.org 57
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer(snapshot):
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    snapshot.assert_match(executed)
    # -*- coding: utf-8 -*-
    # snapshottest: v1
    from __future__ import unicode_literals
    from snapshottest import Snapshot
    snapshots = Snapshot()
    snapshots['test_viewer 1'] = {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }
    Generated automatically
    tests/test_general.py
    tests/snapshots/snap_test_general.py

    View Slide

  58. @syrusakbary github.com/syrusakbary graphene-python.org 58
    • First Introduced by Jest (JS testing framework from FB)
    • Permits fast test iteration, as minimizes repetitive work
    • More tests / More code coverage ✅
    • Happy dev / Happy PM
    Snapshot Testing

    View Slide

  59. @syrusakbary github.com/syrusakbary graphene-python.org 59
    SnapshotTest Demo
    github.com/syrusakbary/snapshottest

    View Slide

  60. W H AT ’ S C O M I N G S O O N

    View Slide

  61. @syrusakbary github.com/syrusakbary graphene-python.org 61
    • Compiled Queries (GraphQL queries compiled to python
    code, ZERO library overhead)
    • Real Time subscriptions
    • Documentation… order of magnitude better!
    Coming Soon

    View Slide

  62. G R A P H Q L I N F R O N T E N D

    View Slide

  63. @syrusakbary github.com/syrusakbary graphene-python.org 63
    React Relay
    facebook.github.io/relay/
    Apollo Client
    dev.apollodata.com/

    View Slide

  64. @syrusakbary github.com/syrusakbary graphene-python.org 64
    React Relay in a Nutshell
    class User extends React.Component {
    render() {
    var {name, talks} = this.props.user;
    return (

    {name}

    {talks.map(talk) => (

    {talk.title} ({talk.time})

    )}


    );
    }
    }
    Component definition
    User = Relay.createContainer(User, {
    fragments: {
    user: () => Relay.QL`
    fragment on User {
    name
    talks {
    title
    time
    }
    }
    `,
    },
    });
    Data Container

    View Slide

  65. @syrusakbary github.com/syrusakbary graphene-python.org 65
    Apollo Client in a Nutshell
    class User extends React.Component {
    render() {
    var {name, talks} = this.props.data.user;
    return (

    {name}

    {talks.map(talk) => (

    {talk.title} ({talk.time})

    )}


    );
    }
    }
    User = graphql(gql`
    query {
    user {
    name,
    talks {
    title
    time
    }
    }
    }
    `)(User);
    Data Container
    Component definition

    View Slide

  66. @syrusakbary github.com/syrusakbary graphene-python.org 66
    • Facebook is behind it (long-term maintained, battle tested)
    • Static analysis of your GraphQL schema
    • Thinner JS bundles as the GraphQL engine is not
    embedded in the js bundle
    • Very performant
    • Only compatible with React
    • Forces the use of Relay in the Backend
    React Relay analysis

    View Slide

  67. @syrusakbary github.com/syrusakbary graphene-python.org 67
    • Simplified usage compared to relay/classic
    • More modularized code and abstracted from React
    • More integrations to come
    React Relay/modern?

    View Slide

  68. @syrusakbary github.com/syrusakbary graphene-python.org 68
    • Easier to use than React-Relay
    • Using Redux under the hood, easier to mutate the internal
    store
    • Doesn’t require to know your GraphQL schema ahead of
    time
    • Compatible with React, AngularJS, Vue, …
    • Apollo Dev tools
    Apollo Client analysis

    View Slide

  69. W H Y U S E G R A P H Q L

    View Slide

  70. @syrusakbary github.com/syrusakbary graphene-python.org 70
    • Easier to maintain than REST API’s
    • 1-click documentation and UI (GraphiQL)
    • Quick integration in Frontend with React thanks to Relay
    • Seamless integration with your Python Backend
    • Much faster iteration/development process
    Why use GraphQL for your API?

    View Slide

  71. Q U E S T I O N S ? : )

    View Slide