Building Scalable APIs with GraphQL

In this presentation we will do an overview into the main paradigm for developing APIs (REST), to then learn what we can improve and researching into GraphQL and how it solves this issues. We will also learn how to make a simple API with Graphene and what are the things that we can adapt for special use cases.

Transcript

1. @syrusakbary github.com/syrusakbary
   GRAPHENE
   Building Scalable APIs with GraphQL
   Syrus Akbary Nieto
   @syrusakbary
   

   View Slide

2. @syrusakbary github.com/syrusakbary graphene-python.org 2
   • CTO @ Try.com
   • Author of:

   pyjade, validate_email, interpy, jsjinja, promise, gdom…
   github.com/syrusakbary
   

   View Slide

3. TA L K O V E RV I E W
   

   View Slide

4. T Y P I C A L P Y T H O N
   P R O J E C T S T R U C T U R E
   

   View Slide

5. @syrusakbary github.com/syrusakbary graphene-python.org 5
   Python
   BACKEND
   Server Side Templates

   Jinja / Django / Mako / …
   FRONTEND
   

   View Slide

6. @syrusakbary github.com/syrusakbary graphene-python.org 6
   

   View Slide

7. @syrusakbary github.com/syrusakbary graphene-python.org 7
   Python
   BACKEND
   Server Side Templates
   Destktop webapps (React.js, Angular…)
   Mobile apps
   FRONTEND
   API
   

   View Slide

8. D O M I N A N T D ATA S Y N C H R O N I Z AT I O N : R E S T
   

   View Slide

9. @syrusakbary github.com/syrusakbary graphene-python.org 9
   REST Example
   Let’s create a Conference App!
   

   View Slide

10. @syrusakbary github.com/syrusakbary graphene-python.org 10
    User
    Talk
    Speaker
    • Title
    • Time schedule
    • Name
    • Avatar
    Talks
    

    View Slide

11. @syrusakbary github.com/syrusakbary graphene-python.org 11
    • Get all the talks
    • What is the title for each talk?
    • What is the name and avatar of the speaker in the talk?
    • …
    Create
    

    View Slide

12. @syrusakbary github.com/syrusakbary graphene-python.org 12
    • /schedule
    • /talk/1
    • /talk/2
    • /talk/3
    • /user/1
    • /user/2
    • /user/3
    HTTP Requests
    

    View Slide

13. @syrusakbary github.com/syrusakbary graphene-python.org 13
    • API Versioning
    • Input Validation
    • Output Validation
    • Data Underfetching / Overfetching
    • Network Errors
    • Network Latency
    Dealing with REST
    

    View Slide

14. @syrusakbary github.com/syrusakbary graphene-python.org 14
    • API Versioning
    • Input Validation
    • Output Validation
    • Data Underfetching / Overfetching
    • Network Errors
    • Network Latency
    Dealing with REST
    Partially solved by serializers and
    views in some API Frameworks
    

    View Slide

15. @syrusakbary github.com/syrusakbary graphene-python.org 15
    • API Versioning
    • Input Validation
    • Output Validation
    • Data Underfetching / Overfetching
    • Network Errors
    • Network Latency
    Dealing with REST
    Glue everything together in one RPC
    endpoint.
    

    View Slide

16. @syrusakbary github.com/syrusakbary graphene-python.org 16
    • /all_talks_with_user_name_and_avatar
    “Could you please add an option to
    get the data back without this field
    but with this extra field for that new
    view that I’m making?”
    • Logic of fetching is moved
    to the server
    • Each time the client
    changes any data
    fetching requirements the
    server has to be updated
    (even if the schema/
    models remain constant)
    • Difficult to scale
    

    View Slide

17. D E C L A R AT I V E D ATA F E T C H I N G
    

    View Slide

18. @syrusakbary github.com/syrusakbary graphene-python.org 18
    GraphQL
    A new look into APIs
    

    View Slide

19. @syrusakbary github.com/syrusakbary graphene-python.org 19
    {
    me {
    name
    }
    }
    GraphQL Query
    

    View Slide

20. @syrusakbary github.com/syrusakbary graphene-python.org 20
    {
    "me": {
    "name": "Syrus Akbary"
    }
    }
    {
    me {
    name
    }
    }
    GraphQL Query GraphQL Response
    

    View Slide

21. @syrusakbary github.com/syrusakbary graphene-python.org 21
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query
    

    View Slide

22. @syrusakbary github.com/syrusakbary graphene-python.org 22
    {
    "me": {
    "name": "Syrus Akbary"
    "talks": [{
    "title": "Graphene: A new…"
    "time": "2016-07-18T17:10"
    }]
    }
    }
    GraphQL Response
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query
    

    View Slide

23. @syrusakbary github.com/syrusakbary graphene-python.org 23
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    type Query {
    me: User
    }
    

    View Slide

24. @syrusakbary github.com/syrusakbary graphene-python.org 24
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    type Query {
    me: User
    }
    type User {
    name: String
    talks: [Talk]
    }
    

    View Slide

25. @syrusakbary github.com/syrusakbary graphene-python.org 25
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    type Query {
    me: User
    }
    type User {
    name: String
    talks: [Talk]
    }
    type Talk {
    title: String
    time: DateTime
    }
    

    View Slide

26. @syrusakbary github.com/syrusakbary graphene-python.org 26
    • Query validation
    • Strictly typed: input and output
    • No data Underfetching / Overfetching
    • Introspection
    • Resolver Context
    • One roundtrip for data fetching
    • Backend agnostic
    Advantages of GraphQL
    

    View Slide

27. @syrusakbary github.com/syrusakbary graphene-python.org 27
    GraphQL implementations
    

    View Slide

28. @syrusakbary github.com/syrusakbary graphene-python.org 28
    Graphene
    The Pythonic way to GraphQL
    

    View Slide

29. @syrusakbary github.com/syrusakbary graphene-python.org 29
    • Most starred GraphQL framework (excluding FB
    implementation) - 1600⭐… much like!
    • Used by 100+ companies in production (Try.com
    included)
    • About 25k installs/month (growing 2x / 4 months)
    • Large community
    • Supports Python 2.7+ and 3.2+ (asyncio included!)
    Graphene Stats
    

    View Slide

30. @syrusakbary github.com/syrusakbary graphene-python.org 30
    Who’s using it?
    

    View Slide

31. C O D E S A M P L E S
    

    View Slide

32. @syrusakbary github.com/syrusakbary graphene-python.org 32
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    class Query(graphene.ObjectType):
    me = graphene.Field(User)
    GraphQL Query Implementation
    

    View Slide

33. @syrusakbary github.com/syrusakbary graphene-python.org 33
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    class Query(graphene.ObjectType):
    me = graphene.Field(User)
    class User(graphene.ObjectType):
    name = graphene.String()
    talks = graphene.List(Talk)
    GraphQL Query Implementation
    

    View Slide

34. @syrusakbary github.com/syrusakbary graphene-python.org 34
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    class Query(graphene.ObjectType):
    me = graphene.Field(User)
    class User(graphene.ObjectType):
    name = graphene.String()
    talks = graphene.List(Talk)
    class Talk(graphene.ObjectType):
    title = graphene.String()
    time = DateTime()
    GraphQL Query Implementation
    

    View Slide

35. @syrusakbary github.com/syrusakbary graphene-python.org 35
    class Query(graphene.ObjectType):
    me = graphene.Field(User)
    class User(graphene.ObjectType):
    name = graphene.String()
    talks = graphene.List(Talk)
    class Talk(graphene.ObjectType):
    title = graphene.String()
    time = DateTime()
    

    View Slide

36. P L AY G R O U N D
    

    View Slide

37. @syrusakbary github.com/syrusakbary graphene-python.org 37
    graphene-python.org/playground
    

    View Slide

38. @syrusakbary github.com/syrusakbary graphene-python.org 38
    • Django
    • Flask
    • WebOb
    • Pyramid
    • Pylons
    • Webapp2
    • Sanic
    Server Integrations
    

    View Slide

39. A U TO M AT I C M A P P I N G F R O M
    O R M M O D E L S ❤
    

    View Slide

40. @syrusakbary github.com/syrusakbary graphene-python.org 40
    • Django
    • SQLAlchemy
    • Google App Engine (NDB)
    • Pewee
    • PynamoDB
    ORM Integrations
    

    View Slide

41. B E S T P R A C T I C E S
    

    View Slide

42. @syrusakbary github.com/syrusakbary graphene-python.org 42
    • No versioning in GraphQL
    • Append-only schema
    • No breaking changes introduced
    • Some fields can be marked as deprecated, throwing
    warnings in the client if used.

    (as long as the deprecated fields are being consumed,
    you will need to maintain them)
    • Delete fields/types only when they have 0 hits
    GraphQL versioning
    

    View Slide

43. @syrusakbary github.com/syrusakbary graphene-python.org 43
    • An addition to the GraphQL spec
    • Node
    • Mechanism to refetch a object from the root given the
    object id.
    • Enforces a unique global id per object
    • Connection
    • Structured types for paginate resources easily
    Relay
    

    View Slide

44. @syrusakbary github.com/syrusakbary graphene-python.org 44
    • Simplified & consistent API over remote data sources via
    batching and caching.
    • Initial implementation inspired by FB one, but there are
    more for python
    • github.com/facebook/dataloader
    • github.com/quora/asynq
    Dataloader
    

    View Slide

45. @syrusakbary github.com/syrusakbary graphene-python.org 45
    Dataloader
    class UserLoader(DataLoader):
    def batch_load_fn(self, ids):
    # Here we return a promise that will result on the
    # corresponding user for each key in keys
    return get_all_users_given_its_ids(ids)
    user_loader = UserLoader()
    user_loader.load(1) # Return as Promise
    user_loader.load(2) # Return as Promise
    get_all_users_given_its_ids([1, 2])
    Dataloader
    will batch the
    calls for you
    

    View Slide

46. G R A P H E N E Q U I R K S
    

    View Slide

47. @syrusakbary github.com/syrusakbary graphene-python.org 47
    • Field resolution Sync by default
    Executors
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query
    1
    2
    3
    4
    5
    Sync
    Execution
    schema.execute(
    query,
    executor=SyncExecutor()
    )
    

    View Slide

48. @syrusakbary github.com/syrusakbary graphene-python.org 48
    • Field resolution Sync by default
    • But can be changed!
    • Parallel field resolution… for
    free!
    • One line of code changed
    Executors
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query
    1
    2
    2
    3
    3
    Parallel
    Execution
    schema.execute(
    query,
    executor=ThreadExecutor()
    )
    

    View Slide

49. @syrusakbary github.com/syrusakbary graphene-python.org 49
    • Affect the evaluation of fields in the query in
    runtime
    Middleware
    {
    me {
    name
    talks {
    title
    time
    }
    }
    }
    GraphQL Query
    context = {
    'counters': defaultdict(int)
    }
    class CounterResolutionMiddleware(object):
    def resolve(self, next, root, args, context, info):
    key = '{}:{}'.format(
    info.parent_type.name,
    info.field_name
    )
    context['counters'][key] += 1
    return next(root, args, context, info)
    1
    1
    1
    5
    5
    Query:me
    User:name
    User:talks
    Talk:title
    Talk:time
    

    View Slide

50. T E S T I N G
    

    View Slide

51. @syrusakbary github.com/syrusakbary graphene-python.org 51
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }
    

    View Slide

52. @syrusakbary github.com/syrusakbary graphene-python.org 52
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }
    

    View Slide

53. @syrusakbary github.com/syrusakbary graphene-python.org 53
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }
    

    View Slide

54. @syrusakbary github.com/syrusakbary graphene-python.org 54
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }
    

    View Slide

55. @syrusakbary github.com/syrusakbary graphene-python.org 55
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer():
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    assert executed == {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }
    Tedious to write
    

    View Slide

56. @syrusakbary github.com/syrusakbary graphene-python.org 56
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer(snapshot):
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    snapshot.assert_match(executed)
    

    View Slide

57. @syrusakbary github.com/syrusakbary graphene-python.org 57
    Graphene Test Client
    from graphene.test import Client
    client = Client(my_schema)
    def test_viewer(snapshot):
    request = MockRequest()
    executed = client.execute('''{
    viewer {
    name
    }
    }''', context_value=request)
    snapshot.assert_match(executed)
    # -*- coding: utf-8 -*-
    # snapshottest: v1
    from __future__ import unicode_literals
    from snapshottest import Snapshot
    snapshots = Snapshot()
    snapshots['test_viewer 1'] = {
    'data': {
    'viewer': {
    'name': 'Syrus'
    }
    }
    }
    Generated automatically
    tests/test_general.py
    tests/snapshots/snap_test_general.py
    

    View Slide

58. @syrusakbary github.com/syrusakbary graphene-python.org 58
    • First Introduced by Jest (JS testing framework from FB)
    • Permits fast test iteration, as minimizes repetitive work
    • More tests / More code coverage ✅
    • Happy dev / Happy PM
    Snapshot Testing
    

    View Slide

59. @syrusakbary github.com/syrusakbary graphene-python.org 59
    SnapshotTest Demo
    github.com/syrusakbary/snapshottest
    

    View Slide

60. W H AT ’ S C O M I N G S O O N
    

    View Slide

61. @syrusakbary github.com/syrusakbary graphene-python.org 61
    • Compiled Queries (GraphQL queries compiled to python
    code, ZERO library overhead)
    • Real Time subscriptions
    • Documentation… order of magnitude better!
    Coming Soon
    

    View Slide

62. G R A P H Q L I N F R O N T E N D
    

    View Slide

63. @syrusakbary github.com/syrusakbary graphene-python.org 63
    React Relay
    facebook.github.io/relay/
    Apollo Client
    dev.apollodata.com/
    

    View Slide

64. @syrusakbary github.com/syrusakbary graphene-python.org 64
    React Relay in a Nutshell
    class User extends React.Component {
    render() {
    var {name, talks} = this.props.user;
    return (
    
    {name}
    
    {talks.map(talk) => (
    
    {talk.title} ({talk.time})
    
    )}
    
    
    );
    }
    }
    Component definition
    User = Relay.createContainer(User, {
    fragments: {
    user: () => Relay.QL`
    fragment on User {
    name
    talks {
    title
    time
    }
    }
    `,
    },
    });
    Data Container
    

    View Slide

65. @syrusakbary github.com/syrusakbary graphene-python.org 65
    Apollo Client in a Nutshell
    class User extends React.Component {
    render() {
    var {name, talks} = this.props.data.user;
    return (
    
    {name}
    
    {talks.map(talk) => (
    
    {talk.title} ({talk.time})
    
    )}
    
    
    );
    }
    }
    User = graphql(gql`
    query {
    user {
    name,
    talks {
    title
    time
    }
    }
    }
    `)(User);
    Data Container
    Component definition
    

    View Slide

66. @syrusakbary github.com/syrusakbary graphene-python.org 66
    • Facebook is behind it (long-term maintained, battle tested)
    • Static analysis of your GraphQL schema
    • Thinner JS bundles as the GraphQL engine is not
    embedded in the js bundle
    • Very performant
    • Only compatible with React
    • Forces the use of Relay in the Backend
    React Relay analysis
    

    View Slide

67. @syrusakbary github.com/syrusakbary graphene-python.org 67
    • Simplified usage compared to relay/classic
    • More modularized code and abstracted from React
    • More integrations to come
    React Relay/modern?
    

    View Slide

68. @syrusakbary github.com/syrusakbary graphene-python.org 68
    • Easier to use than React-Relay
    • Using Redux under the hood, easier to mutate the internal
    store
    • Doesn’t require to know your GraphQL schema ahead of
    time
    • Compatible with React, AngularJS, Vue, …
    • Apollo Dev tools
    Apollo Client analysis
    

    View Slide

69. W H Y U S E G R A P H Q L
    

    View Slide

70. @syrusakbary github.com/syrusakbary graphene-python.org 70
    • Easier to maintain than REST API’s
    • 1-click documentation and UI (GraphiQL)
    • Quick integration in Frontend with React thanks to Relay
    • Seamless integration with your Python Backend
    • Much faster iteration/development process
    Why use GraphQL for your API?
    

    View Slide

71. Q U E S T I O N S ? : )
    

    View Slide