Building Scalable APIs with GraphQL

Building Scalable APIs with GraphQL

In this presentation we will do an overview into the main paradigm for developing APIs (REST), to then learn what we can improve and researching into GraphQL and how it solves this issues. We will also learn how to make a simple API with Graphene and what are the things that we can adapt for special use cases.

Cf18e42df2acc7a1fc1700bcd8e76487?s=128

Syrus Akbary

May 02, 2017
Tweet

Transcript

  1. 2.

    @syrusakbary github.com/syrusakbary graphene-python.org 2 • CTO @ Try.com • Author

    of:
 pyjade, validate_email, interpy, jsjinja, promise, gdom… github.com/syrusakbary
  2. 4.

    T Y P I C A L P Y T

    H O N P R O J E C T S T R U C T U R E
  3. 8.

    D O M I N A N T D ATA

    S Y N C H R O N I Z AT I O N : R E S T
  4. 11.

    @syrusakbary github.com/syrusakbary graphene-python.org 11 • Get all the talks •

    What is the title for each talk? • What is the name and avatar of the speaker in the talk? • … Create
  5. 12.

    @syrusakbary github.com/syrusakbary graphene-python.org 12 • /schedule • /talk/1 • /talk/2

    • /talk/3 • /user/1 • /user/2 • /user/3 HTTP Requests
  6. 13.

    @syrusakbary github.com/syrusakbary graphene-python.org 13 • API Versioning • Input Validation

    • Output Validation • Data Underfetching / Overfetching • Network Errors • Network Latency Dealing with REST
  7. 14.

    @syrusakbary github.com/syrusakbary graphene-python.org 14 • API Versioning • Input Validation

    • Output Validation • Data Underfetching / Overfetching • Network Errors • Network Latency Dealing with REST Partially solved by serializers and views in some API Frameworks
  8. 15.

    @syrusakbary github.com/syrusakbary graphene-python.org 15 • API Versioning • Input Validation

    • Output Validation • Data Underfetching / Overfetching • Network Errors • Network Latency Dealing with REST Glue everything together in one RPC endpoint.
  9. 16.

    @syrusakbary github.com/syrusakbary graphene-python.org 16 • /all_talks_with_user_name_and_avatar “Could you please add

    an option to get the data back without this field but with this extra field for that new view that I’m making?” • Logic of fetching is moved to the server • Each time the client changes any data fetching requirements the server has to be updated (even if the schema/ models remain constant) • Difficult to scale
  10. 17.

    D E C L A R AT I V E

    D ATA F E T C H I N G
  11. 22.

    @syrusakbary github.com/syrusakbary graphene-python.org 22 { "me": { "name": "Syrus Akbary"

    "talks": [{ "title": "Graphene: A new…" "time": "2016-07-18T17:10" }] } } GraphQL Response { me { name talks { title time } } } GraphQL Query
  12. 24.

    @syrusakbary github.com/syrusakbary graphene-python.org 24 { me { name talks {

    title time } } } type Query { me: User } type User { name: String talks: [Talk] }
  13. 25.

    @syrusakbary github.com/syrusakbary graphene-python.org 25 { me { name talks {

    title time } } } type Query { me: User } type User { name: String talks: [Talk] } type Talk { title: String time: DateTime }
  14. 26.

    @syrusakbary github.com/syrusakbary graphene-python.org 26 • Query validation • Strictly typed:

    input and output • No data Underfetching / Overfetching • Introspection • Resolver Context • One roundtrip for data fetching • Backend agnostic Advantages of GraphQL
  15. 29.

    @syrusakbary github.com/syrusakbary graphene-python.org 29 • Most starred GraphQL framework (excluding

    FB implementation) - 1600⭐… much like! • Used by 100+ companies in production (Try.com included) • About 25k installs/month (growing 2x / 4 months) • Large community • Supports Python 2.7+ and 3.2+ (asyncio included!) Graphene Stats
  16. 32.

    @syrusakbary github.com/syrusakbary graphene-python.org 32 { me { name talks {

    title time } } } class Query(graphene.ObjectType): me = graphene.Field(User) GraphQL Query Implementation
  17. 33.

    @syrusakbary github.com/syrusakbary graphene-python.org 33 { me { name talks {

    title time } } } class Query(graphene.ObjectType): me = graphene.Field(User) class User(graphene.ObjectType): name = graphene.String() talks = graphene.List(Talk) GraphQL Query Implementation
  18. 34.

    @syrusakbary github.com/syrusakbary graphene-python.org 34 { me { name talks {

    title time } } } class Query(graphene.ObjectType): me = graphene.Field(User) class User(graphene.ObjectType): name = graphene.String() talks = graphene.List(Talk) class Talk(graphene.ObjectType): title = graphene.String() time = DateTime() GraphQL Query Implementation
  19. 35.

    @syrusakbary github.com/syrusakbary graphene-python.org 35 class Query(graphene.ObjectType): me = graphene.Field(User) class

    User(graphene.ObjectType): name = graphene.String() talks = graphene.List(Talk) class Talk(graphene.ObjectType): title = graphene.String() time = DateTime()
  20. 38.

    @syrusakbary github.com/syrusakbary graphene-python.org 38 • Django • Flask • WebOb

    • Pyramid • Pylons • Webapp2 • Sanic Server Integrations
  21. 39.

    A U TO M AT I C M A P

    P I N G F R O M O R M M O D E L S ❤
  22. 42.

    @syrusakbary github.com/syrusakbary graphene-python.org 42 • No versioning in GraphQL •

    Append-only schema • No breaking changes introduced • Some fields can be marked as deprecated, throwing warnings in the client if used.
 (as long as the deprecated fields are being consumed, you will need to maintain them) • Delete fields/types only when they have 0 hits GraphQL versioning
  23. 43.

    @syrusakbary github.com/syrusakbary graphene-python.org 43 • An addition to the GraphQL

    spec • Node • Mechanism to refetch a object from the root given the object id. • Enforces a unique global id per object • Connection • Structured types for paginate resources easily Relay
  24. 44.

    @syrusakbary github.com/syrusakbary graphene-python.org 44 • Simplified & consistent API over

    remote data sources via batching and caching. • Initial implementation inspired by FB one, but there are more for python • github.com/facebook/dataloader • github.com/quora/asynq Dataloader
  25. 45.

    @syrusakbary github.com/syrusakbary graphene-python.org 45 Dataloader class UserLoader(DataLoader): def batch_load_fn(self, ids):

    # Here we return a promise that will result on the # corresponding user for each key in keys return get_all_users_given_its_ids(ids) user_loader = UserLoader() user_loader.load(1) # Return as Promise user_loader.load(2) # Return as Promise get_all_users_given_its_ids([1, 2]) Dataloader will batch the calls for you
  26. 46.
  27. 47.

    @syrusakbary github.com/syrusakbary graphene-python.org 47 • Field resolution Sync by default

    Executors { me { name talks { title time } } } GraphQL Query 1 2 3 4 5 Sync Execution schema.execute( query, executor=SyncExecutor() )
  28. 48.

    @syrusakbary github.com/syrusakbary graphene-python.org 48 • Field resolution Sync by default

    • But can be changed! • Parallel field resolution… for free! • One line of code changed Executors { me { name talks { title time } } } GraphQL Query 1 2 2 3 3 Parallel Execution schema.execute( query, executor=ThreadExecutor() )
  29. 49.

    @syrusakbary github.com/syrusakbary graphene-python.org 49 • Affect the evaluation of fields

    in the query in runtime Middleware { me { name talks { title time } } } GraphQL Query context = { 'counters': defaultdict(int) } class CounterResolutionMiddleware(object): def resolve(self, next, root, args, context, info): key = '{}:{}'.format( info.parent_type.name, info.field_name ) context['counters'][key] += 1 return next(root, args, context, info) 1 1 1 5 5 Query:me User:name User:talks Talk:title Talk:time
  30. 51.

    @syrusakbary github.com/syrusakbary graphene-python.org 51 Graphene Test Client from graphene.test import

    Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } }
  31. 52.

    @syrusakbary github.com/syrusakbary graphene-python.org 52 Graphene Test Client from graphene.test import

    Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } }
  32. 53.

    @syrusakbary github.com/syrusakbary graphene-python.org 53 Graphene Test Client from graphene.test import

    Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } }
  33. 54.

    @syrusakbary github.com/syrusakbary graphene-python.org 54 Graphene Test Client from graphene.test import

    Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } }
  34. 55.

    @syrusakbary github.com/syrusakbary graphene-python.org 55 Graphene Test Client from graphene.test import

    Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } } Tedious to write
  35. 56.

    @syrusakbary github.com/syrusakbary graphene-python.org 56 Graphene Test Client from graphene.test import

    Client client = Client(my_schema) def test_viewer(snapshot): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) snapshot.assert_match(executed)
  36. 57.

    @syrusakbary github.com/syrusakbary graphene-python.org 57 Graphene Test Client from graphene.test import

    Client client = Client(my_schema) def test_viewer(snapshot): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) snapshot.assert_match(executed) # -*- coding: utf-8 -*- # snapshottest: v1 from __future__ import unicode_literals from snapshottest import Snapshot snapshots = Snapshot() snapshots['test_viewer 1'] = { 'data': { 'viewer': { 'name': 'Syrus' } } } Generated automatically tests/test_general.py tests/snapshots/snap_test_general.py
  37. 58.

    @syrusakbary github.com/syrusakbary graphene-python.org 58 • First Introduced by Jest (JS

    testing framework from FB) • Permits fast test iteration, as minimizes repetitive work • More tests / More code coverage ✅ • Happy dev / Happy PM Snapshot Testing
  38. 60.
  39. 61.

    @syrusakbary github.com/syrusakbary graphene-python.org 61 • Compiled Queries (GraphQL queries compiled

    to python code, ZERO library overhead) • Real Time subscriptions • Documentation… order of magnitude better! Coming Soon
  40. 62.

    G R A P H Q L I N F

    R O N T E N D
  41. 64.

    @syrusakbary github.com/syrusakbary graphene-python.org 64 React Relay in a Nutshell class

    User extends React.Component { render() { var {name, talks} = this.props.user; return ( <div> {name} <ul> {talks.map(talk) => ( <li key={talk.title}> {talk.title} (<em>{talk.time}</em>) </li> )} </ul> </div> ); } } Component definition User = Relay.createContainer(User, { fragments: { user: () => Relay.QL` fragment on User { name talks { title time } } `, }, }); Data Container
  42. 65.

    @syrusakbary github.com/syrusakbary graphene-python.org 65 Apollo Client in a Nutshell class

    User extends React.Component { render() { var {name, talks} = this.props.data.user; return ( <div> {name} <ul> {talks.map(talk) => ( <li key={talk.title}> {talk.title} (<em>{talk.time}</em>) </li> )} </ul> </div> ); } } User = graphql(gql` query { user { name, talks { title time } } } `)(User); Data Container Component definition
  43. 66.

    @syrusakbary github.com/syrusakbary graphene-python.org 66 • Facebook is behind it (long-term

    maintained, battle tested) • Static analysis of your GraphQL schema • Thinner JS bundles as the GraphQL engine is not embedded in the js bundle • Very performant • Only compatible with React • Forces the use of Relay in the Backend React Relay analysis
  44. 67.

    @syrusakbary github.com/syrusakbary graphene-python.org 67 • Simplified usage compared to relay/classic

    • More modularized code and abstracted from React • More integrations to come React Relay/modern?
  45. 68.

    @syrusakbary github.com/syrusakbary graphene-python.org 68 • Easier to use than React-Relay

    • Using Redux under the hood, easier to mutate the internal store • Doesn’t require to know your GraphQL schema ahead of time • Compatible with React, AngularJS, Vue, … • Apollo Dev tools Apollo Client analysis
  46. 70.

    @syrusakbary github.com/syrusakbary graphene-python.org 70 • Easier to maintain than REST

    API’s • 1-click documentation and UI (GraphiQL) • Quick integration in Frontend with React thanks to Relay • Seamless integration with your Python Backend • Much faster iteration/development process Why use GraphQL for your API?