Building Scalable APIs with GraphQL

In this presentation we will do an overview into the main paradigm for developing APIs (REST), to then learn what we can improve and researching into GraphQL and how it solves this issues. We will also learn how to make a simple API with Graphene and what are the things that we can adapt for special use cases.

Transcript

1. @syrusakbary github.com/syrusakbary GRAPHENE Building Scalable APIs with GraphQL Syrus Akbary Nieto @syrusakbary
2. @syrusakbary github.com/syrusakbary graphene-python.org 2 • CTO @ Try.com • Author of:
 pyjade, validate_email, interpy, jsjinja, promise, gdom… github.com/syrusakbary
3. TA L K O V E RV I E W
4. T Y P I C A L P Y T H O N P R O J E C T S T R U C T U R E
5. @syrusakbary github.com/syrusakbary graphene-python.org 5 Python BACKEND Server Side Templates
 Jinja / Django / Mako / … FRONTEND
6. @syrusakbary github.com/syrusakbary graphene-python.org 6
7. @syrusakbary github.com/syrusakbary graphene-python.org 7 Python BACKEND Server Side Templates Destktop webapps (React.js, Angular…) Mobile apps FRONTEND API
8. D O M I N A N T D ATA S Y N C H R O N I Z AT I O N : R E S T
9. @syrusakbary github.com/syrusakbary graphene-python.org 9 REST Example Let’s create a Conference App!
10. @syrusakbary github.com/syrusakbary graphene-python.org 10 User Talk Speaker • Title • Time schedule • Name • Avatar Talks
11. @syrusakbary github.com/syrusakbary graphene-python.org 11 • Get all the talks • What is the title for each talk? • What is the name and avatar of the speaker in the talk? • … Create
12. @syrusakbary github.com/syrusakbary graphene-python.org 12 • /schedule • /talk/1 • /talk/2 • /talk/3 • /user/1 • /user/2 • /user/3 HTTP Requests
13. @syrusakbary github.com/syrusakbary graphene-python.org 13 • API Versioning • Input Validation • Output Validation • Data Underfetching / Overfetching • Network Errors • Network Latency Dealing with REST
14. @syrusakbary github.com/syrusakbary graphene-python.org 14 • API Versioning • Input Validation • Output Validation • Data Underfetching / Overfetching • Network Errors • Network Latency Dealing with REST Partially solved by serializers and views in some API Frameworks
15. @syrusakbary github.com/syrusakbary graphene-python.org 15 • API Versioning • Input Validation • Output Validation • Data Underfetching / Overfetching • Network Errors • Network Latency Dealing with REST Glue everything together in one RPC endpoint.
16. @syrusakbary github.com/syrusakbary graphene-python.org 16 • /all_talks_with_user_name_and_avatar “Could you please add an option to get the data back without this field but with this extra field for that new view that I’m making?” • Logic of fetching is moved to the server • Each time the client changes any data fetching requirements the server has to be updated (even if the schema/ models remain constant) • Difficult to scale
17. D E C L A R AT I V E D ATA F E T C H I N G
18. @syrusakbary github.com/syrusakbary graphene-python.org 18 GraphQL A new look into APIs
19. @syrusakbary github.com/syrusakbary graphene-python.org 19 { me { name } } GraphQL Query
20. @syrusakbary github.com/syrusakbary graphene-python.org 20 { "me": { "name": "Syrus Akbary" } } { me { name } } GraphQL Query GraphQL Response
21. @syrusakbary github.com/syrusakbary graphene-python.org 21 { me { name talks { title time } } } GraphQL Query
22. @syrusakbary github.com/syrusakbary graphene-python.org 22 { "me": { "name": "Syrus Akbary" "talks": [{ "title": "Graphene: A new…" "time": "2016-07-18T17:10" }] } } GraphQL Response { me { name talks { title time } } } GraphQL Query
23. @syrusakbary github.com/syrusakbary graphene-python.org 23 { me { name talks { title time } } } type Query { me: User }
24. @syrusakbary github.com/syrusakbary graphene-python.org 24 { me { name talks { title time } } } type Query { me: User } type User { name: String talks: [Talk] }
25. @syrusakbary github.com/syrusakbary graphene-python.org 25 { me { name talks { title time } } } type Query { me: User } type User { name: String talks: [Talk] } type Talk { title: String time: DateTime }
26. @syrusakbary github.com/syrusakbary graphene-python.org 26 • Query validation • Strictly typed: input and output • No data Underfetching / Overfetching • Introspection • Resolver Context • One roundtrip for data fetching • Backend agnostic Advantages of GraphQL
27. @syrusakbary github.com/syrusakbary graphene-python.org 27 GraphQL implementations
28. @syrusakbary github.com/syrusakbary graphene-python.org 28 Graphene The Pythonic way to GraphQL
29. @syrusakbary github.com/syrusakbary graphene-python.org 29 • Most starred GraphQL framework (excluding FB implementation) - 1600⭐… much like! • Used by 100+ companies in production (Try.com included) • About 25k installs/month (growing 2x / 4 months) • Large community • Supports Python 2.7+ and 3.2+ (asyncio included!) Graphene Stats
30. @syrusakbary github.com/syrusakbary graphene-python.org 30 Who’s using it?
31. C O D E S A M P L E S
32. @syrusakbary github.com/syrusakbary graphene-python.org 32 { me { name talks { title time } } } class Query(graphene.ObjectType): me = graphene.Field(User) GraphQL Query Implementation
33. @syrusakbary github.com/syrusakbary graphene-python.org 33 { me { name talks { title time } } } class Query(graphene.ObjectType): me = graphene.Field(User) class User(graphene.ObjectType): name = graphene.String() talks = graphene.List(Talk) GraphQL Query Implementation
34. @syrusakbary github.com/syrusakbary graphene-python.org 34 { me { name talks { title time } } } class Query(graphene.ObjectType): me = graphene.Field(User) class User(graphene.ObjectType): name = graphene.String() talks = graphene.List(Talk) class Talk(graphene.ObjectType): title = graphene.String() time = DateTime() GraphQL Query Implementation
35. @syrusakbary github.com/syrusakbary graphene-python.org 35 class Query(graphene.ObjectType): me = graphene.Field(User) class User(graphene.ObjectType): name = graphene.String() talks = graphene.List(Talk) class Talk(graphene.ObjectType): title = graphene.String() time = DateTime()
36. P L AY G R O U N D
37. @syrusakbary github.com/syrusakbary graphene-python.org 37 graphene-python.org/playground
38. @syrusakbary github.com/syrusakbary graphene-python.org 38 • Django • Flask • WebOb • Pyramid • Pylons • Webapp2 • Sanic Server Integrations
39. A U TO M AT I C M A P P I N G F R O M O R M M O D E L S ❤
40. @syrusakbary github.com/syrusakbary graphene-python.org 40 • Django • SQLAlchemy • Google App Engine (NDB) • Pewee • PynamoDB ORM Integrations
41. B E S T P R A C T I C E S
42. @syrusakbary github.com/syrusakbary graphene-python.org 42 • No versioning in GraphQL • Append-only schema • No breaking changes introduced • Some fields can be marked as deprecated, throwing warnings in the client if used.
 (as long as the deprecated fields are being consumed, you will need to maintain them) • Delete fields/types only when they have 0 hits GraphQL versioning
43. @syrusakbary github.com/syrusakbary graphene-python.org 43 • An addition to the GraphQL spec • Node • Mechanism to refetch a object from the root given the object id. • Enforces a unique global id per object • Connection • Structured types for paginate resources easily Relay
44. @syrusakbary github.com/syrusakbary graphene-python.org 44 • Simplified & consistent API over remote data sources via batching and caching. • Initial implementation inspired by FB one, but there are more for python • github.com/facebook/dataloader • github.com/quora/asynq Dataloader
45. @syrusakbary github.com/syrusakbary graphene-python.org 45 Dataloader class UserLoader(DataLoader): def batch_load_fn(self, ids): # Here we return a promise that will result on the # corresponding user for each key in keys return get_all_users_given_its_ids(ids) user_loader = UserLoader() user_loader.load(1) # Return as Promise user_loader.load(2) # Return as Promise get_all_users_given_its_ids([1, 2]) Dataloader will batch the calls for you
46. G R A P H E N E Q U I R K S
47. @syrusakbary github.com/syrusakbary graphene-python.org 47 • Field resolution Sync by default Executors { me { name talks { title time } } } GraphQL Query 1 2 3 4 5 Sync Execution schema.execute( query, executor=SyncExecutor() )
48. @syrusakbary github.com/syrusakbary graphene-python.org 48 • Field resolution Sync by default • But can be changed! • Parallel field resolution… for free! • One line of code changed Executors { me { name talks { title time } } } GraphQL Query 1 2 2 3 3 Parallel Execution schema.execute( query, executor=ThreadExecutor() )
49. @syrusakbary github.com/syrusakbary graphene-python.org 49 • Affect the evaluation of fields in the query in runtime Middleware { me { name talks { title time } } } GraphQL Query context = { 'counters': defaultdict(int) } class CounterResolutionMiddleware(object): def resolve(self, next, root, args, context, info): key = '{}:{}'.format( info.parent_type.name, info.field_name ) context['counters'][key] += 1 return next(root, args, context, info) 1 1 1 5 5 Query:me User:name User:talks Talk:title Talk:time
50. T E S T I N G
51. @syrusakbary github.com/syrusakbary graphene-python.org 51 Graphene Test Client from graphene.test import Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } }
52. @syrusakbary github.com/syrusakbary graphene-python.org 52 Graphene Test Client from graphene.test import Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } }
53. @syrusakbary github.com/syrusakbary graphene-python.org 53 Graphene Test Client from graphene.test import Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } }
54. @syrusakbary github.com/syrusakbary graphene-python.org 54 Graphene Test Client from graphene.test import Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } }
55. @syrusakbary github.com/syrusakbary graphene-python.org 55 Graphene Test Client from graphene.test import Client client = Client(my_schema) def test_viewer(): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) assert executed == { 'data': { 'viewer': { 'name': 'Syrus' } } } Tedious to write
56. @syrusakbary github.com/syrusakbary graphene-python.org 56 Graphene Test Client from graphene.test import Client client = Client(my_schema) def test_viewer(snapshot): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) snapshot.assert_match(executed)
57. @syrusakbary github.com/syrusakbary graphene-python.org 57 Graphene Test Client from graphene.test import Client client = Client(my_schema) def test_viewer(snapshot): request = MockRequest() executed = client.execute('''{ viewer { name } }''', context_value=request) snapshot.assert_match(executed) # -*- coding: utf-8 -*- # snapshottest: v1 from __future__ import unicode_literals from snapshottest import Snapshot snapshots = Snapshot() snapshots['test_viewer 1'] = { 'data': { 'viewer': { 'name': 'Syrus' } } } Generated automatically tests/test_general.py tests/snapshots/snap_test_general.py
58. @syrusakbary github.com/syrusakbary graphene-python.org 58 • First Introduced by Jest (JS testing framework from FB) • Permits fast test iteration, as minimizes repetitive work • More tests / More code coverage ✅ • Happy dev / Happy PM Snapshot Testing
59. @syrusakbary github.com/syrusakbary graphene-python.org 59 SnapshotTest Demo github.com/syrusakbary/snapshottest
60. W H AT ’ S C O M I N G S O O N
61. @syrusakbary github.com/syrusakbary graphene-python.org 61 • Compiled Queries (GraphQL queries compiled to python code, ZERO library overhead) • Real Time subscriptions • Documentation… order of magnitude better! Coming Soon
62. G R A P H Q L I N F R O N T E N D
63. @syrusakbary github.com/syrusakbary graphene-python.org 63 React Relay facebook.github.io/relay/ Apollo Client dev.apollodata.com/
64. @syrusakbary github.com/syrusakbary graphene-python.org 64 React Relay in a Nutshell class User extends React.Component { render() { var {name, talks} = this.props.user; return ( <div> {name} <ul> {talks.map(talk) => ( <li key={talk.title}> {talk.title} (<em>{talk.time}</em>) </li> )} </ul> </div> ); } } Component definition User = Relay.createContainer(User, { fragments: { user: () => Relay.QL` fragment on User { name talks { title time } } `, }, }); Data Container
65. @syrusakbary github.com/syrusakbary graphene-python.org 65 Apollo Client in a Nutshell class User extends React.Component { render() { var {name, talks} = this.props.data.user; return ( <div> {name} <ul> {talks.map(talk) => ( <li key={talk.title}> {talk.title} (<em>{talk.time}</em>) </li> )} </ul> </div> ); } } User = graphql(gql` query { user { name, talks { title time } } } `)(User); Data Container Component definition
66. @syrusakbary github.com/syrusakbary graphene-python.org 66 • Facebook is behind it (long-term maintained, battle tested) • Static analysis of your GraphQL schema • Thinner JS bundles as the GraphQL engine is not embedded in the js bundle • Very performant • Only compatible with React • Forces the use of Relay in the Backend React Relay analysis
67. @syrusakbary github.com/syrusakbary graphene-python.org 67 • Simplified usage compared to relay/classic • More modularized code and abstracted from React • More integrations to come React Relay/modern?
68. @syrusakbary github.com/syrusakbary graphene-python.org 68 • Easier to use than React-Relay • Using Redux under the hood, easier to mutate the internal store • Doesn’t require to know your GraphQL schema ahead of time • Compatible with React, AngularJS, Vue, … • Apollo Dev tools Apollo Client analysis
69. W H Y U S E G R A P H Q L
70. @syrusakbary github.com/syrusakbary graphene-python.org 70 • Easier to maintain than REST API’s • 1-click documentation and UI (GraphiQL) • Quick integration in Frontend with React thanks to Relay • Seamless integration with your Python Backend • Much faster iteration/development process Why use GraphQL for your API?
71. Q U E S T I O N S ? : )